Development of a virtual classroom for pre-analytical phase of laboratory medicine for undergraduate medical students using the Delphi technique.

BACKGROUND: Amongst the pre-analytical, analytical, and post-analytical phase of laboratory testing, pre-analytical phase is the most error-prone. Knowledge gaps in understanding of pre-analytical factors are identified in the clinical years amongst undergraduate students due to lack of formal teaching modules on the pre-analytical phase. This study was conducted to seek experts' consensus in Clinical Chemistry on learning objectives and contents using the Delphi technique with an aim to develop an asynchronous virtual classroom for teaching pre-analytical factors of laboratory testing.
METHODS: A mixed method study was conducted at the Aga Khan University. A questionnaire comprising of 16 learning objectives and their associated triggers was developed on Google Docs for developing the case vignettes. A four-point Likert Scale, which included strongly agree, agree, disagree and strongly disagree, was utilized for the learning objectives. An open-ended question was included for experts to suggest new items for inclusion. A cut off of at least 75% agreement was set to establish consensus on each item. A total of 17 Chemical Pathology faculty from 13 institutions across Pakistan were invited to participate in the first round of Delphi. Similar method of response was used in round two to establish consensus on the newly identified items suggested by the faculty in round 1. Later, the agreed-upon objectives and triggers were used to develop interactive scenarios over Moodle to concurrently test and teach medical students in a nonchalant manner.
RESULTS: A total of 17 responses were received in Round 1 of the Delphi process (response rate = 100%), while 12 responses were received in Round 2 (response rate = 71%). In round 1, all 16 learning objectives reached the required consensus (≥ 75%) with no additional learning objectives suggested by the experts. Out of 75 triggers in round 1, 61 (81.3%) reached the consensus to be included while 39 were additionally suggested. In 2nd round, 17 out of 39 newly suggested triggers met the desired consensus. 14 triggers did not reach the consensus after two rounds, and were therefore eliminated. The virtual classroom developed using the agreed-upon learning objectives and triggers consisted of 20 items with a total score of 31 marks. The questions included multiple choice questions, fill in the blanks, drag and drop sequences and read-and-answer comprehensions. Specific learning points were included after each item and graphs and pictures were included for a vibrant experience.
CONCLUSION: We developed an effective and interactive virtual session with expert consensus on the pre-analytical phase of laboratory testing for undergraduate medical students which can be used for medical technologist, graduate students and fellows in Chemical Pathology.

Introduction

Quality and safety in diagnostic testing is essential in providing safe and effective patient care [1, 2]. Laboratory medicine plays a vital role in providing accurate results that are crucial to decision making and patient management in clinical practice [3, 4]. The entire total testing process in laboratory practice can be broadly divided in to three phases; the pre-analytical, analytical and the post- analytical phase [1, 2, 5]. According to the ISO 15189: 2012 standard for laboratory accreditation, the pre-analytical phase can be defined as ‘‘steps starting in chronological order from the clinician’s request and including the examination requisition, preparation of the patient, collection of the primary sample, and transportation to and within the laboratory, and ending when the analytical examination procedure begins” [6]. The analytical phase involves the processes of measurement and analysis of the required analyte, results’ technical validation and its release to the laboratory information management systems [5]. Finally, the interpretation, clinical authorization and communication of results to the requesting clinician constitutes the post-analytical phase [5]. Amongst all three, the pre-analytical phase is regarded as the most error prone phase [3, 4, 7]. According to the literature, the analytical errors account for less than 10%, post analytical errors for 18.5 to 47%, whereas the pre-analytical phase accounts for up to 70% of the total laboratory diagnostic errors [1, 3, 4, 7]. Laboratory errors can occur in any step of these phases and can be defined as “any defect from ordering tests to reporting results and appropriately interpreting and reacting on these" [2, 6]. With the advent of technology, automation and improvement in quality assurance and control measures, literature reports a ten-fold reduction in analytical errors in last two decades but pre-analytical issues is still the area of challenge for the laboratory professionals [1, 3, 6, 7]. Such errors may impact the patient care and outcome in-terms of delayed diagnosis, longer stays at hospital, greater demand on resources and higher cost [3, 5]. Moreover, these errors can lead to more serious hazards of misdiagnosis and mismanagement in clinical practice [5]. The most commonly encountered pre-analytical errors include noncompliance of instructions (like proper fasting, avoiding some special food, adherence of sample collection time, administration of drugs etc.), inaccurate quality and insufficient samples, missing or wrong identification of patient, missing sample or test requisition, inappropriate containers, contamination, inappropriate transport and ineffective storage conditions [2–4, 6].

Despite the increasing attention and importance of prevention of the errors for quality improvement, there is lack of formal education of laboratory medicine in most of the medical school curricula across the globe [8-11]. Studies have reported inadequate knowledge of undergraduate medical students about various phases of pathology testing [9]. Inadequate knowledge in this field, makes medical students more prone to inappropriate test ordering and mistakes in interpreting the results as future health care providers [10]. Lack of formal and sufficient training and knowledge in interns and residents performing the pre-analytical phase, leads to erroneous laboratory testing and results [12]. Moreover, due to insufficient knowledge about pre-analytical variables, the erroneous results if released from the labs may not be recognized as errors by the treating clinicians [11]. Recognition of this gap in education and the mandate from the regulatory bodies necessitate that the students be well equipped with knowledge and trained to identify and control pre-analytical errors.

In educational research, lack of formal curriculum provides the opportunity to use consensus group method to determine the components of a new course development [13]. One of the most widely used consensus method in medical education is the Delphi technique that accounts for approximately 75% studies utilizing consensus group method in this field [13, 14]. The technique was originally developed by Research and Development (RAND) Corporation and is based on ‘‘achieving consensual agreement among expert panelists through repeated iterations of anonymized opinions and of proposed compromise statements from the group moderator” [13, 14]. Moreover, the use of internet and convenience of electronic mails has made Delphi method both time- and cost-effective, thereby increasing its use and popularity among researchers [13, 14]. Therefore, this study was conducted to seek experts’ consensus in Clinical Chemistry on learning objectives (LOs) and content using the Delphi (or e-Delphi) technique. The aim was to develop an asynchronous virtual classroom using virtual learning environment (VLE) with consensus from subject experts from various institutes for teaching pre-analytical factors of laboratory testing to undergraduate medical students.

Methods

A mixed method study was conducted at the medical college of the Aga Khan University, Karachi from December 2020 to May 2021. Exemption from ethical approval for this study was obtained from the institutional ethical review committee (2021-5823-15415). The study was conducted in three phases, Phase I was the Delphi technique for curriculum development, phase II consisted of virtual class creation on VLE and in phase 3 the class was conducted followed by feedback from students.

Phase I: Delphi technique for curriculum development

Two rounds of Delphi were conducted. In the first round, based on the literature search, a questionnaire comprising of 16 learning objectives and their associated triggers for case vignettes was developed on Google Forms. The learning objectives and triggers were extracted after a thorough review of literature on the most common pre-analytical errors in a Chemical Pathology lab [3, 4]. Extracted data was carefully reviewed for relevance to patient safety by the in-house Chemical Pathology faculty and were included as learning objectives. A total of seventeen Chemical Pathology faculty from thirteen institutions across Pakistan were invited to participate in the study. The panelists were registered Chemical Pathology Fellows with College of Physicians and Surgeons of Pakistan and were involved in active clinical and academic practice associated with a tertiary care facility. Feedback on learning objectives and related triggers was obtained from the selected panel to seek consensus via Google Forms questionnaire through email. A written consent was obtained from all the panel members prior to their participation in the Delphi process. The consent was obtained via email which outlined the Delphi process, expectations from the panel members, and how the data will be used afterwards. A four-point Likert Scale, which included strongly agree, agree, disagree and strongly disagree, was utilized for the learning objectives and the associated triggers were to be selected. An open-ended question was included for experts to suggest new items for inclusion. A cut off of at least 75% agreement was set to establish consensus on each item, in similarity to the use of percentage agreement as reported by literature [15, 16]. Similar method of response was used in round 2 to establish consensus on the newly identified items suggested by the faculty in round 1. The data was only accessible to researchers and the anonymity and confidentiality of the participants was ensured.

Phase II: Virtual classroom development

Later, the agreed-upon objectives and triggers were used to develop an asynchronous session with interactive case scenarios over VLE (Moodle) to concurrently test and teach year III medical students in a nonchalant manner. Moodle is a user-friendly tool that can develop vibrant classrooms for active learning of students by increasing their participation through visually appealing question stems and explanations [17, 18]. The virtual classroom was developed by the faculty in the Section of Clinical Chemistry, Department of Pathology & Laboratory Medicine at AKU. The faculty received training for the use of Moodle for classroom development and was aided by the university’s information technology (IT) department in the process. The virtual classroom was developed using the H5P plugin and consisted of formative quiz of various items built within the content and learning material to stimulate learning and activate prior knowledge. Each case scenario was followed by an interactive quiz or game. The interactive quiz included multiple choice questions, fill in the blanks, drag and drop sequences and read-and-answer comprehensions. Specific learning points to reinforce and clarify the concepts were added after each item and graphs and pictures were included for a vibrant experience.

Phase III: Virtual classroom conduction/Post activity survey

By definition, the virtual classroom equates to the process that occurs when teacher, learner, problem and knowledge interact through information and communication technologies for the purpose of learning [19]. Each learning objective was made into a clinical case scenario to represent the most common pre-analytical errors pertinent to that case, while all the triggers with that learning objective were included within each case and represented in the case scenarios. The purpose of developing the case scenarios was to create an engaging simulated environment for the students. With the help of learning objectives real or simulated managerial situations were created for the students to identify potential pre-analytical errors and take action. The questions for which clinical scenarios could not be developed were included as a drag-and-drop sequence. Information related to the learning objective and triggers was included in as a slide right after each question so students can review what they just answered. Visual aids, figures, cartoons and mnemonics were included as learning aid to ease the learning process as compared to the conventional lecture-based classroom. The virtual class was added in a module, Back to Basics, in the beginning of the third year of undergraduate medical; education training. After completion of the virtual class students were directed to a feedback form. Feedback of students was obtained using a questionnaire on VLE, after the completion of session regarding students learning and satisfaction from the session. The students were required to mark their satisfaction on a Likert scale ranging from 1 (not satisfied) to 4 (highly satisfied).

Statistical analysis

The information was derived from Moodle after the completion of the classroom session. Data was transferred from Microsoft excel to SPSS. For statistical analysis IBM SPSS Statistics version 21 was used. We analyzed binary data and quantitative data separately. Frequencies, means and standard deviations were generated for quantitative data.

Results

A total of seventeen responses from the panelist were received in Round 1 of the Delphi process (response rate = 100%), while twelve responses were received in Round 2 (response rate = 71%). The mean work experience of the panelist was found to be 10.8 ± 7.6 SD years.

In round 1, all sixteen learning objectives reached the required consensus (of >75%), with nine LOs (56.25 percent) reaching 100% consensus while the others achieving 82–94% agreement. There were no additional learning objectives suggested by the experts in this round. Out of seventy-five triggers in round 1, sixty-one (81.3 percent) reached the consensus to be included while thirty-nine new triggers were additionally suggested. In 2nd round, seventeen (43.5 percent) of the 39 newly suggested triggers met the desired consensus. Fourteen triggers did not reach the consensus after round one, while 22 triggers did not reach consensus after round two and were therefore eliminated after each round, respectively.

A total of sixteen LOs, with 78 triggers (Table 1) were finalized to develop the virtual session on pre-analytical factors of laboratory testing. The LOs were related to common error prone steps including identification of vacutainer and their properties, correct order of draw, effect of sampling and storage conditions on analytes, effect of certain factors such as temperature, posture, time, food or medicine consumption, etc., on their relevant analytes or procedures, and many more. As shown in Table 1, each LO was further accompanied by the set of most relevant triggers that were agreed upon by the experts and were used to develop the relevant case scenarios and learning material.

Table 1

Agreed-upon learning objectives and triggers to be included in the final classroom.

Learning Objectives	Triggers for Case Vignettes
1. Identify the vacutainers available in clinical laboratory and their properties	• EDTA• Lithium heparin• Serum separator tubes• Sodium fluoride• Don’t take serum calcium in EDTA• Don’t take serum lithium in lithium heparin• Lithium Heparin Syringe for Blood Gases• Citrate Tube for Coagulation
2. Identify the correct order of draw for laboratory investigations	• Blood culture tube• Coagulation tube (blue cap)• Serum tube (no additives, red cap)• Heparin tube (green cap)• EDTA tube (lavender cap)• Glycolysis inhibitor (gray cap)
3. Relate the effect of sampling and storage conditions of specimens on analytes like PTH, porphyrins, ammonia, lactic acid	• PTH and temperature• Porphyrin and sunlight• Ammonia and temperature• Lactic acid and temperature• Delay in transport• Shield from light• Tourniquet on for <1min• Blood gases <3 mins arrival
4. Relate the effect of temperature, heparin, improper mixing and air on ABG results	• Air bubble• Venous blood blue• Improper sample mixing• Delay in transport• Ice
5. Apply International Patient Safety Goal One: Identify patients correctly	• Sample switch• 2 patient identifiers• Bar code• Labelling the sample at the bedside labeling
6. Relate time of blood collection, dosage, and other medications interferences on therapeutic drug monitoring	• Time• Dosage• Peak• Trough• Sample with appropriate information of timings (peak or trough)• Chain of custody
7. Apply the 24-hour urine collection protocol	• Complete 24-hour collection of urine• Low urine creatinine• Assure the right timings of collection followed• Discard first sample in morning and collect the first sample next day
8. Relate the effect of posture, temperature, sampling and storage conditions on renin issues	• Lying• Standing• Ice• Delay in transport• Exercise/stress/sitting before sampling
9. Identify the indications for urine amino acids testing	• LPI• Hartnup• Cystinuria• Fanconis Syndrome• RTA• Dirty sample can clog the HPLC column
10. Identify the Triple Test pre-requisites	• Gestational week 14–20 weeks• Ultrasound scan• Number of pregnancies• Age of mother• History of previous positive child/fetus
11. Define optimal fasting and relate the effect of food consumption and medications on some analytes like triglycerides, amino acids, Metanephrine, 5-HIAA and VMA	• Triglycerides and fasting• Amino acids and fasting• Metanephrine• 5-HIAA• VMA• Chocolates, coffee, tea, banana
12. Relate Circadian rhythm with hormones	• Cortisol fluctuates• FSH, LH and menstrual cycle• ACTH
13. Relate the effect of prolonged tourniquet on serum potassium	• Relate the effect of prolonged tourniquet on serum potassium, calcium• Effect of prolonged tourniquet on Proteins and albumin
14. Relate the values of serum TSH in patients who are critically ill	• Sick thyroid• Avoid TSH sampling in sick patients• After cardiac event
15. Detect intravenous line contamination using the “IF Glucose > 800 mg/dL AND creatinine < 0.6 mg/dL” rule	• High glucose• High sodium• Detect intravenous line contamination using the “IF Glucose > 800 mg/dL AND creatinine < 0.6 mg/dL” rule• Correlate with tests in hematology
16. Identify spectrophotometric estimate of the level of interference from hemoglobin (hemolysis index), bilirubin (icterus index) and lipids and chylomicrons (lipemia index) as indicators for pre-analytical errors.	• Bilirubin interferes with serum creatinine• Sample processing• High triglycerides can cause falsely low electrolytes

Seventy-six students (75.2%) of the class of year III MBBS attended this asynchronous virtual teaching session via Moodle. Feedback was obtained regarding students learning which indicated that 77.6% did not have previous knowledge about pre-analytical phase of laboratory medicine while 93.4% of students learned new information in the session as shown in Figs 1 and 2. Majority of the students (80.4%) were found to be satisfied (57.9%) or highly satisfied (22.4%) from the content and structure of the virtual classroom as shown in Fig 3.

Fig 1

Students’ responses regarding prior knowledge related to the topic.

Fig 2

Survey responses if the class was successful in teaching something new to the students.

Fig 3

Survey responses regarding overall satisfaction of students with the developed classroom.

Discussion

The topic of ‘pre-analytical phase of laboratory medicine’ is not a formal component of undergraduate or post-graduate curriculum of medical education in Pakistan. Lack of a formal structured modules, gave us the opportunity to seek consensus from various experts on the essential learning objectives for this session. To the best of our knowledge, this is the first study, especially in our local context, to utilize the Delphi technique for developing a formal session on pre-analytical phase and errors of laboratory medicine for undergraduate medical students. The Delphi method was suitable for our study due to its consensus-driven methodology, anonymity, flexibility and cost effectiveness [20, 21]. However, some studies have reported the lack of both standardization in methodology and well-defined reporting criteria as its limitation [13]. Comparable to our study, some studies have used the mixed approaches [15, 22] whereas others have used quantitative or qualitative only, while utilizing Delphi for curricular or training content [23-25].

The LOs and associated triggers identified in our study for developing a session on pre-analytical phase(Table 1) are in agreement with the basic principles of laboratory medicine, as reported by Park & Marques [26]. These LOs and triggers are also in agreement with the most common type of errors of pre-analytical phase of laboratory testing as mentioned by various studies [3, 6, 27, 28]. The virtual session was designed for year III medical students as an interactive, case-based session with a quiz and problem-centered approach to effectively engage and motivate the students. Introduction of such training and information at the level of undergraduate curriculum is supported by studies [29, 30] while some have suggested it in even the pre-clinical years [26, 31] or spread over the entire continuum of curriculum as proposed by Smith et al. [30].

It was interesting to note, that despite no previous formal teaching, 22.4 percent of the students reported to have been taught about pre-analytical phase to some extent. This implies that there may be an element of hidden curriculum or informal teaching which imparts knowledge to the students during pre-clinical years or clerk-ship rotations. The level of satisfaction of majority students in our study reflects utility of our session design in terms of both engaging the students, acquiring new knowledge and fostering their learning. Such level of satisfaction is in agreement with other studies with similar introduction of laboratory medicine teachings [9, 11, 32]. The developed classroom can not only be used for undergraduate medical students, but it can also serve as a useful tool for Pathology & Laboratory Medicine residents, fellows and technologists.

This study also had some limitations. Other experts with lack of internet access and technology limitations may have been missed. Moreover, it was a single session, hence, prior knowledge of students and the gain in knowledge post-session was not determined to assess the effect of the teaching session. Although only clinical pathology specialists may attain a truly comprehensive level of proficiency, it is recommended that all health professional should have the basic understanding of essential principles that guide the choice, interpretation and identification of errors of laboratory tests and results. Hence, virtual session like these can serve the purpose not only in time-effective manner but also for a wide range of learners, ranging from undergraduate, postgraduate students to health professionals at all level of expertise across disciplines as a ‘continuing medical education’ program. Research is needed to design similar sessions with integration of hands-on experiences and to assess the impact and outcome of such teaching and training with respect to reduction of pre-analytical errors in practice.

Conclusion

An effective and interactive virtual session with expert consensus on the pre-analytical phase of laboratory testing for undergraduate medical students was developed which can be used for medical technologist, graduate students and fellows in Chemical Pathology and other disciplines.

8 Sep 2021

PONE-D-21-24285

Development of a Virtual Classroom for Pre-Analytical Phase of Laboratory Medicine for Undergraduate Medical Students Using the Delphi Technique

PLOS ONE

Dear Dr. Jafri,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Oct 23 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see:  http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols . Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at  https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols .

We look forward to receiving your revised manuscript.

Kind regards,

Gwo-Jen Hwang

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please amend your current ethics statement to address the following concerns:

a) Did participants provide their written or verbal informed consent to participate in this study?

b) If consent was verbal, please explain i) why written consent was not obtained, ii) how you documented participant consent, and iii) whether the ethics committees/IRB approved this consent procedure.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: No

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: No

Reviewer #2: No

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Dear author,

-methods section, “rounds of Delphi were conducted. In the first round, based on the literature search, a questionnaire comprising of 16 learning objectives and their associated triggers for case vignettes was developed on Google Forms.” But the questionnaire item lack references support and reality and validity, moreover how the development of the study’s related questionnaire is unclear.

-for more contributions, suggest authors add how to Virtual classroom development and are vague, please cite related references.

-a big problem, the study lack a statistic analysis section, to provide the result section.

Reviewer #2: The development of educational content for the pre-analysis phase using Delphi technology, it’s important and meaningful for patient safety. There are some suggestions before publishing.

1. In the first round of Delphi development phase. The authors claim based on the literature search for curriculum development. Finally, a questionnaire comprising of 16 learning objectives and their associated triggers for expert review. Please describe the process of developing the 16 learning objectives and their associated triggers or cite the reference.

2. Of the seventy-five triggers in round 1, 61 (81.3%) reached a consensus for inclusion. However, it is not clear how the triggers that are excluded be treated? Will it be included in the second round of discussion or deleted? As well as how did the 39 new triggers were proposed in 2nd round? In addition to the experts' votes, please describe why they were considered or removed from these triggers.

3. The content of Phase II: Virtual classroom development process is not clearly described. How have the 16 learning objectives and 78 triggers in Table 1 been applied to the learning material in the virtual classroom? How are they different from those of traditional online tests?

3 Nov 2021

1. Reviewer #1: Dear author,

-methods section, “rounds of Delphi were conducted. In the first round, based on the literature search, a questionnaire comprising of 16 learning objectives and their associated triggers for case vignettes was developed on Google Forms.” But the questionnaire item lack references support and reality and validity, moreover how the development of the study’s related questionnaire is unclear.

Author’s response: Details about the development of the questionnaire are added in the methods section. Page 4, Line 26: “The learning objectives and triggers were extracted after a thorough review of literature on the most common pre-analytical errors in a Chemical Pathology lab. [15, 16] Extracted data was carefully reviewed for relevance to patient safety by the in-house Chemical Pathology faculty and were included as learning objectives.”

2. -for more contributions, suggest authors add how to Virtual classroom development and are vague, please cite related references.

Author’s response: Details regarding the development of virtual classroom over Moodle have been added to the methods section. Page 5, Line 10: “Moodle is a user-friendly tool that can develop vibrant classrooms for active learning of students by increasing their participation through visually appealing question stems and explanations. [19, 20] The virtual classroom was developed by the faculty in the Section of Clinical Chemistry, Department of Pathology & Laboratory Medicine at AKU. The faculty received training for the use of Moodle for classroom development and was aided by the university’s information technology (IT) department in the process. The virtual classroom was developed using the H5P plugin and, consisted of a formative quiz of various items built within the content and learning material to stimulate learning and activate prior knowledge. Each case scenario was followed by an interactive quiz or game. The interactive quiz included multiple choice questions, fill in the blanks, drag and drop sequences and read-and-answer comprehensions. Specific learning points to reinforce and clarify the concepts were included added after each item and graphs and pictures were included for a vibrant experience.”

3. -a big problem, the study lack a statistic analysis section, to provide the result section.

Author’s response: As suggested, the statistical analysis section has been appropriately added after the methods section. Page 5, Line 3: “The information was derived from Moodle after the completion of the classroom session. Data was transferred from Microsoft excel to SPSS. For statistical analysis IBM SPSS Statistics version 21 was used. We analyzed binary data and quantitative data separately. Frequencies, means and standard deviations were generated for quantitative data.”

4. Reviewer #2: The development of educational content for the pre-analysis phase using Delphi technology, it’s important and meaningful for patient safety. There are some suggestions before publishing. In the first round of Delphi development phase. The authors claim based on the literature search for curriculum development. Finally, a questionnaire comprising of 16 learning objectives and their associated triggers for expert review. Please describe the process of developing the 16 learning objectives and their associated triggers or cite the reference.

Author’s response: Details about the development of the questionnaire are added in the methods section. Page 4, Line 26: “The learning objectives and triggers were extracted after a thorough review of literature on the most common pre-analytical errors in a Chemical Pathology lab. [15, 16] Extracted data was carefully reviewed for relevance to patient safety by the in-house Chemical Pathology faculty and were included as learning objectives.”

5. Of the seventy-five triggers in round 1, 61 (81.3%) reached a consensus for inclusion. However, it is not clear how the triggers that are excluded be treated? Will it be included in the second round of discussion or deleted? As well as how did the 39 new triggers were proposed in 2nd round? In addition to the experts' votes, please describe why they were considered or removed from these triggers.

Author’s response: The proposed changes have been included the results section. Page 6, line 5: “Fourteen triggers did not reach the consensus after round one, while 22 triggers did not reach consensus after round two and were therefore eliminated after each round, respectively.”

6. The content of Phase II: Virtual classroom development process is not clearly described. How have the 16 learning objectives and 78 triggers in Table 1 been applied to the learning material in the virtual classroom? How are they different from those of traditional online tests?

Author’s response: As suggested, the details about the development of the virtual classroom from the agreed-upon learning objectives and triggers have been included in the methods section.

Page 5, Line 25: "Each learning objective was made into a clinical case scenario to represent the most common pre-analytical errors pertinent to that case, while all the triggers with that learning objective were included within each case and represented in the case scenarios. The purpose of developing the case scenarios was to create an engaging simulated environment for the students. With the help of learning objectives real or simulated managerial situations were created for the students to identify potential pre-analytical errors and take action. The questions for which clinical scenarios could not be developed were included as a drag-and-drop sequence. Information related to the learning objective and triggers was included in as a slide right after each question so students can review what they just answered. Visual aids, figures, cartoons and mnemonics were included as learning aid to ease the learning process as compared to the conventional lecture-based classroom.”

Submitted filename: Response to reviewers.docx

Click here for additional data file.

17 Jan 2022

1 Feb 2022

1/27/2022

Vineet Kumar Rai, PhD

Academic Editor

PLOS ONE

Subject: Rebuttal Letter-'Response to Reviewers': PONE-D-21-24285R1; Development of a Virtual Classroom for Pre-Analytical Phase of Laboratory Medicine for Undergraduate Medical Students Using the Delphi Technique

Dear Editor,

Please find point by point response to all comments and suggestions made by reviewers.

• Journal Requirements: Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice. [Note: HTML markup is below. Please do not edit.]

Author’s Response: All references have been verified.

• Reviewers' comments: Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: (No Response)

Reviewer #2: All comments have been addressed

Author’s Response: Not needed

• 2. Is the manuscript technically sound, and do the data support the conclusions?

Reviewer #1: (No Response)

Reviewer #2: Partly

Author’s Response: Not needed

• 3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: (No Response)

Reviewer #2: N/A

Author’s Response: Not needed

• 4. Have the authors made all data underlying the findings in their manuscript fully available?

Reviewer #1: (No Response)

Reviewer #2: Yes

Author’s Response: Not needed

• 5. Is the manuscript presented in an intelligible fashion and written in standard English?

Reviewer #1: (No Response)\\

Reviewer #2: Yes

Author’s Response: Not needed

• 6. Review Comments to the Author

Reviewer #1: DEAR Authors, I suggest that the review’s add my initial opinion with the revised submission an itemized, point-by-point response to the comments of the reviewers. Thank you.

Reviewer #2: (No Response)

Author’s Response to Reviewer One:

Dear Reviewer these were the suggestions and comments shared by you

“ 1. We note you state consent was obtained from the participants of this study. However, you have not clarified the type of consent provided. Before we can proceed, please address the following prompts:

a) Did participants provide their written or verbal informed consent to participate in this study?

b) If consent was verbal, please explain i) why written consent was not obtained, ii) how you documented participant consent, and iii) whether the ethics committees/IRB approved this consent procedure.”

We have below itemized point-by point response to your comments made then:

Reviewer comment “ 1. We note you state consent was obtained from the participants of this study. However, you have not clarified the type of consent provided. Before we can proceed, please address the following prompts:

a) Did participants provide their written or verbal informed consent to participate in this study?

Author’s Response: Details about the written consent obtained from the participants was included in the manuscript in the methods section. “A written consent was obtained from all the panel members prior to their participation in the Delphi process. The consent was obtained via email which outlined the Delphi process, expectations from the panel members, and how the data will be used afterwards.” (Page 4; Line 35)

b) If consent was verbal, please explain i) why written consent was not obtained, ii) how you documented participant consent, and iii) whether the ethics committees/IRB approved this consent procedure.”

Author’s Response: We did not take verbal consent. Written informed consent was taken from all participants via email and permission from Ethical Review Committee was taken for written consent.

• 7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

Author’s Response: Not needed

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Author’s Response: We have uploaded and improved the figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic to ensure that figures meet PLOS requirements

Thank you

Dr Lena Jafri,

MBBS, FCPS, FAIMER fellow Philadelphia

Department of Pathology and Laboratory Medicine

Medical College, Pakistan, The Aga Khan University

Stadium Road, P.O. Box 3500, Karachi 74800, Pakistan

Submitted filename: Response to Reviewers.docx

Click here for additional data file.

11 Feb 2022

Development of a Virtual Classroom for Pre-Analytical Phase of Laboratory Medicine for Undergraduate Medical Students Using the Delphi Technique

PONE-D-21-24285R2

Dear Dr. Jafri,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Vineet Kumar Rai, PhD

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

9 Mar 2022

PONE-D-21-24285R2

Development of a Virtual Classroom for Pre-Analytical Phase of Laboratory Medicine for Undergraduate Medical Students Using the Delphi Technique

Dear Dr. Jafri:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Vineet Kumar Rai

Academic Editor

PLOS ONE