Effectiveness of Modalities to Teach Evidence Based Medicine to Pediatric Clerkship Students: A Randomized Controlled Trial.

OBJECTIVE: To evaluate the effectiveness of a traditional didactic session (TDS) as compared to a self-paced, interactive, multimedia module (SPM) on the application of evidence-based medicine (EBM) skills among medical students during their inpatient pediatric rotation.
METHODS: We conducted a randomized controlled trial from June, 2017 to June, 2018 at a quaternary care children's hospital. Students were randomized to TDS or SPM during each 2-week block. All students completed a critical appraisal tool (CAT) of evidence related to a clinical question in a standardized appraisal form and self-reflected about the EBM process. The primary outcome was the numeric score of the CAT derived by using the validated Fresno tool. Secondary outcomes of knowledge, attitudes, confidence, and self-reported behaviors related to EBM were measured using validated surveys. Statistical analysis was performed using Student's t test for CAT scores and mixed-model procedure (PROC MIXED), with subject as random effect and time as repeated measure for the secondary outcomes.
RESULTS: One hundred twenty-seven clerkship students were included. Overall, there was no significant difference in mean CAT scores for TDS (n = 59) versus SPM (n = 66) groups (90.3 vs 92.0, P = .65). There were no significant differences between SPM and TDS groups for knowledge (P = .66), attitudes (P = .97), confidence (P  = .55), and accessing evidence (P = .27). Both groups showed significant gains in knowledge, attitudes, confidence, and accessing evidence from baseline to postcourse. Improvements in knowledge and confidence were sustained at 3-months.
CONCLUSION: A SPM learning module is as effective as a TDS module for application of EBM concepts and knowledge to patient care.

RCT Entities:   Population Interventions Outcomes

A self-paced interactive online module is as effective as a traditional didactic session for teaching evidence-based medicine to pediatric clerkship students. No significant differences existed in the critically appraised topic score. Both modalities resulted in improved attitudes, confidence and behaviors.

Alt-text: Unlabelled box

In 2014, the Association of American Medical Colleges created Core Entrustable Professional Activities (EPAs), which serve as a framework for activities that graduating medical students (MS) should be able to perform at the start of residency. EPA 7 is focused on evidence-based medicine (EBM) in which MS are expected to form clinical questions and retrieve evidence to advance patient care. The inclusion of the EPA 7 underscores the need for effective EBM curricula for MS to help them prepare for careers as practicing physicians.

Several studies have evaluated the effectiveness of teaching EBM to undergraduate MS utilizing various methods, but there is no standardized approach to teach EBM in the undergraduate medical setting. Prior methods of EBM teaching have been quite heterogeneous and included problem-based learning, peer-assisted learning, lectures, mini-courses, seminars, or self-instructed learning. , The outcome measures and the results in these studies have varied greatly, further emphasizing a lack in standardization in teaching EBM to undergraduate MS.

Studies report that students are more likely to adopt EBM principles when there is a demonstrable link between classroom teaching and clinical application. , For this reason, integration of EBM teaching is important during clerkships when students engage in patient care. Historically, traditional didactic sessions (TDS) have been a mainstay method of teaching, , , , but TDS may be difficult to sustain during clerkships due to time constraints of students and limited availability of instructors. Furthermore, amid the COVID-19 pandemic, there has also been a rapid shift to promote asynchronous learning through electronic platforms.11, 12, 13 Electronic self-paced modules (SPM) may be a promising alternative given the flexibility it offers students and the appeal it has to the millennial generation.14, 15, 16 Studies have compared TDS to SPM in teaching EBM to undergraduate MS, and have found SPM as effective as TDS in improving knowledge and attitudes. , , However, there are limited data to investigate the effectiveness of SPM for enhancing the application of EBM knowledge and skills to actual patients in the clinical setting. , , , ,

We aimed to compare the effectiveness of a single traditional classroom session (TDS) versus an interactive self-paced, multimedia module (SPM) on the application of EBM skills to an actual patient encounter among pediatric clerkship students. As a secondary aim, we compared changes in knowledge, attitudes, confidence and ability to access evidence between TDS and SPM teaching methods and describe student reflections regarding EBM.

Methods

Population and Setting

We performed a cluster randomized controlled trial at a large, academic, quaternary care children's hospital in Houston, Texas from June, 2017 to June, 2018. The children's hospital is affiliated with Baylor College of Medicine, which has a 4-year medical degree program. MS complete their preclinical training during the first 3 semesters. During the second-half of their second year, students begin their clinical rotations and finish their core rotations by December of their fourth year. Additionally, during the second half of their second year, students attend an EBM course comprised of 9 in-person weekly 1.5-hour sessions occurring at the medical school. Sessions are taught using team-based learning and cover fundamental EBM concepts including PICO (Patient, Intervention, Comparison, Outcome) questions, literature search, and appraisal of different study types.

Within the 6 week Pediatric core rotation, 2 weeks are spent in the inpatient setting on the Pediatric Hospital Medicine (PHM) service. The PHM team is comprised of 1 teaching attending, a senior supervising resident, 2–3 interns, 2–3 clerkship students and during various parts of the year, a PHM fellow and/or subintern. Second, third, and fourth year students on their core pediatrics rotation were enrolled in this study after providing consent. Subinterns were excluded from the study. The curriculum was mandatory for students to complete, however participation in the study surveys was optional. Results from the study were independent from students’ grades. The study was approved by the local Institutional Review Board.

Educational Intervention

The content of the TDS and SPM curriculum for the 2 groups was identical. The goals, objectives, and content were finalized after review of existing EBM curricula and discussion with stakeholders, which included MS, local EBM experts, faculty, and departmental educational leadership.

The content of the curriculum reviewed basic EBM principles, including forming a PICO question, tools to perform efficient literature searches, study design, and critical appraisal basics (Supplemental Document A). Additionally, all students received resources outlining ways to formulate a PICO question, how to perform an efficient literature search, relevant EBM formulas, and a Critically Appraised Topic (CAT) form.

The students randomized to TDS received teaching during the first day of their 2-week PHM block. The TDS included an interactive, in-person teaching session facilitated by faculty involved in this study. PHM faculty that participated in the division's EBM interest group volunteered to facilitate teaching sessions and underwent a training session led by an experienced instructor of EBM (S.W.). Student attendance was tracked with a sign-in sheet. A facilitator's guide was created to orient instructors to the content and format of the session. TDS was approximately 60 minutes in length and consisted of a powerpoint presentation with a case example to illustrate the basic concepts of EBM and its application to clinical practice. Various discussion points were embedded in the lecture to enhance interaction, assess student knowledge, and clarify EBM concepts.

The SPM module was an interactive, multimedia module with voice over narration. It was created using Microsoft PowerPoint and Camtasia screen recorder and video editor, covering the same topics in the TDS. It took approximately 60 hours to develop. Voiceover narration, video clips, and interactive short quizzes were incorporated throughout the module, which was exported as an interactive web page module. Incorrectly answered quiz questions prompted the module to go back to the relevant portion to review the concepts again before proceeding further. Students randomized to the SPM were asked to complete the module during the first week of the rotation. Students received emails at set times during the rotation (day 1 and 7) with reminders to complete the module.

During the second week of their PHM rotation, students in both groups were asked to formulate a clinical question based on a patient they cared for during the rotation. Students were expected to complete a CAT form based on the patient encounter. The form prompted students to create a PICO question, identify an ideal study design for their question, write search terms, select an article, and perform a basic appraisal of the article (Supplemental Document B). The students turned in the CAT form at the end of their rotation along with their history and physical exam note for the patient. To align with EPA 7, students were asked to write a reflection statement at the end of the CAT form asking what practicing EBM meant to them after completing the exercise.

Randomization and Blinding

We created a computer-generated randomization scheme, where each 2-week block of students were randomized to receive either TDS or SPM. The investigator (S.K.) created the scheme and was unaware of student rotation assignments. The rotation schedules for students were made by individuals outside the study team from the Undergraduate Medical Education office, who were unaware of the randomization schedule. Students were notified of their educational intervention at the start of their rotation. To avoid contamination between groups, only students assigned to the SPM were given instructions on how to access the learning module. Due to the nature of the intervention, we were unable to blind students and TDS facilitators. Blinding was utilized for assessment of the primary outcome. A single evaluator (A.D.), blinded to the group assignment and student names, graded the forms. AD did not facilitate any teaching sessions nor work on the inpatient service during the study period.

Measurements of Outcomes

The primary outcome was a numeric score given to the CAT forms based on the previously validated Fresno tool (Supplemental Document B). The Fresno tool was modified to include the domains taught within the session and to stay in-line with EPA 7. The modified Fresno tool evaluated 6 domains on the CAT form (PICO question, study design, search terms, validity, magnitude/statistical significance, and relevance; Supplemental Document C). Each section ranged from a maximum score of 16 to 24 points, and overall maximum score was 124. A single investigator (A.D.) scored the CAT forms. Prior to beginning scoring, A.D. participated in a training session with the primary investigator (T.H.) for which they reviewed four sample CATs together. A posthoc analysis for the reliability of the scoring was performed by independent review of 10% of the CATs by a second investigator (S.W.) and showed good inter-rater reliability (intraclass correlation coefficient 0.77). S.W. was blinded to the original scores.

Secondary outcomes were measured via a survey administered to the students before the educational intervention, at the end of the 2-week block, and 3 months posteducational intervention. The survey questions focused on knowledge, attitudes, confidence, and accessing evidence of various data sources. The survey was modeled after a previously validated KACE questionnaire (Supplemental Document C). The KACE questionnaire was originally developed for dental students, so minor modifications were performed to fit our study population. To assess the frequency of accessing evidence, questions from the “Critical Appraisal Skills Programme Workshop Evaluation Questionnaire” were also added to the survey. The survey was piloted with faculty within the study team and also cognitive interviews were done with 3 MS prior to the start of the study to ensure clarity and understandability of the questions.

Statistical Analysis

Descriptive statistics were used to describe the characteristics of students in the TDS and SPM groups. Characteristics were compared as per the intervention received using Pearson's chi-squared test and Fisher's exact test. The primary outcome of mean CAT score in each group was compared using the student's t test. Prior to using the t test, Levene's test was used to determine the homogeneity of variance and met the assumption (P = .55). The mean CAT scores were also compared in the subgroup of MS-2 students that had not yet taken the EBM course at the medical school using the Wilcoxon Rank Sum test. The secondary outcomes were average scores on knowledge, as well as 5-point Likert scale items for attitudes, confidence, and frequency of accessed data sources were analyzed using repeated-measures analysis using the mixed model procedures (PROC MIXED with REPEATED statement to take into account the covariance structure of the data). The protocol defined in PROC MIXED used the REML (restricted maximum likelihood) option with subjects as random effect and time as repeated measure with an autoregressive heterogeneous covariance matrix option. The autoregressive heterogeneous covariance matrix was used because the time period between sessions varied in the protocol. The least square means (LS means) statement of SAS PROC MIXED was used and compared between sessions and also sessions at each time point using the Student's t test. Missing data were assumed missing at random. To assess for attrition bias due to missing data at 3 months, a sensitivity analysis was performed with inclusion of only students who completed all 3 time points of data for secondary outcomes. A P level of ≤.05 was considered significant. Data analysis was performed using SAS version 9.4 (SAS Institute, NC).

A summative content analysis was performed to describe reflection statements and quantify categories of content based on key words. Two investigators (T.H. and S.W.) reviewed reflection statements to develop a code book of categories. TH quantified the proportion of reflections that contained key words within categories. Data was organized using Excel.

Results

One hundred twenty-seven of 130 students consented and enrolled in the study (Fig. 1 ). Sixty four students were randomized to the TDS group and 63 students to SPM. One session with 4 students was reassigned from TDS to SPM due to the lack of instructor availability during the week of a national holiday. There was 100% attendance for students randomized to TDS. Of the students enrolled, 49% were MS2s, 41% were MS3s, and 10% were MS4s. The characteristics of students in TDS and SPM groups were similar except for field of interest, where TDS had a larger percentage of students with multiple interests or undecided and SPM had a larger percentage of students with interest in internal medicine (Table 1 ).

Figure 1

Participant flow diagram. SPM indicates self-paced multimedia; TDS, traditional didactic session, and CAT, critically appraised topic.

Table 1

Participant Characteristics in SPM and TDS Groups

	SPM (%) n = 67	TDS (%) n = 60	P Value
Year in Medical School			.58*
MS2	33 (49)	30 (50)
MS3	26 (39)	26 (43)
MS4	8 (12)	4 (7)
Advanced degree	103 MPH, 3 MBA, 4 PhD	73 MPH, 4 PhD	.69*
Field of interest			.04†
Pediatrics	7 (10)	9 (15)
Surgery	12 (18)	9 (15)
Internal medicine	16 (24)	7 (12)
Obstetrics/gynecology	4 (6)	2 (3)
Multiple interests	0 (0)	7 (12)
Undecided	12 (18)	16 (27)
Other	15 (22)	12 (20)
Core rotations completed			.88*
0	8 (12)	5 (8)
1	15 (22)	11 (18)
2	10 (17)	14 (23)
3	13 (19)	11 (18)
4	9 (13)	9 (15)
5	12 (18)	10 (17)

SPM indicates self-paced multimedia; TDS, traditional didactic session; MS, medical school year; MPH, master of public health; MBA, master of business administration, and PhD, doctor of philosophy.

Pearson's chi-squared test.

Fisher's exact test

CAT forms were completed in 125 of the 127 students enrolled. Students reported spending an average of 90 minutes to complete it. There was no significant difference in mean CAT scores between the TDS and the SPM sessions (90.3, 95% confidence interval [CI], 84.9–95.6 vs 92.0, 95% CI 86.6–97.4, mean difference = 1.7, t value = −0.45, P = .65). For the subgroup of MS-2 students (n = 11) who did not complete the medical college EBM course prior to participation, the mean CAT scores were 73.0 (95% CI, 59.7–86.2) and 82.5 (95% CI, 59.7–105.3) for SPM and TDS, respectively (P= .78). Secondary outcomes were available via questionnaire for 100% (127/127) precourse, 95% (121/127) postcourse and 33% (42/127) 3 months postcourse. Baseline pretest knowledge, attitudes, confidence, and accessing evidence were not statistically different between groups (Table 2 ). Small but statistically significant improvements were seen in all outcomes from precourse to postcourse in both TDS and SPM interventions (Table 2, Fig. 2 ). Improvements were sustained from precourse to 3-month postintervention for knowledge and confidence for SPM and TDS (Table 2, Fig. 2). Accessing evidence did improve immediately postintervention but was not sustained at 3 months (Table 2, Fig. 2). Similarly, attitudes toward EBM had a small but statistically significant improvement overall but there was a slight decline in attitudes between postintervention and 3 months. There were no significant differences in knowledge, attitudes, confidence, and accessing evidence between TDS and SPM groups across time points (Table 2, Supplemental Table 1). UpToDate and colleagues were the most commonly used resources, while metasearch engines were used least frequently.

Table 2

Least Square Mean Difference of Modules at Each Time Point and Between Modules

		Knowledge		Attitudes		Confidence		Accessing Evidence
Time 1	Time 2	Change*	P Value	Change*	P Value	Change*	P Value	Change*	P Value
Modules at each time point
Traditional
Pre	Post	11.5	<.01	.45	<.01	.84	<.01	.12	.03
Pre	3-mo Post	10.4	.01	.22	.05	.92	<.01	.06	.60
Post	3-mo Post	−1.2	.74	−.23	.02	.79	.57	−.06	.58
Self-paced
Pre	Post	7.0	<.01	.39	<.01	.77	<.01	.15	<0.01
Pre	3-mo Post	11.2	<.01	.18	.06	.72	<.01	.05	.65
Post	3-mo Post	4.1	.17	−.21	.02	−.05	.70	−.11	.26
Comparison between modules
Traditional	Self-paced
Pre	Pre	−.10	.98	−.03	.67	−.02	.85	.09	.24
Post	Post	4.4	.11	.03	.71	.04	.69	.06	.47
3-mo Post	3-mo Post	−.89	.85	.006	.96	.17	.36	.10	.49

For modules at each time point: Change* = Time 2 – Time 1; a positive change indicates improvement and negative change indicates worsening of the outcome.

Pre- to 3 month postcomparison was used to assess sustainment of outcomes and post to 3 month postcomparison was used to assess any decline in the outcome.

P values <.05 are bolded.

Figure 2

Average responses (with error bars) of knowledge, attitudes, confidence, and self-reported behavior for traditional (TDS) and self-paced modules (SPM) pre-, post-, and 3 months postcourse. Knowledge was evaluated based on 8 questions on pre and post questionnaires. Attitude, confidence and accessing evidence were evaluated using 5-point Likert scale questions.

Due to low 3 month postcourse survey completion, a sensitivity analysis was performed on the 41/127 (32.2%) students who completed surveys at all time points, pre-, post-, and 3-months post. The results were similar to the full population with improvement in knowledge, attitudes, confidence, and accessing evidence between pre and post course time points in each group (Supplemental Fig. 1). Interestingly, improvements in confidence and accessing evidence postcourse were greater in TDS compared to SPM (P = .03, P = .05, respectively; Supplemental Fig. 1). Confidence was sustained at 3 months in TDS and SPM. However, just as with the full population, accessing evidence was not sustained at 3 months and attitudes declined (Supplemental Fig. 1).

To keep in-line with EPA7, students provided their perspectives on the EBM process in reflection statements. Summative content analysis was performed on 115 reflection statements and 165 key words were created. Six categories emerged: utility of EBM (n = 49, 30%), goal setting (n = 45, 28%), efficiency of integrating EBM in practice (n = 30, 18%), importance of EBM (n = 20, 12%), clinical application (n = 14, 9%), and the usefulness of the assignment (n = 7, 4%; Table 3 ).

Table 3

Content Analysis of Reflection Comments

Category	Keywords		Illustrative Quotes
Importance of EBM	- important- valid, objective- significant impact- continuously practiced- center of practice	- vital part- useful- routine part- cornerstone- opened my eyes	-“EBM is a vital part of practicing medicine and is becoming more and more relevant to our patients. As a result, it is important to include EBM in daily practice.” -“EBM should be the cornerstone of any practicing physician today.”
Goal setting	- incorporate- plan to use/keep up - will ask questions - intend to read/use/find answers/practice-will read	- will ask questions- look at guidelines - set a goal - need to improve - will critically appraise - build a habit- continue to apply	- “I will read at least a few articles per week to keep up with the latest advances in the field.” - “It will take more practice to effectively interpret such studies and trials, but my hope is that the more I incorporate into my own practice, the easier and more meaningful it will become.”
Utility of EBM	- apply PICO questions- provide the highest value of care- recognize gaps- use with mentees- reduce harm- comparing interventions- guide decisions - teaching current data	- uncertainty in medicine- promote patients' health- informed decisions - broaden knowledge - supplement knowledge - guide treatment choices- patient centered care- quality improvement	-“I hope to practice in an academic setting, I will be responsible for teaching current data to medical students and residents.” -“I plan to use EBM throughout my career as a guide to clinical decision making. I believe staying up to date with EBM is the best way to ensure that my patients overall get the care (they) deserve.”-“I plan to use EBM to supplement my knowledge for areas I am unsure, answer prognostic and epidemiologic questions, and continue learning about evolving treatment areas.”
Efficiency of Integrating EBM into Practice	- search tools- incorporation of evidence- synthesized evidence based resources- various search engines- literature review	- PICO based questions- clinical guidelines - peer reviewed resources- systematic reviews/meta-analysis- search techniques- PubMed and Cochrane	- “Efficient ways to access evidence-based medicine include Cochrane reviews, hospital-specific clinical guidelines and algorithms, meta-analyses, and systematic reviews. Another good resource is the RCT, especially if the magnitude of effect and quality of evidence is significant enough to warrant a change of guidelines. I intend to use the above mentioned resources in my practice.”
Clinical application	- management- diagnostic testing- diagnosis- counseling	- risk factors- management- imaging- antibiotic usage	- “Given the results of this study, I will carefully consider the adverse side effects and potential consequences of obtaining CT imaging in pediatric patients.”
Usefulness of assignment	- great learning tool- valuable - helpful	- will benefit me- useful	- “The skills that I have learned through this process will benefit me in the future as I can better evaluate sources I find.”- “This exercise was helpful because it make me think about a patient who wasn't my own and allowed me to really feel like I was a part of her care.”

Discussion

Our study is unique in that we compared two different teaching modalities via randomization to identify the most effective way to teach students how to apply EBM principles to real patient cases. We found no significant difference in CAT scores between the TDS and SPM groups, which suggests that SPM is as effective in teaching and applying EBM principles as the TDS in students who have had previous exposure to EBM. This finding is important for educators who aim to integrate EBM teaching in the clinical setting. In our setting, the TDS and SPM sessions followed by the CAT assignment served as review of fundamental EBM concepts plus application of the concepts to a real patient case carried by the student. We were pleased to see that the students’ general reflections encompassed not only the importance of EBM but strategies for efficient integration into practice, the utility of EBM and even specific future goals. The reflections show how teaching application of EBM in the clinical setting puts EBM knowledge into meaningful context for the students. When determining which modality of teaching is best suited for a program, educators should weigh the challenges of developing SPM with the challenges in sustaining TDS. TDS requires availability of instructors to sustain this mode of teaching whereas SPM comes with the cost of the software, skills needed to use the software and time.

For the secondary outcomes, our results showed small but statistically significant gains in knowledge, attitudes, confidence, and behaviors accessing evidence from pre- to immediately postcourse with SPM and TDS. Knowledge and confidence gains were sustained at 3 months postcourse in SPM and TDS. However, there was a lack of improvement for accessing evidence and mild decline in the attitudes toward EBM after 3 months. This suggests that a teaching session in conjunction with application of principles during clerkship can reinforce concepts and make students more comfortable applying EBM in the clinical setting, but more longitudinal teaching may be necessary to change behaviors and attitudes toward EBM over time.

Several studies exist of EBM curricula implemented during clerkship years with similar results. Modalities of teaching varied from a seminar-based series to on-line education in these clerkship curricula.26, 27, 28, 29, 30, 31 These studies demonstrated similar results to our study with increased knowledge, attitudes, usage of clinical evidence, and improved critical appraisal skills.27, 28, 29, 30 Fewer studies exist to examine on-line methods of teaching clerkship students fundamental EBM skills. , One study by Schilling, et al implemented an online curriculum during a family medicine clerkship rotation and compared EBM skills in students that received the online modules to those that did not. The intervention group reported improved confidence in information retrieval, and overall had higher quality search strategies and information retrieval compared to the control group. Aronoff et al utilized a novel longitudinal approach for on-line learning over 18 weeks. Students then completed four EBM exercises over the subsequent 24 weeks of rotations. Competency in EBM was measured via Fresno score and showed overall significant improvement in scores post course. Outcomes related to confidence, attitudes and behaviors were not reported. In our study, we directly compared traditional and on-line methods of teaching, which were studied separately in these prior studies and found no difference in how well students applied EBM skills to real patient cases. Gains in knowledge, confidence, attitudes, and behaviors were also similar across both groups, which further support the use of either SPM or TDS for teaching EBM. The longitudinal approach to on-line learning utilized by Aronoff et al seems to be a promising method to help sustain positive attitudes and behaviors throughout the clerkship, as they assimilate EBM skills into their practice.

The need for longitudinal curricula across medical school clerkships is supported by our findings 3 months after our intervention. Students reported a mild decline in attitudes toward EBM and a decline in the frequency of accessing evidence. By including EBM curricula throughout clerkships, students could continue to practice strategies to identify evidence relevant to their clinical questions and become more comfortable with the uncertainty that can arise when only small studies or lower quality studies are available. Furthermore, suboptimal role models for evidence-based practice are a barrier to teaching. Certainly, this barrier would be more pronounced when trying to implement longitudinal EBM curricula. The SPM mode for asynchronous teaching and utilization of academic health librarians as instructors could be tools to help overcome this barrier. Integrating a librarian led session to teach and reinforce efficient search strategies related to clinical questions could have a sustainable impact on students. Institutions with limited resources could seek faculty with EBM expertise from other institutions and hold virtual webinars or identify interested faculty to train in EBM instruction through courses such as those offered through the Centre of Evidence-Based Medicine. Further studies are needed to evaluate effective methods to teach EBM longitudinally, which are sustainable, and focus on the application of EBM principles that engage millennial learners.14, 15, 16 Lastly, due to COVID-19 pandemic, educators have had to create innovative educational opportunities and there has been a quick shift toward virtual learning. The shift to distanced learning may serve as a footing for continued virtual teaching options even in the postpandemic era.11, 12, 13

Since completion of this study, the SPM has been incorporated into the clerkship curriculum and feedback has been positive. We have several strengths and limitations to our study. One strength is the design of our study with randomization and blinding. Our study, however, utilizes a single center with students from one medical college and these students do receive one EBM course during their second year of medical school. It is uncertain whether the results would differ for MS who do not receive any early education in EBM. It is possible that one mode of learning may be more effective over the other for these students. We had a small sample of MS-2 students (n = 11) that did not complete the course at the medical school prior to participation and their mean scores were lower. Further studies are needed to evaluate the most effective mode of teaching for students that receive EBM education for the first time during clerkship. Additionally, we were limited to a 1-year time frame for the study before adoption of SPM formally as part of the curriculum. We did not have literature available from prior research to guide a sample size calculation but our estimates of the primary outcome, the CAT scores, are very similar between groups with overlapping, narrow CIs making type II error unlikely. Another limitation of this study is the low 3-month response rate for the survey results. We are reassured, however, that outcomes measured in students who completed all 3 times points of surveys were similar to the full population making the likelihood of attrition bias low. Additionally, we conducted analyses as per the modality of teaching received rather than the group assigned but there was minimal cross-over between groups and the characteristics of students in SPM and TDS remained similar, making bias unlikely. Blinding of facilitators or students was not possible due to the nature of the interventions but the students were unaware of the grading rubric used for the CATs and the grader for the CATs was blinded to group assignment and student identity. Lastly, in students randomized to the SPM, adherence to completion of the module, the timing of viewing the module during the rotation and time spent on the module was not formally tracked.

Conclusions

SPM is as effective in teaching and applying EBM principles as TDS during medical school clerkship rotations. Institutions should balance the availability of resources to determine the preferable mode of teaching at individual programs. Further studies should address whether longitudinal EBM curricula through clinical clerkships can be used to sustain favorable attitudes and behaviors towards evidence-based practice.