A small group learning model for evidence-based medicine.

BACKGROUND: Evidence-based medicine (EBM) skills are invaluable tools for residents and practicing physicians. The purpose of this study is to evaluate the effectiveness of small-group learning models in teaching fundamental EBM skills.
METHODS: The intervention consisted of an EBM bootcamp divided into four 2-hour sessions across 4-week rotations. Residents worked in small groups of three to four to explore fundamentals of EBM through interactive dialogue and mock clinical scenario practice. The intervention's effectiveness was evaluated using pre- and post-assessments.
RESULTS: A total of 40 (93.0%) residents out of a potential 43 participated in the EBM bootcamps across the 3 years. There was significant improvement of 3.28 points on self-assessed EBM skills from an average of 9.66-12.945 out of a maximum score of 15 (P=0.000). There was significant improvement of 1.68 points on the EBM skills test from an average of 6.02-7.71 out of a maximum score of 9 (P=0.00). All residents (100%) agreed or strongly agreed that EBM is important for a physician's clinical practice. This view did not change after the training.
CONCLUSION: A brief small-group interactive workshop in EBM basic skills at the start of residency was effective in developing fundamental EBM skills.

Background

Evidence-based medicine (EBM) is defined as “the conscientious, explicit, and judicious use of the current best evidence in making decisions about the care of individual patients.”1 Although the concepts of EBM have been discussed in medical education for the past 20 years, many medical learners and practicing physicians still lack the knowledge and skills needed to incorporate EBM into their patient care.2 Lack of time, of skills, and knowledge, inadequate access to resources, and cost are identified as common barriers to learning and practicing EBM.3

While the principles and benefits of EBM are well-established, there is a relatively small body of literature addressing EBM instruction in medical education.4 A few studies have explored instructional methods, from teaching in clinical settings to seminars, workshops, and short courses.5,6 Others looked at pedagogical efficacy by comparing in-person and online teaching, discussion-based and lecture formats, and single-discipline and interprofessional groups.4 Many of these studies, however, had methodological or outcome evaluation limitations.5–7

Recently, scholars have called for educational approaches to EBM that are authentic, engaging, and student-centered.8 Small-group learning (SGL), sometimes called cooperative learning, has been used in secondary and higher education for many decades to promote student engagement.9–12 SGL has been successfully adopted in medical education to enhance skills and knowledge in a variety of areas, including interpreting and applying evidence in clinical practice.13–18 SGL utilizes an expert facilitator to guide face-to-face educational experiences and support learners to take active roles in their training.8,14–16,19

The purpose of this study is to evaluate the effectiveness of SGL in teaching the fundamental EBM skills.

Methods

This study was set at a university-affiliated family medicine residency program located in a large Midwestern city. In the three academic years between 2013 and 2015, 40 incoming interns participated in the educational intervention. The workshop series, referred to as “EBM bootcamp”, was integrated into the Introduction to Family Medicine rotation, held each year during the interns’ first month in residency.

The workshop

EBM bootcamp was divided into four 2-hour sessions spread across the 4-week rotation. Residents worked in small groups of three to four to explore the fundamentals of EBM through interactive dialogue and mock clinical scenario practice. The overarching educational goal of the training was to identify clinically relevant questions and then formulate evidence-based answers. At the end of the training, learners were expected to be able to: 1) ask clinically relevant answerable questions, 2) find the evidence by searching the medical literature, 3) assess the quality of the evidence, 4) summarize medical articles, and 5) synthesize a conclusion to present an evidence-based answer. Details of training objectives and activities will be provided by the authors upon request. The principal investigator, who was also the instructor in the workshop, completed the grading of the learners’ assessments. The Indiana University Institutional Review Board reviewed and approved this study which was exempted from participant consent as the study reports on educational activity.

Study design

The intervention’s effectiveness was evaluated using pre- and post-assessments. At the start of the workshop, participants reported their prior experiences with EBM in medical school, their evidence-based resource use in clinical practice, their literature-reading habits, and their prior experience presenting in journal clubs or similar activities.

Assessments included both subjective and objective components. Interns completed a pre-test self-assessment and an EBM skills test at the start of the workshop series and again 4 weeks after completing the workshop. The self-assessment tool explored three primary domains of EBM: 1) formulating clinical questions, 2) searching the literature, and 3) identifying appropriate study designs. On the EBM skills test, the residents were presented with a clinical scenario and were asked to formulate a question, identify and summarize an article that addresses the question, and finally, explain their choice of best study design. Attitudes toward EBM were also assessed before and after the workshop. Besides this intervention, no further EBM training was taught between the completion of the course and the post test. The study was reviewed and deemed exempt by the university’s Institutional Review Board. The assessment tools, adapted from the original Fresno test,20,21 are available in Supplementary materials. Supplementary materials contain the rubric used to evaluate resident responses. We adapted the Fresno test since some of the skills assessed by the original test (such as validity, significance, etc) were beyond the objectives of this training.

Statistical analysis

Performance on the EBM exercise was analyzed using Student’s t-test to compare each item before and after the training. Self-efficacy for using patient, intervention, comparison, and outcomes,22 identifying resources, and summarizing articles were evaluated on Likert scales. A self-efficacy score was generated from the summation of the Likert scale ratings for these three items. We used Student’s t-test to compare before and after scores.

We used regression analysis to examine the correlation between the self-assessment and EBM tests scores (dependent variables) and the residents’ characteristics, prior experiences, and baseline attitudinal indicators (independent variables). In addition, we used regression analysis to explain the improvement of scores using multiple models. We used a P-value of 0.05 as a cutoff to determine statistical significance. STATA 14 was used for the quantitative analysis.

Results

A total of 40 residents out of a potential 43 (93.0%) participated in the EBM bootcamps over the 3 years of the study. The three interns excluded from the study started residency 2–3 months late and thus could not participate in the training. The final analysis included all residents except in the case of the post-workshop assessment, for which two tests were lost. The baseline characteristics of the participants, the self-assessment scores, and the EBM skills test scores are presented in Table 1. When examining predictors of baseline performance, the practice of reviewing journals weekly was significantly associated with the EBM test score (P=0.02).

Table 1

Participants’ characteristics and baseline mean scores on the self-efficacy and the EBM test

	Number (% of total)	Mean score on self-assessment test (SD)	Mean score on the EBM test (SD)
Class
2016	14 (35)	9.14 (1.23)	5.46 (1.97)
2017	13 (32)	10.30 (1.60)	6.68 (2.65)
2018	13 (32)	9.15 (2.76)	6.07 (1.27)
Gender
Male	25 (62.5)	9.28 (1.92)	6.02 (1.79)
Female	15 (37.5)	9.93 (2.08)	6.03 (2.41)
Using evidence-based once weekly or more	21 (52.5)	9.85 (1.59)	5.7 (1.80)
Reviewing current medical journals once weekly or more	5 (12.5)	9.6 (2.40)	7.37 (1.49)*
“I present journal clubs well”
Strongly agree	2 (5)	10.5 (3.53)	5.5 (2.12)
Agree or strongly agree	7 (17.5)	10.85 (2.03)	5.76 (1.64)
Never presented a journal club before	19 (47.5)	9.42 (2.03)	5.833 (2.23)
“EBM was a priority in my school”
Strongly agree	7 (17.5)	10.57 (1.98)	6.35 (0.99)
Agree or strongly agree	21 (52.5)	10.75 (1.81)	6.18 (1.23)
“EBM is important for every physician”
Strongly agree	31 (77.5)	9.67 (2.11)	6.21 (1.97)
Agree or strongly agree	40 (100)	9.52 (1.98)	6.03 (2.01)

Note:

P<0.05.

Abbreviations: EBM, evidence-based medicine; SD, standard deviation.

Individual self-efficacy

There was a significant improvement of 3.28 points (P<0.05) in the average self-assessed EBM skills score (from 9.66 to 12.95, out of a maximum score of 15). Improvement in the self-assessment score was inversely correlated with baseline self-assessment: the lower the baseline scores, the higher the improvement (P<0.05) (Figure 1).

Figure 1

Improvement in self-assessment as a function of baseline self-assessment score.

EBM skills assessment

There was a significant improvement of 1.68 points (P<0.05) in the average EBM skills test score (from 6.02 to 7.71, out of a maximum score of 9). Similar to the self-assessment scores, improvement on the EBM skills test was inversely correlated with baseline scores (P<0.05) (Figure 2). There was no correlation between subjective and objective score improvement (r=0.06, P=0.73).

Figure 2

EBM skills improvement as a function of baseline score.

Abbreviation: EBM, evidence-based medicine.

Attitude toward EBM

All residents (100%) agreed or strongly agreed that EBM is important for a physician’s clinical practice. This view did not change after the training.

Resident evaluation of the workshop

The vast majority of residents (97.37%) agreed or strongly agreed that the EBM bootcamp was helpful in improving their EBM skills. Similarly, a majority of residents (86.84%) agreed or strongly agreed that the workshop series should always be a component of the orientation. The majority of residents (84.21%) said they would most likely use the skills learned during the series in their daily practice.

Discussion

The small-group EBM workshop yielded improvements in both EBM test scores and perception of skills. The workshop was highly acceptable to the residents. Residents with the lowest initial scores appeared to gain the most benefit from the intervention. The fact that poor initial performers on both tests improved more than their peers suggests that this brief intervention may help get everyone “up to speed” with EBM skills at the start of residency.

Our study is consistent with previous research examining the impact of focused training on EBM skills and knowledge.4,23 For example, Dinkevich et al found a marked improvement in pediatrics interns’ EBM skills after 3 or 4 weekly seminars.23 In their literature review, Hecht et al documented the effectiveness of various methods for learning EBM such as journal clubs, conferences, workshops, and courses.5 However, many such studies are undermined by significant potential bias.4 While our study does not compare different approaches, it is unique in documenting the effectiveness of one important learning method. The SGL model employed in this workshop series has many advantages for residents beyond the mastery of EBM basic skills. SGL facilitates working in teams, an essential skill for today’s residents and practicing physicians.19

Although our study was conducted at a single center and may thus not be generalizable to all other settings, including all the residents of three cohorts lend substance to the conclusion. Our study used a pre-and post-design, which does not provide strong evidence for causal inference between the intervention and the outcomes. However, the short period of time (4 weeks only) between the pre- and post-assessments and the absence of any other exposure to EBM concepts during this time support the validity of the findings. Future studies should use a multi-center experimental design to compare small-group learning to more commonly used methods such as didactic teaching to determine the best option in EBM education.

Conclusion

A small-group workshop yields improvements in skills and self-efficacy in EBM for starting residents.