Teaching Residents Clinical Practice Guidelines Using a Flipped Classroom Model.

INTRODUCTION: Prior studies have demonstrated poor guideline compliance by pediatricians, and there is no published curriculum on how to teach clinical guidelines. Furthermore, in a national survey of pediatric residency training programs conducted in 2015, only two had a formal curriculum for teaching clinical guidelines. This module provides a framework for teaching residents clinical guidelines through a modified flipped classroom approach. Associated materials include a guide for faculty facilitators, sample slides and worksheet, and pictures of the classroom setup.
METHODS: In this module, the guidelines for acute otitis media (AOM), obstructive sleep apnea syndrome (OSAS), and attention deficit-hyperactivity disorder (ADHD) are taught in three sessions and evaluated with a pre-/posttest assessing knowledge, attitudes, self-efficacy, and satisfaction. Each guideline is delivered in a 30-minute session, with five learners per group. Faculty training requires approximately 30 minutes of preparation. The intervention groups (n = 9 for OSAS, 10 each for AOM and ADHD) received three weekly, half-hour flipped classroom lessons. The control group (n = 19) had no formal guideline education.
RESULTS: Pre-/posttests showed a statistically significant improvement in knowledge and attitudes in the group of interns who received this educational intervention over the control group. The learners rated the sessions as highly effective. DISCUSSION: This module provides an efficient and effective way of utilizing a modified flipped classroom approach to teach learners the correct use of clinical guidelines, a skill residents must master to provide evidence-based care. This curriculum has been successfully incorporated into our pediatric residency program.

Educational Objectives

By the end of this module, learners will be able to:

Discuss one key action statement in a clinical practice guideline for acute otitis media (AOM), obstructive sleep apnea syndrome (OSAS), and attention deficit-hyperactivity disorder (ADHD).

Describe one relevant clinical example for their assigned key action statement.

Teach back a minimum of one key point of the key action statement to their small group.

Create a diagnostic or treatment algorithm for the management of AOM, OSAS, and ADHD using relevant clinical practice guidelines.

Introduction

Studies show that although adherence to and use of clinical practice guidelines vary widely in pediatric practice, these guidelines are generally used less frequently than similar ones in adult medicine. In Flores, Lee, Bauchner, and Kastner's evaluation of 1,088 pediatricians, practice guidelines were used by only 35%, despite the fact that 89% of respondents agreed that practice guidelines improve outcomes and that a vast majority (94%) desired to improve quality.[1] In another study, Forrest and colleagues evaluated the use of a clinical decision support system. Before their intervention, it was found that only 15% of acute otitis media (AOM) and 5% of otitis media with effusion guidelines were followed.[2] Rushton, Fant, and Clark described similar results when evaluating the use of the American Academy of Pediatrics (AAP) diagnostic criteria for attention deficit-hyperactivity disorder (ADHD), reporting that only 25.8% of clinicians indicated routine use of all four diagnostic components in the survey.[3] Little evidence has been produced and few explanations made as to why pediatricians neither follow nor adhere to clinical practice guidelines. Meanwhile, there is no published curriculum on how to teach clinical guidelines. Furthermore, in a national survey of pediatric residency training programs we conducted in 2015, only two indicated they had a formal curriculum for teaching the guidelines. This is a true gap in resident education, and we hypothesize that if residents are taught how to read and apply the clinical guidelines during their training, they may be more likely to follow them when they go into practice.

While medical educators and physicians struggle to teach and implement clinical practice guidelines, there is a movement to change the way medicine is taught that offers a tangible solution. Medical educators currently advocate moving away from the traditional classroom model of didactics and towards a flipped classroom approach (also called blended learning or an inverted classroom), where the learners engage with the material beforehand (prework) and participate in small active-learning groups during a facilitated session.[4-7] The flipped classroom has been shown to be equally as effective, if not more so, in knowledge acquisition across multiple settings (e.g., medical school core courses such as biochemistry and core clerkships like surgery).[8-12] Additionally, learners tend to prefer this model of self-directed learning over the traditional classroom.[12,4]

Residency training, however, poses a challenge to the flipped classroom model due to the intense clinical demands on trainees that limit their ability and willingness to complete the required prework.[14] In an effort to ensure pediatric residents are learning essential clinical practice guidelines during their training, we propose a hybrid flipped classroom model that incorporates the prework directly into the small active-learning groups. Because the prework is built into the facilitated session, residents are able to briefly engage with the material on their own and then work in a small group to apply, create, and analyze the clinical practice guidelines to develop their own diagnostic or treatment algorithm for the condition.

Methods

Hybrid Flipped Classroom Intervention Development

Guideline selection

We developed a pilot study to evaluate the effectiveness of three 30-minute hybrid flipped classrooms to teach pediatric interns three clinical practice guidelines. The clinical practice guidelines selected for this study were those for AOM, obstructive sleep apnea syndrome (OSAS), and ADHD.[15-17] Through expert consensus, we distilled five key learning points for each guideline from the guidelines' key action statements (Table 1). This was done by having three of the authors independently review the three guidelines and select the most pertinent key action statements to teach in each session. Mediation by the senior author was used to address discrepancies. These learning points were then made the focus of the modified flipped classroom sessions.

Table 1.

Guideline Action Statements Divided Into Learning Points

Learning Point	Key Action Statements
	Obstructive Sleep Apnea Syndrome	Acute Otitis Media	Attention Deficit-Hyperactivity Disorder
1	1	1A	1
2	2A + 2B	1B + 1C	2
3	3 + 4	3A + 3B + 3C + 3D	3
4	5A + 5B	4A + 4B + 4C	5A + 5B + 5C
5	6 + 7	5A + 5B	6

Intervention design

The intervention groups (n = 9 for OSAS, 10 each for AOM and ADHD) received weekly, half-hour flipped classroom lessons for the three guidelines, while the control group (n = 19) had no formal education about guidelines. The intervention consisted of a 30-minute structured small-group activity led by a facilitator. The facilitator led the session with a simple PowerPoint (Appendix A for OSAS, Appendix B for AOM, or Appendix C for ADHD) to advance the groups through each portion of the intervention.

Intervention Implementation Guide

Classroom setup

Prior to the session, the room is arranged with tables and chairs forming groups of five, with a printed copy of the clinical practice guidelines of that session's selected topic (OSAS, AOM, or ADHD) at each seat, as demonstrated in Appendix D.[15-17] Each printed copy of the clinical practice guideline has a post-it note with an assigned learning point derived from the key action statements (e.g., Learning Point 1 for Group Member 1, Learning Point 2 for Group Member 2, etc.). As stated previously, the five assigned learning points were identified earlier through expert consensus and reflect the content on the pre/post assessment test. Each small group receives one guideline summary sheet (Appendix E), a piece of chart paper, and markers.

Introduction and objectives

All three sessions begin with the facilitator reviewing the learning objectives and introducing the clinical practice guideline for the session using the presentation found in Appendix A, B, or C. Facilitators should conduct the presentation only through slide seven for Appendix A, slide six for Appendix B, and slide seven for Appendix C. The remaining slides for each presentation are used later in the session. Typically, the facilitator should set aside 2 minutes for this session. Please note that during the students' initial session, the pretest (Appendix F) is administered first to assess student presession knowledge of clinical guidelines for OSAS, AOM, and ADHD. The facilitator should allow an additional 5–10 minutes to administer this pretest during the first session.

Prework

Residents are instructed to quietly and individually read their assigned learning point. They are prompted to identify the main point they think is most relevant for all pediatric residents to know and develop a relevant clinical example. This is the prework portion of the flipped classroom model that we adapted to occur during our hybrid sessions. The facilitator allots 3 minutes for this portion.

Teach back

The facilitator assigns 12 minutes for this portion. Residents are asked to teach back the main point of their assigned learning point to their group in approximately 2 minutes (in sequential order of learning guideline). A scribe is assigned to record the main points for the group on the guideline summary sheet (Appendix E). The residents are encouraged to highlight key tables, figures, or charts while explaining their relevance. This generates small-group discussion as the remaining group members can ask clarifying questions and compare or contrast their prior knowledge to the action statement. The process is repeated roughly every 2 minutes until each group member has taught his or her assigned learning point to the small group.

Create diagnostic flowchart/algorithm

During the remaining 10 minutes of the session, each group compiles the content gathered on the guideline summary sheet (Appendix E) into a diagnostic and treatment algorithm or flowchart, a sample of which can be found in Appendix G. Additional examples are found in Appendices A–C, and these may be revisited if needed. The facilitators can circulate the room but should refrain from interjecting unless a group is off task and needs to be redirected. The purpose of this time-constrained portion is to challenge group members to apply the knowledge they have just learned to create a tool they can reference in the future. If the group needs prompting, possible questions can include “How will you use this information next time you are in clinic?”, “What component was most challenging or interesting?”, and “What is the first step to diagnosing this condition?” The end products may vary in their design and structure (e.g., a flowchart vs. a table vs. a diagram), but the content should reflect the learning points.

Check for understanding—apply to case

The facilitator asks the large group a series of clinical questions that each small group must answer using its diagnostic flowchart/algorithm. These questions are provided in the facilitator PowerPoints and begin on slide eight of Appendix A, slide seven of Appendix B, and slide eight of Appendix C. This allows the facilitator to reinforce the main learning points and find any major gaps in the flowcharts/algorithms before the end of the session. Any misinformation can be quickly corrected and retaught to ensure the residents do not leave with incorrect information. Three minutes are generally allotted for this component. If the students are on their final session, they are given the posttest evaluation (Appendix F). Students should be given an additional 5–10 minutes to complete the posttest.

Results

A deidentified 32-question test (Appendix F) was developed to assess pediatric interns' knowledge, attitudes, and general use of clinical practice guidelines both prior to and after they had experienced the three sessions. For the purpose of the pilot study, the test was administered to 29 pediatric interns (PGY1) during Week 1 (preintervention) and repeated on Week 4 (postintervention). Ten interns received the intervention, and 19 received the control (no-guidelines curriculum). A single investigator graded all tests using a scoring rubric. Evaluation of the quality of the facilitation, content, and structure of the three flipped classroom sessions was obtained at Week 4 (postintervention) using Likert scales and open-ended feedback for improvement (Appendix H). The study was Stanford IRB exempt.

All of our pediatric interns (N = 29, 22 female, seven male) were included in the study. Descriptive statistics and unpaired, two-tailed t tests (p < .05) were used for analyses. Before the intervention, three out of 29 interns indicated on the pretest (Appendix F) that they had received teaching on OSAS, 13 out of 29 on AOM, and eight out of 29 on ADHD, respectively. Only 20 of 29 interns knew there were any clinical practice guidelines published by the AAP, and these 20 were able to name on average only three of the 10 guidelines. On a 5-point Likert scale (1 = very important, 5 = not important), 28 interns rated the importance of a pediatrician knowing and using the guidelines as very important; the other intern rated these as somewhat important.

The interns had an average knowledge score of 44% on the pretest. The score breakdown by guideline was 45% for OSAS, 11% for AOM, and 79% for ADHD. A convenience sample was used to select the intervention group based on the interns who were at the main training site (Lucile Packard Children's Hospital). The number of interns in each intervention group was nine for OSAS, 10 for AOM, and 10 for ADHD.

When the posttest was adminsistered, the intervention interns had a statistically significant increase in knowledge over the control interns: 21% versus 10% for OSAS (p = .03), 16% versus 4% for AOM (p = .009), and 21% versus −5% for ADHD (p = .005; Table 2). There was a statistically significant increase in self-efficacy posttest versus pretest but no difference between intervention and control groups.

Table 2.

Percentage Score Change on Pre- Versus Posttests

Disorder	Control Groupa	Intervention Group	p Value
Obstructive sleep apnea syndromeb	9.92	20.96	0.03
Acute otitis mediac	4.11	16.13	0.009
Attention deficit-hyperactivity disorderd	−5.26	21.05	0.005

n = 19.

n = 9.

n = 10.

n = 10.

All participants filled out anonymous evaluations (Appendix H) after the session to assess facilitation, handouts, comfort using the guideline, and overall session. A 5-point Likert scale (1 = poor, 5 = excellent) was used. Overall, the sessions were rated 4.4 for OSAS, 4.4 for AOM, and 4.2 for ADHD. The sessions received unanimously high marks for all categories (facilitation, handouts, and comfort; Table 3).

Table 3.

Resident Likert-Scale Evaluation of Flipped Classroom Sessions

Session	Facilitation	Handouts	Comfort	Overall
Obstructive sleep apnea syndrome	4.5	4.375	3.875	4.375
Acute otitis media	4.375	4.5	3.75	4.375
Attention deficit-hyperactivity disorder	4.18	4	3.45	4.18

Five-point Likert scale (1 = poor, 5 = excellent).

Discussion

We developed and implemented a modified flipped classroom approach to teach clinical guidelines, which successfully improved intern knowledge and attitudes when compared with standard education. These sessions were well received. In fact, our PGY2 and PGY3 residents insisted the sessions be taught to all levels of learners. Furthermore, it was noticed that the interns were incorporating the guideline knowledge into their actual patient care in clinic and on the wards and at times knew the guidelines better than the PGY2 and PGY3 residents. This created further interest among the PGY2 and PGY3 residents, which led to us offering this module to all levels in later months. The curriculum was expanded to include all 10 published AAP guidelines, and it is now a core component of our primary care conference series. In addition, Drs. Peterson, Louden, and Gribben presented this work at both the Association of Pediatric Program Directors Chief Resident Forum and the Academic Pediatric Association Education Committee's national meeting in 2016.

Although this curriculum was created to teach clinical guidelines, the teaching methods can also be adapted to teach any topic to medical students, residents, and fellows. This work successfully showed that a modified flipped classroom approach can be used for busy clinical learners who might not have sufficient time to prepare ahead of class. The interactive nature of the sessions allowed learners to solidify their baseline knowledge and apply it to real clinical scenarios. In this way, the key concepts of the clinical guideline were reinforced at least four times throughout the session.

A few revisions have been made to the curriculum since inception. These include the creation of advanced and modified sessions with slight variations. For example, in the hyperbilirubinemia guidelines, each group works on one action statement and reports out (a jigsaw technique).[18] For AOM, skills stations have been added as a form of case-based application, and as the guidelines are discussed, residents match the category of tympanic membrane disease with pictures of tympanic membranes. Regarding challenges in implementing the curriculum, although there was initially student discomfort with learning via a new model and some uncertainty as to the appropriate level of participation, the students quickly learned how to participate. In addition, given the short duration of our morning report (30 minutes), we found that the facilitator needs to stick closely to the time line so as not to get off schedule.

Despite the curriculum's rigorous design and implementation, a few limitations do exist. A convenience sample of interns at a particular site was studied, and although the curriculum was required of all interns at that site, residents could self-select into receiving or not receiving it. Moreover, this was a small pilot study at a single institution. Follow-up studies should investigate the sustainability of the increased knowledge and attitudes. Our educational intervention provides a robust framework for teaching how to use clinical guidelines and demonstrates an interactive teaching method (the modified flipped classroom) that is a more feasible construct than the true flipped classroom for busy clinical learners.

A. Facilitator Guide Presentation OSAS.pptx

B. Facilitator Guide Presentation AOM.pptx

C. Facilitator Guide Presentation ADHD.pptx

D. Room Setup.pptx

E. Guideline Summary Sheet.docx

F. Pre-Posttest.pdf

G. Sample Final Algorithm.pptx

H. Learner Evaluation.docx

All appendices are peer reviewed as integral parts of the Original Publication.