Acute Otitis Media Curriculum.

INTRODUCTION: Studies have shown that clinicians do a poor job of diagnosing middle ear infections, and an inaccurate diagnosis may result in inappropriate treatment, such as overuse of antibiotics. Antibiotic overuse is known to lead to increased bacterial resistance and puts patients at risk of deleterious side effects, including allergic reactions, vomiting, diarrhea, and rash. In this curriculum, we teach three important topics for achieving a successful middle ear exam and diagnosis of pathology.
METHODS: The topics covered during the session are middle ear examination techniques, diagnosis of middle ear pathology and criteria for diagnosing acute otitis media, and treatment of acute otitis media. Each topic has a 30-minute lesson plan with broad goals, specific and measurable learning objectives, interactive learning activities, and assessment measures. Teaching activities include a mix of knowledge, skill, and attitude activities. Assessment measures aim at the highest level of Miller's pyramid of assessment when possible. Also included is a scholarship map that relates how this learning activity can fit into the assessment of learners at the program level by linking learning objectives with pediatric developmental milestones.
RESULTS: The interactive nature of these lessons has been very well received by learners, and pediatric residents have had a self-perceived increase in skills. DISCUSSION: This curriculum provides a standardized approach to teaching the middle ear exam structured by educational rubrics and allows for accurate assessment of learners.

Educational Objectives

By the end of this resource, learners will be able to:

Perform a middle ear exam correctly on children of all ages, minimizing the child's discomfort and maximizing the likelihood of a successful exam.

Correctly identify normal middle ear anatomy on children of all ages.

Correctly identify abnormal middle ear anatomy on children of all ages.

Correctly make the diagnosis of acute otitis media and otitis media with effusion.

Appropriately manage a patient with acute otitis media or otitis media with effusion.

Introduction

Studies have shown that clinicians do a poor job at clinically diagnosing middle ear infections.[1,2] Even when presented with pictures online of normal tympanic membranes/middle ears, otitis media with effusion, and acute otitis media (AOM), clinicians do a poor job at assessing middle ear pathology in actual patients.[3] Inaccurate diagnosis may result in inappropriate treatment, including overuse of antibiotics. Antibiotic overuse is known to lead to increased bacterial resistance to antibiotics and puts patients at risk of deleterious side effects, including allergic reactions, rashes, and diarrhea.

This AOM curriculum is one topic in the Pediatric Primary Care Curriculum (PPCC) at the Children's Hospital Colorado residency program.[4] The PPCC consists of the following:

Twenty 30-minute teaching sessions given every morning during a clinic month that repeat for new learners every month of the year.

Forty 15-minute teaching sessions given before each resident's continuity clinic. The same topic is taught every day weekly for each continuity resident regardless of his or her continuity day.

Six 3-hour teaching sessions during our interns’ 3-month centennial block. During this block, interns spend 1 contiguous month in a primary care clinic and then split the remainder of the other 2 months in service learning between a behavior and development clinic and rotating in the emergency department.

The PPCC repeats itself annually for deliberate redundancy and to account for missed topics due to resident clinical duties. This AOM curriculum has been taught in two 30-minute teaching sessions during the clinic month but was recently shifted to a 3-hour block during the intern's centennial block to allow for more time to discuss and practice AOM skills. Residents practice the skills taught in this session during their clinic month and continuity clinics, with some AOM objectives assessed by their clinic preceptors throughout the year. There are typically five to 10 learners per teaching session with the AOM curriculum at Children's Hospital Colorado, which allows for individual attention to the learners.

At our institution, we have found that the Yale Primary Care Pediatrics Curriculum is an affordable and high-quality resource with an excellent breadth of primary care topics.[5] The AOM topic has a seven-page case study with salient associated journal article readings.[6] That resource, however, focuses on discussion of topics aimed at changes in learner knowledge and attitudes only, without a clear plan for improving learner skills. In addition, it sometimes lacks learning objectives that lend themselves to specific assessment of outcome measures of the learner.

Kern's curriculum development model was used in a systematic way to develop this AOM curriculum within the larger PPCC.[7] The general needs assessment consisted of a literature search that revealed a prevalence of overdiagnosis and misdiagnosis of AOM in the US with overuse of antibiotics. Targeted needs assessment was based on conversations with attendings and meetings with the PPCC resident steering committee, which revealed that residents had little confidence in the middle ear exam, even in their third year of residency, and that attendings felt that many residents were unable to make an accurate diagnosis of AOM. In addition, many attendings were discouraged by the inaccurate diagnoses of AOM that came to our clinic for follow-up.

Broad goals were formulated to meet the needs of the learners from a variety of training contexts, including pediatric and family medicine residents, physician assistant students, and medical students. Learning objectives were created that are specific and measurable and take into account feasibility of educational strategies and learner assessment. At our institution, each resident spends time in a primary care clinic for 1 month during his or her first year and 1 month during the second year, physician assistant students spend 1 month in their third year with us, and many medical students spend 2 to 4 weeks with us during their third-year clerkship. There is dedicated teaching time from 8:00 am to 8:30 am each morning at our primary teaching outpatient clinic. Teaching lessons are made to fit these time slots, and assessment of the learner is made to fit into this context when possible.

Learning objectives were formulated to achieve learning goals with the philosophical approach of considering “What would you be embarrassed that your learner did not know?” and keeping in mind that it is not feasible to teach everything on a given subject. At our institution, learners prepare by reading a seven page case-based overview on AOM through the Yale Primary Care Pediatric Curriculum. The 30-minute teaching sessions then focus on interactive educational strategies, with some assessment of the learner using a flipped classroom model. Alternatively, the curriculum can be modified to have the learners read the American Academy of Pediatrics (AAP) clinical practice guideline before coming to the teaching sessions.

Methods

Our AOM curriculum is written as three separate 30-minute teaching sessions; however, at times, we have modified it to fit into two 30-minute teaching sessions or into one 2-hour session. We created short lesson plans based upon the Yale AOM case study and associated readings by developing learning activities that are interactive and may be used to assess the learner. The lesson plans include our own SMART (specific, measurable, actionable, relevant, and time-bound) learning objectives that are different from the Yale case study learning objectives. The lesson plans should be easy for clinical teachers to pick up and use with the basic pediatric knowledge that most general pediatricians have, with little preparation required.

Lesson plans for each AOM topic (Appendices A, B, & C) include preparation for the session, goals, specific learning objectives, educational activities with key discussion points, and estimated time for each activity to keep the teacher on track. The teacher can peruse the lesson plan in 10–15 minutes any time prior to the lesson to help organize and strategize. We recommend writing the learning objectives in a conspicuous place (e.g., on a whiteboard) before the teaching session so that all learners have a road map of the session. Some learning objectives are written so that assessment takes place by the teacher during the teaching session. A Preceptor Assessment Worksheet (Appendix D) is included to allow for workplace-based assessment when feasible, which is aligned with the respective learning objectives from the lesson plans. The Preceptor Assessment Worksheet is a formative assessment tool, meant to stimulate discussion of what the learner did well and what areas may need further development. It includes an assessment level based on Miller's pyramid paradigm of professional authenticity,[8] which can help programs build data for summative assessment at the program level when coupled with their own assessment data collection methods.

The normal and abnormal middle ear anatomy presentation (Appendix E) should be used with the Socratic method (i.e., asking questions to elicit understanding of the content) for teaching and assessing learners during the lesson on diagnosis of AOM (Appendix B). Each slide includes comments at the bottom to guide teaching points.

Accompanying the Treatment Lesson Plan (Appendix C), the Bacteria Learning Activity (Appendix F), AAP Recommendations for Initial Management for Uncomplicated AOM (Appendix G), and Number Needed to Treat to Benefit or Harm (Appendix H) files should be printed out and cut into pieces prior to the lesson. Laminating the pieces will help make them durable for future lessons.

For Appendix F, we arrange some common bacteria on the table in one column and percentages in another column. We then ask the learners to identify the three most common bacteria that cause AOM and the corresponding percentage of time that they cause AOM according to recent epidemiologic data.[9-10]

For Appendix G, we reconstruct Table 4 from the AAP practice guideline by arranging the patient presentations on the horizontal axis and the ages on the vertical axis and then ask the learners to place the treatment choice according to AAP guidelines.

For Appendix H, before placing anything on the table, we use the Socratic method to first ask why antibiotics are used to treat AOM. As the learners state each reason (e.g., to treat contralateral AOM episodes), we put the number needed to treat to benefit (NNTB) cards down on the table. When all treatment reasons are placed on the table, we put the answers to the NNTB on the table and ask the learners to align the NNTB with their corresponding answers. We give time for the learners to struggle through this exercise, as this can be thought of as esoteric information, but allow them to critically think about the relatively small added benefit of antibiotics. Answers are revealed when the teacher feels that the learners have reflected adequately on this epidemiologic data.

A scholarship map (Appendix I) was created to align learning objectives with assessment of the learner through the Pediatric Milestones Project.[11] In addition, at the request of the learners, learning objectives were also mapped to Pediatric Board Examination Content.[12] Possible tools for implementing this curriculum include the Welch Allyn Digital MacroView Otoscope[13] and a faculty development curriculum we created for training on the use of this tool.[14]

For curriculum evaluation, a short Likert scale with space available for written comments (Appendix K) is given at the end of the curriculum to evaluate learner satisfaction. The evaluation is based on level one of Kirkpatrick's Evaluating Training Programs.[15] In addition, a retrospective pre- and postcurriculum self-assessment (Appendix J) is used to evaluate changes at Kirkpatrick's third level, learner behavior. The self-assessment is written as one reflective statement that aligns to each lesson and measures the learner's perceived increase in skills. It is to be completed by the learners at the end of their rotation but may be filled out any time in the future to evaluate for retention of behavior change.

Results

This curriculum has been used at our institution for the last 3 years and has been evaluated for learner satisfaction by using the tool in Appendix K (see Table 1). The 5-point Likert scale evaluation, with 5 being the best score, significantly rated the AOM teaching higher (4.96) than all other topics taught in the PPCC delivered during clinic month (4.57; two-tailed p value < .0001). Representative comments on this evaluation include “I enjoyed the interactive nature of the lesson” and “I enjoyed the review of bugs and drugs and the great information about NNTB to help explain why we don't treat all AOM.”

Table 1.

Overall Course Evaluation

Topic	Ma	SD	N
AOM teaching	4.96	0.2	72
All other topics	4.57	0.69	956

Abbreviation: AOM, acute otitis media.

Scale of 1 to 5, with 5 being best.

A retrospective pre- and postcurriculum self-evaluation (1 = Strongly Disagree, 5 = Strongly Agree) completed at our institution revealed a perceived increase in the skills aligned with all three lesson plans, indicating a Kirkpatrick level three behavior change (see Table 2). For example, for the middle ear examination lesson plan, learners only slightly agreed that before the curriculum they could examine the middle ear of a child appropriately, including knowing where to position themselves, the patient, and the equipment to optimize the chances of visualizing the tympanic membrane and minimizing the discomfort of the child. The average score among learners was 3.42. However, after the curriculum, this self-perceived skill increased to 4.79 (paired t-test p value < .0001). In addition, the skill of identifying normal middle ear anatomy, including the malleus, incus, lateral malleolar process of the malleus, umbo, light reflex, pars flaccida, and pars tensa, increased from 2.89 to 4.63 (paired t-test p value < .0001). Lastly, the skill of identifying middle ear pathology in patients and making appropriate diagnoses and treatment decisions based on the AAP guidelines increased from 3.32 to 4.58 (p value < .0001).

Table 2.

Middle Ear Examination Retrospective Pre and Post Self-Evaluation

Statement	Self-Perceived Confidence in Performing Taska		p
	Before Curriculum	After Curriculum
Examine the middle ear of a child appropriately; including knowing how to position myself, the patient, and the equipment to optimize the chances of visualizing the tympanic membrane and minimizing the discomfort of the child.	3.42	4.79	<.0001
Identify normal middle ear anatomy; including the malleus, incus, lateral malleolar process of the malleus, umbo, light reflex, pars flaccida, and pars tensa.	2.89	4.63	<.0001
Identify middle ear pathology in my patients and make appropriate diagnoses and treatment decisions based on the AAP guidelines.	3.32	4.58	<.0001

Abbreviation: AAP, American Academy of Pediatrics.

5-point Likert scale (1 = Strongly Disagree, 5 = Strongly Agree).

Discussion

This resource has been developed over the last 3 years with monthly implementation and rapid prototyping based on feedback. It is complementary to reading and meant to engage the learner in interactive learning activities. Feedback has been overwhelmingly positive, and learners have offered the most praise for the structured interactive learning, the digital macroscopic otoscopic tool, and the dedicated time and deliberate teaching.

When possible, learning objectives were written to assess the learner at the highest possible level of Miller's pyramid. Several resources are included for assessment. The instructor can assess learner comprehension through the Socratic method (Appendices A, B, & C), a University of Pittsburgh quiz that assesses diagnosing AOM and otitis media with effusion (Appendix B),[16] a preceptor worksheet for workplace-based assessment (Appendix D), and a retrospective pre- and postcurriculum self-assessment (Appendix J) that can also be collected and used for curriculum evaluation. A standardized checklist for otoscopy performance evaluation is also available on MedEdPORTAL to assess learner skills.[17]

Workplace-based assessment is challenging, and we have had difficulty getting faculty to incorporate tools into their workflow consistently. One helpful strategy is to give the preceptor assessment worksheet to the learners and have them initiate their assessment with their preceptor in the work setting. The digital macroscopic otoscope is a good tool, but it costs between $800 and $900 and requires a user device running Windows XP or later operating system, a viewing screen (one cannot look through it as one would a standard otoscope), and a USB 2.0 port. We currently use a computer on wheels as the viewing screen and bring it into the room, but it takes time to wheel the equipment in and start it up. One also needs to consider if the software will be compatible with information technology firewall standards in one's workplace, and there is also the storage of the equipment to consider. While learners have given feedback that the digital macroscopic otoscope has enhanced their learning, it is awkward to manipulate the otoscope and use the computer as a viewing screen, and it takes practice to improve upon this skill. There are new technologies available for simultaneous middle ear viewing, including a CellScope ($299), which allows learners to use their phones as an otoscope with an attachment for the speculum. This allows pictures or videos to be taken, but using a phone as an otoscope is awkward and requires some practice to master.

In addition, we use Epic software at our institution, and it has been challenging to get data from Epic to evaluate the curriculum at the level of changes in practice behavior, such as fewer inappropriate antibiotic prescriptions. Evaluation at Kirkpatrick's first level, learner satisfaction, and third level, learner behavior, has been overwhelmingly positive.

This curriculum is written in a clear, flexible manner in which specific learning objectives, learning activities, and assessments can be modified by the teacher to fit the feasibility and needs of any program. We have used this curriculum successfully with medical students, physician assistant students, family medicine residents, and pediatric residents. The scholarship map may be used to think about ways to adjust the curriculum's learning objectives and assessment methods to meet the needs of any educational program. It also may be used to map assessments to learners at the program level.

A. Middle Ear Examination Lesson Plan.docx

B. Diagnosis Lesson Plan.docx

C. Treatment Lesson Plan.docx

D. Preceptor Assessment Worksheet.docx

E. Pictures of Normal and Abnormal Middle Ear Anatomy.pptx

F. Bacteria Learning Activity.docx

G. AAP Recommendations for Initial Management for Uncomplicated AOM.docx

H. Number Needed to Treat to Benefit or Harm.docx

I. Acute Otitis Media Curriculum Scholarship Map.docx

J. Retrospective Pre and Post Self-Assessment.docx

K. Curriculum Evaluation.docx

All appendices are peer reviewed as integral parts of the Original Publication.