Jaime Jordan1,2, Kaushal Shah3, Andrew W Phillips4, Nicholas Hartman5, Jeffrey Love6, Michael Gottlieb7. 1. UCLA Department of Emergency Medicine Ronald Reagan UCLA Medical Center Los Angeles CA. 2. David Geffen School of Medicine at University of California Los Angeles Los Angeles CA. 3. Department of Emergency Medicine Icahn School of Medicine at Mount Sinai Medical Center New York NY. 4. Department of Emergency Medicine University of North Carolina Chapel Hill NC. 5. Department of Emergency Medicine Wake Forest School of Medicine Winston-Salem NC. 6. Department of Emergency Medicine George Washington University Washington DC. 7. Department of Emergency Medicine Rush University Medical Center Chicago IL.
Abstract
BACKGROUND: There are a limited number of emergency medicine (EM) physicians with expertise in education research. The Harvard Macy "step-back" method is an emerging model utilized to gather group feedback. Despite its use in multiple educational settings, there are little published data demonstrating effectiveness. OBJECTIVES: Our objective was to create and evaluate a national faculty development session providing consultation in education research utilizing the step-back method. METHODS: This was a pilot study. EM experts in education research from across the country served as facilitators for a faculty development session held at the 2018 Council of Emergency Medicine Residency Directors Academic Assembly. Small groups consisting of two or three facilitators and one or two participants were formed and each participant underwent a step-back consultation for their education research study. Participants wrote their study question before and after the session. After the session, facilitators and participants completed an evaluative survey consisting of multiple-choice, Likert-type, and free-response items. Descriptive statistics were reported. Qualitative analysis using a thematic approach was performed on free-response data. Participant study questions were assessed by the PICO (population, intervention, comparison, outcome) and FINER (feasible, interesting, novel, ethical, relevant) criteria. Both scales were evaluated using a two-way random-consistency intraclass correlation. Before and after scores were evaluated with a paired t-test. RESULTS: Twenty-four facilitators and 13 participants completed the step-back session. Evaluations from 20 facilitators and nine participants were submitted and analyzed. Sixteen of 20 facilitators felt that the step-back method "greatly facilitated" their ability to share their education research expertise. All facilitators and participants recommended that the session be provided at a future academic assembly. Regarding suggestions for improvement, qualitative analysis revealed three major themes: praise for the session, desire for additional time, and a room set up more conducive to small group work. Seven of nine responding participants felt that the session was "very valuable" for improving the strength of their study methods. Qualitative analysis regarding change in study as a result of the step-back session yielded four major themes: refinement of study question, more specific outcomes and measurements, improvement in study design, and greater understanding of study limitations. Both FINER and PICO scale comparisons showed improvement pre- and postintervention (PICO 60% relative increase; FINER 16% relative increase). Neither achieved statistical significance (PICO t(5) = -1.835, p = 0.126; and FINER t(5) = -1.305, p = 0.249). CONCLUSION: A national-level education research consultation utilizing the step-back method was feasible to implement and highly valued by facilitators and participants. Potential positive outcomes include refinement of study question, more specific outcomes and measurements, improvement in study design, and greater understanding of limitations. These results may inform others who want to utilize this method.
BACKGROUND: There are a limited number of emergency medicine (EM) physicians with expertise in education research. The Harvard Macy "step-back" method is an emerging model utilized to gather group feedback. Despite its use in multiple educational settings, there are little published data demonstrating effectiveness. OBJECTIVES: Our objective was to create and evaluate a national faculty development session providing consultation in education research utilizing the step-back method. METHODS: This was a pilot study. EM experts in education research from across the country served as facilitators for a faculty development session held at the 2018 Council of Emergency Medicine Residency Directors Academic Assembly. Small groups consisting of two or three facilitators and one or two participants were formed and each participant underwent a step-back consultation for their education research study. Participants wrote their study question before and after the session. After the session, facilitators and participants completed an evaluative survey consisting of multiple-choice, Likert-type, and free-response items. Descriptive statistics were reported. Qualitative analysis using a thematic approach was performed on free-response data. Participant study questions were assessed by the PICO (population, intervention, comparison, outcome) and FINER (feasible, interesting, novel, ethical, relevant) criteria. Both scales were evaluated using a two-way random-consistency intraclass correlation. Before and after scores were evaluated with a paired t-test. RESULTS: Twenty-four facilitators and 13 participants completed the step-back session. Evaluations from 20 facilitators and nine participants were submitted and analyzed. Sixteen of 20 facilitators felt that the step-back method "greatly facilitated" their ability to share their education research expertise. All facilitators and participants recommended that the session be provided at a future academic assembly. Regarding suggestions for improvement, qualitative analysis revealed three major themes: praise for the session, desire for additional time, and a room set up more conducive to small group work. Seven of nine responding participants felt that the session was "very valuable" for improving the strength of their study methods. Qualitative analysis regarding change in study as a result of the step-back session yielded four major themes: refinement of study question, more specific outcomes and measurements, improvement in study design, and greater understanding of study limitations. Both FINER and PICO scale comparisons showed improvement pre- and postintervention (PICO 60% relative increase; FINER 16% relative increase). Neither achieved statistical significance (PICO t(5) = -1.835, p = 0.126; and FINER t(5) = -1.305, p = 0.249). CONCLUSION: A national-level education research consultation utilizing the step-back method was feasible to implement and highly valued by facilitators and participants. Potential positive outcomes include refinement of study question, more specific outcomes and measurements, improvement in study design, and greater understanding of limitations. These results may inform others who want to utilize this method.
Authors: Lalena M Yarris; Amy Miller Juve; Anthony R Artino; Gail M Sullivan; Steven Rougas; Barbara Joyce; Kevin Eva Journal: J Grad Med Educ Date: 2014-09
Authors: Darcy A Reed; David A Cook; Thomas J Beckman; Rachel B Levine; David E Kern; Scott M Wright Journal: JAMA Date: 2007-09-05 Impact factor: 56.272
Authors: Gail M Sullivan; Deborah Simpson; David A Cook; Nicole M DeIorio; Kathryn Andolsek; Lawrence Opas; Ingrid Philibert; Lalena M Yarris Journal: J Grad Med Educ Date: 2014-09
Authors: Jaime Jordan; Wendy C Coates; Samuel Clarke; Daniel P Runde; Emilie Fowlkes; Jacqueline Kurth; Lalena M Yarris Journal: West J Emerg Med Date: 2016-12-05
Authors: Jaime Jordan; Wendy C Coates; Samuel Clarke; Daniel Runde; Emilie Fowlkes; Jaqueline Kurth; Lalena Yarris Journal: West J Emerg Med Date: 2018-03-13
Authors: Stephanie N Stapleton; Ambrose H Wong; Jessica M Ray; Ashley C Rider; Tiffany Moadel; Suzanne Bentley; Michael Cassara Journal: AEM Educ Train Date: 2020-10-21