Matthew B Weinger1, Arna Banerjee, Amanda R Burden, William R McIvor, John Boulet, Jeffrey B Cooper, Randolph Steadman, Matthew S Shotwell, Jason M Slagle, Samuel DeMaria, Laurence Torsher, Elizabeth Sinz, Adam I Levine, John Rask, Fred Davis, Christine Park, David M Gaba. 1. From the Center for Experiential Learning and Assessment (M.B.W., A.B.), Vanderbilt University School of Medicine (M.B.W., M.S.S., J.M.S.), Nashville, Tennessee; Center for Research and Innovation in Systems Safety, Vanderbilt University Medical Center, Nashville, Tennessee (M.B.W., A.B., J.M.S.); Geriatric Research Education and Clinical Center, VA Tennessee Valley Healthcare System, Nashville, Tennessee (M.B.W.); Cooper Medical School, Rowan University, Cooper University Hospital, Camden, New Jersey (A.R.B.); University of Pittsburgh Medical Center and Winter Institute for Simulation Education and Research, Pittsburgh, Pennsylvania (W.R.M.); Foundation for Advancement of International Medical Education and Research, Philadelphia, Pennsylvania (J.B.); Harvard Medical School, Massachusetts General Hospital, Boston, Massachusetts (J.B.C.); Center for Medical Simulation, Boston, Massachusetts (J.B.C., F.D.); Department of Anesthesiology, University of California Los Angeles, Los Angeles, California (R.S.); Icahn School of Medicine at Mt. Sinai, New York, New York (S.D., A.I.L.); Mayo Clinic, Rochester, Minnesota (L.T.); Pennsylvania State University College of Medicine, Hershey, Pennsylvania (E.S.); Department of Anesthesiology and Critical Care Medicine and University of New Mexico Basic and Advanced Trauma Computer Assisted Virtual Experience Simulation Center, University of New Mexico School of Medicine, Albuquerque, New Mexico (J.R.); Department of Anesthesiology, Feinberg School of Medicine, Northwestern University, Chicago, Illinois (C.P.); Center for Immersive and Simulation-based Learning, Stanford University School of Medicine, Stanford, California (D.M.G.); VA Palo Alto Health Care System, Palo Alto, California (D.M.G.).
Abstract
BACKGROUND: We sought to determine whether mannequin-based simulation can reliably characterize how board-certified anesthesiologists manage simulated medical emergencies. Our primary focus was to identify gaps in performance and to establish psychometric properties of the assessment methods. METHODS: A total of 263 consenting board-certified anesthesiologists participating in existing simulation-based maintenance of certification courses at one of eight simulation centers were video recorded performing simulated emergency scenarios. Each participated in two 20-min, standardized, high-fidelity simulated medical crisis scenarios, once each as primary anesthesiologist and first responder. Via a Delphi technique, an independent panel of expert anesthesiologists identified critical performance elements for each scenario. Trained, blinded anesthesiologists rated video recordings using standardized rating tools. Measures included the percentage of critical performance elements observed and holistic (one to nine ordinal scale) ratings of participant's technical and nontechnical performance. Raters also judged whether the performance was at a level expected of a board-certified anesthesiologist. RESULTS: Rater reliability for most measures was good. In 284 simulated emergencies, participants were rated as successfully completing 81% (interquartile range, 75 to 90%) of the critical performance elements. The median rating of both technical and nontechnical holistic performance was five, distributed across the nine-point scale. Approximately one-quarter of participants received low holistic ratings (i.e., three or less). Higher-rated performances were associated with younger age but not with previous simulation experience or other individual characteristics. Calling for help was associated with better individual and team performance. CONCLUSIONS: Standardized simulation-based assessment identified performance gaps informing opportunities for improvement. If a substantial proportion of experienced anesthesiologists struggle with managing medical emergencies, continuing medical education activities should be reevaluated.
BACKGROUND: We sought to determine whether mannequin-based simulation can reliably characterize how board-certified anesthesiologists manage simulated medical emergencies. Our primary focus was to identify gaps in performance and to establish psychometric properties of the assessment methods. METHODS: A total of 263 consenting board-certified anesthesiologists participating in existing simulation-based maintenance of certification courses at one of eight simulation centers were video recorded performing simulated emergency scenarios. Each participated in two 20-min, standardized, high-fidelity simulated medical crisis scenarios, once each as primary anesthesiologist and first responder. Via a Delphi technique, an independent panel of expert anesthesiologists identified critical performance elements for each scenario. Trained, blinded anesthesiologists rated video recordings using standardized rating tools. Measures included the percentage of critical performance elements observed and holistic (one to nine ordinal scale) ratings of participant's technical and nontechnical performance. Raters also judged whether the performance was at a level expected of a board-certified anesthesiologist. RESULTS: Rater reliability for most measures was good. In 284 simulated emergencies, participants were rated as successfully completing 81% (interquartile range, 75 to 90%) of the critical performance elements. The median rating of both technical and nontechnical holistic performance was five, distributed across the nine-point scale. Approximately one-quarter of participants received low holistic ratings (i.e., three or less). Higher-rated performances were associated with younger age but not with previous simulation experience or other individual characteristics. Calling for help was associated with better individual and team performance. CONCLUSIONS: Standardized simulation-based assessment identified performance gaps informing opportunities for improvement. If a substantial proportion of experienced anesthesiologists struggle with managing medical emergencies, continuing medical education activities should be reevaluated.
Authors: Sean Coeckelenbergh; Cedrick Zaouter; Brenton Alexander; Maxime Cannesson; Joseph Rinehart; Jacques Duranteau; Philippe Van der Linden; Alexandre Joosten Journal: J Anesth Date: 2019-09-25 Impact factor: 2.078
Authors: M H Andreae; A Dudak; V Cherian; P Dhar; P G Dalal; W Po; M Pilipovic; B Shah; W Hazard; D L Rodgers; E H Sinz Journal: Data Brief Date: 2020-07-15
Authors: Thomas J Caruso; Asheen Rama; Lynda J Knight; Ralph Gonzales; Farrukh Munshey; Curtis Darling; Michael Chen; Paul J Sharek Journal: Pediatr Qual Saf Date: 2019-04-12