Andrea L O'Boyle1, Richard B Mink, Nick G Anas, Tom C Kallay. 1. 1Division of Pediatric Critical Care Medicine, Department of Pediatrics, Harbor-UCLA Medical Center and Los Angeles BioMedical Research Institute, Torrance, CA. 2Division of Pediatric Critical Care Medicine, Department of Pediatrics, Children's Hospital of Orange County, Orange, CA. 3Department of Pediatrics, David Geffen School of Medicine at University of California Los Angeles, Los Angeles, CA. 4Department of Pediatrics, University of California, Irvine, CA.
Abstract
OBJECTIVES: Graduate medical education is shifting toward an outcome-based paradigm, where physicians are evaluated for competency using well-defined criteria. Our aim was to learning objectives and a testing tool to assess competency in the management of mechanical ventilation for infants, children, and adolescents and to verify that the test was reliable and valid. DESIGN: Prospective reliability and validity study. SETTING: Large, university-affiliated academic hospital. SUBJECTS: Sixty-one total subjects from five different academic centers divided into three groups of varying experience. The groups were second- and third-year pediatric residents (Novice), second- and third-year pediatric critical care fellows (Advanced), and pediatric critical care faculty (Expert). INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: Ten learning objectives considered important for the management of pediatric mechanical ventilation were developed from expert opinion and current evidence. Based on these objectives, a 35-question multiple choice, knowledge- and case-based test was created. Content validity was achieved by consensus of three experts in pediatric critical care medicine evaluating whether the questions reflected the learning objectives and the responses were consistent with current practice and evidence-based medicine. The test was then administered to the three groups to establish construct validity. The "Novice" group scored a mean of 34.6% (95% CI, 28-41%), the "Advanced" group a mean of 59.4% (95% CI, 53-65%), and the "Expert" group a mean of 74.8% (95% CI, 69-80%), with p less than 0.01 for all comparisons. As determined by Hoyt's analysis, the reliability coefficient was 0.89, reflecting excellent reliability. CONCLUSIONS: This is the first description of specific learning objectives for management of pediatric mechanical ventilation and the first validated and reliable testing tool for assessing knowledge. This tool could be used by fellowship programs to assess fellow competency and identify knowledge gaps in this area prior to completion of training. Further work must be done to determine the criteria for determination of competency.
OBJECTIVES: Graduate medical education is shifting toward an outcome-based paradigm, where physicians are evaluated for competency using well-defined criteria. Our aim was to learning objectives and a testing tool to assess competency in the management of mechanical ventilation for infants, children, and adolescents and to verify that the test was reliable and valid. DESIGN: Prospective reliability and validity study. SETTING: Large, university-affiliated academic hospital. SUBJECTS: Sixty-one total subjects from five different academic centers divided into three groups of varying experience. The groups were second- and third-year pediatric residents (Novice), second- and third-year pediatric critical care fellows (Advanced), and pediatric critical care faculty (Expert). INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: Ten learning objectives considered important for the management of pediatric mechanical ventilation were developed from expert opinion and current evidence. Based on these objectives, a 35-question multiple choice, knowledge- and case-based test was created. Content validity was achieved by consensus of three experts in pediatric critical care medicine evaluating whether the questions reflected the learning objectives and the responses were consistent with current practice and evidence-based medicine. The test was then administered to the three groups to establish construct validity. The "Novice" group scored a mean of 34.6% (95% CI, 28-41%), the "Advanced" group a mean of 59.4% (95% CI, 53-65%), and the "Expert" group a mean of 74.8% (95% CI, 69-80%), with p less than 0.01 for all comparisons. As determined by Hoyt's analysis, the reliability coefficient was 0.89, reflecting excellent reliability. CONCLUSIONS: This is the first description of specific learning objectives for management of pediatric mechanical ventilation and the first validated and reliable testing tool for assessing knowledge. This tool could be used by fellowship programs to assess fellow competency and identify knowledge gaps in this area prior to completion of training. Further work must be done to determine the criteria for determination of competency.
Authors: Jonathan M Keller; Dru Claar; Juliana Carvalho Ferreira; David C Chu; Tanzib Hossain; William Graham Carlos; Jeffrey A Gold; Stephanie A Nonas; Nitin Seam Journal: J Grad Med Educ Date: 2019-08