Tess H Aulet1, Jesse S Moore2, Peter W Callas3, Cate Nicholas4, Michael Hulme5. 1. Department of Surgery, University of Vermont Medical Center, 111 Colchester Ave, Burlington, VT, 05401, USA. Electronic address: Tess.aulet@uvmhealth.org. 2. Department of Surgery, University of Vermont Medical Center, 111 Colchester Ave, Mailstop, 320FL4, Burlington, VT, 05401, USA. Electronic address: Jesse.Moore@uvmhealth.org. 3. University of Vermont, Medical Biostatistics, 24C Hills Building, Burlington, VT, 05405, USA. Electronic address: pcallas@uvm.edu. 4. Clinical Simulation Laboratory at the University of Vermont, 237 Rowell Hall, 106 Carrigan Drive, Burlington, VT, 05405, USA. Electronic address: cate.nicholas@med.uvm.edu. 5. Wake Forest School of Medicine, General Surgery Office, 5th Floor Watlington Hall, Medical Center Boulevard, Winston-Salem, NC, 27157, USA. Electronic address: mhulme@wakehealth.edu.
Abstract
BACKGROUND: The AAMC developed 13 Core Entrustable Professional Activities (EPAs) for graduating medical students. EPA 5 is: Document a clinical encounter in the patient record. Our goal was to develop an assessment rubric and gather evidence to support its validity in measuring progress towards entrustability. METHODS: A rubric was developed for EPA 5. During the 2017 surgery clerkship, 57 students wrote a note for each of two standardized patient (SP) encounters. These notes were prospectively collected and assessed by two physician raters. Messick's validity framework was used to gather validity data. RESULTS: Inter-rater reliability with two raters was excellent, ICC = 0.86 (ICC 95%, confidence interval (CI) 0.80-0.90) for overall note score. Correlation between note items and SP checklists ranged 0.39-0.46 (p < 0.05) and between note items and clinical evaluations 0.28-0.39 (p < 0.05). CONCLUSIONS: There is initial reliability evidence supporting the use of our rubric for assessing progress towards entrustability of EPA 5.
BACKGROUND: The AAMC developed 13 Core Entrustable Professional Activities (EPAs) for graduating medical students. EPA 5 is: Document a clinical encounter in the patient record. Our goal was to develop an assessment rubric and gather evidence to support its validity in measuring progress towards entrustability. METHODS: A rubric was developed for EPA 5. During the 2017 surgery clerkship, 57 students wrote a note for each of two standardized patient (SP) encounters. These notes were prospectively collected and assessed by two physician raters. Messick's validity framework was used to gather validity data. RESULTS: Inter-rater reliability with two raters was excellent, ICC = 0.86 (ICC 95%, confidence interval (CI) 0.80-0.90) for overall note score. Correlation between note items and SP checklists ranged 0.39-0.46 (p < 0.05) and between note items and clinical evaluations 0.28-0.39 (p < 0.05). CONCLUSIONS: There is initial reliability evidence supporting the use of our rubric for assessing progress towards entrustability of EPA 5.