Jason A Post1, Christopher M Wittich2, Kris G Thomas2, Denise M Dupras2, Andrew J Halvorsen2, Jay N Mandrekar3, Amy S Oxentenko2, Thomas J Beckman2. 1. College of Medicine, Department of Internal Medicine, Mayo Clinic, 200 1st St SW, Rochester, MN, 55905, USA. post.jason@mayo.edu. 2. College of Medicine, Department of Internal Medicine, Mayo Clinic, 200 1st St SW, Rochester, MN, 55905, USA. 3. College of Medicine, Department of Health Sciences Research, Mayo Clinic, 200 1st St SW, Rochester, MN, 55905, USA.
Abstract
BACKGROUND: Entrustable professional activities (EPAs) have been developed to assess resident physicians with respect to Accreditation Council for Graduate Medical Education (ACGME) competencies and milestones. Although the feasibility of using EPAs has been reported, we are unaware of previous validation studies on EPAs and potential associations between EPA quality scores and characteristics of educational programs. OBJECTIVES: Our aim was to validate an instrument for assessing the quality of EPAs for assessment of internal medicine residents, and to examine associations between EPA quality scores and features of rotations. DESIGN: This was a prospective content validation study to design an instrument to measure the quality of EPAs that were written for assessing internal medicine residents. PARTICIPANTS: Residency leadership at Mayo Clinic, Rochester participated in this study. This included the Program Director, Associate program directors and individual rotation directors. INTERVENTIONS: The authors reviewed salient literature. Items were developed to reflect domains of EPAs useful for assessment. The instrument underwent further testing and refinement. Each participating rotation director created EPAs that they felt would be meaningful to assess learner performance in their area. These 229 EPAs were then assessed with the QUEPA instrument to rate the quality of each EPA. MAIN MEASURES: Performance characteristics of the QUEPA are reported. Quality ratings of EPAs were compared to the primary ACGME competency, inpatient versus outpatient setting and specialty type. KEY RESULTS: QUEPA tool scores demonstrated excellent reliability (ICC range 0.72 to 0.94). Higher ratings were given to inpatient versus outpatient (3.88, 3.66; p = 0.03) focused EPAs. Medical knowledge EPAs scored significantly lower than EPAs assessing other competencies (3.34, 4.00; p < 0.0001). CONCLUSIONS: The QUEPA tool is supported by good validity evidence and may help in rating the quality of EPAs developed by individual programs. Programs should take care when writing EPAs for the outpatient setting or to assess medical knowledge, as these tended to be rated lower.
BACKGROUND: Entrustable professional activities (EPAs) have been developed to assess resident physicians with respect to Accreditation Council for Graduate Medical Education (ACGME) competencies and milestones. Although the feasibility of using EPAs has been reported, we are unaware of previous validation studies on EPAs and potential associations between EPA quality scores and characteristics of educational programs. OBJECTIVES: Our aim was to validate an instrument for assessing the quality of EPAs for assessment of internal medicine residents, and to examine associations between EPA quality scores and features of rotations. DESIGN: This was a prospective content validation study to design an instrument to measure the quality of EPAs that were written for assessing internal medicine residents. PARTICIPANTS: Residency leadership at Mayo Clinic, Rochester participated in this study. This included the Program Director, Associate program directors and individual rotation directors. INTERVENTIONS: The authors reviewed salient literature. Items were developed to reflect domains of EPAs useful for assessment. The instrument underwent further testing and refinement. Each participating rotation director created EPAs that they felt would be meaningful to assess learner performance in their area. These 229 EPAs were then assessed with the QUEPA instrument to rate the quality of each EPA. MAIN MEASURES: Performance characteristics of the QUEPA are reported. Quality ratings of EPAs were compared to the primary ACGME competency, inpatient versus outpatient setting and specialty type. KEY RESULTS: QUEPA tool scores demonstrated excellent reliability (ICC range 0.72 to 0.94). Higher ratings were given to inpatient versus outpatient (3.88, 3.66; p = 0.03) focused EPAs. Medical knowledge EPAs scored significantly lower than EPAs assessing other competencies (3.34, 4.00; p < 0.0001). CONCLUSIONS: The QUEPA tool is supported by good validity evidence and may help in rating the quality of EPAs developed by individual programs. Programs should take care when writing EPAs for the outpatient setting or to assess medical knowledge, as these tended to be rated lower.
Entities:
Keywords:
medial education–graduate; medical education–assessment/evaluation; medical education–quantitative methods
Authors: Paul A Harris; Robert Taylor; Robert Thielke; Jonathon Payne; Nathaniel Gonzalez; Jose G Conde Journal: J Biomed Inform Date: 2008-09-30 Impact factor: 6.317
Authors: Karen E Hauer; Olle Ten Cate; Christy Boscardin; David M Irby; William Iobst; Patricia S O'Sullivan Journal: Adv Health Sci Educ Theory Pract Date: 2013-07-27 Impact factor: 3.853
Authors: Eric J Warm; Bradley R Mathis; Justin D Held; Savita Pai; Jonathan Tolentino; Lauren Ashbrook; Cheryl K Lee; David Lee; Sharice Wood; Carl J Fichtenbaum; Daniel Schauer; Ryan Munyon; Caroline Mueller Journal: J Gen Intern Med Date: 2014-02-21 Impact factor: 5.128
Authors: Amy L Pittenger; Brenda L Gleason; Stuart T Haines; Stephen Neely; Melissa S Medina Journal: Am J Pharm Educ Date: 2019-11 Impact factor: 2.047
Authors: Andrew Coyle; Ira Helenius; Christina M Cruz; E Allison Lyons; Natalie May; John Andrilli; M Merav Bannet; Rachel Pinotti; David C Thomas Journal: J Grad Med Educ Date: 2019-04
Authors: Alicia Hamui-Sutton; Ana María Monterrosas-Rojas; Armando Ortiz-Montalvo; Felipe Flores-Morones; Uri Torruco-García; Andrea Navarrete-Martínez; Araceli Arrioja-Guerrero Journal: BMC Med Educ Date: 2017-08-25 Impact factor: 2.463
Authors: Michael W Cullen; Kyle W Klarich; Amy S Oxentenko; Andrew J Halvorsen; Thomas J Beckman Journal: BMC Med Educ Date: 2020-07-28 Impact factor: 2.463
Authors: Fares Alahdab; Andrew J Halvorsen; Jayawant N Mandrekar; Brianna E Vaa; Victor M Montori; Colin P West; M Hassan Murad; Thomas J Beckman Journal: BMJ Open Date: 2020-12-15 Impact factor: 2.692
Authors: Ryan A Meverden; Jason H Szostek; Saswati Mahapatra; Cathy D Schleck; Jayawant N Mandrekar; Thomas J Beckman; Christopher M Wittich Journal: BMC Med Educ Date: 2018-06-04 Impact factor: 2.463