BACKGROUND: The Pediatrics Milestone Project uses behavioral anchors, narrative descriptions of observable behaviors, to describe learner progression through the Accreditation Council for Graduate Medical Education competencies. Starting June 2014, pediatrics programs were required to submit milestone reports for their trainees semiannually. Likert-type scale assessment tools were not designed to inform milestone reporting, creating a challenge for Clinical Competency Committees. OBJECTIVE: To determine if milestone-based assessments better stratify trainees by training level compared to Likert-type assessments. METHODS: We compared assessment results for 3 subcompetencies after changing from a 5-point Likert scale to milestone-based behavioral anchors in July 2013. Program leadership evaluated the new system by (1) comparing PGY-1 mean scores on Likert-type versus milestone-based assessments; and (2) comparing mean scores on the Likert-type versus milestone-based assessments across PGY levels. RESULTS: Mean scores for PGY-1 residents were significantly higher on the prior year's Likert-type assessments than milestone-based assessments for all 3 subcompetencies (P < .01). Stratification by PGY level was not observed with Likert-type assessments (eg, interpersonal and communication skills 1 [ICS1] mean score for PGY-1, 3.99 versus PGY-3, 3.98; P = .98). In contrast, milestone-based assessments demonstrated stratification by PGY level (eg, the ICS1 mean score was 3.06 for PGY-1, 3.83 for PGY-2, and 3.99 for PGY-3; P < .01 for PGY-1 versus PGY-3). Significantly different means by trainee level were noted across 21 subcompetencies on milestone-based assessments (P < .01 for PGY-1 versus PGY-3). CONCLUSIONS: Initial results indicate milestone-based assessments stratify trainee performance by level better than Likert-type assessments. Average PGY-level scores from milestone-based assessments may ultimately provide guidance for determining whether trainees are progressing at the expected pace.
BACKGROUND: The Pediatrics Milestone Project uses behavioral anchors, narrative descriptions of observable behaviors, to describe learner progression through the Accreditation Council for Graduate Medical Education competencies. Starting June 2014, pediatrics programs were required to submit milestone reports for their trainees semiannually. Likert-type scale assessment tools were not designed to inform milestone reporting, creating a challenge for Clinical Competency Committees. OBJECTIVE: To determine if milestone-based assessments better stratify trainees by training level compared to Likert-type assessments. METHODS: We compared assessment results for 3 subcompetencies after changing from a 5-point Likert scale to milestone-based behavioral anchors in July 2013. Program leadership evaluated the new system by (1) comparing PGY-1 mean scores on Likert-type versus milestone-based assessments; and (2) comparing mean scores on the Likert-type versus milestone-based assessments across PGY levels. RESULTS: Mean scores for PGY-1 residents were significantly higher on the prior year's Likert-type assessments than milestone-based assessments for all 3 subcompetencies (P < .01). Stratification by PGY level was not observed with Likert-type assessments (eg, interpersonal and communication skills 1 [ICS1] mean score for PGY-1, 3.99 versus PGY-3, 3.98; P = .98). In contrast, milestone-based assessments demonstrated stratification by PGY level (eg, the ICS1 mean score was 3.06 for PGY-1, 3.83 for PGY-2, and 3.99 for PGY-3; P < .01 for PGY-1 versus PGY-3). Significantly different means by trainee level were noted across 21 subcompetencies on milestone-based assessments (P < .01 for PGY-1 versus PGY-3). CONCLUSIONS: Initial results indicate milestone-based assessments stratify trainee performance by level better than Likert-type assessments. Average PGY-level scores from milestone-based assessments may ultimately provide guidance for determining whether trainees are progressing at the expected pace.
Authors: Patricia J Hicks; Robert Englander; Daniel J Schumacher; Ann Burke; Bradley J Benson; Susan Guralnick; Stephen Ludwig; Carol Carraccio Journal: J Grad Med Educ Date: 2010-12
Authors: Patricia J Hicks; Daniel J Schumacher; Bradley J Benson; Ann E Burke; Robert Englander; Susan Guralnick; Stephen Ludwig; Carol Carraccio Journal: J Grad Med Educ Date: 2010-09
Authors: Susan R Swing; Michael S Beeson; Carol Carraccio; Michael Coburn; William Iobst; Nathan R Selden; Peter J Stern; Kay Vydareny Journal: J Grad Med Educ Date: 2013-03
Authors: Carol Carraccio; Bradley Benson; Ann Burke; Robert Englander; Susan Guralnick; Patricia Hicks; Stephen Ludwig; Daniel Schumacher; Jerry Vasilias Journal: J Grad Med Educ Date: 2013-03
Authors: Daniel J Schumacher; Nancy D Spector; Sharon Calaman; Daniel C West; Mario Cruz; John G Frohna; Javier Gonzalez Del Rey; Kristina K Gustafson; Sue Ellen Poynter; Glenn Rosenbluth; W Michael Southgate; Robert J Vinci; Theodore C Sectish Journal: Pediatrics Date: 2014-04-14 Impact factor: 7.124
Authors: Eric S Holmboe; Kenji Yamazaki; Laura Edgar; Lisa Conforti; Nicholas Yaghmour; Rebecca S Miller; Stanley J Hamstra Journal: J Grad Med Educ Date: 2015-09
Authors: Shalini T Reddy; Ara Tekian; Steven J Durning; Shanu Gupta; Justin Endo; Brenda Affinati; Yoon Soo Park Journal: J Grad Med Educ Date: 2018-06
Authors: Ara Tekian; Yoon Soo Park; Sarette Tilton; Patrick F Prunty; Eric Abasolo; Fred Zar; David A Cook Journal: Acad Med Date: 2019-12 Impact factor: 6.893