BACKGROUND: In 2013, milestone ratings became a reporting requirement for emergency medicine (EM) residency programs. Programs rate each resident in the fall and spring on 23 milestone subcompetencies. OBJECTIVE: This study examined the incidence of straight line scoring (SLS) for EM Milestone ratings, defined as a resident being assessed the same score across the milestone subcompetencies. METHODS: This descriptive analysis measured the frequencies of SLS for all Accreditation Council for Graduate Medical Education (ACGME)-accredited EM programs during the 2015-2016 academic year. Outcomes were the frequency of SLS in the fall and spring milestone assessments, changes in the number of SLS reports, and reporting trends. Chi-square analysis compared nominal variables. RESULTS: There were 6257 residents in the fall and 6588 in the spring. Milestone scores were reported for 6173 EM residents in the fall (99% of 6257) and spring (94% of 6588). In the fall, 93% (5753 residents) did not receive SLS ratings and 420 (7%) did, with no significant difference compared with the spring (5776 [94%] versus 397 [6%]). Subgroup analysis showed higher SLS results for residents' first ratings (183 of 2136 versus 237 of 4220, P < .0001) and for their final ratings (200 of 2019 versus 197 of 4354, P < .0001). Twenty percent of programs submitted 10% or more SLS ratings, and a small percentage submitted more than 50% of ratings as SLS. CONCLUSIONS: Most programs did not submit SLS ratings. Because of the statistical improbability of SLS, any SLS ratings reduce the validity assertions of the milestone assessments.
BACKGROUND: In 2013, milestone ratings became a reporting requirement for emergency medicine (EM) residency programs. Programs rate each resident in the fall and spring on 23 milestone subcompetencies. OBJECTIVE: This study examined the incidence of straight line scoring (SLS) for EM Milestone ratings, defined as a resident being assessed the same score across the milestone subcompetencies. METHODS: This descriptive analysis measured the frequencies of SLS for all Accreditation Council for Graduate Medical Education (ACGME)-accredited EM programs during the 2015-2016 academic year. Outcomes were the frequency of SLS in the fall and spring milestone assessments, changes in the number of SLS reports, and reporting trends. Chi-square analysis compared nominal variables. RESULTS: There were 6257 residents in the fall and 6588 in the spring. Milestone scores were reported for 6173 EM residents in the fall (99% of 6257) and spring (94% of 6588). In the fall, 93% (5753 residents) did not receive SLS ratings and 420 (7%) did, with no significant difference compared with the spring (5776 [94%] versus 397 [6%]). Subgroup analysis showed higher SLS results for residents' first ratings (183 of 2136 versus 237 of 4220, P < .0001) and for their final ratings (200 of 2019 versus 197 of 4354, P < .0001). Twenty percent of programs submitted 10% or more SLS ratings, and a small percentage submitted more than 50% of ratings as SLS. CONCLUSIONS: Most programs did not submit SLS ratings. Because of the statistical improbability of SLS, any SLS ratings reduce the validity assertions of the milestone assessments.
Authors: Michael S Beeson; Wallace A Carter; Theodore A Christopher; Jonathan W Heidt; James H Jones; Lynne E Meyer; Susan B Promes; Kevin G Rodgers; Philip H Shayne; Susan R Swing; Mary Jo Wagner Journal: Acad Emerg Med Date: 2013-06-19 Impact factor: 3.451
Authors: Susan R Swing; Michael S Beeson; Carol Carraccio; Michael Coburn; William Iobst; Nathan R Selden; Peter J Stern; Kay Vydareny Journal: J Grad Med Educ Date: 2013-03
Authors: Stanley J Hamstra; Kenji Yamazaki; Melissa A Barton; Sally A Santen; Michael S Beeson; Eric S Holmboe Journal: Acad Med Date: 2019-10 Impact factor: 6.893
Authors: Stanley J Hamstra; Monica M Cuddy; Daniel Jurich; Kenji Yamazaki; John Burkhardt; Eric S Holmboe; Michael A Barone; Sally A Santen Journal: Acad Med Date: 2021-09-01 Impact factor: 7.840
Authors: Danielle Hart; Douglas Franzen; Michael Beeson; Rahul Bhat; Miriam Kulkarni; Lorraine Thibodeau; Moshe Weizberg; Susan Promes Journal: West J Emerg Med Date: 2018-11-30