BACKGROUND: The increasing use of workplace-based assessments (WBAs) in competency-based medical education has led to large data sets that assess resident performance longitudinally. With large data sets, problems that arise from missing data are increasingly likely. OBJECTIVE: The purpose of this study is to examine (1) whether data are missing at random across various WBAs, and (2) the relationship between resident performance and the proportion of missing data. METHODS: During 2012-2013, a total of 844 WBAs of CanMEDs Roles were completed for 9 second-year emergency medicine residents. To identify whether missing data were randomly distributed across various WBAs, the total number of missing data points was calculated for each Role. To examine whether the amount of missing data was related to resident performance, 5 faculty members rank-ordered the residents based on performance. A median rank score was calculated for each resident and was correlated with the proportion of missing data. RESULTS: More data were missing for Health Advocate and Professional WBAs relative to other competencies (P < .001). Furthermore, resident rankings were not related to the proportion of missing data points (r = 0.29, P > .05). CONCLUSIONS: The results of the present study illustrate that some CanMEDS Roles are less likely to be assessed than others. At the same time, the amount of missing data did not correlate with resident performance, suggesting lower-performing residents are no more likely to have missing data than their higher-performing peers. This article discusses several approaches to dealing with missing data.
BACKGROUND: The increasing use of workplace-based assessments (WBAs) in competency-based medical education has led to large data sets that assess resident performance longitudinally. With large data sets, problems that arise from missing data are increasingly likely. OBJECTIVE: The purpose of this study is to examine (1) whether data are missing at random across various WBAs, and (2) the relationship between resident performance and the proportion of missing data. METHODS: During 2012-2013, a total of 844 WBAs of CanMEDs Roles were completed for 9 second-year emergency medicine residents. To identify whether missing data were randomly distributed across various WBAs, the total number of missing data points was calculated for each Role. To examine whether the amount of missing data was related to resident performance, 5 faculty members rank-ordered the residents based on performance. A median rank score was calculated for each resident and was correlated with the proportion of missing data. RESULTS: More data were missing for Health Advocate and Professional WBAs relative to other competencies (P < .001). Furthermore, resident rankings were not related to the proportion of missing data points (r = 0.29, P > .05). CONCLUSIONS: The results of the present study illustrate that some CanMEDS Roles are less likely to be assessed than others. At the same time, the amount of missing data did not correlate with resident performance, suggesting lower-performing residents are no more likely to have missing data than their higher-performing peers. This article discusses several approaches to dealing with missing data.
Authors: William F Iobst; Jonathan Sherbino; Olle Ten Cate; Denyse L Richardson; Deepak Dath; Susan R Swing; Peter Harris; Rani Mungroo; Eric S Holmboe; Jason R Frank Journal: Med Teach Date: 2010 Impact factor: 3.650
Authors: C P M van der Vleuten; L W T Schuwirth; F Scheele; E W Driessen; B Hodges Journal: Best Pract Res Clin Obstet Gynaecol Date: 2010-05-26 Impact factor: 5.237
Authors: Jonathan Sherbino; Kulamakan Kulasegaram; Andrew Worster; Geoffrey R Norman Journal: Adv Health Sci Educ Theory Pract Date: 2013-01-11 Impact factor: 3.853
Authors: J M W Moonen-van Loon; K Overeem; H H L M Donkers; C P M van der Vleuten; E W Driessen Journal: Adv Health Sci Educ Theory Pract Date: 2013-03-15 Impact factor: 3.853
Authors: Teresa M Chan; Stefanie S Sebok-Syer; Warren J Cheung; Martin Pusic; Christine Stehman; Michael Gottlieb Journal: AEM Educ Train Date: 2020-11-05