Tim Dwyer1, Susan Glover Takahashi2, Melissa Kennedy Hynes2, Jodi Herold3, David Wasserstein3, Markku Nousiainen4, Peter Ferguson5, Veronica Wadey6, M Lucas Murnaghan7, Tim Leroux3, John Semple8, Brian Hodges9, Darrell Ogilvie-Harris10. 1. The Women's College Hospital, Mt. Sinai Hospital, Toronto, Ont. 2. The Postgraduate Medicine Office, University of Toronto, Toronto, Ont. 3. The University of Toronto, Toronto, Ont. 4. The Holland Orthopaedic and Arthritic Centre, Toronto, Ont. 5. The Mt. Sinai Hospital, Toronto, Ont. 6. The Sunnybrook Health Sciences Centre, Toronto, Ont. 7. The Women's College Hospital, University of Toronto Orthopaedics Sports Medicine, and the Hospital for Sick Children, Toronto, Ont. 8. The Women's College Hospital, Toronto, Ont. 9. The Toronto General Hospital, Toronto, Ont. 10. The Women's College Hospital, The Hospital for Sick Children, and Toronto Western Hospital, Toronto, Ont.
Abstract
BACKGROUND: Assessing residents' understanding and application of the 6 intrinsic CanMEDS roles (communicator, professional, manager, collaborator, health advocate, scholar) is challenging for postgraduate medical educators. We hypothesized that an objective structured clinical examination (OSCE) designed to assess multiple intrinsic CanMEDS roles would be sufficiently reliable and valid. METHODS: The OSCE comprised 6 10-minute stations, each testing 2 intrinsic roles using case-based scenarios (with or without the use of standardized patients). Residents were evaluated using 5-point scales and an overall performance rating at each station. Concurrent validity was sought by correlation with in-training evaluation reports (ITERs) from the last 12 months and an ordinal ranking created by program directors (PDs). RESULTS: Twenty-five residents from postgraduate years (PGY) 0, 3 and 5 participated. The interstation reliability for total test scores (percent) was 0.87, while reliability for each of the communicator, collaborator, manager and professional roles was greater than 0.8. Total test scores, individual station scores and individual CanMEDS role scores all showed a significant effect by PGY level. Analysis of the PD rankings of intrinsic roles demonstrated a high correlation with the OSCE role scores. A correlation was seen between ITER and OSCE for the communicator role, while the ITER medical expert and total scores highly correlated with the communicator, manager and professional OSCE scores. CONCLUSION: An OSCE designed to assess the intrinsic CanMEDS roles was sufficiently valid and reliable for regular use in an orthopedic residency program.
BACKGROUND: Assessing residents' understanding and application of the 6 intrinsic CanMEDS roles (communicator, professional, manager, collaborator, health advocate, scholar) is challenging for postgraduate medical educators. We hypothesized that an objective structured clinical examination (OSCE) designed to assess multiple intrinsic CanMEDS roles would be sufficiently reliable and valid. METHODS: The OSCE comprised 6 10-minute stations, each testing 2 intrinsic roles using case-based scenarios (with or without the use of standardized patients). Residents were evaluated using 5-point scales and an overall performance rating at each station. Concurrent validity was sought by correlation with in-training evaluation reports (ITERs) from the last 12 months and an ordinal ranking created by program directors (PDs). RESULTS: Twenty-five residents from postgraduate years (PGY) 0, 3 and 5 participated. The interstation reliability for total test scores (percent) was 0.87, while reliability for each of the communicator, collaborator, manager and professional roles was greater than 0.8. Total test scores, individual station scores and individual CanMEDS role scores all showed a significant effect by PGY level. Analysis of the PD rankings of intrinsic roles demonstrated a high correlation with the OSCE role scores. A correlation was seen between ITER and OSCE for the communicator role, while the ITER medical expert and total scores highly correlated with the communicator, manager and professional OSCE scores. CONCLUSION: An OSCE designed to assess the intrinsic CanMEDS roles was sufficiently valid and reliable for regular use in an orthopedic residency program.
Authors: Joseph D Zuckerman; Justin P Holder; John J Mercuri; Donna P Phillips; Kenneth A Egol Journal: J Bone Joint Surg Am Date: 2012-04-18 Impact factor: 5.284
Authors: Tim Dwyer; Veronica Wadey; Douglas Archibald; William Kraemer; Jesse Slade Shantz; John Townley; Darrell Ogilvie-Harris; Massimo Petrera; Peter Ferguson; Markku Nousiainen Journal: Clin Orthop Relat Res Date: 2016-04 Impact factor: 4.176
Authors: Abraham M Goch; Raj Karia; David Taormina; Adina Kalet; Joseph Zuckerman; Kenneth A Egol; Donna Phillips Journal: J Grad Med Educ Date: 2018-04