Balakrishnan Kichu R Nair1, Joyce Mw Moonen-van Loon2, Mulavana Parvathy3, Brian C Jolly4, Cees Pm van der Vleuten2. 1. Centre for Medical Professional Development, John Hunter Hospital, Newcastle, NSW kichu.nair@newcastle.edu.au. 2. Maastricht University, Maastricht, The Netherlands. 3. Centre for Medical Professional Development, John Hunter Hospital, Newcastle, NSW. 4. University of Newcastle, Newcastle, NSW.
Abstract
OBJECTIVE: The fitness to practise of international medical graduates (IMGs) is usually evaluated with standardised assessment tests. The performance rather than the competency of practising doctors should, however, be assessed, for which reason workplace-based assessment (WBA) has gained increasing attention. Our aim was to assess the composite reliability of WBA instruments for assessing IMGs. DESIGN AND SETTING: Between June 2010 and April 2015, 142 IMGs were assessed by 99 calibrated assessors; each was assessed in the workplace over 6 months. The IMGs completed 970 case-based discussions (CBDs), 1741 mini-clinical examination exercises (mini-CEX), and 1020 multi-source feedback (MSF) assessments. PARTICIPANTS: 103 male and 39 female candidates from 28 countries (Africa, Asia, Europe, South America, South Pacific) in urban and rural hospitals of the Hunter New England Health region. MAIN OUTCOME MEASURES: The composite reliability across the three WBA tools, expressed as the standard error of measurement (SEM). RESULTS: In our WBA program, a combination of five CBD and 12 mini-CEX assessments achieved an SEM of 0.33, greater than the threshold 0.26 of a scale point. Adding six MSF results to the assessment package reduced the SEM to 0.24, which is adequately precise. CONCLUSIONS: Combining data from different WBA assessment instruments achieves acceptable reliability for assessing IMGs, provided that the panel of WBA assessment types are carefully selected and the assessors are calibrated.
OBJECTIVE: The fitness to practise of international medical graduates (IMGs) is usually evaluated with standardised assessment tests. The performance rather than the competency of practising doctors should, however, be assessed, for which reason workplace-based assessment (WBA) has gained increasing attention. Our aim was to assess the composite reliability of WBA instruments for assessing IMGs. DESIGN AND SETTING: Between June 2010 and April 2015, 142 IMGs were assessed by 99 calibrated assessors; each was assessed in the workplace over 6 months. The IMGs completed 970 case-based discussions (CBDs), 1741 mini-clinical examination exercises (mini-CEX), and 1020 multi-source feedback (MSF) assessments. PARTICIPANTS: 103 male and 39 female candidates from 28 countries (Africa, Asia, Europe, South America, South Pacific) in urban and rural hospitals of the Hunter New England Health region. MAIN OUTCOME MEASURES: The composite reliability across the three WBA tools, expressed as the standard error of measurement (SEM). RESULTS: In our WBA program, a combination of five CBD and 12 mini-CEX assessments achieved an SEM of 0.33, greater than the threshold 0.26 of a scale point. Adding six MSF results to the assessment package reduced the SEM to 0.24, which is adequately precise. CONCLUSIONS: Combining data from different WBA assessment instruments achieves acceptable reliability for assessing IMGs, provided that the panel of WBA assessment types are carefully selected and the assessors are calibrated.
Entities:
Keywords:
Clinical competence; Educational measurement; Quality of health care
Authors: Kaushik Baliga; Andrew Coggins; Sandra Warburton; Divya Mathias; Nicole K Yamada; Janene H Fuerch; Louis P Halamek Journal: BMC Med Educ Date: 2022-08-22 Impact factor: 3.263