Yen-Yuan Chen1, Yu-Chun Chiu2, Tzong-Shinn Chu3, Hong-Yuan Hsu4, Huey-Ling Chen4, Chau-Chung Wu3, Tien-Shang Huang5. 1. Department of Medical Education, National Taiwan University Hospital, #7, Rd. Chong-Shan S., Taipei, 10002, Taiwan; Graduate Institute of Medical Education & Bioethics, National Taiwan University College of Medicine, #1, Rd. Ren-Ai Sec. 1, Taipei, 10051, Taiwan. 2. Department of Medical Education, National Taiwan University Hospital, #7, Rd. Chong-Shan S., Taipei, 10002, Taiwan; Department of Pediatrics, National Taiwan University Hospital, #7, Rd. Chong-Shan S., Taipei, 10002, Taiwan. 3. Graduate Institute of Medical Education & Bioethics, National Taiwan University College of Medicine, #1, Rd. Ren-Ai Sec. 1, Taipei, 10051, Taiwan; Department of Internal Medicine, National Taiwan University Hospital, #7, Rd. Chong-Shan S., Taipei, 10002, Taiwan. 4. Graduate Institute of Medical Education & Bioethics, National Taiwan University College of Medicine, #1, Rd. Ren-Ai Sec. 1, Taipei, 10051, Taiwan; Department of Pediatrics, National Taiwan University Hospital, #7, Rd. Chong-Shan S., Taipei, 10002, Taiwan. 5. Department of Medical Education, Cathay Hospital, #280, Rd. Ren-Ai Sec. 4, Taipei, 10630, Taiwan. Electronic address: huangts@ntu.edu.tw.
Abstract
PURPOSE: Whether the rating result of mini-clinical evaluation exercise (Mini-CEX) for rating clinical skills is reliable is of a medical trainee's great concerns. The objectives of this study were to analyze the test-retest reliability, interrater reliability and internal consistency reliability of Mini-CEX. METHODS: Three clinical scenarios, each played by a standardized patient and resident, were developed and videotaped. A group of assessors were recruited to rate the resident's clinical skills using Mini-CEX with a nine-point grading scale in each videotaped clinical scenario. Each assessor was required: (1) to watch the videotaped clinical scenarios a sequential order; (2) to rate each medical trainee's clinical skills in each clinical scenario for two rating sessions, and there must be a minimum three-week interval between the first and the second Mini-CEX rating session. RESULTS: A total of 38 assessors participated in this study. This study showed that: (1) an assessor carried out similar rating reuslts under the same clinical performance based on an acceptable test-retest reliability (Pearson's correlation coefficients = 0.24-0.76, P value=<0.01-0.14); (2) assessors gave similar rating results to a medical trainee's clinical performance based on a good interrater reliability (intra-class correlation coefficient = 0.57-0.83, P value=<0.01-0.03); and (3) the items reflected unidimensionally a construct-a medical trainee's clinical skills based on an excellent internal consistency reliability (Cronbach's alpha = 0.92-0.97). CONCLUSION: This study convincingly showed that Mini-CEX is a reliable assessment tool for rating clinical skills, and can be widely used to assess medical trainees' clinical skills.
PURPOSE: Whether the rating result of mini-clinical evaluation exercise (Mini-CEX) for rating clinical skills is reliable is of a medical trainee's great concerns. The objectives of this study were to analyze the test-retest reliability, interrater reliability and internal consistency reliability of Mini-CEX. METHODS: Three clinical scenarios, each played by a standardized patient and resident, were developed and videotaped. A group of assessors were recruited to rate the resident's clinical skills using Mini-CEX with a nine-point grading scale in each videotaped clinical scenario. Each assessor was required: (1) to watch the videotaped clinical scenarios a sequential order; (2) to rate each medical trainee's clinical skills in each clinical scenario for two rating sessions, and there must be a minimum three-week interval between the first and the second Mini-CEX rating session. RESULTS: A total of 38 assessors participated in this study. This study showed that: (1) an assessor carried out similar rating reuslts under the same clinical performance based on an acceptable test-retest reliability (Pearson's correlation coefficients = 0.24-0.76, P value=<0.01-0.14); (2) assessors gave similar rating results to a medical trainee's clinical performance based on a good interrater reliability (intra-class correlation coefficient = 0.57-0.83, P value=<0.01-0.03); and (3) the items reflected unidimensionally a construct-a medical trainee's clinical skills based on an excellent internal consistency reliability (Cronbach's alpha = 0.92-0.97). CONCLUSION: This study convincingly showed that Mini-CEX is a reliable assessment tool for rating clinical skills, and can be widely used to assess medical trainees' clinical skills.