Victor Lee1, Keira Brain, Jenepher Martin. 1. V. Lee is codirector of emergency medicine training, Department of Emergency Medicine, Austin Health, Heidelberg, Victoria, Australia.K. Brain is doctor, South West Healthcare, Warrnambool, Victoria, Australia.J. Martin is associate professor and director, Medical Student Programs, Monash University and Deakin University, Eastern Health Clinical School, Box Hill, Victoria, Australia.
Abstract
PURPOSE: At present, little is known about how mini-clinical evaluation exercise (mini-CEX) raters translate their observations into judgments and ratings. The authors of this systematic literature review aim both to identify the factors influencing mini-CEX rater judgments in the medical education setting and to translate these findings into practical implications for clinician assessors. METHOD: The authors searched for internal and external factors influencing mini-CEX rater judgments in the medical education setting from 1980 to 2015 using the Ovid MEDLINE, PsycINFO, ERIC, PubMed, and Scopus databases. They extracted the following information from each study: country of origin, educational level, study design and setting, type of observation, occurrence of rater training, provision of feedback to the trainee, research question, and identified factors influencing rater judgments. The authors also conducted a quality assessment for each study. RESULTS: Seventeen articles met the inclusion criteria. The authors identified both internal and external factors that influence mini-CEX rater judgments. They subcategorized the internal factors into intrinsic rater factors, judgment-making factors (conceptualization, interpretation, attention, and impressions), and scoring factors (scoring integration and domain differentiation). CONCLUSIONS: The current theories of rater-based judgment have not helped clinicians resolve the issues of rater idiosyncrasy, bias, gestalt, and conflicting contextual factors; therefore, the authors believe the most important solution is to increase the justification of rater judgments through the use of specific narrative and contextual comments, which are more informative for trainees. Finally, more real-world research is required to bridge the gap between the theory and practice of rater cognition.
PURPOSE: At present, little is known about how mini-clinical evaluation exercise (mini-CEX) raters translate their observations into judgments and ratings. The authors of this systematic literature review aim both to identify the factors influencing mini-CEX rater judgments in the medical education setting and to translate these findings into practical implications for clinician assessors. METHOD: The authors searched for internal and external factors influencing mini-CEX rater judgments in the medical education setting from 1980 to 2015 using the Ovid MEDLINE, PsycINFO, ERIC, PubMed, and Scopus databases. They extracted the following information from each study: country of origin, educational level, study design and setting, type of observation, occurrence of rater training, provision of feedback to the trainee, research question, and identified factors influencing rater judgments. The authors also conducted a quality assessment for each study. RESULTS: Seventeen articles met the inclusion criteria. The authors identified both internal and external factors that influence mini-CEX rater judgments. They subcategorized the internal factors into intrinsic rater factors, judgment-making factors (conceptualization, interpretation, attention, and impressions), and scoring factors (scoring integration and domain differentiation). CONCLUSIONS: The current theories of rater-based judgment have not helped clinicians resolve the issues of rater idiosyncrasy, bias, gestalt, and conflicting contextual factors; therefore, the authors believe the most important solution is to increase the justification of rater judgments through the use of specific narrative and contextual comments, which are more informative for trainees. Finally, more real-world research is required to bridge the gap between the theory and practice of rater cognition.
Authors: Susan Humphrey-Murto; Timothy J Wood; Shelly Ross; Walter Tavares; Brent Kvern; Ravi Sidhu; Joan Sargeant; Claire Touchie Journal: J Grad Med Educ Date: 2017-12
Authors: Annabel K Frank; Patricia O'Sullivan; Lynnea M Mills; Virginie Muller-Juge; Karen E Hauer Journal: J Gen Intern Med Date: 2019-05 Impact factor: 5.128
Authors: Catherine Hyde; Sarah Yardley; Janet Lefroy; Simon Gay; Robert K McKinley Journal: Adv Health Sci Educ Theory Pract Date: 2020-01-29 Impact factor: 3.853
Authors: Ana G Palis; Jesús Barrio-Barrio; Eduardo P Mayorga; Ilhem Mili-Boussen; Christelle D Noche; Meenakshi Swaminathan; Karl C Golnik Journal: Indian J Ophthalmol Date: 2021-01 Impact factor: 1.848