Juliane E Kämmer1,2, Wolf E Hautz3, Maren März4. 1. Inistitute of Health and Nursing Science, Charité - Universitätsmedizin Berlin, Berlin, Germany. 2. Centre for Adaptive Rationality, Max Planck Institute for Human Development, Berlin, Germany. 3. Department of Emergency Medicine, Inselspital University Hospital, University of Bern, Bern, Switzerland. 4. AG Progress Test Medicine, Charité - Universitätsmedizin Berlin, Berlin, Germany.
Abstract
CONTEXT: Accurate self-assessment of one's performace on a moment-by-moment basis (ie, accurate self-monitoring) is vital for the self-regulation of practising physicians and indeed for the effective regulation of self-directed learning during medical education. However, little is currently known about the functioning of self-monitoring and its co-development with medical knowledge across medical education. This study is the first to simultaneously investigate a number of relevant aspects and measures that have so far been studied separately: different measures of self-monitoring for a broad area of medical knowledge across 10 different performance levels. METHODS: This study assessed the self-monitoring accuracy of medical students (n = 3145) across 10 semesters. Data collected during the administration of the formative Berlin Progress Test Medicine (PTM) were analysed. The PTM comprises 200 multiple-choice questions covering all major medical disciplines and organ systems. A self-report indicator (ie, confidence) and two behavioural indicators of self-monitoring accuracy (ie, response time and the likelihood of changing an initial answer to a correct rather than an incorrect item) were examined for their development over semesters. RESULTS: Analyses of more than 390 000 observations (of approximately 250 students per semester) showed that confidence was higher for correctly than for incorrectly answered items and that 86% of items answered with high confidence were indeed correct. Response time and the likelihood of the initial answer being changed were higher when the initial answer was incorrect than when it was correct. Contrary to expectations, no differences in self-monitoring accuracy were observed across semesters. CONCLUSIONS: Convergent evidence from different measures of self-monitoring suggests that medical students self-monitor their knowledge on a question-by-question basis well, although not perfectly, and to the same degree as has been found in studies outside medicine. Despite large differences in performance, no variations in self-monitoring across semesters (with the exception of the first semester) were observed.
CONTEXT: Accurate self-assessment of one's performace on a moment-by-moment basis (ie, accurate self-monitoring) is vital for the self-regulation of practising physicians and indeed for the effective regulation of self-directed learning during medical education. However, little is currently known about the functioning of self-monitoring and its co-development with medical knowledge across medical education. This study is the first to simultaneously investigate a number of relevant aspects and measures that have so far been studied separately: different measures of self-monitoring for a broad area of medical knowledge across 10 different performance levels. METHODS: This study assessed the self-monitoring accuracy of medical students (n = 3145) across 10 semesters. Data collected during the administration of the formative Berlin Progress Test Medicine (PTM) were analysed. The PTM comprises 200 multiple-choice questions covering all major medical disciplines and organ systems. A self-report indicator (ie, confidence) and two behavioural indicators of self-monitoring accuracy (ie, response time and the likelihood of changing an initial answer to a correct rather than an incorrect item) were examined for their development over semesters. RESULTS: Analyses of more than 390 000 observations (of approximately 250 students per semester) showed that confidence was higher for correctly than for incorrectly answered items and that 86% of items answered with high confidence were indeed correct. Response time and the likelihood of the initial answer being changed were higher when the initial answer was incorrect than when it was correct. Contrary to expectations, no differences in self-monitoring accuracy were observed across semesters. CONCLUSIONS: Convergent evidence from different measures of self-monitoring suggests that medical students self-monitor their knowledge on a question-by-question basis well, although not perfectly, and to the same degree as has been found in studies outside medicine. Despite large differences in performance, no variations in self-monitoring across semesters (with the exception of the first semester) were observed.
Authors: Martin Müller; Wolf E Hautz; Tanja Birrenbach; Michele Hoffmann; Stefanie C Hautz; Juliane E Kämmer; Aristomenis K Exadaktylos; Thomas C Sauter Journal: BMC Emerg Med Date: 2022-06-15
Authors: Stefanie C Hautz; Daniel L Oberholzer; Julia Freytag; Aristomenis Exadaktylos; Juliane E Kämmer; Thomas C Sauter; Wolf E Hautz Journal: BMC Med Educ Date: 2020-06-23 Impact factor: 2.463
Authors: Wolf E Hautz; Moritz M Kündig; Roger Tschanz; Tanja Birrenbach; Alexander Schuster; Thomas Bürkle; Stefanie C Hautz; Thomas C Sauter; Gert Krummrey Journal: Diagnosis (Berl) Date: 2021-10-21
Authors: Stefan K Schauber; Stefanie C Hautz; Juliane E Kämmer; Fabian Stroben; Wolf E Hautz Journal: Adv Health Sci Educ Theory Pract Date: 2021-05-11 Impact factor: 3.853