H A Emam1, C A Jatana1, S Wade2, D Hamamoto3. 1. Division of Oral and Maxillofacial Surgery and Dental Anesthesiology, College of Dentistry, The Ohio State University, Columbus, OH, USA. 2. College of Dentistry, The Ohio State University, Columbus, OH, USA. 3. Division of Oral Pathology and Radiology, Academic Affairs, College of Dentistry, The Ohio State University, Columbus, OH, USA.
Abstract
PURPOSE: To develop a standardized competency exam to evaluate dental student knowledge of patients with complex medical histories. Analysis was performed to determine if there is a difference between dental student self-assessment of the exam compared to oral surgery faculty. The overall goal is to enhance student comprehension of advanced medical patients in the pre-doctoral oral and maxillofacial surgery clinic and make changes based on student responses. METHODS: The exam took place in a simulation laboratory containing two-way mirrors where the student could not see the evaluator. Three standardized patients (trained actors) were given different medical history scenarios to learn prior to the exam. Students were randomly assigned to interview one patient. The graders consisted of senior and junior level faculty. In real time (live) randomized manner, faculty assessed each student using a standardized rubric to assign a score of 1 to 4. Immediately afterwards, students assessed their own performance using the same scale. RESULTS: Students gave themselves significantly higher grades (P < 0.001) on the exam (10.7 ± 0.12) compared to the faculty (9.2 ± 0.17). There was no significant difference in student (P = 0.16) or faculty assessment (P = 0.29) between the three different medical scenarios. There was no significant difference between faculty in assessment of student performances (P = 0.16). CONCLUSION: These results suggested that students overestimated their performance compared to the faculty assessment. This could be due to that students assessed their performance based on memory and self-confidence; that exceeds their expectations. Internal calibration of student performances and faculty grading using standardized criteria may produce more accurate and reliable outcomes in student scoring.
RCT Entities:
PURPOSE: To develop a standardized competency exam to evaluate dental student knowledge of patients with complex medical histories. Analysis was performed to determine if there is a difference between dental student self-assessment of the exam compared to oral surgery faculty. The overall goal is to enhance student comprehension of advanced medical patients in the pre-doctoral oral and maxillofacial surgery clinic and make changes based on student responses. METHODS: The exam took place in a simulation laboratory containing two-way mirrors where the student could not see the evaluator. Three standardized patients (trained actors) were given different medical history scenarios to learn prior to the exam. Students were randomly assigned to interview one patient. The graders consisted of senior and junior level faculty. In real time (live) randomized manner, faculty assessed each student using a standardized rubric to assign a score of 1 to 4. Immediately afterwards, students assessed their own performance using the same scale. RESULTS: Students gave themselves significantly higher grades (P < 0.001) on the exam (10.7 ± 0.12) compared to the faculty (9.2 ± 0.17). There was no significant difference in student (P = 0.16) or faculty assessment (P = 0.29) between the three different medical scenarios. There was no significant difference between faculty in assessment of student performances (P = 0.16). CONCLUSION: These results suggested that students overestimated their performance compared to the faculty assessment. This could be due to that students assessed their performance based on memory and self-confidence; that exceeds their expectations. Internal calibration of student performances and faculty grading using standardized criteria may produce more accurate and reliable outcomes in student scoring.