S-L Oh1, L Liberman1, O Mishler1. 1. Department of Advanced Oral Sciences and Therapeutics, School of Dentistry, University of Maryland, Baltimore, MD, USA.
Abstract
PURPOSE: The purpose of this study was to investigate the effect of faculty calibration and students' self-assessments on students' performances in a periodontal practical examination. METHODS: Before a new instructional rubric was implemented in the second-year periodontics course, faculty calibration was conducted with a pilot group of 32 third-year dental students. Afterwards, the new rubric was implemented in the second-year periodontics course. Second-year students used the rubric for their self-assessments before the practical examination. An intraclass correlation coefficient was used to test the reliability of the faculty members. A paired t test was used to compare scores between self-assessments of the pilot group (third-year students) and faculty evaluation. A two-way analysis of variance was performed to compare scores between self-assessments of second-year students and faculty evaluations. Chi-square tests were used to compare overall failure rates amongst four different classes. RESULTS: The reliability amongst the faculty members was strong (the ICC = 0.75 at the first and 0.97 at the second calibration). The mean self-assessment score from the pilot group was significantly higher than the faculty evaluation. However, the mean self-assessment score of second-year students was significantly lower than the faculty evaluation. The class that practiced self-assessments with the validated instructional rubric exhibited the lowest overall failure rate compared to three past classes. CONCLUSIONS: Using an instructional rubric and conducting faculty calibration improved the process of the periodontal practical examination. Improving the examination process and practicing self-assessments with feedback from faculty may have a positive impact on students' performances in the examination.
PURPOSE: The purpose of this study was to investigate the effect of faculty calibration and students' self-assessments on students' performances in a periodontal practical examination. METHODS: Before a new instructional rubric was implemented in the second-year periodontics course, faculty calibration was conducted with a pilot group of 32 third-year dental students. Afterwards, the new rubric was implemented in the second-year periodontics course. Second-year students used the rubric for their self-assessments before the practical examination. An intraclass correlation coefficient was used to test the reliability of the faculty members. A paired t test was used to compare scores between self-assessments of the pilot group (third-year students) and faculty evaluation. A two-way analysis of variance was performed to compare scores between self-assessments of second-year students and faculty evaluations. Chi-square tests were used to compare overall failure rates amongst four different classes. RESULTS: The reliability amongst the faculty members was strong (the ICC = 0.75 at the first and 0.97 at the second calibration). The mean self-assessment score from the pilot group was significantly higher than the faculty evaluation. However, the mean self-assessment score of second-year students was significantly lower than the faculty evaluation. The class that practiced self-assessments with the validated instructional rubric exhibited the lowest overall failure rate compared to three past classes. CONCLUSIONS: Using an instructional rubric and conducting faculty calibration improved the process of the periodontal practical examination. Improving the examination process and practicing self-assessments with feedback from faculty may have a positive impact on students' performances in the examination.