Ryan Edwards1, Noura Alsufyani2, Giseon Heo3, Carlos Flores-Mir4. 1. Private practice, Red Deer, Alberta, Canada. 2. Assistant professor, Division of Oral Medicine, Pathology and Radiology, School of Dentistry, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, Alberta, Canada. 3. Associate professor, Division of Orthodontics, School of Dentistry, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, Alberta, Canada. 4. Professor and head, Division of Orthodontics, School of Dentistry, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, Alberta, Canada. Electronic address: carlosflores@ualberta.ca.
Abstract
INTRODUCTION: In this study, we aimed to assess interrater and intrarater agreement among orthodontic clinicians in their assessments of reported incidental findings in regard to both the need for additional follow-up and the impact on future orthodontic treatment in large-field maxillofacial cone-beam computed tomography (CBCT) imaging. METHODS: The study sample consisted of 18 nonrandomly selected large-field maxillofacial CBCT volumes containing a reported total of 88 radiographic findings. All scans were associated with formal radiologic reports. However, the suggestions of further follow-up were removed from the radiologic reports so as to not bias the 3 evaluating orthodontists in their subsequent decision making. The evaluators had on average 7.6 years of CBCT usage and self-interpretation experience. Reliability was determined by quantifying the level of agreement between the evaluators' assessments for both research questions for all 88 findings using a binary response (yes/no) as the outcome measure. The Cohen kappa statistic was calculated to quantify intrarater and interrater agreement globally for both statements. RESULTS: Although interrater agreement was considerable, potential decisions with clinical impact were not consistent. This needs to be considered when interpreting maxillofacial incidental findings. Evaluators demonstrated higher levels of agreement for dentoalveolar findings compared with all other extragnathic regions when assessing clinical significance. CONCLUSIONS: Among the evaluators who were considered experienced in CBCT, "fair-to-good" interrater agreement and "excellent" intrarater agreement were demonstrated in terms of the need for further follow-up and their potential impact on future orthodontic treatment.
INTRODUCTION: In this study, we aimed to assess interrater and intrarater agreement among orthodontic clinicians in their assessments of reported incidental findings in regard to both the need for additional follow-up and the impact on future orthodontic treatment in large-field maxillofacial cone-beam computed tomography (CBCT) imaging. METHODS: The study sample consisted of 18 nonrandomly selected large-field maxillofacial CBCT volumes containing a reported total of 88 radiographic findings. All scans were associated with formal radiologic reports. However, the suggestions of further follow-up were removed from the radiologic reports so as to not bias the 3 evaluating orthodontists in their subsequent decision making. The evaluators had on average 7.6 years of CBCT usage and self-interpretation experience. Reliability was determined by quantifying the level of agreement between the evaluators' assessments for both research questions for all 88 findings using a binary response (yes/no) as the outcome measure. The Cohen kappa statistic was calculated to quantify intrarater and interrater agreement globally for both statements. RESULTS: Although interrater agreement was considerable, potential decisions with clinical impact were not consistent. This needs to be considered when interpreting maxillofacial incidental findings. Evaluators demonstrated higher levels of agreement for dentoalveolar findings compared with all other extragnathic regions when assessing clinical significance. CONCLUSIONS: Among the evaluators who were considered experienced in CBCT, "fair-to-good" interrater agreement and "excellent" intrarater agreement were demonstrated in terms of the need for further follow-up and their potential impact on future orthodontic treatment.