Amy Franklin1, Elsbeth Kalenderian2, Nutan Hebballi3, Veronique Delattre3, Jini Etoule4, Joel White2, Ram Vaderhobli2, Denice Stewart5, Karla Kent6, Alfa Yansane4, Muhammad Walji3. 1. From the School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, Texas. 2. Department of Preventive and Restorative Dental Sciences, School of Dentistry, University of California San Francisco, San Francisco, California. 3. School of Dentistry, The University of Texas Health Science Center at Houston, Houston, Texas. 4. Oral Health Policy and Epidemiology, School of Dental Medicine, Harvard University, Boston, Massachusetts. 5. Community Dentistry. 6. Integrative Biosciences, School of Dentistry, Oregon Health and Science University, Portland, Oregon.
Abstract
BACKGROUND: To achieve high-quality health care, adverse events (AEs) must be proactively recognized and mitigated. However, there is often ambiguity in applying guidelines and definitions. We describe the iterative calibration process needed to achieve a shared definition of AEs in dentistry. Our alignment process includes both independent and consensus building approaches. OBJECTIVE: We explore the process of defining dental AEs and the steps necessary to achieve alignment across different care providers. METHODS: Teams from 4 dental institutions across the United States iteratively reviewed patient records after identification of charts using an automated trigger tool. Calibration across teams was supported through negotiated definition of AEs and standardization of evidence provided in review. Interrater reliability was assessed using descriptive and κ statistics. RESULTS: After 5 iterative cycles of calibration, the teams (n = 8 raters) identified 118 cases. The average percent agreement for AE determination was 82.2%. Furthermore, the average, pairwise prevalence and bias-adjusted κ (PABAK) was 57.5% (κ = 0.575) for determining AE presence. The average percent agreement for categorization of the AE type was 78.5%, whereas the PABAK was 48.8%. Lastly, the average percent agreement for categorization of AE severity was 82.2% and the corresponding PABAK was 71.7%. CONCLUSIONS: Successful calibration across reviewers is possible after consensus building procedures. Higher levels of agreement were found when categorizing severity (of identified events) rather than the events themselves. Our results demonstrate the need for collaborative procedures as well as training for the identification and severity rating of AEs.
BACKGROUND: To achieve high-quality health care, adverse events (AEs) must be proactively recognized and mitigated. However, there is often ambiguity in applying guidelines and definitions. We describe the iterative calibration process needed to achieve a shared definition of AEs in dentistry. Our alignment process includes both independent and consensus building approaches. OBJECTIVE: We explore the process of defining dental AEs and the steps necessary to achieve alignment across different care providers. METHODS: Teams from 4 dental institutions across the United States iteratively reviewed patient records after identification of charts using an automated trigger tool. Calibration across teams was supported through negotiated definition of AEs and standardization of evidence provided in review. Interrater reliability was assessed using descriptive and κ statistics. RESULTS: After 5 iterative cycles of calibration, the teams (n = 8 raters) identified 118 cases. The average percent agreement for AE determination was 82.2%. Furthermore, the average, pairwise prevalence and bias-adjusted κ (PABAK) was 57.5% (κ = 0.575) for determining AE presence. The average percent agreement for categorization of the AE type was 78.5%, whereas the PABAK was 48.8%. Lastly, the average percent agreement for categorization of AE severity was 82.2% and the corresponding PABAK was 71.7%. CONCLUSIONS: Successful calibration across reviewers is possible after consensus building procedures. Higher levels of agreement were found when categorizing severity (of identified events) rather than the events themselves. Our results demonstrate the need for collaborative procedures as well as training for the identification and severity rating of AEs.
Authors: Elsbeth Kalenderian; Rachel L Ramoni; Joel M White; Meta E Schoonheim-Klein; Paul C Stark; Nicole S Kimmes; Gregory G Zeller; George P Willis; Muhammad F Walji Journal: J Dent Educ Date: 2011-01 Impact factor: 2.264
Authors: A R Localio; S L Weaver; J R Landis; A G Lawthers; T A Brenhan; L Hebert; T J Sharp Journal: Ann Intern Med Date: 1996-09-15 Impact factor: 25.391
Authors: A K Jha; G J Kuperman; J M Teich; L Leape; B Shea; E Rittenberg; E Burdick; D L Seger; M Vander Vliet; D W Bates Journal: J Am Med Inform Assoc Date: 1998 May-Jun Impact factor: 4.497
Authors: T A Brennan; L L Leape; N M Laird; L Hebert; A R Localio; A G Lawthers; J P Newhouse; P C Weiler; H H Hiatt Journal: N Engl J Med Date: 1991-02-07 Impact factor: 91.245
Authors: P Daniel Patterson; Judith R Lave; Matthew D Weaver; Francis X Guyette; Robert M Arnold; Christian Martin-Gill; Jon C Rittenberger; David Krackhardt; Vincent N Mosesso; Ronald N Roth; Richard J Wadas; Donald M Yealy Journal: Prehosp Emerg Care Date: 2014-05-30 Impact factor: 3.077