OBJECTIVE: To evaluate the inter-rater agreement of the record review process of the Dutch Adverse Event study, which we aimed to improve by the involvement of two independent physician reviewers per record instead of one including a consensus procedure in case of disagreement. METHODS: The inter-rater agreement within pairs of physicians (independent review between physician A+B) and between pairs of physicians (independent review between physician A+B and C+D) was measured to evaluate the record review process with two physicians including a consensus procedure, with 4,272 and 119 records, respectively. RESULTS: The inter-rater agreement within pairs of physicians was substantial for the determination of adverse events (AEs) with a kappa of 0.64 (95% confidence interval [CI]: 0.61, 0.68). The inter-rater agreement between pairs of physicians was fair for the determination of AEs with a kappa of 0.25 (95% CI: 0.05, 0.45). CONCLUSION: A record review process with two physicians per record including a consensus procedure to assess AEs is not more reliable than a record review process with one physician. Retrospective estimates of incidence of AEs from record review studies should be interpreted with caution. Improvement of the method is necessary for monitoring incidence of AEs over time at a national level.
OBJECTIVE: To evaluate the inter-rater agreement of the record review process of the Dutch Adverse Event study, which we aimed to improve by the involvement of two independent physician reviewers per record instead of one including a consensus procedure in case of disagreement. METHODS: The inter-rater agreement within pairs of physicians (independent review between physician A+B) and between pairs of physicians (independent review between physician A+B and C+D) was measured to evaluate the record review process with two physicians including a consensus procedure, with 4,272 and 119 records, respectively. RESULTS: The inter-rater agreement within pairs of physicians was substantial for the determination of adverse events (AEs) with a kappa of 0.64 (95% confidence interval [CI]: 0.61, 0.68). The inter-rater agreement between pairs of physicians was fair for the determination of AEs with a kappa of 0.25 (95% CI: 0.05, 0.45). CONCLUSION: A record review process with two physicians per record including a consensus procedure to assess AEs is not more reliable than a record review process with one physician. Retrospective estimates of incidence of AEs from record review studies should be interpreted with caution. Improvement of the method is necessary for monitoring incidence of AEs over time at a national level.
Authors: Thomas M Atkinson; Yuelin Li; Charles W Coffey; Laura Sit; Mary Shaw; Dawn Lavene; Antonia V Bennett; Mike Fruscione; Lauren Rogak; Jennifer Hay; Mithat Gönen; Deborah Schrag; Ethan Basch Journal: Qual Life Res Date: 2011-10-08 Impact factor: 4.147
Authors: Lilian H F Hoonhout; Martine C de Bruijne; Cordula Wagner; Henk Asscheman; Gerrit van der Wal; Maurits W van Tulder Journal: Drug Saf Date: 2010-10-01 Impact factor: 5.606
Authors: P Daniel Patterson; Judith R Lave; Matthew D Weaver; Francis X Guyette; Robert M Arnold; Christian Martin-Gill; Jon C Rittenberger; David Krackhardt; Vincent N Mosesso; Ronald N Roth; Richard J Wadas; Donald M Yealy Journal: Prehosp Emerg Care Date: 2014-05-30 Impact factor: 3.077
Authors: Benjamin A Rodwin; Victor P Bilan; Naseema B Merchant; Catherine G Steffens; Alyssa A Grimshaw; Lori A Bastian; Craig G Gunderson Journal: J Gen Intern Med Date: 2020-01-21 Impact factor: 5.128
Authors: Helen Hogan; Frances Healey; Graham Neale; Richard Thomson; Charles Vincent; Nick Black Journal: BMJ Qual Saf Date: 2012-09 Impact factor: 7.035