Chantal B van Houten1, Christiana A Naaktgeboren2, Liat Ashkenazi-Hoffnung3, Shai Ashkenazi4, Wim Avis5, Irena Chistyakov6, Teresa Corigliano7, Annick Galetto7, Iker Gangoiti8, Alain Gervaix7, Daniel Glikman9, Inga Ivaskeviciene10, Amir A Kuperman11, Laurence Lacroix7, Yvette Loeffen1, Fanny Luterbacher7, Clemens B Meijssen12, Santiago Mintegi8, Basheer Nasrallah6, Cihan Papan13, Annemarie M C van Rossum14, Henriette Rudolph12, Michal Stein15, Roie Tal16, Tobias Tenenbaum12, Vytautas Usonis10, Wouter de Waal17, Stefan Weichert12, Joanne G Wildenbeest1, Karin M de Winter-de Groot18, Tom F W Wolfs1, Niv Mastboim19, Tanya M Gottlieb19, Asi Cohen19, Kfir Oved19, Eran Eden19, Paul D Feigin20, Liran Shani19, Louis J Bont21. 1. Division of Pediatric Immunology and Infectious Diseases, University Medical Centre Utrecht, Utrecht University, Utrecht, The Netherlands. 2. Division Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, The Netherlands. 3. Schneider Children's Medical Center, Petach Tikva, Sackler Faculty of Medicine, Tel Aviv, Israel. 4. Adelson School of Medicine, Ariel University, Schneider Children's Medical Center, Petach Tikva, Israel. 5. Department of Pediatrics, Gelderse Vallei Hospital, Ede, The Netherlands. 6. Department of Pediatrics, Bnai Zion Medical Centre, Haifa, Israel. 7. Department of Pediatrics, Geneva University Hospitals, Geneva, Switzerland. 8. Department of Pediatric Emergency Medicine, Cruces University Hospital, Bilbao, Spain. 9. Infectious Diseases Unit, Padeh Poria Medical Center and the Azrieli faculty of Medicine in the Galilee, Bar-Ilan University, Safed, Israel. 10. Clinic of Children Diseases, Institute of Clinical medicine, Faculty of Medicine, Vilnius University Vilnius, Lithuania. 11. Blood Coagulation Service and Pediatric Hematology Clinic, Galilee Medical Centre, Nahariya, and Azrieli Faculty of Medicine, Bar-Ilan University, Safed, Israel. 12. Department of Pediatrics, Meander Medical Centre, Amersfoort, The Netherlands. 13. Pediatric Infectious Diseases, University Children's Hospital Mannheim, Medical Faculty Mannheim, Heidelberg University, Mannheim, Germany. 14. Department of Pediatrics, Erasmus MC University, Medical Centre Rotterdam, The Netherlands. 15. Department of Pediatrics, Hillel Yaffe Medical Centre, Hadera, Israel. 16. Department of Pediatrics, Galilee Medical Centre, Nahariya and Azrieli Faculty of Medicine, Bar-Ilan University, Safed, Israel. 17. Department of Pediatrics, Diakonessenhuis, Utrecht, The Netherlands. 18. Department of Pediatric Respiratory Medicine, University Medical Centre Utrecht, Utrecht University, Utrecht, The Netherlands. 19. MeMed, Tirat Carmel, Israel. 20. Faculty of Industrial Engineering and Management, Technion-Israel Institute of Technology, Haifa, Israel. 21. Division of Pediatric Immunology and Infectious Diseases, University Medical Centre Utrecht, Utrecht University, Utrecht, The Netherlands. Electronic address: l.bont@umcutrecht.nl.
Abstract
OBJECTIVE: If a gold standard is lacking in a diagnostic test accuracy study, expert diagnosis is frequently used as reference standard. However, interobserver and intraobserver agreements are imperfect. The aim of this study was to quantify the reproducibility of a panel diagnosis for pediatric infectious diseases. STUDY DESIGN AND SETTING: Pediatricians from six countries adjudicated a diagnosis (i.e., bacterial infection, viral infection, or indeterminate) for febrile children. Diagnosis was reached when the majority of panel members came to the same diagnosis, leaving others inconclusive. We evaluated intraobserver and intrapanel agreement with 6 weeks and 3 years' time intervals. We calculated the proportion of inconclusive diagnosis for a three-, five-, and seven-expert panel. RESULTS: For both time intervals (i.e., 6 weeks and 3 years), intrapanel agreement was higher (kappa 0.88, 95%CI: 0.81-0.94 and 0.80, 95%CI: NA) compared to intraobserver agreement (kappa 0.77, 95%CI: 0.71-0.83 and 0.65, 95%CI: 0.52-0.78). After expanding the three-expert panel to five or seven experts, the proportion of inconclusive diagnoses (11%) remained the same. CONCLUSION: A panel consisting of three experts provides more reproducible diagnoses than an individual expert in children with lower respiratory tract infection or fever without source. Increasing the size of a panel beyond three experts has no major advantage for diagnosis reproducibility.
OBJECTIVE: If a gold standard is lacking in a diagnostic test accuracy study, expert diagnosis is frequently used as reference standard. However, interobserver and intraobserver agreements are imperfect. The aim of this study was to quantify the reproducibility of a panel diagnosis for pediatric infectious diseases. STUDY DESIGN AND SETTING: Pediatricians from six countries adjudicated a diagnosis (i.e., bacterial infection, viral infection, or indeterminate) for febrile children. Diagnosis was reached when the majority of panel members came to the same diagnosis, leaving others inconclusive. We evaluated intraobserver and intrapanel agreement with 6 weeks and 3 years' time intervals. We calculated the proportion of inconclusive diagnosis for a three-, five-, and seven-expert panel. RESULTS: For both time intervals (i.e., 6 weeks and 3 years), intrapanel agreement was higher (kappa 0.88, 95%CI: 0.81-0.94 and 0.80, 95%CI: NA) compared to intraobserver agreement (kappa 0.77, 95%CI: 0.71-0.83 and 0.65, 95%CI: 0.52-0.78). After expanding the three-expert panel to five or seven experts, the proportion of inconclusive diagnoses (11%) remained the same. CONCLUSION: A panel consisting of three experts provides more reproducible diagnoses than an individual expert in children with lower respiratory tract infection or fever without source. Increasing the size of a panel beyond three experts has no major advantage for diagnosis reproducibility.
Authors: Cihan Papan; Katharina Reifenrath; Katharina Last; Andishe Attarbaschi; Norbert Graf; Andreas H Groll; Johannes Huebner; Hans-Jürgen Laws; Thomas Lehrnbecher; Johannes Liese; Luise Martin; Tobias Tenenbaum; Stefan Weichert; Simon Vieth; Ulrich von Both; Markus Hufnagel; Arne Simon Journal: JMIR Res Protoc Date: 2022-06-20
Authors: Adam Ryason; Emil R Petrusa; Uwe Kruger; Zhaohui Xia; Vanessa T Wong; Daniel B Jones; Suvranu De; Stephanie B Jones Journal: J Educ Perioper Med Date: 2020-01-01
Authors: Emily R Ko; Ricardo Henao; Katherine Frankey; Elizabeth A Petzold; Pamela D Isner; Anja K Jaehne; Nakia Allen; Jayna Gardner-Gray; Gina Hurst; Jacqueline Pflaum-Carlson; Namita Jayaprakash; Emanuel P Rivers; Henry Wang; Irma Ugalde; Siraj Amanullah; Laura Mercurio; Thomas H Chun; Larissa May; Robert W Hickey; Jacob E Lazarus; Shauna H Gunaratne; Daniel J Pallin; Guruprasad Jambaulikar; David S Huckins; Krow Ampofo; Ravi Jhaveri; Yunyun Jiang; Lauren Komarow; Scott R Evans; Geoffrey S Ginsburg; L Gayani Tillekeratne; Micah T McClain; Thomas W Burke; Christopher W Woods; Ephraim L Tsalik Journal: JAMA Netw Open Date: 2022-04-01