Trevor A McGrath1, Matthew D F McInnes2, Felipe W Langer3, Jiho Hong4, Daniël A Korevaar5, Patrick M M Bossuyt6. 1. Faculty of Medicine, University of Ottawa, Ottawa, ON, Canada. Electronic address: tmcgr043@uottawa.ca. 2. University of Ottawa, Department of Radiology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Room c159 Ottawa Hospital Civic Campus, 1053 Carling Ave., Ottawa, ON, K1Y 4E9, Canada. Electronic address: mmcinnes@toh.on.ca. 3. Faculty of Medicine, Federal University of Santa Maria, Rio Grande do Sul, Brazil. Electronic address: felipewlanger@gmail.com. 4. Faculty of Medicine, University of Ottawa, Ottawa, ON, Canada. Electronic address: jhong030@uottawa.ca. 5. Department of Clinical Epidemiology, Biostatistics and Bioinformatics, Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands. Electronic address: d.a.korevaar@amc.uva.nl. 6. Department of Clinical Epidemiology, Biostatistics and Bioinformatics, Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands. Electronic address: p.m.bossuyt@amc.uva.nl.
Abstract
OBJECTIVE: To evaluate the handling of multiple readers in imaging diagnostic accuracy systematic reviews-meta-analyses. METHODS: Search was performed for imaging diagnostic accuracy systematic reviews that performed meta-analysis from 2005-2015. Handling of multiple readers was classified as: 1) averaged; 2) 'best' reader; 3) 'most experienced' reader; 4) each reader counted individually; 5) random; 6) other; 7) not specified. Incidence and reporting of multiple reader data was assessed in primary diagnostic accuracy studies that were included in a random sample of reviews. RESULTS: Only 28/296 (9.5%) meta-analyses specified how multiple readers were handled: 7/28 averaged results, 2/28 included the best reader, 14/28 treated each reader as a separate data set, 1/28 randomly selected a reader, 4/28 used other methods. Sample of 27/268 'not specified' reviews generated 442 primary studies. 270/442 (61%) primary studies had multiple readers: 164/442 (37%) reported consensus reading, 87/442 (20%) reported inter-observer variability, 9/442 (2%) reported independent datasets for each reader. 26/27 (96%) meta-analyses contained at least one primary study with multiple readers. CONCLUSIONS: Reporting how multiple readers were treated in imaging systematic reviews-meta-analyses is uncommon and method used varied widely. This may result from a lack of guidance, unavailability of appropriate statistical methods for handling multiple readers in meta-analysis, and sub-optimal primary study reporting.
OBJECTIVE: To evaluate the handling of multiple readers in imaging diagnostic accuracy systematic reviews-meta-analyses. METHODS: Search was performed for imaging diagnostic accuracy systematic reviews that performed meta-analysis from 2005-2015. Handling of multiple readers was classified as: 1) averaged; 2) 'best' reader; 3) 'most experienced' reader; 4) each reader counted individually; 5) random; 6) other; 7) not specified. Incidence and reporting of multiple reader data was assessed in primary diagnostic accuracy studies that were included in a random sample of reviews. RESULTS: Only 28/296 (9.5%) meta-analyses specified how multiple readers were handled: 7/28 averaged results, 2/28 included the best reader, 14/28 treated each reader as a separate data set, 1/28 randomly selected a reader, 4/28 used other methods. Sample of 27/268 'not specified' reviews generated 442 primary studies. 270/442 (61%) primary studies had multiple readers: 164/442 (37%) reported consensus reading, 87/442 (20%) reported inter-observer variability, 9/442 (2%) reported independent datasets for each reader. 26/27 (96%) meta-analyses contained at least one primary study with multiple readers. CONCLUSIONS: Reporting how multiple readers were treated in imaging systematic reviews-meta-analyses is uncommon and method used varied widely. This may result from a lack of guidance, unavailability of appropriate statistical methods for handling multiple readers in meta-analysis, and sub-optimal primary study reporting.
Authors: Trevor A McGrath; Robert A Frank; Nicola Schieda; Brian Blew; Jean-Paul Salameh; Patrick M M Bossuyt; Matthew D F McInnes Journal: Eur Radiol Date: 2020-01-24 Impact factor: 5.315
Authors: Mostafa Alabousi; Isabelle D Gauthier; Nicole Li; Gonçalo Mf Dos Santos; Dmitry Golev; Michael N Patlas; Abdullah Alabousi Journal: Emerg Radiol Date: 2019-06-17
Authors: Sanam Ebrahimzadeh; Nayaar Islam; Haben Dawit; Jean-Paul Salameh; Sakib Kazi; Nicholas Fabiano; Lee Treanor; Marissa Absi; Faraz Ahmad; Paul Rooprai; Ahmed Al Khalil; Kelly Harper; Neil Kamra; Mariska Mg Leeflang; Lotty Hooft; Christian B van der Pol; Ross Prager; Samanjit S Hare; Carole Dennie; René Spijker; Jonathan J Deeks; Jacqueline Dinnes; Kevin Jenniskens; Daniël A Korevaar; Jérémie F Cohen; Ann Van den Bruel; Yemisi Takwoingi; Janneke van de Wijgert; Junfeng Wang; Elena Pena; Sandra Sabongui; Matthew Df McInnes Journal: Cochrane Database Syst Rev Date: 2022-05-16
Authors: Nayaar Islam; Sanam Ebrahimzadeh; Jean-Paul Salameh; Sakib Kazi; Nicholas Fabiano; Lee Treanor; Marissa Absi; Zachary Hallgrimson; Mariska Mg Leeflang; Lotty Hooft; Christian B van der Pol; Ross Prager; Samanjit S Hare; Carole Dennie; René Spijker; Jonathan J Deeks; Jacqueline Dinnes; Kevin Jenniskens; Daniël A Korevaar; Jérémie F Cohen; Ann Van den Bruel; Yemisi Takwoingi; Janneke van de Wijgert; Johanna Aag Damen; Junfeng Wang; Matthew Df McInnes Journal: Cochrane Database Syst Rev Date: 2021-03-16
Authors: Trevor A McGrath; Mostafa Alabousi; Becky Skidmore; Daniël A Korevaar; Patrick M M Bossuyt; David Moher; Brett Thombs; Matthew D F McInnes Journal: Syst Rev Date: 2017-10-10