Anahita Dehmoobad Sharifabadi1, Mariska Leeflang2, Lee Treanor1, Noemie Kraaijpoel3, Jean-Paul Salameh4, Mostafa Alabousi5, Nabil Asraoui6, Jade Choo-Foo6, Yemisi Takwoingi7,8, Jonathan J Deeks7,8, Matthew D F McInnes9. 1. Department of Radiology-Faculty of Medicine, University of Ottawa, Ottawa, Canada. 2. Amsterdam UMC, Clinical Epidemiology, Biostatistics and Bioinformatics, Amsterdam Public Health, University of Amsterdam, Meibergdreef 9, Amsterdam, Netherlands. 3. Academic Medical Center, Meibergdreef 9, 1105 AZ, Amsterdam, The Netherlands. 4. Clinical Epidemiology Program, Ottawa Hospital Research Institute, School of Epidemiology and Public Health, University of Ottawa, Ottawa, Canada. 5. McMaster University, Hamilton, Canada. 6. Faculty of Medicine, University of Ottawa, Ottawa, Canada. 7. Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, B15 2TT, UK. 8. NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, B15 2TT, UK. 9. Department of Radiology, University of Ottawa, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Room c159 Ottawa Hospital Civic Campus, 1053 Carling Ave., Ottawa, ON, K1Y 4E9, Canada. mmcinnes@toh.on.ca.
Abstract
PURPOSE: The purpose of this methodological review was to determine the extent to which comparative imaging systematic reviews of diagnostic test accuracy (DTA) use primary studies with comparative or non-comparative designs. METHODS: MEDLINE was used to identify DTA systematic reviews published in imaging journals between January 2000 and May 2018. INCLUSION CRITERIA: systematic reviews comparing at least two index tests (one of which was imaging-based); review characteristics were extracted. Study design and other characteristics of primary studies included in the systematic reviews were evaluated. RESULTS: One hundred three comparative imaging reviews were included; 11 (11%) included only comparative studies, 12 (11%) included only non-comparative primary studies, and 80 (78%) included both comparative and non-comparative primary studies. For reviews containing both comparative and non-comparative primary studies, the median proportion of non-comparative primary studies was 81% (IQR 57-90%). Of 92 reviews that included non-comparative primary studies, 86% did not recognize this as a limitation. Furthermore, among 4182 primary studies, 3438 (82%) were non-comparative and 744 (18%) were comparative in design. CONCLUSION: Most primary studies included in comparative imaging reviews are non-comparative in design and awareness of the risk of bias associated with this is low. This may lead to incorrect conclusions about the relative accuracy of diagnostic tests and be counter-productive for informing guidelines and funding decisions about imaging tests. KEY POINTS: • Few comparative accuracy imaging reviews include only primary studies with optimal comparative study designs. Among the rest, few recognize the risk of bias conferred from inclusion of primary studies with non-comparative designs. • The demand for accurate comparative accuracy data combined with minimal awareness of valid comparative study designs may lead to counter-productive research and inadequately supported clinical decisions for diagnostic tests. • Using comparative accuracy imaging reviews with a high risk of bias to inform guidelines and funding decisions may have detrimental impacts on patient care.
PURPOSE: The purpose of this methodological review was to determine the extent to which comparative imaging systematic reviews of diagnostic test accuracy (DTA) use primary studies with comparative or non-comparative designs. METHODS: MEDLINE was used to identify DTA systematic reviews published in imaging journals between January 2000 and May 2018. INCLUSION CRITERIA: systematic reviews comparing at least two index tests (one of which was imaging-based); review characteristics were extracted. Study design and other characteristics of primary studies included in the systematic reviews were evaluated. RESULTS: One hundred three comparative imaging reviews were included; 11 (11%) included only comparative studies, 12 (11%) included only non-comparative primary studies, and 80 (78%) included both comparative and non-comparative primary studies. For reviews containing both comparative and non-comparative primary studies, the median proportion of non-comparative primary studies was 81% (IQR 57-90%). Of 92 reviews that included non-comparative primary studies, 86% did not recognize this as a limitation. Furthermore, among 4182 primary studies, 3438 (82%) were non-comparative and 744 (18%) were comparative in design. CONCLUSION: Most primary studies included in comparative imaging reviews are non-comparative in design and awareness of the risk of bias associated with this is low. This may lead to incorrect conclusions about the relative accuracy of diagnostic tests and be counter-productive for informing guidelines and funding decisions about imaging tests. KEY POINTS: • Few comparative accuracy imaging reviews include only primary studies with optimal comparative study designs. Among the rest, few recognize the risk of bias conferred from inclusion of primary studies with non-comparative designs. • The demand for accurate comparative accuracy data combined with minimal awareness of valid comparative study designs may lead to counter-productive research and inadequately supported clinical decisions for diagnostic tests. • Using comparative accuracy imaging reviews with a high risk of bias to inform guidelines and funding decisions may have detrimental impacts on patient care.
Entities:
Keywords:
Comparative effectiveness research; Diagnostic test, routine; Sensitivity and specificity
Authors: Trevor A McGrath; Patrick M Bossuyt; Paul Cronin; Jean-Paul Salameh; Noémie Kraaijpoel; Nicola Schieda; Matthew D F McInnes Journal: J Magn Reson Imaging Date: 2018-08-13 Impact factor: 4.813
Authors: Matthew D F McInnes; David Moher; Brett D Thombs; Trevor A McGrath; Patrick M Bossuyt; Tammy Clifford; Jérémie F Cohen; Jonathan J Deeks; Constantine Gatsonis; Lotty Hooft; Harriet A Hunt; Christopher J Hyde; Daniël A Korevaar; Mariska M G Leeflang; Petra Macaskill; Johannes B Reitsma; Rachel Rodin; Anne W S Rutjes; Jean-Paul Salameh; Adrienne Stevens; Yemisi Takwoingi; Marcello Tonelli; Laura Weeks; Penny Whiting; Brian H Willis Journal: JAMA Date: 2018-01-23 Impact factor: 56.272
Authors: M Alabousi; A Alabousi; T A McGrath; K D Cobey; B Budhram; R A Frank; F Nguyen; J P Salameh; A Dehmoobad Sharifabadi; M D F McInnes Journal: Eur Radiol Date: 2018-07-26 Impact factor: 5.315
Authors: Wytze Laméris; Adrienne van Randen; Shandra Bipat; Patrick M M Bossuyt; Marja A Boermeester; Jaap Stoker Journal: Eur Radiol Date: 2008-06-04 Impact factor: 5.315
Authors: Penny F Whiting; Anne W S Rutjes; Marie E Westwood; Susan Mallett; Jonathan J Deeks; Johannes B Reitsma; Mariska M G Leeflang; Jonathan A C Sterne; Patrick M M Bossuyt Journal: Ann Intern Med Date: 2011-10-18 Impact factor: 25.391
Authors: Steve Halligan; Shedrack F Kenis; Oshaani Abeyakoon; Andrew A O Plumb; Susan Mallett Journal: Eur Radiol Date: 2020-02-21 Impact factor: 5.315