BACKGROUND: We examined whether and to what extent different strategies of defining and incorporating quality of included studies affect the results of metaanalyses of diagnostic accuracy. METHODS: We evaluated the methodological quality of 487 diagnostic-accuracy studies in 30 systematic reviews with the QUADAS (Quality Assessment of Diagnostic-Accuracy Studies) checklist. We applied 3 strategies that varied both in the definition of quality and in the statistical approach to incorporate the quality-assessment results into metaanalyses. We compared magnitudes of diagnostic odds ratios, widths of their confidence intervals, and changes in a hypothetical clinical decision between strategies. RESULTS: Following 2 definitions of quality, we concluded that only 70 or 72 of 487 studies were of "high quality". This small number was partly due to poor reporting of quality items. None of the strategies for accounting for differences in quality led systematically to accuracy estimates that were less optimistic than ignoring quality in metaanalyses. Limiting the review to high-quality studies considerably reduced the number of studies in all reviews, with wider confidence intervals as a result. In 18 reviews, the quality adjustment would have resulted in a different decision about the usefulness of the test. CONCLUSIONS: Although reporting the results of quality assessment of individual studies is necessary in systematic reviews, reader wariness is warranted regarding claims that differences in methodological quality have been accounted for. Obstacles for adjusting for quality in metaanalyses are poor reporting of design features and patient characteristics and the relatively low number of studies in most diagnostic reviews.
BACKGROUND: We examined whether and to what extent different strategies of defining and incorporating quality of included studies affect the results of metaanalyses of diagnostic accuracy. METHODS: We evaluated the methodological quality of 487 diagnostic-accuracy studies in 30 systematic reviews with the QUADAS (Quality Assessment of Diagnostic-Accuracy Studies) checklist. We applied 3 strategies that varied both in the definition of quality and in the statistical approach to incorporate the quality-assessment results into metaanalyses. We compared magnitudes of diagnostic odds ratios, widths of their confidence intervals, and changes in a hypothetical clinical decision between strategies. RESULTS: Following 2 definitions of quality, we concluded that only 70 or 72 of 487 studies were of "high quality". This small number was partly due to poor reporting of quality items. None of the strategies for accounting for differences in quality led systematically to accuracy estimates that were less optimistic than ignoring quality in metaanalyses. Limiting the review to high-quality studies considerably reduced the number of studies in all reviews, with wider confidence intervals as a result. In 18 reviews, the quality adjustment would have resulted in a different decision about the usefulness of the test. CONCLUSIONS: Although reporting the results of quality assessment of individual studies is necessary in systematic reviews, reader wariness is warranted regarding claims that differences in methodological quality have been accounted for. Obstacles for adjusting for quality in metaanalyses are poor reporting of design features and patient characteristics and the relatively low number of studies in most diagnostic reviews.
Authors: Jeltsje S Cnossen; Karlijn C Vollebregt; Nynke de Vrieze; Gerben ter Riet; Ben W J Mol; Arie Franx; Khalid S Khan; Joris A M van der Post Journal: BMJ Date: 2008-05-14
Authors: Mariska M G Leeflang; Jonathan J Deeks; Constantine Gatsonis; Patrick M M Bossuyt Journal: Ann Intern Med Date: 2008-12-16 Impact factor: 25.391
Authors: Jeltsje S Cnossen; Rachel K Morris; Gerben ter Riet; Ben W J Mol; Joris A M van der Post; Arri Coomarasamy; Aeilko H Zwinderman; Stephen C Robson; Patrick J E Bindels; Jos Kleijnen; Khalid S Khan Journal: CMAJ Date: 2008-03-11 Impact factor: 8.262
Authors: Pedro E A A Brasil; Liane De Castro; Alejandro M Hasslocher-Moreno; Luiz H C Sangenis; José U Braga Journal: BMC Infect Dis Date: 2010-11-25 Impact factor: 3.090
Authors: Janneke Van't Hooft; Johanna H van der Lee; Brent C Opmeer; Cornelieke S H Aarnoudse-Moens; Arnold G E Leenders; Ben Willem J Mol; Timo R de Haan Journal: Syst Rev Date: 2015-05-17
Authors: Pedro Emmanuel Alvarenga Americano do Brasil; Rodolfo Castro; Liane de Castro Journal: Mem Inst Oswaldo Cruz Date: 2016-01 Impact factor: 2.743