T M Koster1, J Wetterslev2,3, C Gluud2, F Keus1, I C C van der Horst1. 1. Department of Critical Care, University of Groningen, University Medical Center Groningen, Groningen, The Netherlands. 2. The Copenhagen Trial Unit (CTU), Centre for Clinical Intervention Research, Copenhagen, Denmark. 3. Centre for Research in Intensive Care, Copenhagen, Denmark.
Abstract
RATIONALE: Meta-analysed intervention effect estimates are perceived to represent the highest level of evidence. However, such effects and the randomized clinical trials which are included in them need critical appraisal before the effects can be trusted. OBJECTIVE: Critical appraisal of a predefined set of all meta-analyses on interventions in intensive care medicine to assess their quality and assessed the risks of bias in those meta-analyses having the best quality. METHODS: We conducted a systematic search to select all meta-analyses of randomized clinical trials on interventions used in intensive care medicine. Selected meta-analyses were critically appraised for basic scientific criteria, (1) presence of an available protocol, (2) report of a full search strategy, and (3) use of any bias risk assessment of included trials. All meta-analyses which qualified these criteria were scrutinized by full "Risk of Bias in Systematic Reviews" ROBIS evaluation of 4 domains of risks of bias, and a "Preferred Reporting Items for Systematic Reviews and Meta-Analyses" PRISMA evaluation. RESULTS: We identified 467 meta-analyses. A total of 56 meta-analyses complied with these basic scientific criteria. We scrutinized the risks of bias in the 56 meta-analyses by full ROBIS evaluation and a PRISMA evaluation. Only 4 meta-analyses scored low risk of bias in all the 4 ROBIS domains and 41 meta-analyses reported all 27 items of the PRISMA checklist. CONCLUSION: In contrast with what might be perceived as the highest level of evidence only 0.9% of all meta-analyses were judged to have overall low risk of bias.
RATIONALE: Meta-analysed intervention effect estimates are perceived to represent the highest level of evidence. However, such effects and the randomized clinical trials which are included in them need critical appraisal before the effects can be trusted. OBJECTIVE: Critical appraisal of a predefined set of all meta-analyses on interventions in intensive care medicine to assess their quality and assessed the risks of bias in those meta-analyses having the best quality. METHODS: We conducted a systematic search to select all meta-analyses of randomized clinical trials on interventions used in intensive care medicine. Selected meta-analyses were critically appraised for basic scientific criteria, (1) presence of an available protocol, (2) report of a full search strategy, and (3) use of any bias risk assessment of included trials. All meta-analyses which qualified these criteria were scrutinized by full "Risk of Bias in Systematic Reviews" ROBIS evaluation of 4 domains of risks of bias, and a "Preferred Reporting Items for Systematic Reviews and Meta-Analyses" PRISMA evaluation. RESULTS: We identified 467 meta-analyses. A total of 56 meta-analyses complied with these basic scientific criteria. We scrutinized the risks of bias in the 56 meta-analyses by full ROBIS evaluation and a PRISMA evaluation. Only 4 meta-analyses scored low risk of bias in all the 4 ROBIS domains and 41 meta-analyses reported all 27 items of the PRISMA checklist. CONCLUSION: In contrast with what might be perceived as the highest level of evidence only 0.9% of all meta-analyses were judged to have overall low risk of bias.