Padhraig S Fleming1, Despina Koletsi2, John P A Ioannidis3, Nikolaos Pandis4. 1. Oral Growth and Development, Institute of Dentistry, Barts and The London School of Medicine and Dentistry, Queen Mary University of London, London E1 2AD, England. Electronic address: padhraig.fleming@gmail.com. 2. Department of Orthodontics, University of Athens, 2 Thivon Street, 115 27, Goudi, Athens, Greece. 3. Department of Medicine, Stanford Prevention Research Center, Stanford University School of Medicine, Stanford, CA, USA; Department of Health Research and Policy, Stanford University School of Medicine, Stanford, CA, USA; Department of Statistics, Stanford University School of Humanities and Sciences, Stanford, CA, USA; Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, CA, USA. 4. Department of Orthodontics and Dentofacial Orthopedics, Dental School, Medical Faculty, University of Bern, Hochschulstrasse 4, 3012 Bern, Switzerland; Private Practice, 29 P. Zafiropoulou Street, Corfu 49100, Greece.
Abstract
OBJECTIVES: To appraise the quality of evidence in systematic reviews (SRs) within the Cochrane Database of Systematic Reviews (CDSRs) across diverse topics and to explore the relationship between the strength of evidence using Grades of Recommendation, Assessment, Development, and Evaluation (GRADE) and the probability that authors would interpret that an intervention may be of value. STUDY DESIGN AND SETTING: We evaluated the SRs published on the CDSR from January 1, 2013, to June 30, 2014. Two authors identified relevant SRs by independent searching of the Cochrane register. We further focused on SRs that incorporated tables with GRADE [summary of findings (SoF)]. Data were extracted independently by two authors. The quality of the evidence for the first listed primary outcome in SoF tables in each review and reasons for upgrade or downgrade were recorded. RESULTS: Overall, 1,394 SRs were identified. Of these, 608 (43.6%) incorporated GRADE. Within these reviews, only 13.5% (n = 82) reported a high quality and 30.8% (n = 187) a moderate quality of evidence for the first listed primary outcome, whereas 31.7% (n = 193) had low level and 24% (n = 146) had very low level of evidence. High quality of evidence was more common in updated compared to new reviews and in pharmacologic than other types of interventions. Even when all outcomes listed in the SoFs were considered, only 116/608 (19.1%) of SRs had at least one outcome with high quality of evidence. Overall, only 4.1% (25/608) of SRs incorporating GRADE in SoF tables had high quality of evidence, allied both to significant results and a favorable interpretation of the intervention by the reviewers. CONCLUSION: Evidence of high quality is uncommon for medical and health-related interventions assessed with GRADE within the CDSR, and favorable evidence of high quality is even more uncommon.
OBJECTIVES: To appraise the quality of evidence in systematic reviews (SRs) within the Cochrane Database of Systematic Reviews (CDSRs) across diverse topics and to explore the relationship between the strength of evidence using Grades of Recommendation, Assessment, Development, and Evaluation (GRADE) and the probability that authors would interpret that an intervention may be of value. STUDY DESIGN AND SETTING: We evaluated the SRs published on the CDSR from January 1, 2013, to June 30, 2014. Two authors identified relevant SRs by independent searching of the Cochrane register. We further focused on SRs that incorporated tables with GRADE [summary of findings (SoF)]. Data were extracted independently by two authors. The quality of the evidence for the first listed primary outcome in SoF tables in each review and reasons for upgrade or downgrade were recorded. RESULTS: Overall, 1,394 SRs were identified. Of these, 608 (43.6%) incorporated GRADE. Within these reviews, only 13.5% (n = 82) reported a high quality and 30.8% (n = 187) a moderate quality of evidence for the first listed primary outcome, whereas 31.7% (n = 193) had low level and 24% (n = 146) had very low level of evidence. High quality of evidence was more common in updated compared to new reviews and in pharmacologic than other types of interventions. Even when all outcomes listed in the SoFs were considered, only 116/608 (19.1%) of SRs had at least one outcome with high quality of evidence. Overall, only 4.1% (25/608) of SRs incorporating GRADE in SoF tables had high quality of evidence, allied both to significant results and a favorable interpretation of the intervention by the reviewers. CONCLUSION: Evidence of high quality is uncommon for medical and health-related interventions assessed with GRADE within the CDSR, and favorable evidence of high quality is even more uncommon.