Lun Li1, Jinhui Tian, Hongliang Tian, Rao Sun, Yali Liu, Kehu Yang. 1. Evidence-Based Medicine Center, School of Basic Medical Sciences, Lanzhou University, Lanzhou, China The First Clinical College of Lanzhou University, Lanzhou, China.
Abstract
OBJECTIVE: To evaluate the reporting and methodological quality of overviews of systematic reviews. METHOD: We developed an 18-item assessment tool for overviews of systematic reviews. We then performed a systematic search for such overviews using the terms ('overview' AND ('meta analys*' OR 'systematic review*')) OR 'umbrella review' in the title. We only included those overviews that were limited to systematic reviews or meta-analyses. Their methodological and reporting quality were assessed by two independent reviewers using the checklist, and differences were resolved by a third reviewer. Data analyses was conducted by SPSS version 15.0 for Windows. RESULTS: We identified 41 overviews of systematic reviews whose mean total reporting score was 10.78 (SD 3.84) and methodological score 3.05 (SD 2.09). Some important items were not adequately reported: only 69% reported defined eligibility criteria, 76% reported search strategy, 49% reported the process of review selection, 44% reported the data collection process, 5% reported evaluating the reporting quality, 46% reported evaluating methodological quality, and 20% reported assessing the evidence level for each outcome. CONCLUSION: The reporting and methodological quality of overviews of systematic reviews was very poor, and there is still much room for improvement. A checklist for overviews of systematic reviews should be developed and used.
OBJECTIVE: To evaluate the reporting and methodological quality of overviews of systematic reviews. METHOD: We developed an 18-item assessment tool for overviews of systematic reviews. We then performed a systematic search for such overviews using the terms ('overview' AND ('meta analys*' OR 'systematic review*')) OR 'umbrella review' in the title. We only included those overviews that were limited to systematic reviews or meta-analyses. Their methodological and reporting quality were assessed by two independent reviewers using the checklist, and differences were resolved by a third reviewer. Data analyses was conducted by SPSS version 15.0 for Windows. RESULTS: We identified 41 overviews of systematic reviews whose mean total reporting score was 10.78 (SD 3.84) and methodological score 3.05 (SD 2.09). Some important items were not adequately reported: only 69% reported defined eligibility criteria, 76% reported search strategy, 49% reported the process of review selection, 44% reported the data collection process, 5% reported evaluating the reporting quality, 46% reported evaluating methodological quality, and 20% reported assessing the evidence level for each outcome. CONCLUSION: The reporting and methodological quality of overviews of systematic reviews was very poor, and there is still much room for improvement. A checklist for overviews of systematic reviews should be developed and used.