Livia Puljak1, Irma Ramic2, Coral Arriola Naharro3, Jana Brezova4, Yi-Chen Lin5, Andrada-Alexandra Surdila6, Ester Tomajkova7, Inês Farias Medeiros8, Mishela Nikolovska9, Tina Poklepovic Pericic10, Ognjen Barcot11, Maria Suarez Salvado12. 1. Center for Evidence-Based Medicine and Health Care, Catholic University of Croatia, Zagreb, Croatia. Electronic address: livia.puljak@unicath.hr. 2. Department of Cardiac Anesthesia at Heart Center, Sarajevo, Bosnia and Herzegovina. 3. University of Alcala Faculty of Medicine, Alcala de Henares, Spain. 4. Jessenius Faculty of Medicine in Martin, Comenius University in Bratislava, Slovakia. 5. National Yang-Ming University, School of Medicine, Taipei, Taiwan. 6. University of Laval School of Medicine, Quebec, Canada. 7. Faculty of Medicine of Pavol Jozef Safarik University in Kosice, Kosice, Slovakia. 8. Abel Salazar Institute for Biomedical Sciences, University of Porto (ICBAS-UP), Porto, Portugal. 9. Medical Faculty of Skopje, Ss Cyril and Methodius University, Skopje, North Macedonia. 10. Cochrane Croatia, University of Split School of Medicine, Split, Croatia. 11. Department of Surgery, University Hospital Split, Split, Croatia. 12. University of Cadiz, School of Medicine, Cadiz, Spain.
Abstract
OBJECTIVES: To analyze how many non-Cochrane systematic reviews (NCSRs) used Cochrane's risk of bias (RoB) tool, domains they used, and whether judgments and comments about RoB were in line with Cochrane Handbook. METHODS: This was a methodological (research-on-research) study. We retrieved NCSRs from PubMed, extracted information about methods used for RoB assessment, and if they used 2011 Cochrane RoB tool, we analyzed their RoB methods and compared them with Cochrane Handbook guidance. RESULTS: We included 508 NCSRs; 431 (85%) reported they analyzed RoB, and 269 (53%) used Cochrane RoB tool. Only 16 of those 269 (5.9%) reported both a judgment and a supporting comment in the Cochrane RoB table in the manuscript (N = 4) or in a supplementary file (N = 12). Fifteen reviews, with 158 included trials, used judgments low/high/unclear; 41% of analyzed available judgments were inadequate, either because judgment was not in line with comment or comment was missing. CONCLUSIONS: Most NCSRs use Cochrane RoB tool to assess RoB, but most of them reported it incompletely, with high prevalence of inadequate judgments. Authors, editors, and peer-reviewers should make an effort to improve completeness and adequacy of Cochrane RoB assessment in non-Cochrane reviews.
OBJECTIVES: To analyze how many non-Cochrane systematic reviews (NCSRs) used Cochrane's risk of bias (RoB) tool, domains they used, and whether judgments and comments about RoB were in line with Cochrane Handbook. METHODS: This was a methodological (research-on-research) study. We retrieved NCSRs from PubMed, extracted information about methods used for RoB assessment, and if they used 2011 Cochrane RoB tool, we analyzed their RoB methods and compared them with Cochrane Handbook guidance. RESULTS: We included 508 NCSRs; 431 (85%) reported they analyzed RoB, and 269 (53%) used Cochrane RoB tool. Only 16 of those 269 (5.9%) reported both a judgment and a supporting comment in the Cochrane RoB table in the manuscript (N = 4) or in a supplementary file (N = 12). Fifteen reviews, with 158 included trials, used judgments low/high/unclear; 41% of analyzed available judgments were inadequate, either because judgment was not in line with comment or comment was missing. CONCLUSIONS: Most NCSRs use Cochrane RoB tool to assess RoB, but most of them reported it incompletely, with high prevalence of inadequate judgments. Authors, editors, and peer-reviewers should make an effort to improve completeness and adequacy of Cochrane RoB assessment in non-Cochrane reviews.
Authors: Ognjen Barcot; Matija Boric; Svjetlana Dosenovic; Marija Cavar; Antonia Jelicic Kadic; Tina Poklepovic Pericic; Ivana Vukicevic; Ivana Vuka; Livia Puljak Journal: BMC Med Res Methodol Date: 2020-09-29 Impact factor: 4.615