Reint Meursinge Reynders1, Luisa Ladu2, Nicola Di Girolamo3. 1. Department of Oral and Maxillofacial Surgery, Academic Medical Center, University of Amsterdam, Meibergdreef 9, Amsterdam 1105 AZ, The Netherlands; Department of Orthodontics, Academisch Centrum Tandheelkunde Amsterdam (ACTA), University of Amsterdam, Gustav Mahlerlaan 3004, Amsterdam 1081 LA, The Netherlands; Private Practice of Orthodontics, Milan, Italy. Electronic address: reyndersmail@gmail.com. 2. Private Practice of Orthodontics, Milan, Italy. 3. Center for Veterinary Health Sciences, Oklahoma State University, 2065 W. Farm Road, Stillwater, OK 74078, USA; EBMVet, Via Sigismondo Trecchi 20, Cremona CR 26100, Italy.
Abstract
OBJECTIVES: The objective of the study was to assess the prevalence, the reporting quality, the need, and the consequences of contacting of authors by Cochrane reviewers to obtain additional information for their reviews. STUDY DESIGN AND SETTING: Cross-sectional study and survey on all new Cochrane intervention reviews published between January 1, 2016 and January 31, 2017. RESULTS: The cross-sectional study found that reviewers had contacted or had tried to contact studies to obtain additional information in 73.4% (234/319) of reviews but reported poorly on the methods, outcomes, and consequences of this procedure. Most eligible studies in the reviews were poorly reported, but few reviewers 21.2% (65/306) reported that they had contacted these studies. The survey showed that risk of bias scores, Grading of Recommendations, Assessment, Development and Evaluation scores, the summary primary or secondary outcomes, and the summary effect size of the primary outcome of the review were changed as a consequence of contacting of authors. Thirty-five of one hundred and thirty (26.9%) reviews scored opposite outcomes for the same question in the cross-sectional study compared with the survey. CONCLUSIONS: Our findings on contacting of authors by Cochrane reviewers showed relevant shortcomings in the current standards and transparency of Cochrane reviews. These shortcomings can compromise the validity and reproducibility of these reviews and affect a wide audience.
OBJECTIVES: The objective of the study was to assess the prevalence, the reporting quality, the need, and the consequences of contacting of authors by Cochrane reviewers to obtain additional information for their reviews. STUDY DESIGN AND SETTING: Cross-sectional study and survey on all new Cochrane intervention reviews published between January 1, 2016 and January 31, 2017. RESULTS: The cross-sectional study found that reviewers had contacted or had tried to contact studies to obtain additional information in 73.4% (234/319) of reviews but reported poorly on the methods, outcomes, and consequences of this procedure. Most eligible studies in the reviews were poorly reported, but few reviewers 21.2% (65/306) reported that they had contacted these studies. The survey showed that risk of bias scores, Grading of Recommendations, Assessment, Development and Evaluation scores, the summary primary or secondary outcomes, and the summary effect size of the primary outcome of the review were changed as a consequence of contacting of authors. Thirty-five of one hundred and thirty (26.9%) reviews scored opposite outcomes for the same question in the cross-sectional study compared with the survey. CONCLUSIONS: Our findings on contacting of authors by Cochrane reviewers showed relevant shortcomings in the current standards and transparency of Cochrane reviews. These shortcomings can compromise the validity and reproducibility of these reviews and affect a wide audience.
Authors: Ursula W de Ruijter; Z L Rana Kaplan; Wichor M Bramer; Frank Eijkenaar; Daan Nieboer; Agnes van der Heide; Hester F Lingsma; Willem A Bax Journal: J Gen Intern Med Date: 2022-01-11 Impact factor: 6.473