Ani Movsisyan1, G J Melendez-Torres2, Paul Montgomery2. 1. Centre for Evidence-Based Intervention, Department of Social Policy and Intervention, University of Oxford, Barnett House, 32 Wellington Square, Oxford OX1 2ER, UK. Electronic address: ani.movsisyan@spi.ox.ac.uk. 2. Centre for Evidence-Based Intervention, Department of Social Policy and Intervention, University of Oxford, Barnett House, 32 Wellington Square, Oxford OX1 2ER, UK.
Abstract
OBJECTIVES: To investigate the application of the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach and the quality of evidence ratings in systematic reviews of complex interventions. STUDY DESIGN AND SETTING: This study examined all 40 systematic reviews published in three Cochrane Review Groups from 2013 to May 2014: Cochrane Developmental, Psychosocial and Learning Problems Group (CDPLPG); Cochrane Public Health Group (CPHG); and Cochrane Depression, Anxiety, and Neurosis Group (CCDAN). The reviews were coded and classified into "complex" (n = 24) and "simple" (n = 16) intervention review groups based on the predefined complexity dimensions from the extant literature mapped into the PICOTS framework. All the data were analyzed in these two groups to help identify specific patterns of the GRADE ratings in the reviews of complex interventions. RESULTS: Outcomes of complex intervention reviews had higher proportions of "very low" quality of evidence ratings compared with those of simple intervention reviews (37.5% vs. 9.1% for the primary benefit outcomes) and were more frequently downgraded for inconsistency, performance bias, and study design. None of the outcomes of complex intervention reviews (0%) were given "high" GRADE ratings. CONCLUSION: Results suggest that the GRADE assessment may not adequately describe the evidence base of complex interventions.
OBJECTIVES: To investigate the application of the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach and the quality of evidence ratings in systematic reviews of complex interventions. STUDY DESIGN AND SETTING: This study examined all 40 systematic reviews published in three Cochrane Review Groups from 2013 to May 2014: Cochrane Developmental, Psychosocial and Learning Problems Group (CDPLPG); Cochrane Public Health Group (CPHG); and Cochrane Depression, Anxiety, and Neurosis Group (CCDAN). The reviews were coded and classified into "complex" (n = 24) and "simple" (n = 16) intervention review groups based on the predefined complexity dimensions from the extant literature mapped into the PICOTS framework. All the data were analyzed in these two groups to help identify specific patterns of the GRADE ratings in the reviews of complex interventions. RESULTS: Outcomes of complex intervention reviews had higher proportions of "very low" quality of evidence ratings compared with those of simple intervention reviews (37.5% vs. 9.1% for the primary benefit outcomes) and were more frequently downgraded for inconsistency, performance bias, and study design. None of the outcomes of complex intervention reviews (0%) were given "high" GRADE ratings. CONCLUSION: Results suggest that the GRADE assessment may not adequately describe the evidence base of complex interventions.
Authors: Jennyfer Wolf; Sydney Hubbard; Michael Brauer; Argaw Ambelu; Benjamin F Arnold; Robert Bain; Valerie Bauza; Joe Brown; Bethany A Caruso; Thomas Clasen; John M Colford; Matthew C Freeman; Bruce Gordon; Richard B Johnston; Andrew Mertens; Annette Prüss-Ustün; Ian Ross; Jeffrey Stanaway; Jeff T Zhao; Oliver Cumming; Sophie Boisson Journal: Lancet Date: 2022-07-02 Impact factor: 202.731
Authors: Eileen Fs Kaner; Fiona R Beyer; Colin Muirhead; Fiona Campbell; Elizabeth D Pienaar; Nicolas Bertholet; Jean B Daeppen; John B Saunders; Bernard Burnand Journal: Cochrane Database Syst Rev Date: 2018-02-24
Authors: Jade Benjamin-Chung; Jaynal Abedin; David Berger; Ashley Clark; Veronica Jimenez; Eugene Konagaya; Diana Tran; Benjamin F Arnold; Alan E Hubbard; Stephen P Luby; Edward Miguel; John M Colford Journal: Int J Epidemiol Date: 2017-08-01 Impact factor: 9.685
Authors: Celia C Kamath; Claudia C Dobler; Rozalina G McCoy; Michelle A Lampman; Atieh Pajouhi; Patricia J Erwin; John Matulis; Muhamad Elrashidi; Joseph Darcel; Mouaz Alsawas; Zhen Wang; Nilay D Shah; M Hassan Murad; Bjorg Thorsteinsdottir Journal: J Gen Intern Med Date: 2020-10-26 Impact factor: 5.128