Dimitris Mavridis1, Anna Chaimani2, Orestis Efthimiou2, Stefan Leucht3, Georgia Salanti2. 1. Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece Department of Primary Education, University of Ioannina, Ioannina, Greece. 2. Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece. 3. Department of Psychiatry and Psychotherapy, Technische Universität München, München, Germany Department of Psychiatry, University of Oxford, Oxford, UK.
Abstract
OBJECTIVE: Missing outcome data are a common problem in clinical trials and systematic reviews, as it compromises inferences by reducing precision and potentially biasing the results. Systematic reviewers often assume that the missing outcome problem has been resolved at the trial level. However, in many clinical trials a complete case analysis or suboptimal imputation techniques are employed and the problem is accumulated in a quantitative synthesis of trials via meta-analysis. The risk of bias due to missing data depends on the missingness mechanism. Most statistical analyses assume missing data to be missing at random, which is an unverifiable assumption. The aim of this paper is to present methods used to account for missing outcome data in a systematic review and meta-analysis. METHODS: The following methods to handle missing outcome data are presented: (1) complete cases analysis, (2) imputation methods from observed data, (3) best/worst case scenarios, (4) uncertainty interval for the summary estimate and (5) a statistical model that makes assumption about how treatment effects in missing data are connected to those in observed data. Examples are used to illustrate all the methods presented. RESULTS: Different methods yield different results. A complete case analysis leads to imprecise and potentially biased results. The best-case/worst-case scenarios give unrealistic estimates, while the uncertainty interval produces very conservative results. Imputation methods that replace missing data with values from the observed data do not properly account for the uncertainty introduced by the unobserved data and tend to underestimate SEs. Employing a statistical model that links treatment effects in missing and observed data, unlike the other methods, reduces the weight assigned to studies with large missing rates. CONCLUSIONS: Unlike clinical trials, in systematic reviews and meta-analyses we cannot adapt pre-emptive methods to account for missing outcome data. There are statistical techniques implemented in commercial software (eg, STATA) that quantify the departure from the missing at random assumption and adjust results appropriately. A sensitivity analysis with increasingly stringent assumptions on how parameters in the unobserved and observed data are related is a sensible way to evaluate robustness of results. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.
OBJECTIVE: Missing outcome data are a common problem in clinical trials and systematic reviews, as it compromises inferences by reducing precision and potentially biasing the results. Systematic reviewers often assume that the missing outcome problem has been resolved at the trial level. However, in many clinical trials a complete case analysis or suboptimal imputation techniques are employed and the problem is accumulated in a quantitative synthesis of trials via meta-analysis. The risk of bias due to missing data depends on the missingness mechanism. Most statistical analyses assume missing data to be missing at random, which is an unverifiable assumption. The aim of this paper is to present methods used to account for missing outcome data in a systematic review and meta-analysis. METHODS: The following methods to handle missing outcome data are presented: (1) complete cases analysis, (2) imputation methods from observed data, (3) best/worst case scenarios, (4) uncertainty interval for the summary estimate and (5) a statistical model that makes assumption about how treatment effects in missing data are connected to those in observed data. Examples are used to illustrate all the methods presented. RESULTS: Different methods yield different results. A complete case analysis leads to imprecise and potentially biased results. The best-case/worst-case scenarios give unrealistic estimates, while the uncertainty interval produces very conservative results. Imputation methods that replace missing data with values from the observed data do not properly account for the uncertainty introduced by the unobserved data and tend to underestimate SEs. Employing a statistical model that links treatment effects in missing and observed data, unlike the other methods, reduces the weight assigned to studies with large missing rates. CONCLUSIONS: Unlike clinical trials, in systematic reviews and meta-analyses we cannot adapt pre-emptive methods to account for missing outcome data. There are statistical techniques implemented in commercial software (eg, STATA) that quantify the departure from the missing at random assumption and adjust results appropriately. A sensitivity analysis with increasingly stringent assumptions on how parameters in the unobserved and observed data are related is a sensible way to evaluate robustness of results. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.
Authors: Elie A Akl; Lara A Kahale; Thomas Agoritsas; Romina Brignardello-Petersen; Jason W Busse; Alonso Carrasco-Labra; Shanil Ebrahim; Bradley C Johnston; Ignacio Neumann; Ivan Sola; Xin Sun; Per Vandvik; Yuqing Zhang; Pablo Alonso-Coello; Gordon Guyatt Journal: Syst Rev Date: 2015-07-23
Authors: Evan Mayo-Wilson; Susan Hutfless; Tianjing Li; Gillian Gresham; Nicole Fusco; Jeffrey Ehmsen; James Heyward; Swaroop Vedula; Diana Lock; Jennifer Haythornthwaite; Jennifer L Payne; Theresa Cowley; Elizabeth Tolbert; Lori Rosman; Claire Twose; Elizabeth A Stuart; Hwanhee Hong; Peter Doshi; Catalina Suarez-Cuervo; Sonal Singh; Kay Dickersin Journal: Syst Rev Date: 2015-11-02
Authors: Toshi A Furukawa; Georgia Salanti; Lauren Z Atkinson; Stefan Leucht; Henricus G Ruhe; Erick H Turner; Anna Chaimani; Yusuke Ogawa; Nozomi Takeshima; Yu Hayasaka; Hissei Imai; Kiyomi Shinohara; Aya Suganuma; Norio Watanabe; Sarah Stockton; John R Geddes; Andrea Cipriani Journal: BMJ Open Date: 2016-07-08 Impact factor: 2.692