OBJECTIVES: Widespread use of increasingly complex statistical methods makes it ever more challenging to adequately assess the results reported and conclusions drawn in meta-analytic research. This paper aims to identify potential fallacies by in-depth examination of recent publications on mood disorders. METHODS: Three meta-analyses were selected based on availability of data and representativeness of methods employed. By means of detailed re-analysis, several widespread methodological problems were identified, and the example data were used to illustrate and discuss them. RESULTS: General points addressed include clear formulation of the research question, choice of effect size measures, and general choice of model. Data quality problems like missing data and publication bias are discussed along with methods to deal with them. Furthermore, aspects of meta-analytic modelling like the use of fixed or random effects, data aggregation, as well as the use of subgroups are explained, and issues of excessive complexity and data dredging pointed out. Finally, the benefit of diagnostic tools like confidence bands and the importance of transparency regarding data and methodology for the interpretation of meta-analytic results are highlighted. CONCLUSIONS: Practically relevant quality criteria for readers to bear in mind when dealing with meta-analytic publications are summarized in a ten point checklist.
OBJECTIVES: Widespread use of increasingly complex statistical methods makes it ever more challenging to adequately assess the results reported and conclusions drawn in meta-analytic research. This paper aims to identify potential fallacies by in-depth examination of recent publications on mood disorders. METHODS: Three meta-analyses were selected based on availability of data and representativeness of methods employed. By means of detailed re-analysis, several widespread methodological problems were identified, and the example data were used to illustrate and discuss them. RESULTS: General points addressed include clear formulation of the research question, choice of effect size measures, and general choice of model. Data quality problems like missing data and publication bias are discussed along with methods to deal with them. Furthermore, aspects of meta-analytic modelling like the use of fixed or random effects, data aggregation, as well as the use of subgroups are explained, and issues of excessive complexity and data dredging pointed out. Finally, the benefit of diagnostic tools like confidence bands and the importance of transparency regarding data and methodology for the interpretation of meta-analytic results are highlighted. CONCLUSIONS: Practically relevant quality criteria for readers to bear in mind when dealing with meta-analytic publications are summarized in a ten point checklist.
Authors: Roland N Boubela; Wolfgang Huf; Klaudius Kalcher; Ronald Sladky; Peter Filzmoser; Lukas Pezawas; Siegfried Kasper; Christian Windischberger; Ewald Moser Journal: MAGMA Date: 2011-11-16 Impact factor: 2.310
Authors: Johanna Michl; Christian Scharinger; Miriam Zauner; Siegfried Kasper; Michael Freissmuth; Harald H Sitte; Gerhard F Ecker; Lukas Pezawas Journal: Eur Neuropsychopharmacol Date: 2014-07-03 Impact factor: 4.600
Authors: Roland N Boubela; Klaudius Kalcher; Wolfgang Huf; Claudia Kronnerwetter; Peter Filzmoser; Ewald Moser Journal: Front Hum Neurosci Date: 2013-05-01 Impact factor: 3.169