Peer H Voss1, Eva A Rehfuess. 1. Institute for Medical Informatics, Biometry and Epidemiology, University of Munich, Marchioninistrasse 15, 81377 Munich, Germany.
Abstract
INTRODUCTION: Systematic reviews are a cornerstone of evidence-based public health. The method of appraising the quality of different intervention and observational study designs in such reviews remains an important challenge. This article examines the applicability of selected quality appraisal tools (QATs) and the impact of choice of tool on the meta-analysis of a published systematic review. METHODS: The authors selected a systematic review on the effectiveness of hand washing with soap in preventing diarrhoea, covering a range of epidemiological study designs. 6 QATs were used to assess 13 studies meeting their inclusion criteria; component sections/questions were coded numerically to derive a summary score between -1 (low quality) and +1 (high quality) for each QAT and study. Heterogeneity in study quality was evaluated graphically using traffic light schemes and spider charts. Random effects meta-analysis was undertaken for all studies; sensitivity analyses for each QAT included only those studies with a score of 0 or above. RESULTS: The authors found substantial heterogeneity in summary scores for a given study. Their main meta-analysis yielded an OR of 0.60 (95% CI 0.47 to 0.77) with most sensitivity analyses giving similar pooled effect sizes with wider CIs. DISCUSSION: The six QATs differ greatly in applicability across study designs, approach to quality appraisal (ie, scale vs checklist, presence/absence of summary score), coverage of domains and quality of component questions and answers. Learning from advantages and disadvantages of each QAT, we recommend research into the development of a reliable QAT with a broad applicability across study designs.
INTRODUCTION: Systematic reviews are a cornerstone of evidence-based public health. The method of appraising the quality of different intervention and observational study designs in such reviews remains an important challenge. This article examines the applicability of selected quality appraisal tools (QATs) and the impact of choice of tool on the meta-analysis of a published systematic review. METHODS: The authors selected a systematic review on the effectiveness of hand washing with soap in preventing diarrhoea, covering a range of epidemiological study designs. 6 QATs were used to assess 13 studies meeting their inclusion criteria; component sections/questions were coded numerically to derive a summary score between -1 (low quality) and +1 (high quality) for each QAT and study. Heterogeneity in study quality was evaluated graphically using traffic light schemes and spider charts. Random effects meta-analysis was undertaken for all studies; sensitivity analyses for each QAT included only those studies with a score of 0 or above. RESULTS: The authors found substantial heterogeneity in summary scores for a given study. Their main meta-analysis yielded an OR of 0.60 (95% CI 0.47 to 0.77) with most sensitivity analyses giving similar pooled effect sizes with wider CIs. DISCUSSION: The six QATs differ greatly in applicability across study designs, approach to quality appraisal (ie, scale vs checklist, presence/absence of summary score), coverage of domains and quality of component questions and answers. Learning from advantages and disadvantages of each QAT, we recommend research into the development of a reliable QAT with a broad applicability across study designs.
Authors: Jacob Burns; Hanna Boogaard; Stephanie Polus; Lisa M Pfadenhauer; Anke C Rohwer; Annemoon M van Erp; Ruth Turley; Eva Rehfuess Journal: Cochrane Database Syst Rev Date: 2019-05-20
Authors: Thomas Harder; Anja Takla; Eva Rehfuess; Alex Sánchez-Vivar; Dorothea Matysiak-Klose; Tim Eckmanns; Gérard Krause; Helena de Carvalho Gomes; Andreas Jansen; Simon Ellis; Frode Forland; Roberta James; Joerg J Meerpohl; Antony Morgan; Holger Schünemann; Teun Zuiderent-Jerak; Ole Wichmann Journal: BMC Med Res Methodol Date: 2014-05-21 Impact factor: 4.615
Authors: Tanith C Rose; Natalie Adams; David C Taylor-Robinson; Benjamin Barr; Jeremy Hawker; Sarah O'Brien; Mara Violato; Margaret Whitehead Journal: Syst Rev Date: 2016-01-21