K S Khan1, S Daya, A Jadad. 1. International Scholars Programme, Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario, Canada.
Abstract
BACKGROUND: Traditional and largely qualitative reviews of evidence are now giving way to much more structured systematic overviews that use a quantitative method to calculate the overall effect of treatment. The latter approach is dependent on the quality of primary studies, which may introduce bias if they are of poor methodologic quality. OBJECTIVE: To test the hypothesis that the inclusion of poor-quality trials in meta-analyses would bias the conclusions and produce incorrect estimates of treatment effect. METHODS: An overview of randomized trials of antiestrogen therapy in subfertile men with oligospermia was performed to test the hypothesis. Data sources included online searching of MEDLINE and Science Citation Index databases between 1966 and 1994, scanning the bibliography of known primary studies and review articles, and contacting experts in the field. After independent, blind assessment, nine of 149 originally identified studies met the inclusion criteria and were selected. We assessed study quality independently. Outcome data from each study were pooled and statistically summarized. RESULTS: There was a marginal improvement in pregnancy rate with antiestrogen treatment (odds ratio, 1.6; 95% confidence interval, 0.9 to 2.6). Sensitivity analyses on the basis of methodologic quality demonstrated that poor-quality studies produced a positive effect with treatment, whereas no benefit was observed with high-quality studies. CONCLUSION: The results of a meta-analysis are influenced by the quality of the primary studies included. Methodologically, poor studies tend to exaggerate the overall estimate of treatment effect and may lead to incorrect inferences.
BACKGROUND: Traditional and largely qualitative reviews of evidence are now giving way to much more structured systematic overviews that use a quantitative method to calculate the overall effect of treatment. The latter approach is dependent on the quality of primary studies, which may introduce bias if they are of poor methodologic quality. OBJECTIVE: To test the hypothesis that the inclusion of poor-quality trials in meta-analyses would bias the conclusions and produce incorrect estimates of treatment effect. METHODS: An overview of randomized trials of antiestrogen therapy in subfertile men with oligospermia was performed to test the hypothesis. Data sources included online searching of MEDLINE and Science Citation Index databases between 1966 and 1994, scanning the bibliography of known primary studies and review articles, and contacting experts in the field. After independent, blind assessment, nine of 149 originally identified studies met the inclusion criteria and were selected. We assessed study quality independently. Outcome data from each study were pooled and statistically summarized. RESULTS: There was a marginal improvement in pregnancy rate with antiestrogen treatment (odds ratio, 1.6; 95% confidence interval, 0.9 to 2.6). Sensitivity analyses on the basis of methodologic quality demonstrated that poor-quality studies produced a positive effect with treatment, whereas no benefit was observed with high-quality studies. CONCLUSION: The results of a meta-analysis are influenced by the quality of the primary studies included. Methodologically, poor studies tend to exaggerate the overall estimate of treatment effect and may lead to incorrect inferences.
Authors: Mohit Bhandari; P J Devereaux; Victor Montori; Claudio Cinà; Ved Tandan; Gordon H Guyatt Journal: Can J Surg Date: 2004-02 Impact factor: 2.089
Authors: Thomas W Hamilton; Vassilis Athanassoglou; Stephen Mellon; Louise H Strickland; Marialena Trivella; David Murray; Hemant G Pandit Journal: Cochrane Database Syst Rev Date: 2017-02-01