Orestis A Panagiotou1, John P A Ioannidis. 1. Department of Hygiene and Epidemiology, Clinical and Molecular Epidemiology Unit and Clinical Trials and Evidence-Based Medicine Unit, University of Ioannina School of Medicine, Ioannina 45110, Greece.
Abstract
OBJECTIVE: To assess the interpretation of a highly heterogeneous meta-analysis by authors of primary studies and by methodologists. STUDY DESIGN AND SETTING: We surveyed the authors of studies on the association between insulin-like growth factor 1 (IGF-1) and prostate cancer, and 20 meta-analysis methodologists. Authors and methodologists presented with the respective meta-analysis results were queried about the effect size and potential causality of the association. We evaluated whether author responses correlated with the number of IGF-related articles they had published and their study results included in the meta-analysis. We also compared authors' and methodologists' responses. RESULTS: Authors who had published more IGF-related papers offered more generous effect size estimates for the association (ρ(s)=0.61, P=0.01) and higher likelihood that the odds ratio (OR) was greater than 1.20 (ρ(s)=0.63, P=0.01). Authors who had published themselves studies with statistically significant effects for a positive association were more likely to believe that the true OR is greater than 1.20 compared with methodologists (median likelihood 50% versus 2.5%, P=0.01). CONCLUSION: Researchers are influenced by their own investment in the field, when interpreting a meta-analysis that includes their own study. Authors who published significant results are more likely to believe that a strong association exists compared with methodologists.
OBJECTIVE: To assess the interpretation of a highly heterogeneous meta-analysis by authors of primary studies and by methodologists. STUDY DESIGN AND SETTING: We surveyed the authors of studies on the association between insulin-like growth factor 1 (IGF-1) and prostate cancer, and 20 meta-analysis methodologists. Authors and methodologists presented with the respective meta-analysis results were queried about the effect size and potential causality of the association. We evaluated whether author responses correlated with the number of IGF-related articles they had published and their study results included in the meta-analysis. We also compared authors' and methodologists' responses. RESULTS: Authors who had published more IGF-related papers offered more generous effect size estimates for the association (ρ(s)=0.61, P=0.01) and higher likelihood that the odds ratio (OR) was greater than 1.20 (ρ(s)=0.63, P=0.01). Authors who had published themselves studies with statistically significant effects for a positive association were more likely to believe that the true OR is greater than 1.20 compared with methodologists (median likelihood 50% versus 2.5%, P=0.01). CONCLUSION: Researchers are influenced by their own investment in the field, when interpreting a meta-analysis that includes their own study. Authors who published significant results are more likely to believe that a strong association exists compared with methodologists.