John P A Ioannidis1, Orestis A Panagiotou. 1. Prevention Research Center, Department of Medicine, Stanford University School of Medicine, Stanford, California 94305, USA. jioannid@stanford.edu
Abstract
CONTEXT: Many biomarkers are proposed in highly cited studies as determinants of disease risk, prognosis, or response to treatment, but few eventually transform clinical practice. OBJECTIVE: To examine whether the magnitude of the effect sizes of biomarkers proposed in highly cited studies is accurate or overestimated. DATA SOURCES: We searched ISI Web of Science and MEDLINE until December 2010. STUDY SELECTION: We included biomarker studies that had a relative risk presented in their abstract. Eligible articles were those that had received more than 400 citations in the ISI Web of Science and that had been published in any of 24 highly cited biomedical journals. We also searched MEDLINE for subsequent meta-analyses on the same associations (same biomarker and same outcome). DATA EXTRACTION: In the highly cited studies, data extraction was focused on the disease/outcome, biomarker under study, and first reported relative risk in the abstract. From each meta-analysis, we extracted the overall relative risk and the relative risk in the largest study. Data extraction was performed independently by 2 investigators. RESULTS: We evaluated 35 highly cited associations. For 30 of the 35 (86%), the highly cited studies had a stronger effect estimate than the largest study; for 3 the largest study was also the highly cited study; and only twice was the effect size estimate stronger in the largest than in the highly cited study. For 29 of the 35 (83%) highly cited studies, the corresponding meta-analysis found a smaller effect estimate. Only 15 of the associations were nominally statistically significant based on the largest studies, and of those only 7 had a relative risk point estimate greater than 1.37. CONCLUSION: Highly cited biomarker studies often report larger effect estimates for postulated associations than are reported in subsequent meta-analyses evaluating the same associations.
CONTEXT: Many biomarkers are proposed in highly cited studies as determinants of disease risk, prognosis, or response to treatment, but few eventually transform clinical practice. OBJECTIVE: To examine whether the magnitude of the effect sizes of biomarkers proposed in highly cited studies is accurate or overestimated. DATA SOURCES: We searched ISI Web of Science and MEDLINE until December 2010. STUDY SELECTION: We included biomarker studies that had a relative risk presented in their abstract. Eligible articles were those that had received more than 400 citations in the ISI Web of Science and that had been published in any of 24 highly cited biomedical journals. We also searched MEDLINE for subsequent meta-analyses on the same associations (same biomarker and same outcome). DATA EXTRACTION: In the highly cited studies, data extraction was focused on the disease/outcome, biomarker under study, and first reported relative risk in the abstract. From each meta-analysis, we extracted the overall relative risk and the relative risk in the largest study. Data extraction was performed independently by 2 investigators. RESULTS: We evaluated 35 highly cited associations. For 30 of the 35 (86%), the highly cited studies had a stronger effect estimate than the largest study; for 3 the largest study was also the highly cited study; and only twice was the effect size estimate stronger in the largest than in the highly cited study. For 29 of the 35 (83%) highly cited studies, the corresponding meta-analysis found a smaller effect estimate. Only 15 of the associations were nominally statistically significant based on the largest studies, and of those only 7 had a relative risk point estimate greater than 1.37. CONCLUSION: Highly cited biomarker studies often report larger effect estimates for postulated associations than are reported in subsequent meta-analyses evaluating the same associations.
Authors: Valentina Gallo; Matthias Egger; Valerie McCormack; Peter B Farmer; John P A Ioannidis; Micheline Kirsch-Volders; Giuseppe Matullo; David H Phillips; Bernadette Schoket; Ulf Stromberg; Roel Vermeulen; Christopher Wild; Miquel Porta; Paolo Vineis Journal: Eur J Epidemiol Date: 2011-10-29 Impact factor: 8.082
Authors: Eric D Weinhandl; Jiannong Liu; David T Gilbertson; Thomas J Arneson; Allan J Collins Journal: J Am Soc Nephrol Date: 2012-02-23 Impact factor: 10.121
Authors: S E Bushnell; Z Zhao; C C Stebbins; D Cadavid; A M Buko; E T Whalley; J A Davis; E M Versage; J R Richert; R C Axtell; L Steinman; R Medori Journal: Neurology Date: 2012-05-09 Impact factor: 9.910
Authors: S H Faro; F B Mohamed; J A Helpern; J H Jensen; K R Thulborn; I C Atkinson; H I Sair; D J Mikulis Journal: AJNR Am J Neuroradiol Date: 2013-10-17 Impact factor: 3.825