Stefania I Papatheodorou1, Konstantinos K Tsilidis2, Evangelos Evangelou2, John P A Ioannidis3. 1. Department of Health Sciences, Cyprus Institute for Environmental and Public Health, Cyprus University of Technology, Eirinis 95 Str, 3041, Limassol, Cyprus. 2. Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, University Campus, 45110 Ioannina, Greece. 3. Stanford Prevention Research Center, Department of Medicine, Stanford University School of Medicine, Stanford 94305, CA, USA; Department of Health Research and Policy, Stanford University School of Medicine, Stanford 94305, CA, USA; Department of Statistics, Stanford University School of Humanities and Sciences, Stanford 94305, CA, USA. Electronic address: jioannid@stanford.edu.
Abstract
OBJECTIVES: Meta-analyses of biomarkers often present spurious significant results and large effects. We applied sensitivity analyses with the use of credibility ceilings to assess whether and how the results of meta-analyses of biomarkers and cancer risk would change. STUDY DESIGN AND SETTING: We evaluated 98 meta-analyses, 43 (44%) of which had nominally statistically significant results. We assumed that any single study cannot give more than a maximum certainty 100 - c% (c, credibility ceiling) that the effect estimate [odds ratio (OR)] exceeds 1 (null) or 1.2. RESULTS: Nominal statistical significance was maintained for 21 (21%) meta-analyses, for c = 10% and OR >1, and these proportions changed to 7%, 3%, and 6% with ceilings of 20%, 30%, and 40%, respectively. For ceilings for OR >1.2, the respective proportions were 37%, 21%, 7%, and 3%. Seven meta-analyses on infectious agents retained statistical significance even with a high ceiling of c = 20% for OR >1.00. Meta-analyses without other hints of bias (large between-study heterogeneity, small-study effects, excess significance) were more likely to retain statistical significance than those that had such hints of bias. CONCLUSION: Credibility ceilings may be helpful in meta-analyses of biomarkers to understand the robustness of the results to different levels of uncertainty.
OBJECTIVES: Meta-analyses of biomarkers often present spurious significant results and large effects. We applied sensitivity analyses with the use of credibility ceilings to assess whether and how the results of meta-analyses of biomarkers and cancer risk would change. STUDY DESIGN AND SETTING: We evaluated 98 meta-analyses, 43 (44%) of which had nominally statistically significant results. We assumed that any single study cannot give more than a maximum certainty 100 - c% (c, credibility ceiling) that the effect estimate [odds ratio (OR)] exceeds 1 (null) or 1.2. RESULTS: Nominal statistical significance was maintained for 21 (21%) meta-analyses, for c = 10% and OR >1, and these proportions changed to 7%, 3%, and 6% with ceilings of 20%, 30%, and 40%, respectively. For ceilings for OR >1.2, the respective proportions were 37%, 21%, 7%, and 3%. Seven meta-analyses on infectious agents retained statistical significance even with a high ceiling of c = 20% for OR >1.00. Meta-analyses without other hints of bias (large between-study heterogeneity, small-study effects, excess significance) were more likely to retain statistical significance than those that had such hints of bias. CONCLUSION: Credibility ceilings may be helpful in meta-analyses of biomarkers to understand the robustness of the results to different levels of uncertainty.
Authors: Tai Lim Kim; Gwang Hun Jeong; Jae Won Yang; Keum Hwa Lee; Andreas Kronbichler; Hans J van der Vliet; Giuseppe Grosso; Fabio Galvano; Dagfinn Aune; Jong Yeob Kim; Nicola Veronese; Brendon Stubbs; Marco Solmi; Ai Koyanagi; Sung Hwi Hong; Elena Dragioti; Eunyoung Cho; Leandro F M de Rezende; Edward L Giovannucci; Jae Il Shin; Gabriele Gamerith Journal: Adv Nutr Date: 2020-11-16 Impact factor: 8.701
Authors: Maria Kyrgiou; Ilkka Kalliala; Georgios Markozannes; Marc J Gunter; Evangelos Paraskevaidis; Hani Gabra; Pierre Martin-Hirsch; Konstantinos K Tsilidis Journal: BMJ Date: 2017-02-28
Authors: André F Carvalho; Marco Solmi; Marcos Sanches; Myrela O Machado; Brendon Stubbs; Olesya Ajnakina; Chelsea Sherman; Yue Ran Sun; Celina S Liu; Andre R Brunoni; Giorgio Pigato; Brisa S Fernandes; Beatrice Bortolato; Muhammad I Husain; Elena Dragioti; Joseph Firth; Theodore D Cosco; Michael Maes; Michael Berk; Krista L Lanctôt; Eduard Vieta; Diego A Pizzagalli; Lee Smith; Paolo Fusar-Poli; Paul A Kurdyak; Michele Fornaro; Jürgen Rehm; Nathan Herrmann Journal: Transl Psychiatry Date: 2020-05-18 Impact factor: 6.222
Authors: Myrela O Machado; Nicola Veronese; Marcos Sanches; Brendon Stubbs; Ai Koyanagi; Trevor Thompson; Ioanna Tzoulaki; Marco Solmi; Davy Vancampfort; Felipe B Schuch; Michael Maes; Giovanni A Fava; John P A Ioannidis; André F Carvalho Journal: BMC Med Date: 2018-07-20 Impact factor: 8.775
Authors: Jong Yeob Kim; Johanna Steingroever; Keum Hwa Lee; Jun Oh; Min Jae Choi; Jiwon Lee; Nicholas G Larkins; Franz Schaefer; Sung Hwi Hong; Gwang Hun Jeong; Jae Il Shin; Andreas Kronbichler Journal: J Clin Med Date: 2020-02-01 Impact factor: 4.241