Maya B Mathur1, Tyler J VanderWeele2. 1. Quantitative Sciences Unit, Stanford University, Stanford, CA, USA. Electronic address: mmathur@stanford.edu. 2. Department of Epidemiology, Harvard University, Boston, MA, USA.
Abstract
BACKGROUND AND OBJECTIVES: The "credibility ceiling" method was proposed to conduct sensitivity analysis for unmeasured confounding and other forms of bias in meta-analyses and has been used in umbrella reviews to grade evidence strength. However, we explain that the method has fundamental statistical flaws. METHODS: We use statistical reasoning to assess the method's validity, providing intuition for our findings by presenting simple applied examples in which the method yields clearly incorrect conclusions. RESULTS: The credibility ceiling is not a valid bias correction, as we show mathematically and illustrate using examples in which, for example, the method incorrectly "adjusts" the meta-analytic point estimate in the wrong direction. Although the originators describe the method as limiting the credibility of any given observational study to a fixed ceiling, we show why this interpretation in fact bears little relation to what the method actually does. CONCLUSION: Given the fundamental problems with the credibility ceiling method and its demonstrated potential for misleading conclusions, we recommend against its use.
BACKGROUND AND OBJECTIVES: The "credibility ceiling" method was proposed to conduct sensitivity analysis for unmeasured confounding and other forms of bias in meta-analyses and has been used in umbrella reviews to grade evidence strength. However, we explain that the method has fundamental statistical flaws. METHODS: We use statistical reasoning to assess the method's validity, providing intuition for our findings by presenting simple applied examples in which the method yields clearly incorrect conclusions. RESULTS: The credibility ceiling is not a valid bias correction, as we show mathematically and illustrate using examples in which, for example, the method incorrectly "adjusts" the meta-analytic point estimate in the wrong direction. Although the originators describe the method as limiting the credibility of any given observational study to a fixed ceiling, we show why this interpretation in fact bears little relation to what the method actually does. CONCLUSION: Given the fundamental problems with the credibility ceiling method and its demonstrated potential for misleading conclusions, we recommend against its use.
Authors: Stefania I Papatheodorou; Konstantinos K Tsilidis; Evangelos Evangelou; John P A Ioannidis Journal: J Clin Epidemiol Date: 2014-11-26 Impact factor: 6.437
Authors: Cristiano A Köhler; Evangelos Evangelou; Brendon Stubbs; Marco Solmi; Nicola Veronese; Lazaros Belbasis; Beatrice Bortolato; Matias C A Melo; Camila A Coelho; Brisa S Fernandes; Mark Olfson; John P A Ioannidis; André F Carvalho Journal: J Psychiatr Res Date: 2018-05-25 Impact factor: 4.791
Authors: Leandro Fórnias Machado de Rezende; Thiago Hérick de Sá; Georgios Markozannes; Juan Pablo Rey-López; I-Min Lee; Konstantinos K Tsilidis; John P A Ioannidis; José Eluf-Neto Journal: Br J Sports Med Date: 2017-11-16 Impact factor: 13.800