Manuel R Blum1,2,3, Yuan Jin Tan1,3, John P A Ioannidis1,3,4,5,6. 1. Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, CA, USA. 2. Department of General Internal Medicine, Inselspital, Bern University Hospital, University of Bern, Bern, Switzerland. 3. Department of Epidemiology and Population Health, Stanford University School of Medicine, Stanford, CA, USA. 4. Stanford Prevention Research Center, Department of Medicine, Stanford University School of Medicine, Stanford, CA, USA. 5. Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, CA, USA. 6. Department of Statistics, Stanford University School of Humanities and Science, Stanford, CA, USA.
Abstract
BACKGROUND: E-values are a recently introduced approach to evaluate confounding in observational studies. We aimed to empirically assess the current use of E-values in published literature. METHODS: We conducted a systematic literature search for all publications, published up till the end of 2018, which cited at least one of two inceptive E-value papers and presented E-values for original data. For these case publications we identified control publications, matched by journal and issue, where the authors had not calculated E-values. RESULTS: In total, 87 papers presented 516 E-values. Of the 87 papers, 14 concluded that residual confounding likely threatens at least some of the main conclusions. Seven of these 14 named potential uncontrolled confounders. 19 of 87 papers related E-value magnitudes to expected strengths of field-specific confounders. The median E-value was 1.88, 1.82, and 2.02 for the 43, 348, and 125 E-values where confounding was felt likely to affect the results, unlikely to affect the results, or not commented upon, respectively. The 69 case-control publication pairs dealt with effect sizes of similar magnitude. Of 69 control publications, 52 did not comment on unmeasured confounding and 44/69 case publications concluded that confounding was unlikely to affect study conclusions. CONCLUSIONS: Few papers using E-values conclude that confounding threatens their results, and their E-values overlap in magnitude with those of papers acknowledging susceptibility to confounding. Facile automation in calculating E-values may compound the already poor handling of confounding. E-values should not be a substitute for careful consideration of potential sources of unmeasured confounding. If used, they should be interpreted in the context of expected confounding in specific fields.
BACKGROUND: E-values are a recently introduced approach to evaluate confounding in observational studies. We aimed to empirically assess the current use of E-values in published literature. METHODS: We conducted a systematic literature search for all publications, published up till the end of 2018, which cited at least one of two inceptive E-value papers and presented E-values for original data. For these case publications we identified control publications, matched by journal and issue, where the authors had not calculated E-values. RESULTS: In total, 87 papers presented 516 E-values. Of the 87 papers, 14 concluded that residual confounding likely threatens at least some of the main conclusions. Seven of these 14 named potential uncontrolled confounders. 19 of 87 papers related E-value magnitudes to expected strengths of field-specific confounders. The median E-value was 1.88, 1.82, and 2.02 for the 43, 348, and 125 E-values where confounding was felt likely to affect the results, unlikely to affect the results, or not commented upon, respectively. The 69 case-control publication pairs dealt with effect sizes of similar magnitude. Of 69 control publications, 52 did not comment on unmeasured confounding and 44/69 case publications concluded that confounding was unlikely to affect study conclusions. CONCLUSIONS: Few papers using E-values conclude that confounding threatens their results, and their E-values overlap in magnitude with those of papers acknowledging susceptibility to confounding. Facile automation in calculating E-values may compound the already poor handling of confounding. E-values should not be a substitute for careful consideration of potential sources of unmeasured confounding. If used, they should be interpreted in the context of expected confounding in specific fields.
Authors: Scholastic Ashaba; Christine Cooper-Vince; Samuel Maling; Emily N Satinsky; Charles Baguma; Dickens Akena; Denis Nansera; Francis Bajunirwe; Alexander C Tsai Journal: J Affect Disord Rep Date: 2021-01-23
Authors: Scholastic Ashaba; Bernard Kakuhikire; Charles Baguma; Emily N Satinsky; Jessica M Perkins; Justin D Rasmussen; Christine E Cooper-Vince; Phionah Ahereza; Patrick Gumisiriza; Justus Kananura; David R Bangsberg; Alexander C Tsai Journal: SSM Ment Health Date: 2022-01-29
Authors: Timothy L Lash; Thomas P Ahern; Lindsay J Collin; Matthew P Fox; Richard F MacLehose Journal: Am J Epidemiol Date: 2021-08-01 Impact factor: 4.897
Authors: Danica J Gomes; Benjamin Kaufman; Adam R Aluisio; Scott Kendall; Vladimir Thomas; Christina Bloem Journal: Ann Glob Health Date: 2020-07-03 Impact factor: 2.462