Lingzhi Chu1, John P A Ioannidis2,3,4,5, Alex C Egilman6, Vasilis Vasiliou1, Joseph S Ross7,8,9,10, Joshua D Wallach1,6. 1. Department of Environmental Health Sciences, Yale School of Public Health, New Haven, CT, USA. 2. Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, CA, USA. 3. Department of Health Research and Policy, Stanford University School of Medicine, Stanford, CA, USA. 4. Stanford Prevention Research Center, Department of Medicine, Stanford University School of Medicine, Stanford, CA, USA. 5. Department of Statistics, Stanford University School of Humanities and Sciences, Stanford, CA, USA. 6. Collaboration for Research Integrity and Transparency (CRIT), Yale Law School, New Haven, CT, USA. 7. Section of General Medicine, Department of Internal Medicine, Yale School of Medicine, New Haven, CT, USA. 8. National Clinician Scholars Program, Yale School of Medicine, New Haven, CT, USA. 9. Department of Health Policy and Management, Yale School of Public Health, New Haven, CT, USA. 10. Center for Outcomes Research and Evaluation, Yale-New Haven Health System, New Haven, CT, USA.
Abstract
BACKGROUND: Different analytical approaches can influence the associations estimated in observational studies. We assessed the variability of effect estimates reported within and across observational studies evaluating the impact of alcohol on breast cancer. METHODS: We abstracted largest harmful, largest protective and smallest (closest to the null value of 1.0) relative risk estimates in studies included in a recent alcohol-breast cancer meta-analysis, and recorded how they differed based on five model specification characteristics, including exposure definition, exposure contrast levels, study populations, adjustment covariates and/or model approaches. For each study, we approximated vibration of effects by dividing the largest by the smallest effect estimate [i.e. ratio of odds ratio (ROR)]. RESULTS: Among 97 eligible studies, 85 (87.6%) reported both harmful and protective relative effect estimates for an alcohol-breast cancer relationship, which ranged from 1.1 to 17.9 and 0.0 to 1.0, respectively. The RORs comparing the largest and smallest estimates in value ranged from 1.0 to 106.2, with a median of 3.0 [interquartile range (IQR) 2.0-5.2]. One-third (35, 36.1%) of the RORs were based on extreme effect estimates with at least three different model specification characteristics; the vast majority (87, 89.7%) had different exposure definitions or contrast levels. Similar vibrations of effect were observed when only extreme estimates with differences based on study populations and/or adjustment covariates were compared. CONCLUSIONS: Most observational studies evaluating the impact of alcohol on breast cancer report relative effect estimates for the same associations that diverge by >2-fold. Therefore, observational studies should estimate the vibration of effects to provide insight regarding the stability of findings.
BACKGROUND: Different analytical approaches can influence the associations estimated in observational studies. We assessed the variability of effect estimates reported within and across observational studies evaluating the impact of alcohol on breast cancer. METHODS: We abstracted largest harmful, largest protective and smallest (closest to the null value of 1.0) relative risk estimates in studies included in a recent alcohol-breast cancer meta-analysis, and recorded how they differed based on five model specification characteristics, including exposure definition, exposure contrast levels, study populations, adjustment covariates and/or model approaches. For each study, we approximated vibration of effects by dividing the largest by the smallest effect estimate [i.e. ratio of odds ratio (ROR)]. RESULTS: Among 97 eligible studies, 85 (87.6%) reported both harmful and protective relative effect estimates for an alcohol-breast cancer relationship, which ranged from 1.1 to 17.9 and 0.0 to 1.0, respectively. The RORs comparing the largest and smallest estimates in value ranged from 1.0 to 106.2, with a median of 3.0 [interquartile range (IQR) 2.0-5.2]. One-third (35, 36.1%) of the RORs were based on extreme effect estimates with at least three different model specification characteristics; the vast majority (87, 89.7%) had different exposure definitions or contrast levels. Similar vibrations of effect were observed when only extreme estimates with differences based on study populations and/or adjustment covariates were compared. CONCLUSIONS: Most observational studies evaluating the impact of alcohol on breast cancer report relative effect estimates for the same associations that diverge by >2-fold. Therefore, observational studies should estimate the vibration of effects to provide insight regarding the stability of findings.
Authors: B M Psaty; T D Koepsell; D Lin; N S Weiss; D S Siscovick; F R Rosendaal; M Pahor; C D Furberg Journal: J Am Geriatr Soc Date: 1999-06 Impact factor: 5.562
Authors: Rafael Dal-Ré; John P Ioannidis; Michael B Bracken; Patricia A Buffler; An-Wen Chan; Eduardo L Franco; Carlo La Vecchia; Elisabete Weiderpass Journal: Sci Transl Med Date: 2014-02-19 Impact factor: 17.956
Authors: Rachel Visontay; Matthew Sunderland; Tim Slade; Jack Wilson; Louise Mewton Journal: BMC Med Res Methodol Date: 2022-01-14 Impact factor: 4.615