Does publication bias explain the divergent findings on menopausal hormone therapy and cardioprotection in the literature?

As women traverse the menopause transition, they lose the ability to produce estradiol, and their risk of developing cardiovascular disease (CVD) increases. The use of menopausal hormone therapy (HT) has been viewed as a way to counteract ovarian aging and the accompanying elevation in CVD risk. Initial observational studies of HT use in the 1980s and 1990s strongly supported this argument. In data from the Nurses’ Health Study among 48 470 postmenopausal women (30–63 years old) followed for 10 years, a reduction in the incidence of coronary heart disease (CHD) as well as in cardiovascular disease (CVD) mortality, was observed with current use of HT. Of 16 prospective studies on this subject, 15 found decreased relative risks of CHD among women using HT compared to nonusers, supporting a protective association with estrogen therapy. These favorable findings led to an endorsement of HT use for cardiovascular health, even appearing in some clinical guidelines. The findings also inspired two landmark double‐blind and placebo‐controlled randomized clinical trials (RCTs) to test HT use (conjugated equine estrogens [CEEs]/with and without medroxyprogesterone acetate) for primary (Women’s Health Initiative [WHI]) and secondary prevention (Heart and Estrogen/Progestin Replacement Study [HERS]) of CVD in the 1990s. Surprisingly for the medical and research communities, both trials did not confirm the positive findings from previous observational studies, , casting doubt on a cardioprotective effect and even suggesting a harmful effect of HT use for primary or secondary prevention of CVD.

Since that time, researchers have actively sought answers to explain the discrepancy between positive findings of observational studies and the negative outcome of the WHI/HERS trials. In this issue of Research and Practice in Thrombosis and Haemostasis (RPTH), Berntsen et al shift the focus of this comparison from observational studies in humans to animal studies. Most animal studies testing effects of estrogens on atherosclerosis and vascular disease had shown beneficial effects, and these positive results further bolstered a case for HT and cardioprotection. Berntsen et al conducted an elegant systematic review and meta‐analysis of published animal studies comparing estradiol and its natural metabolites or CEE, with controls for effects on measures of atherosclerosis. The authors assessed whether confirmation and/or publication bias could explain the discrepancy between the favorable findings for HT in animal studies versus the neutral or adverse effects found in major RCTs.

Confirmation bias refers to the seeking or interpreting of evidence in ways that are partial to existing beliefs, expectations, or a hypothesis in hand. The authors hypothesized that this bias may have resulted in interpreting findings from animal studies on estrogen use differently before versus after the landmark WHI publication. Contrary to the authors’ hypothesis, no evidence was found of a change in researchers’ interpretations of their own findings before versus after WHI. Strikingly, 75% (95% confidence interval [CI], 67%‐81%) of animal studies conducted before WHI concluded that estrogens had a protective effect on atherosclerosis compared with 78% (95% CI, 71%‐83%) of animal studies conducted after WHI. This reported finding is strong evidence that experimental animal research has been consistent in showing a protective effect of estrogen on the cardiovascular system. The consistent findings from these studies over time calls for additional efforts to better understand the divergent findings from RCTs of HT in postmenopausal women. Interestingly, when authors compared general statements made by authors about exogenous estrogens in animal studies before versus after WHI, the percentage of those statements referring to estrogen as cardioprotective decreased from 70% before to 40% after WHI. However, such general statements may have been influenced by reviewers’ and editors’ requests for text modifications, or anticipation of such feedback, during the peer‐review process.

A provocative finding from Berntsen et al is the suggestion of publication bias, as detected by extremely skewed funnel plots and significant Egger’s tests that were more pronounced after 2002. Interestingly, once authors adjusted for this bias, the overall estimate of estrogen’s effects on atherosclerosis was described as close to null, making findings from animal studies in line with those from RCTs. However, this observation was not relevant to studies of cynomolgus monkeys, one of the best primate animal models of human atherosclerosis, which did not show any sign of publication bias. It is critical to point out that funnel plot asymmetry could be a statistical artifact rather than an indication of the presence of publication bias. This is mainly relevant when an outcome of interest is a continuous measure that is found to be dependent on baseline risk (effect of interest in the control group). On average, studies with higher baseline risk will have larger standard deviations, and, if effect estimates are also dependent on baseline risk, this may cause correlation between mean differences (x axis) and standard errors (y axis). Such correlation can result in funnel plot asymmetry even in the absence of publication bias. Adjusting for baseline risk treatment interactions and regressing on inverse sample size (rather than standard error) could help determine if funnel plot asymmetry is due to statistical artifact or not. What remains unknown in the Berntsen et al study was whether the main effect of interest was dependent on baseline risk, resulting in artificially skewed funnel plots.

The novel analytic approach used in this paper, however, does not address the evolving clinical trial data in support of the timing hypothesis. This “timing” or “critical window” hypothesis posits that the negative findings of the WHI and HERS are related to the older age of study participants and the long duration between menopause onset and HT initiation. When estrogen is provided shortly after menopause, it produces anti‐inflammatory, vasodilatory, and cardioprotective effects. However, if estrogen is provided later in life after a long period of estradiol deficiency, its cardioprotective effects are abolished. In‐depth analyses of the WHI data by participant age and time since menopause have supported this hypothesis by showing patterns of favorable or neutral effects on CHD events in recently menopausal women and adverse effects in older women randomized to estrogen therapy (see Table 1). , Data from a separate RCT designed specifically to test the “timing” hypothesis, the Early Versus Late Intervention Trial (ELITE), provided additional support, by demonstrating that progression of atherosclerosis (assessed by carotid intima‐media thickness [CIMT]) was slowed by estradiol in recently menopausal women but not among women at least a decade past menopause. The Kronos Early Estrogen Study (KEEPS), however, showed neutral effects of HT on CIMT progression in a newly menopausal cohort but may have lacked statistical power. Interestingly, the timing hypothesis is not limited to human studies. A loss of anti‐inflammatory features and vascular protective effects of exogenous estrogens was observed in older ovariectomized rats, when compared with younger and recently ovariectomized animals. Most recently, vascular reactivity and G protein–coupled estrogen receptor (GPER) protein expression were assessed in female mice of varying ages (adult, middle‐aged, and aged male and female C57BL/6 mice). Vasodilation in response to estrogen and the GPER agonist G‐1 were reduced in aging female mice and accompanied by downregulation of GPER protein. It would have been of great interest if Berntsen et al had assessed the “timing” hypothesis as a potential explanation for the divergent findings from animal studies compared with RCTs.

TABLE 1

Health outcomes in the Women's Health Initiative estrogen‐alone trial, according to age at study entry, intervention phase

Outcome	Estrogen‐alone trial		Difference b Per 10 000 PY	HR	95% CI	P value
	CEE Events per 10 000 PY	Placebo Events per 10 000 PY
Myocardial infarction						.02
50–59 y	14	25	−11	0.55	0.31–1.00
60–69 y	46	48	−2	0.95	0.69–1.30
70–79 y	83	69	14	1.24	0.88–1.75
All‐cause mortality						.04
50–59 y	29	40	−11	0.70	0.46–1.09
60–69 y	78	77	0	1.01	0.79–1.29
70–79 y	155	129	26	1.21	0.95–1.56

Numbers may not add precisely due to rounding error.

Adapted from Manson et al. ,

Abbreviations: CEE, conjugated equine estrogens; CI, confidence interval; HR, hazard ratio; PY, person‐years.

Median length of randomized treatment 7.2 years for estrogen alone.

Difference = events per 10 000 women per year in the hormone group − events per 10 000 women per year in the placebo group.

Difference

In recent RCTs of HT, different estrogen formulations, doses, and routes of administration are being tested. Evolving lines of evidence suggest potential differential effects based on these factors. , , However, such evidence is generally limited to observational studies, and RCTs are needed. The work by Berntsen et al confirms the ongoing need for more rigorous research and analysis to advance science, including elucidating the divergent findings from observational studies, animal research, and RCTs of HT use in postmenopausal women.

AUTHOR CONTRIBUTIONS

Both authors contributed to the drafting and approval of the final manuscript.

RELATIONSHIP DISCLOSURE

The authors declare no conflicts of interest.