Risk of cardiovascular serious adverse events associated with varenicline use for tobacco cessation: systematic review and meta-analysis.

OBJECTIVE: To examine the risk of treatment emergent, cardiovascular serious adverse events associated with varenicline use for tobacco cessation.
DESIGN: Meta-analysis comparing study effects using four summary estimates. DATA SOURCES: Medline, Cochrane Library, online clinical trials registries, and reference lists of identified articles. REVIEW
METHODS: We included randomised controlled trials of current tobacco users of adult age comparing use of varenicline with an inactive control and reporting adverse events. We defined treatment emergent, cardiovascular serious adverse events as occurring during drug treatment or within 30 days of discontinuation, and included any ischaemic or arrhythmic adverse cardiovascular event (myocardial infarction, unstable angina, coronary revascularisation, coronary artery disease, arrhythmias, transient ischaemic attacks, stroke, sudden death or cardiovascular related death, or congestive heart failure).
RESULTS: We identified 22 trials; all were double blinded and placebo controlled; two included participants with active cardiovascular disease and 11 enrolled participants with a history of cardiovascular disease. Rates of treatment emergent, cardiovascular serious adverse events were 0.63% (34/5431) in the varenicline groups and 0.47% (18/3801) in the placebo groups. The summary estimate for the risk difference, 0.27% (95% confidence interval -0.10 to 0.63; P = 0.15), based on all 22 trials, was neither clinically nor statistically significant. For comparison, the relative risk (1.40, 0.82 to 2.39; P = 0.22), Mantel-Haenszel odds ratio (1.41, 0.82 to 2.42; P = 0.22), and Peto odds ratio (1.58, 0.90 to 2.76; P = 0.11), all based on 14 trials with at least one event, also indicated a non-significant difference between varenicline and placebo groups.
CONCLUSIONS: This meta--analysis--which included all trials published to date, focused on events occurring during drug exposure, and analysed findings using four summary estimates-found no significant increase in cardiovascular serious adverse events associated with varenicline use. For rare outcomes, summary estimates based on absolute effects are recommended and estimates based on the Peto odds ratio should be avoided.

Introduction

Tobacco use accounts for 440 000 deaths in the United States every year, killing nearly one in two long term smokers, with the leading cause of death being cardiovascular disease.1 2 Quitting smoking has immediate cardiovascular benefits,3 reducing the risk of recurrence of coronary events to that of a non-smoker within three years and reducing mortality after a heart attack by up to 50% over three to five years.4 5 6

US clinical practice guidelines for treating tobacco dependence recommend the use of cessation pharmacotherapy for all smokers interested in quitting, unless contraindicated.7 The US Food and Drug Administration has approved three first line classes of cessation pharmacotherapy: nicotine replacement therapy; bupropion (Zyban, Wellbutrin), an antidepressant; and the most recently approved option, varenicline (Chantix, Champix), a partial agonist to nicotine receptors. Varenicline binds with high affinity and selectivity to α4β2 nicotinic acetylcholine receptors in neurones. The partial agonist activity induces modest receptor stimulation that attenuates the symptoms of nicotine withdrawal. In addition, by blocking the ability of nicotine to activate α4β2 nicotinic acetylcholine receptors, varenicline inhibits the surges of dopamine release that are believed to be responsible for the reinforcement and reward associated with tobacco use.8 Meta-analyses and comparative trials have demonstrated the effectiveness of varenicline for quitting smoking and sustaining abstinence relative both to placebo and to bupropion.9

For smokers with cardiovascular disease, cessation offers critical health benefits and yet has been a challenge to achieve, particularly in the long term.10 A recent placebo controlled trial of varenicline in 714 smokers with stable cardiovascular disease reported sustained abstinence of 47% at the end of treatment and 19% at one year follow-up in the varenicline group, compared with 14% and 7% in the placebo group, respectively; the group difference was significant at both time points.11 Incidence of cardiovascular serious adverse events in the varenicline group during the 52 week trial was low (7.0%) and the 95% confidence interval ruled out an excess greater than 5% (1.4%, −2.3% to 5.0%). Nonetheless, the FDA has called for systematic review of all randomised clinical trials of varenicline for tobacco cessation to determine its association with cardiovascular risk.12

Singh and colleagues conducted a meta-analysis of the safety of varenicline and concluded that the drug increased the risk of cardiovascular serious adverse events by 72%.13 However, doubts about the researchers’ conclusions have been raised,14 15 16 17 18 19 owing to several methodological issues, namely, the inclusion of adverse events well beyond the treatment period; exclusion of trials with no events, thereby biasing findings against the null; and use of the Peto odds ratio, which has shown bias under conditions of imbalanced design and rare events, which were present in most of the reviewed trials. Inclusion of the adverse events beyond the treatment period is of concern, because of the elevated risk for cardiovascular disease among study participants associated with their chronic use of tobacco and because of differential drop out by condition. In all but one20 of the 14 studies reviewed by Singh and colleagues, retention was lower in the placebo group than in the varenicline group, thereby reducing the likelihood of detecting adverse events in the placebo arm.

We conducted a systematic review and meta-analysis of treatment emergent, cardiovascular serious adverse events in all published, randomised controlled trials of varenicline for tobacco cessation. We defined these events as occurring during drug treatment or within 30 days of discontinuation. We chose a 30 day window because it was biologically relevant for detecting a drug toxicity effect, while still being conservative; the half life of varenicline is 24 hours, and any direct pharmacological effect should be gone within seven days. Other reports of serious adverse events to the FDA for drugs with a similar half life have used a discontinuation period of seven days.21

In consideration of the low event rate and imbalanced study designs, we summarised the study effects with the risk difference, an estimate of absolute effect. Summaries based on the risk difference are easily interpretable and ideal for conveying the clinical effect of a treatment. The risk difference provides an unbiased estimate of treatment effect, has been shown to be particularly appropriate in examining rare event data, and can accommodate trials with no events.22 23 For comparison, we also summarised the study effects with the relative risk, Mantel-Haenszel odds ratio, and Peto odds ratio.

Methods

Literature search

Our systematic review entailed computer based searches of Medline, the Cochrane Library, and online clinical trials registries (ClinicalTrials.gov and the industry sponsored Clinical Study Results registry) to identify randomised controlled trials evaluating varenicline for tobacco cessation. The search covered January 2005 (the year when articles on varenicline were first published) to September 2011, and included articles available online ahead of print publication. The search strings were “varenicline and randomised” in Medline and “SR-tobacco and varenicline” in the Cochrane Central Register of Controlled Trials. We manually searched bibliographies of relevant research and review articles.

Studies included in the meta-analysis met the following criteria: randomised controlled design, study sample of current tobacco users of adult age, comparing use of varenicline with that of an inactive control, and reporting adverse events. Exclusion criteria included use of a quasi-experimental or crossover design, laboratory studies with no follow-up, studies with adolescents, studies of non-smokers, studies in which all participants received varenicline, and comparisons of varenicline with another active drug (for example, nicotine replacement). Study inclusion criteria, data extraction, and methods of the analysis were specified in advance and documented in a protocol.

Data extraction

Two reviewers independently conducted article data extraction and quality assessment for each study meeting the inclusion criteria. Data extraction included descriptive characteristics of the study samples; varenicline dose; duration of drug treatment; study duration; sample attrition; and the number of participants with treatment emergent, cardiovascular serious adverse events in the inactive and active drug conditions. For comparability, we used the same primary outcome as Singh and colleagues, defining cardiovascular serious adverse events as any ischaemic or arrhythmic adverse cardiovascular event (myocardial infarction, unstable angina, coronary revascularisation, coronary artery disease, arrhythmias, transient ischaemic attacks, stroke, sudden death or cardiovascular related death, or congestive heart failure).13

We obtained details on timing of the cardiovascular serious adverse events from the study publications; the online clinical trials registries; the Chantix product label; and for three trials, for which the information was not publicly available, we contacted the medical adviser for varenicline at Pfizer11 24 or the study lead author directly.25 For one additional study, we contacted the lead author to confirm that no serious adverse events had occurred in the trial because only adverse events were reported.26 Discrepant findings between reviewers were settled by discussion, further review of the article, and (if necessary) consultation with a third reviewer.

We assessed study quality using a three item method developed by Jadad and colleagues27 that evaluated adequacy of randomisation, concealment of randomisation, and completeness of follow-up. We assessed two additional items, relevant to the area of interest: adequate reporting and adjudication of the cardiovascular serious adverse events. No quality scoring system has proven to correlate consistently with treatment outcomes, and it is recognised that general quality scales often need to be supplemented with more problem specific items for each particular meta-analysis.28

Statistical analysis

We described trial characteristics in terms of publication date; sample size and allocation; and participants’ exposures to tobacco, cardiovascular disease, and study treatments. For each trial included in the meta-analysis, we cross classified study participants by treatment group and by any occurrence of a treatment emergent, cardiovascular serious adverse event. For trials that examined multiple doses of varenicline, we combined the active treatment groups.

Using Comprehensive Meta-Analysis (version 2 professional edition; Biostat) and fixed effects estimation, we summarised evidence of an increased risk of cardiovascular serious adverse events associated with varenicline use via four summary statistics: the Mantel-Haenszel versions of the risk difference, relative risk, and odds ratio; and the Peto odds ratio.29 30 For each statistic, we reported the mean effect, 95% confidence interval, P value testing the null hypothesis of no effect, and I2 statistic estimating heterogeneity across trials.31 We confirmed all results using the “metan” and “funnel” routines added to Stata version 11. The three relative statistics excluded trials in which event counts were zero in both arms.32 For trials in which one event count was equal to zero, both software packages (Stata and Comprehensive Meta-Analysis) added 0.5 to each of the four cell counts before estimating the relative risk or Mantel-Haenszel odds ratio.

We planned to do random effects meta-analyses and subgroup analyses if levels of heterogeneity exceeded 50%. We plotted the cumulative evidence by date of trial publication33 and examined asymmetry in a funnel plot as an indicator of publication bias. Finally, we compared the four summary statistics at the trial level grouping the studies by presence (v absence) of events and equal (v unequal) numbers of events, ordering the groups by increasing evidence of a varenicline effect. We expected this comparison would provide insight into the most suitable summary statistic in the present setting.

Results

Study characteristics

The Medline search yielded 133 citations; of these, 21 met the inclusion criteria, which were all published in 2006 or later (fig 1). We identified one additional unpublished study from the online clinical trials registries. Searching in the Cochrane Central Register of Controlled Trials yielded 83 results which, along with manual searches of the bibliographies of relevant research and review articles, did not yield any additional studies.

Fig 1 Literature search results and study selection

In total, we identified 22 randomised controlled trials of varenicline use for tobacco cessation; all were double blind and placebo controlled, and collectively included 9232 participants (5431 randomised to varenicline, 3801 to placebo; table 1). The median overall sample size was 404 (range 31-1210). Seven trials had notable imbalances in sample size by condition—four allocated participants to varenicline and placebo groups in a 2:1 ratio, and three studied multiple varenicline doses or regimens. In 21 trials, the varenicline dose was 1 mg twice daily; three of these trials also studied lower doses of varenicline (table 1). The median duration of study treatment was 12 weeks, the median duration of follow-up for treatment emergent, cardiovascular serious adverse events was 16 weeks, and the median duration of the study period after randomisation was 25 weeks.

Table 1

 Characteristics of randomised controlled trials of varenicline included in the meta-analysis

Study	N	Male participants (%)	White participants (%)	No of cigarettes per day (mean)	Duration of tobacco use (years; mean)	Cardiac exclusions	Varenicline dose	Duration of treatment (weeks)	Study duration (weeks)
Rigotti et al11	714	78.7	80.5	22.5	39.5	Cardiovascular procedure in past two months or cardiovascular instability (myocardial infarction or unstable angina), uncontrolled hypertension, significant neurological sequelae of cerebrovascular disease, peripheral vascular disease with previous amputation, or severe congestive heart failure	1 mg twice daily	12	52
Steinberg et al25	79	59	72	20.0	Not reported	No cardiac exclusions, 57% of sample had a diagnosis at admission related to cardiovascular disease	1 mg twice daily	12	24
Protocol A3051072	128	76.6	59.4	23.1	24.1	Clinically significant cardiovascular disease in past six months	1 mg twice daily	12	24
Bolliger et al37	593	60.4	30.6	23.8	25.7	Serious or unstable disease in past six months (for example, cardiovascular disease, cerebrovascular disorders), uncontrolled hypertension or systolic blood pressure >150 mm Hg or diastolic blood pressure >95 mm Hg	1 mg twice daily	12	24
Ebbert et al34	76	100	97.4	Not applicable	18.8	Unstable angina, myocardial infarction in past three months, hypertension with systolic blood pressure ≥200 mm Hg or diastolic blood pressure ≥100 mm Hg, cardiac dysrhythmia other than drug controlled atrial fibrillation or paroxysmal supraventricu­lar tachycardia	1 mg twice daily	12	24
Garza et al41	110	66.4	75.5	22.3	16.9	Unstable cardiovascular disease in past six months	1 mg twice daily	12	16
Gonzales et al42	696	52.0	77.9	21.3	24.5	Clinically significant cardiovascular disease in past six months; uncontrolled hypertension	1 mg twice daily	12	52
Jorenby et al43	685	56.6	85.3	22	25.8	Clinically significant cardiovascular disease in past six months; uncontrolled hypertension or systolic blood pressure >150 mm Hg or diastolic blood pressure >95mm Hg	1 mg twice daily	12	52
Rennard et al44	659	60	“Mostly white”	Not reported	Not reported	Unstable serious disease in past six months (cardiovascular disease not specified)	1 mg twice daily	12	24
Tashkin et al45	504	62.3	83.0	24.4	40.5	Abnormal electrocardiogram, unstable cardiovascular disease or history of disease in past six months; uncontrolled hypertension or systolic blood pressure >160 mm Hg or diastolic blood pressure >95 mm Hg; cerebrovascular events (for example, stroke, transient ischaemic attack) in past six months	1 mg twice daily	12	52
Tonstad et al36	1210	49.3	96.9	20.7	28.1	Cardiovascular disease within past six months, uncontrolled hypertension	1 mg twice daily	12	40
Wang et al46	333	96.7	0	20.8	20	Cardiovascular or cerebrovascular disease in past six months, uncontrolled hypertension or systolic blood pressure >150 mm Hg or diastolic blood pressure >95 mm Hg) at baseline	1 mg twice daily	12	24
Williams et al47	377	49.9	88.6	23.3	30.4	Cardiovascular disease in past six months, history of hypertension	1 mg twice daily	52	53
Fagerstrom et al24	432	89.3	99.3	Not applicable	21	Any serious medical condition (cardiovascular disease and timing not specified)	1 mg twice daily	12	26
Hong et al26	41	66	Not reported	17.7	18.8	Major medical conditions, atrioventricular block identified on electrocardiogram (timing not specified)	1 mg once daily	8	10
Hughes et al48	218	59.1	91.7	18.5	26.3	No current or history of medical problems that would place the participant at substantial risk of an adverse event	1 mg twice daily	2-8	24
Nakamura et al20	619	74.8	0	23.9	20.9	History of cardiovascular and cerebrovascular disease	0.25 mg twice daily, 0.5 mg twice daily, 1 mg twice daily	12	40
Niaura et al49	320	51.9	90.7	22.2	25.3	History of cardiovascular disease, abnormal electrocardiogram, systolic blood pressure >160 mm Hg or diastolic blood pressure >95 mm Hg	1 mg twice daily	12	52
Nides et al50	510	49	76	20.2	24.3	History of cardiovascular disease, uncontrolled hypertension	0.3 once daily, 1 mg once daily, 1 mg twice daily	6	52
Oncken et al51	647	49.5	80.5	20.9	25.2	Cardiovascular disease (timing not specified)	0.5 mg twice daily, 1 mg twice daily*	12	52
Poling et al52	31	80.6	61.3	18.7	Not reported	Major cardiovascular disease (timing not specified)	1 mg twice daily	12	12
Tsai et al53	250	88.8	0	23.1	21.1	Present or history of cardiovascular or cerebrovascular disease	1 mg twice daily	12	24

Studies ordered according to cardiovascular disease inclusions: current disease, past disease, no history of disease, or timing not specified.

*Two regimens per dose were studied, fixed and progressive.

The samples tended to have a majority of male and white participants. Two trials studied smokeless tobacco users24 34 and 20 studied cigarette smokers. Among the trials of cigarette smokers, participants averaged 21.5 cigarettes (standard deviation 1.9) per day at study screening and 25.1 years (6.3) of tobacco use. Thirteen trials included patients with current or past cardiovascular disease. Of these trials, one was conducted among smokers admitted to hospital, of whom 57% had an admitting diagnosis that was cardiovascular25; another was conducted specifically among people with stable cardiovascular disease11; and 11 trials included people with a past cardiovascular event (table 1). Nine trials excluded people with any history of cardiovascular disease or the timing for exclusion was not specified. Study quality was strong overall, with all 22 trials being of double blind design and providing adequate descriptions of randomisation, loss to follow-up, and cardiovascular serious adverse events. However, only one trial adjudicated the serious adverse events.11

Risk of treatment emergent, cardiovascular serious adverse events

Across the 22 studies, the crude rates of treatment emergent, cardiovascular serious adverse events were 0.63% (34/5431) for the varenicline group and 0.47% (18/3801) for the placebo group. No events occurred in eight trials, including three trials with more than 100 participants per arm. The summary risk difference was 0.27% (−0.10% to 0.63%, P=0.15, I2=0%; fig 2), with no indication of publication bias in the funnel plot. For comparison, based on 14 studies with at least one event, the relative risk was 1.40 (0.82 to 2.39, P=0.22, I2=0%; table 2), the Mantel-Haenszel odds ratio was 1.41 (0.82 to 2.42, P=0.22, I2=0%), and the Peto odds ratio was 1.58 (0.90 to 2.76, P=0.11, I2=0%).

Fig 2 Difference in risk of treatment emergent, cardiovascular serious adverse events associated with varenicline use in 22 double blinded, placebo controlled, randomised trials. Studies grouped by presence (v absence) of events and equal (v unequal) numbers of events with groups ordered by increasing evidence of a varenicline effect

Table 2

 Risk of treatment emergent, cardiovascular serious adverse events associated with varenicline use in double blind, placebo controlled randomised trials of tobacco cessation

Study	Events/randomised participants (no)		Allocation ratio between study groups	Treatment effect (95% confidence interval)
	Varenicline	Placebo		Peto odds ratio	Odds ratio	Relative risk	Risk difference (%)
Fagerstrom et al24	0/214	1/218	1:1	0.14 (0.00 to 6.95)	0.34 (0.14 to 8.34)	0.34 (0.01 to 8.29)	−0.46 (−1.73 to 0.81)
Rennard et al44	0/493	0/166	3:1	UC	UC	UC	0 (−0.87 to 0.87)
Protocol A305107240	0/85	0/43	2:1	UC	UC	UC	0 (−3.52 to 3.52)
Hong et al26	0/20	0/21	1:1	UC	UC	UC	0 (−9.00 to 9.00)
Ebbert et al34	0/38	0/38	1:1	UC	UC	UC	0 (−4.99 to 4.99)
Garza et al41	0/55	0/55	1:1	UC	UC	UC	0 (−3.48 to 3.48)
Hughes et al48	0/107	0/111	1:1	UC	UC	UC	0 (−1.78 to 1.78)
Wang et al46	0/165	0/168	1:1	UC	UC	UC	0 (−1.17 to 1.17)
Poling et al52	0/13	0/18	0.7:1	UC	UC	UC	0 (−12.10 to 12.10)
Steinberg et al25	1/40	1/39	1:1	0.98 (0.06 to 15.87)	0.97 (0.06 to 16.15)	0.98 (0.06 to 15.05)	−0.06 (−0.07 to 6.87)
Jorenby et al43	1/344	1/341	1:1	0.99 (0.06 to 15.88)	0.99 (0.06 to 15.91)	0.99 (0.06 to 15.78)	0.00 (−0.81 to 0.81)
Gonzales et al42	2/352	2/344	1:1	0.98 (0.14 to 6.97)	0.98 (0.14 to 6.98)	0.98 (0.14 to 6.90)	−0.01 (−1.14 to 1.11)
Rigotti et al11	10/355	10/359	1:1	1.01 (0.42 to 2.46)	1.01 (0.42 to 2.46)	1.01 (0.43 to 2.40)	0.03 (−2.39 to 2.45)
Oncken et al51	2/518	0/129	4:1	3.49 (0.11 to 112.44)	1.25 (0.06 to 26.27)	1.25 (0.06 to 25.93)	0.39 (−0.83 to 1.61)
Nides et al50	1/383	0/127	3:1	3.79 (0.04 to 352.09)	1.00 (0.04 to 24.70)	1.00 (0.04 to 24.39)	0.26 (−0.99 to 1.51)
Nakamura et al20	1/465	0/154	3:1	3.79 (0.04 to 352.44)	1.00 (0.04 to 24.62)	1.00 (0.04 to 24.37)	0.22 (−0.82 to 1.25)
Bolliger et al37	1/394	0/199	2:1	4.50 (0.07 to 285.96)	1.52 (0.06 to 37.51)	1.52 (0.06 to 37.12)	0.25 (−0.67 to 1.17)
Tsai et al53	1/126	0/124	1:1	7.27 (0.14 to 366.57)	2.98 (0.12 to 73.76)	2.95 (0.12 to 71.79)	0.79 (−1.39 to 2.97)
Niaura et al49	2/160	0/160	1:1	7.44 (0.46 to 119.40)	5.06 (0.24 to 106.30)	5.00 (0.24 to 103.33)	1.25 (−0.84 to 3.34)
Tonstad et al36	2/603	0/607	1:1	7.45 (0.47 to 119.26)	5.05 (0.24 to 105.41)	5.03 (0.24 to 104.62)	0.33 (−0.23 to 0.89)
Williams et al47	6/251	1/126	2:1	2.40 (0.49 to 11.67)	3.06 (0.37 to 25.71)	3.01 (0.37 to 24.75)	1.60 (−0.85 to 4.04)
Tashkin et al45	4/250	2/254	1:1	1.99 (0.40 to 9.95)	2.05 (0.37 to 11.29)	2.03 (0.38 to 10.99)	0.81 (−1.08 to 2.71)
All trials combined	34/5431	18/3801	—	1.58 (0.90 to 2.76)	1.41 (0.82 to 2.42)	1.40 (0.82 to 2.39)	0.27 (−0.10 to 0.63)

Studies grouped by presence (v absence) of events and equal (v unequal) numbers of events; groups ordered by increasing evidence of a varenicline effect. UC=unable to calculate using Peto odds ratio, Mantel-Haenszel odds ratio, and relative risk because no events in either group.

Neither the individual trials nor the summary estimates showed a significant treatment effect; consequently, we did not calculate the number needed to harm.35 We found no evidence of heterogeneity according to the I2 statistic or the cumulative estimated effect of varenicline on cardiovascular serious adverse events (fig 3).

Fig 3 Cumulative estimated difference in risk of cardiovascular serious adverse events attributable to varenicline use. Studies ordered by publication year

We conducted four sensitivity analyses excluding trials of participants with active cardiovascular disease11 25 (risk difference 0.29% (95% confidence interval −0.04 to 0.62), I2=0%); trials of smokeless tobacco users24 34 (0.31% (−0.07 to 0.69), I2=0%); one trial in which all participants were initially exposed to varenicline and then randomised to placebo or to a 12 week maintenance phase of varenicline36 (0.26% (−0.16 to 0.67, I2=0%)); and one unpublished trial40 (0.27% (−0.10 to 0.64, I2=0%)). All four sensitivity analyses differed minimally from the full analysis.

After grouping the trials by presence (v absence) of events and equal (v unequal) numbers of events, we obtained five groups (table 2, fig 2). Group 1 included the one trial with more cardiovascular serious adverse events in the placebo arm than in the varenicline group. Group 2 had no events in either arm. Group 3 had an equal number of events per arm. Group 4 had one or more events in the varenicline arm but none on the placebo arm. Finally, group 5 had events occurring in both arms, with more occurring in the varenicline arm.

For group 2 (no event on either arm; eight trials), “no effect” was estimated by a risk difference of 0%, but not by relative effects equal to 1, because the relative summary statistics exclude trials with no events. For groups 3 and 5 (at least one event on each arm; six trials), the evidence for and against the null hypothesis of no treatment effect was similar across the four statistics in group 3, but the Peto odds ratio seemed to underestimate treatment effects in group 5.

For every trial in groups 1 and 4 (eight trials), in terms of both mean effects and confidence intervals, the Peto odds ratio was far stronger than the relative risk, but the Mantel-Haenszel odds ratio was nearly identical to the relative risk. In all eight trials, the Peto odds ratio exceeded 3.4 (or its inverse) despite very few events, large sample sizes, and differences of only one to two events by arm. For example, the trial by Bolliger and colleagues37 saw one event among 394 participants receiving varenicline (0.25%) compared with no events among 199 participants receiving placebo (0%) (risk difference 0.25%, relative risk of 1.52, Mantel-Haenszel odds ratio of 1.52, Peto odds ratio of 4.50).

Discussion

This meta-analysis included all published randomised, placebo controlled trials of varenicline for tobacco cessation, examined events occurring during drug exposure or within 30 days of discontinuation, and analysed findings via four summary measures. None of these measures identified a significantly elevated risk of treatment emergent, cardiovascular serious adverse events with varenicline use, and the 95% confidence interval of the risk difference excluded an increase larger than 0.63%. In response to the FDA’s call for analysis of cardiovascular serious adverse events attributed to varenicline use, this meta-analysis of 22 independent trials and more than 9000 individuals had high power to detect a significant treatment effect and found negligible variation in the evidence across the trials.

Study participants tended to be chronic, heavy smokers, averaging more than one pack of cigarettes a day for more than two decades, placing them at elevated risk of serious adverse events related to cardiovascular disease. Most trials included individuals with current (two trials) or past (11 trials) cardiovascular disease. More than a third of studies did not observe a cardiovascular serious adverse event; among these trials, five of eight included participants with past cardiovascular disease.34 40 41 44 46

Comparison with other studies

In their meta-analysis of the safety of varenicline, Singh and colleagues reported a Peto odds ratio of 1.72 (95% confidence interval 1.09 to 2.71) and concluded that “The use of varenicline among tobacco users was associated with a 72% increased risk of serious adverse cardiovascular events.” They questioned the safety of this medication,13 and in subsequent press interviews called for withdrawal of varenicline from the market.38 The researchers’ analysis of cardiovascular serious adverse events at any time during the trial duration was, on average, twice the duration of study drug exposure and did not account for longer follow-up in the varenicline group than in the placebo group.

The discrepancy between the conclusions of our meta-analysis and those of Singh and colleagues’ study is explained not only by differing periods over which events were collected but also by the choice of statistics used to summarise the results, which affected the trials included in the meta-analyses. For direct comparison in our study, we analysed data from all 22 trials using the full study follow-up, because the observation period of interest and the calculated risk difference was 0.47% (95% confidence interval 0.04 to 0.91), far less inflammatory than the previous meta-analysis’s reported risk based on the Peto odds ratio, yet still biased in favour of finding an effect due to the differential inclusion of events related to disease and not just treatment. By contrast, our calculated risk difference of treatment emergent, cardiovascular serious events was 0.27% (−0.10 to 0.63).

Choice of summary statistics

When study participants are selected by outcome status (for example, case or control), an odds ratio must be used to summarise an association with exposure status. However, when participants are selected by exposure status (for example, active or placebo treatment), a risk difference or relative risk can be used. These statistics are more natural choices for randomised trials because they explicitly estimate and contrast effects of interest—namely, event rates in the active and placebo arms. Treatment effects based on relative risks always are as or less extreme than those based on odds ratios.30 The (absolute) risk difference has a further advantage because it can be calculated for trials in which zero events occur, whereas relative statistics cannot be calculated for these trials and therefore can bias summaries against the null hypothesis of “no effect.”

Furthermore, relative statistics are unitless, which hides the fact that a low response rate remains very low even when scaled up by a seemingly large effect; by contrast, the risk difference retains the units of the measurement scale, showing that a difference between low response rates is itself very small. Vandermeer and colleagues’ comprehensive reanalysis of findings from 1613 meta-analyses of safety data indicated that the Peto odds ratio statistic was particularly biased.22 We further demonstrate that, regardless of sample size allocation, when all events are in one study arm, Mantel-Haenszel odds ratios match relative risks well whereas Peto odds ratios are far more extreme. For clinical considerations and in the setting of rare events, the risk difference most clearly conveys the relevant effect.22 23 39

Conclusions and clinical implications

Meta-analysis is an important analytical technique for synthesising treatment effects across trials for maximum power and is particularly useful for analysis of serious adverse events, which can occur with low frequency. Bias in methods, however, is a real concern. Our comparison of four summary statistics identified conditions under which the Peto odds ratio produced extreme estimates that did not reflect the underlying event rates, and identified cases in which it produced smaller estimates than it theoretically should have done. Our results accord with other reports that the Peto statistic can lead to incorrect conclusions.22 29 The consequence of inflated risk estimates, such as those from Singh and colleagues’ meta-analysis concerning the effect of varenicline on serious adverse events related to cardiovascular disease,13 can be unnecessary public alarm and real harm, since patients may discontinue their drug treatment out of fear of adverse effects and clinicians may recommend cessation treatments of reduced efficacy or discourage use of the drug treatment altogether.

Smoking is the leading preventable cause of death worldwide. Half of long term smokers die from their tobacco use, and smokers die from cardiovascular disease more than from any other cause.1 2 3 Varenicline is a first line treatment for quitting smoking, and quitting smoking is central to the prevention of cardiovascular disease. Our meta-analysis of treatment emergent, cardiovascular serious adverse events, with attention to bias and critical design issues, indicates that the risk of these events associated with varenicline use is small, and statistically and clinically insignificant.

There have been drug safety concerns about the use of varenicline for tobacco cessation and the emergence of cardiovascular serious adverse events

However, this association has since been called into question, owing to less than optimal methodology used, and the US FDA has called for further analysis

Our meta-analysis of all published, randomised controlled trials of varenicline use for tobacco cessation included 50% more studies than a previous meta-analysis by Singh and colleagues; used an unbiased summary estimate and compared findings with three other estimates; and examined events that occurred during drug treatment, which is more biologically relevant and obviates problems with differential drop out

All four summary estimates indicated no significant increase in the risk of treatment emergent, cardiovascular serious adverse events attributed to varenicline use