Diminishing benefit of smoking cessation medications during the first year: a meta-analysis of randomized controlled trials.

BACKGROUND AND AIMS: Although smoking cessation medications have shown effectiveness in increasing abstinence in randomized controlled trials (RCTs), it is unclear to what extent benefits persist over time. This paper assesses whether the benefits of smoking cessation medications decline over the first year.
METHODS: We selected studies from three systematic reviews published by the Cochrane Collaboration. RCTs of first-line smoking cessation medications, with 6- and 12-month follow-up, were eligible for inclusion. Meta-analysis was used to synthesize information on sustained abstinence (SA) at 6 versus 12 months and 3 versus 6 months, using the risk difference (RD) ('net benefit') between intervention and control group quit rates, the relative risk (RR) and the odds ratio (OR).
RESULTS: Sixty-one studies (27 647 participants) were included. Fewer than 40% of intervention group participants were sustained abstinent at 3 months (bupropion: 37.1%; nicotine replacement therapy (NRT): 34.8%; varenicline: 39.3%); approximately a quarter were sustained abstinent at 6 months (bupropion: 25.9%; NRT: 26.6%; varenicline: 25.4%), and approximately a fifth were sustained abstinent at 12 months (bupropion: 19.9%; NRT: 19.8%%; varenicline: 18.7%). There was only a small decline in RR (3 months: 1.95 [95% confidence interval (CI) = 1.74-2.18, P < 0.0001]; 6 months: 1.87 (95% CI = 1.67-2.08 P < 0.0001); 12 months: 1.75 (95% CI = 1.56-1.95, P < 0.0001) between intervention and control groups over time, but a substantial decline in net benefit [3 months: RD = 17.3% (14.5-20.1%); 6 months: RD = 11.8% (10.0-13.7%); 12 months: RD = 8.2% (6.8-9.6%)]. The decline in net benefit was statistically significant between 3 and 6 [RD = 4.95% (95% CI = 3.49-6.41%), P < 0.0001] and 6 and 12 months [RD = 3.00% (95% CI = 2.36%-3.64%), P < 0.0001)] for medications combined and individual medications.
CONCLUSIONS: The proportion of smokers who use smoking cessation medications who benefit from doing so decreases during the course of the first year, but a net benefit still remains at 12 months.

Introduction

Tobacco use is the most common cause of preventable death in the world today 1, 2, causing more than 7 million deaths per year 3. The fastest way to reduce the burden of smoking is through smoking cessation by current smokers 4. The effectiveness of US FDA‐recommended first‐line smoking cessation medications 5 is undisputed when considering evidence from randomized controlled trials (RCTs). Systematic reviews and meta‐analyses conducted by the Cochrane Collaboration 6, 7, 8, the US Public Health Services, 5 England's National Institute for Health and Care Excellence (NICE) 9 and others 10, 11, 12, 13 show consistently that smoking cessation medications help smokers quit.

However, the magnitude of benefit to individual smokers and populations is less well understood. There are two sources of confusion. First, it is possible that the benefits of medications decline, even in the context of RCTs. Many smokers who quit relapse 14, 15, 16, 17, leading to declines in quit rates over time 18, 19, 20, 21. This is well illustrated by a recent study (n = 1086), which found initial quit rates of at least 24 hours’ duration to be 70–80%, with declines over time to approximately 20% for varenicline, nicotine patch and combination nicotine groups by 1 year 22. Original reports of studies often describe quit rates at a single time‐point, with 6 months used as a standard, on the grounds that it is possible to ‘make reliable estimates of permanent cessation’ on the basis of 6‐month continued abstinence 23. Reviews and meta‐analyses usually provide a single summary measure based on quit rates at a single time‐point, or as a summary based on several time‐points (Cochrane Reviews: 6 months or more, 6–24 months 6, 7, 8; the US Public Health Service: 6 months 5; independent reviewers: 6 or 12 months 10, 12.). Representing smoking cessation at a single or composite time‐point obscures the more complex reality of declining quit rates: neither the benefit to an individual as time progresses nor to a population is easily grasped. The problem is compounded because few studies continue beyond 1 year 6, 7, 8.

The second reason for confusion is the nearly exclusive use of the relative risk (RR) (the ratio of the probability of success in intervention versus control groups) or odds ratio (OR) (the ratio of the odds of success in intervention versus control groups) to describe intervention effects. These measures have been shown to be stable over time and settings in smoking cessation 24, 25. However, like all summary measures, they provide only partial information. An RR of 2 indicates that medication doubles the probability of quitting; it could equally well describe an intervention group quit rate of 2% and a control group quit rate of 1% or an intervention quit rate of 80% and a control quit rate of 40%. These represent vastly different scenarios in terms of benefit to individuals or populations. The risk difference (RD), another important summary measure, overcomes this problem: it presents differences in probabilities of success between intervention and control groups, allowing consideration of the magnitude of benefit. The corresponding RDs for our examples are 1 and 40%, respectively. However, the RD is used far less often than the RR or the OR. Cochrane Reviews generally present RRs in the results of smoking cessation medication reviews 6, 7, 8, while NICE 9 and others 10, 11, 12, 13 present ORs. The European guideline on cardiovascular disease prevention describes relative benefit based on the OR: ‘Overall, NRT and bupropion help ∼80% more people to quit than placebo’ (26, p. 92). Information on the percentage of people who benefit from treatment is often absent 26 or unclear 7, 8.

In this study, we assessed effects and changes in effects of cessation medication over time from published RCTs, with several complementary measures. Using studies included in Cochrane Collaboration Systematic Reviews, we present quit rates, RRs, RDs, and ORs from RCTs of first‐line smoking cessation medications at three time‐points (3, 6 and 12 months), and test whether benefits decreased over time.

Methods

Our primary goal was to examine changes in effectiveness of Food and Drug Administration (FDA) first‐line smoking cessation medications [bupropion, nicotine replacement therapy (NRT) (all five approved forms) and varenicline] 5 over time among participants in RCTs. The specific objectives were (1) to compare the effects of the intervention versus control on the proportion of sustained abstainers (SA) from smoking at 6 and 12 months, and on the proportion of people who were quit at 6 and 12 months [point prevalence quit rate (PP)] and (2) to measure the interaction effect (i.e. the change) with time across these two time‐points (6 and 12 months), on the scale of risk differences, relative risks and ORs for the medications. A secondary goal was to examine changes from 3 to 6 months.

Data sources and study eligibility

We used the Cochrane network meta‐analysis of reviews on smoking cessation 27 to identify relevant Cochrane Reviews which assessed first‐line smoking cessation medications recommended by the US Public Health Service 2008 5. Three relevant reviews were found: Stead 8, Cahill 6 and Hughes 7. All studies included in the reviews were considered for inclusion in this meta‐analysis.

Eligibility criteria were as follows: (1) Intervention—the drug must have been a first‐line smoking cessation medication, as defined by the US Public Health Services in 2008: bupropion, one of five forms of NRT (gum, inhaler, nasal spray, lozenge or patch) or varenicline 5; (2) Goal—the goal of the intervention must have been smoking cessation (not merely smoking reduction); (3) Study design, comparison—the study must have been a RCT comparing intervention and control groups, in which the treatment arms differed only on provision of the medication being tested and in which the control group received no active medication, including low‐dose medication. Trials were not required to be placebo‐controlled; (4) Population—the target population must have been non‐pregnant adults; (5) Outcome—data on quit rates must have been collected at 6 and 12 months, with reporting at both times of either point prevalence (PP) or sustained or continuous abstinence (SA) rates; (6) Reporting—the trial results must have been published in a peer‐reviewed paper, data must have been reported in numerical form and at least 95% of randomized participants must have been included in the analyses. Assuming that dropouts were continuing smokers did not preclude inclusion; and (7) Language—the paper must have been published in English. When more than one eligibility criterion was not met, we assigned a single reason for exclusion, with priority given to the earlier number on this list.

Data extraction

Relevant studies were identified independently by three authors (L.J.R, J.R.K, M.A.G) and then compared. Data were extracted independently by the same three authors and compared. Differences were resolved with discussion. No inter‐rater reliability of coding was performed.

Intent‐to‐treat (ITT) approach

We followed the assumption made by most investigators that individuals lost to follow‐up be considered continuing smokers 10, 28, 29. We found that this definition was not applied uniformly in the field: some researchers included only participants who received an initial dose of medication, or excluded participants for non‐compliance with protocol or other reasons. We applied a strict, uniform, ITT approach, using the following method: for all quit rates, the number randomized to treatment or control was the denominator. For numerators, the number of quitters was used if provided. Otherwise, if the authors had specified an ITT analysis, we back‐calculated the numerator based on the reported quit rate and the number randomized to the group and rounded to the nearest whole number. If a number other than the number randomized was reported as the denominator, we used that denominator to back‐calculate the numerator, rounded to the nearest whole number, and then divided by the number randomized to that group.

Comparisons

Some studies included more than two treatment groups. For 2 × 2 factorial designs that included a medical and a non‐medical (behavioural/ psychological) intervention, two separate comparisons were performed: one for the active non‐medical intervention and one for the inactive non‐medical intervention. In cases where high‐ and low‐dependent smoker subgroups were randomized separately to intervention and control arms, each subgroup was treated in the meta‐analysis as a separate study. In studies with multiple doses and a single control, participants assigned any non‐zero dose were combined into a single intervention group. For trials with multiple intervention arms which did not have a clear 2 × 2 design, we compared the two groups that were identical on all aspects of the intervention except for provision of medication. Information on included treatment groups is presented in Supporting information, Table S1.

Outcomes

The primary outcome was sustained abstinence (SA) from smoking. We used SA to denote any measure of long‐term abstinence including, but not restricted to, the terms ‘sustained abstinence’, ‘prolonged abstinence’ (PA) or ‘continuous abstinence’ (CA). We did not differentiate between whether slips were allowed or whether a grace period was allowed. The secondary outcome was point prevalence (PP) of smoking. Quit rates could be based on biochemical validation or self‐report. When both were available, the former were used. We also performed a sensitivity analysis including only biochemically validated SA quit rates.

Statistical methods

Linear random‐effects meta‐analysis models with restricted maximum‐likelihood estimation were used for estimation and testing. This method essentially estimates separate ‘effects’ from each treatment–control comparison and combines them using a weighting that reflects the relative precision of each estimate. Across‐study quit rates (SA and PP) for different treatment groups were estimated and plotted over time. Each of the three drugs (NRT, varenicline, bupropion) was compared to the control at each time‐point (3, 6 and 12 months) using three different measures: the RD, i.e. the difference in the proportion of quitters, RR, i.e. the ratio of the proportion of quitters and OR, i.e. the ratio of the odds of quitting. The interaction with time, i.e. the change over time, was measured as the difference between RDs or between log RRs at two time‐points (principally 6 versus 12 months, but also 3 versus 6 months). The log RR is used in such analyses to allow the application of the linear random‐effects model. The asymptotic variance of the interaction estimate was developed analytically and included the dependency between two sustained abstinence rates measured in the same cohort at different time‐points 30. Detailed formulae can be found in the Supporting information, Statistical Supplement. Pooled estimates of the change in RD and log RR over time and their standard errors and P‐values were obtained. For RRs, estimates and CIs were obtained by back‐transforming from the logarithmic to the original scale. Sensitivity analysis of the test for interaction (change over time) used the r‐value, the maximum P‐value obtained when leaving out one of the studies from the meta‐analysis 31. This provides protection against drawing conclusions that are based on the results of a single dominant study. The analysis was conducted in the R environment 32 using the packages metaphor 33 for meta‐analysis, ggplot2 34 for visualization, and other packages for reproducible research (including knitr, readxl, dplyr, tidyr, broom).

Due to weighting procedures implicit in the random‐effects model, the differences in RDs across time may not be numerically identical to those found using simple subtraction of differences between quit rates from one time‐point to the next.

We also calculated the number of people needed to treat (NNT) (the inverse of the absolute risk difference) to obtain one extra sustained quitter at different times after start of treatment, using the standard convention of rounding‐up to a positive whole number 28.

Pre‐registered hypothesis

The hypotheses for this paper were registered in the Open Science Framework prior to running the analyses (https://osf.io/n5h4g/).

Results

A total of 269 studies (NRT: 155, bupropion: 90, varenicline: 24) were listed in the ‘Studies included in this review’ sections of the three Cochrane Reviews. Two hundred and four studies (NRT: 114, bupropion: 72, varenicline: 18) were excluded from this analysis. Of the 65 studies which met our inclusion criteria (18 bupropion trials, 41 NRT trials and six varenicline trials), four (Gonzales 2006, Jorenby 1999, Jorenby 2006, Nides 2006) were included in two reviews, leaving 61 distinct studies (n = 27 647 participants). Forty‐nine studies, including 19 549 participants, were included in the main SA analysis (6–12 m). In each of three NRT studies and two bupropion studies, there were two comparisons which were treated as separate studies in the meta‐analysis. Forty‐two studies, including 18 073 participants, were included in the secondary SA analysis (3–6 m).

Reasons for exclusion were: time (data on either SA or PP were not available at both 6 and 12 months) (71 studies), design (the trial was not an RCT with a comparable non‐medicated control group) (68 studies), medication (the trial did not test bupropion, NRT or varenicline) (27 studies), reporting (the paper was not published in a peer‐reviewed journal, or data on quit rates and number randomized were either not reported or not reported in numerical form (e.g. figures only) or there was incomplete reporting of participant outcomes) (24 studies), language (non‐English) (five studies), population (pregnant) (four studies), trial excluded by Cochrane (three studies) and goals (cessation was not the goal) (two studies). A flow‐chart, characteristics of included studies and reasons for exclusion can be found in Supporting information, Fig. S1 and Tables S1 and S2.

For PP, most studies reported biochemically validated quit rates. The exceptions were one bupropion study (Planer 2011) and three NRT studies (Fee 1982, Hughes 1989, Perng 1998). Authors described long‐term quitting in different ways, with no consistent distinction between use of the terms ‘continuous’, ‘prolonged’ and ‘sustained’. The most commonly used term, ‘continuous abstinence’ (CA) was used by all the varenicline trials, many of the bupropion trials and some of the NRT trials. Most CA trials allowed a grace period after beginning treatment. Authors approached biochemical validation of SA quit rates in different ways, usually with a mix of biochemical validation at observed time‐points and self‐report.

SA quit rates

SA quit rates over time are shown in Fig. 1. Mean quit rates were higher in intervention than control groups, and quit rates in almost all studies decreased over time. Of the intervention group participants, 36.5% were sustained abstinent at 3 months (bupropion: 37.1%; NRT: 34.8%; varenicline: 39.3%); 26.6% at 6 months (bupropion: 25.9%; NRT: 26.6%; varenicline: 25.4%) and 19.9% at 12 months (bupropion: 19.9%; NRT: 19.8%; varenicline: 18.7%). In the control groups, the sustained abstinence rates were 18.8% at 3 months, 14.3% at 6 months and 11.4% at 12 months. For all medications together, control group quit rates were just more than half of the intervention group quit rates at all time‐points, and 12‐month quit rates were a little more than half of the 3‐month quit rates. Quit rates overall and per medication can be found in Table 1.

Figure 1

Sustained abstinence quit rates (QRs), risk differences (RDs) and relative risks (RRs) over time and treatment arm, for individual medications. NRT = nicotine replacement therapy; ITT = intent‐to‐treat

Table 1

Sustained abstinence quit rates (QRs), risk differences (RDs), relative risks (RRs) and odds ratios (ORs) by time, overall and for individual medications.

Drug	Time	Quit rate intervention group (CI: lower, upper)	Quit rate control group (CI: lower, upper)	n Studies	Risk dDifference (CI: lower, upper), P‐value	Relative risk (CI: lower, upper), P‐value	Odds ratio (CI: lower, upper), P‐value
All	3 months	36.45% (33.12%, 39.78%)	18.84% (16.27%, 21.42%)	42	17.30% (14.54 20.07%) P < 0.0001	1.95 (1.74, 2.18) P < 0.0001	2.55 (2.21, 2.95) P < 0.0001
Bupropion	3 months	37.12% (31.30%, 42.94%)	20.23% (15.97%, 24.48%)	12	15.72% (12.29%, 19.16%) P < 0.0001	1.79 (1.57, 2.05) P < 0.0001	2.29 (1.96, 2.68) P < 0.0001
NRT	3 months	34.81% (30.33%, 39.29%)	19.38% (15.86%, 22.89%)	27	14.94% (11.77%, 18.11%) P < 0.0001	1.81 (1.59, 2.06) P < 0.0001	2.29 (1.94, 2.71) P < 0.0001
Varenicline	3 months	39.26% (31.08%, 47.44%)	13.3% (10.16%, 16.44%)	6	26.09% (19.01%, 33.17%) P < 0.0001	2.96 (2.36, 3.71) P < 0.0001	4.34 (3.20, 5.88) P < 0.0001
All	6 months	26.59% (24.10%, 29.08%)	14.29% (12.33%, 16.25%)	49	11.84% (9.95%, 13.73%) P < 0.0001	1.87 (1.67, 2.08) P < 0.0001	2.22 (1.95, 2.52) P < 0.0001
Bupropion	6 months	25.88% (21.77%, 29.99%)	15.42% (11.09%, 19.75%)	13	10.38% (7.26%, 13.50%) P < 0.0001	1.70 (1.40, 2.06) P < 0.0001	1.96 (1.58, 2.45) P < 0.0001
NRT	6 months	26.60% (23.08%, 30.13%)	14.56% (12.17%, 16.95%)	33	11.05% (8.56%, 13.53%) P < 0.0001	1.78 (1.57, 2.02) P < 0.0001	2.11 (1.80, 2.46) P < 0.0001
Varenicline	6 months	25.40% (19.99%, 30.80%)	9.43% (7.60%, 11.26%)	6	16.09% (11.78%, 20.41%) P < 0.0001	2.71 (2.29, 3.21) P < 0.0001	3.37 (2.77, 4.11) P < 0.0001
All	12 months	19.90% (17.96%, 21.84%)	11.43% (9.81%,13.05%)	49	8.19% (6.79%, 9.59%) P < 0.0001	1.75 (1.56, 1.95) P < 0.0001	1.97 (1.73, 2.23) P < 0.0001
Bupropion	12 months	19.88% (16.31%, 23.45%)	12.07% (8.68%, 15.45%)	13	7.53% (4.89%, 10.18%) P < 0.0001	1.63 (1.34, 1.98) P < 0.0001	1.82 (1.47, 2.25) P < 0.0001
NRT	12 months	19.84% (17.10%, 22.59%)	11.84% (9.80%, 13.88%)	33	7.28% (5.49%, 9.06%) P < 0.0001	1.65 (1.44, 1.89) P < 0.0001	1.84 (1.57, 2.17) P < 0.0001
Varenicline	12 months	18.65% (14.26%, 23.04%)	7.31% (5.75%, 8.86%)	6	11.31% (8.14%, 14.48%) P < 0.0001	2.56 (2.11, 3.12) P < 0.0001	2.97 (2.38, 3.71) P < 0.0001

CI = confidence interval; NRT = nicotine replacement therapy.

SA RDs, RRs and ORs

Differences in quit rates between intervention and control groups over time are presented in Fig. 1. Table 1 presents RDs, RRs and ORs of quit rates for intervention versus control groups over time.

There was a slight decline in RR between intervention and control groups over time [3 months: 1.95 (95% CI = 1.74–2.18, P < 0.0001); 6 months: 1.87 (95% CI = 1.67–2.08, P < 0.0001); 12 months: 1.75 (95% CI = 1.56–1.95, P < 0.0001)] and a substantial decline in net benefit, i.e. RD [(3 months: 17.3% (95% CI = 14.5–20.1%, P < 0.0001); 6 months: 11.8% (95% CI = 10.0–13.7%, P < 0.0001); 12 months: 8.2% (95% CI = 6.8–9.6%, P < 0.0001)]. These correspond to NNTs of 6, 9 and 13 patients, respectively, at 3, 6 and 12 months.

Table 2 presents the changes in RDs and RRs from 3 to 6 months and from 6 to 12 months. Over all medications, the estimated RD declined by 4.95% (95% CI = 3.49–6.41%, P < 0.0001) points from 3 to 6 months and by a further 3.0% (95% CI = 2.36–3.64%, P < 0.0001) points from 6 to 12 months.

Table 2

Changes over time in sustained abstinence quit rates: risk differences, relative risks and odds ratios, overall and for individual medications.

Drug	Comparison	n Studies/n participants	Risk difference		Odds ratio		Relative risk
			Estimate (CI)	P‐value, r‐value	Estimate (CI)	P‐value, r‐value	Estimate (CI)	P‐value, r‐value
All	3 versus 6	Studies: 42 Participants, control: 6825 Participants, intervention: 11 248 Participants, total: 18 073	4.95% (3.49%, 6.41%)	P < 0.0001 r < 0.0001	1.12 (1.05, 1.20)	P = 0.0008 r = 0.0020	1.03 (0.99, 1.08)	P = 0.1448 r = 0.2406
Bupropion	3 versus 6	Studies: 12 Participants, control: 1852 Participants, intervention: 2271 Participants, total: 4123	4.22% (2.17%, 6.28%)	P < 0.0001 r = 0.0009	1.09 (0.96, 1.25)	P = 0.1829 r = 0.3686	1.01 (0.92, 1.12)	P = 0.7674 r = 0.9990
NRT	3 versus 6	Studies: 27 Participants, control: 4200 Participants, intervention: 7133 Participants, total: 11 333	3.80% (2.28%, 5.32%)	P < 0.0001 r < 0.0001	1.09 (1.01, 1.18)	P = 0.0294 r = 0.0717	1.03 (0.97, 1.08)	P = 0.3883 r = 0.5736
Varenicline	3 versus 6	Studies: 6 Participants, control: 1585 Participants, intervention: 1844 Participants, total: 3429	9.79% (6.26%, 13.31%)	P < 0.0001 r < 0.0001	1.31 (1.13, 1.52)	P = 0.0003 r = 0.0066	1.10 (0.98, 1.23)	P = 0.1049 r = 0.3028
All	6 versus 12	Studies: 49 Participants, control: 7543 Participants, intervention: 12 006 Participants, total: 19 549	3.00% (2.36%, 3.64%)	P < 0.0001 r < 0.0001	1.12 (1.07, 1.17)	P < 0.0001 r < 0.0001	1.07 (1.04, 1.11)	P = 0.0001 r = 0.0002
Bupropion	6 versus 12	Studies: 13 Participants, control: 2052 Participants, intervention: 2463 Participants, total: 4515	2.65% (1.54%, 3.75%)	P < 0.0001 r < 0.0001	1.09 (1.01, 1.19)	P = 0.0327 r = 0.1481	1.05 (0.98, 1.13)	P = 0.1369 r = 0.4482
NRT	6 versus 12	Studies: 33 Participants, control: 4718 Participants, intervention: 7699 Participants, total: 12 417	2.64% (1.88%, 3.39%)	P < 0.0001 r < 0.0001	1.13 (1.07, 1.19)	P < 0.0001 r < 0.0001	1.08 (1.04, 1.13)	P = 0.0003 r = 0.0010
Varenicline	6 versus 12	Studies: 6 Participants, control: 1585 Participants, intervention: 1844 Participants, total: 3429	4.60% (3.06%, 6.14%)	P < 0.0001 r < 0.0001	1.14 (1.02, 1.28)	P = 0.0218 r = 0.0740	1.06 (0.96, 1.17)	P = 0.2213 r = 0.3790

CI = confidence interval; NRT = nicotine replacement therapy.

This pattern was observed for RD, with statistical significance, for all individual medications at 3–6 and 6–12 months. The forest plots from the meta‐analyses are presented in Fig. 2. The I 2 measure of heterogeneity across studies for the 6–12‐month comparisons was 20% for RD and 0% for RR. This indicates a fair degree of consistency in results across studies. Associated funnel plots 28 are presented in Supporting information, Fig. S2. Funnel plots display the results of individual comparisons, with the ‘funnel’ (the white area) defining the bounds of expected results when there is no between‐study heterogeneity. The funnel plot in Fig. 2 reveals no comparisons that are clearly outliers. ‘Holes’ in funnel plots indicate possibly missing results associated with lack of statistical significance (publication bias). No such holes are seen in Supporting information, Fig. S2. This is unsurprising, as interactions of treatment effects with time are not the primary concern in most publications.

Figure 2

Forest plots for meta‐analyses of interaction terms (time by intervention arm), overall and for individual medications, for sustained abstinence quit rates 6‐12 months

Statistical significance for changes in RDs was found for all medications combined and for individual medications even after performing sensitivity analysis (r < 0.0001), indicating that results were not driven by a single dominant study.

Small declines in relative risks can be observed in Figure 1. Table 2 shows that overall, RRs decreased slightly over time between 6 and 12 months (P < 0.0001). This decrease was driven by NRT, while no significant declines were observed for varenicline or Bupropion. No significant difference was observed for RRs between times 3 and 6.

PP and OR analyses

The pattern of decreasing PP quit rates over time (Supporting information, Tables S3, S4) was similar to that for SA quit rates. Most, but not all, PP quit rates were higher than the corresponding SA quit rates. ORs were strictly greater than RRs. ORs are presented in Tables 1 and 2 and Supporting information, Tables S3 and S4. The change in the PP RD was significant in the 3‐ and 6‐month analyses, and the change in the PP RR was significant in the 6‐month analysis.

Sensitivity analysis using biochemically validated quit rates only

Results (Supporting information, Table S5) changed very little by excluding studies with only self‐reported data.

Discussion

Based on 27 647 participants in 61 trials of first‐line recommended smoking cessation medications, this study showed that the benefit of medications diminished during the course of the first year. The net benefit to the intervention declined by 4.95% (95% CI = 3.49–6.41%, P < 0.0001) points from 3 to 6 months and by 3.00% (95% CI = 2.36–3.64%, P < 0.0001) points from 6 to 12 months. This decline was observed for each main category of smoking cessation medications: bupropion, NRT and varenicline. Although a decline in SA RR was found for all medications combined and for NRT at 6‐12 months, the magnitude of the decline was very small, so that relative risks essentially remained constant.

These empirical findings, which show large changes over time in RD and only very slight changes in RR, have a clear theoretical basis: it is a mathematical fact that under conditions of constant relative risks and decreasing quit rates the net benefit, as expressed by the RD, will decrease 35. This is illustrated with hypothetical data in Fig. 3. Of note is that if cessation medications were to induce permanent cessation—the true goal as stated by Stapleton nearly 20 years ago 35—then the declining quit rates in the control groups would not be matched by declining quit rates in intervention groups, and both RR and RD would increase over time.

Figure 3

Illustrative example using hypothetical data: how net benefit of medications decreases over time with declining quit rates and stable relative risk

The net benefit of medications was 11.8% (95% CI = 10.0–13.7%, P < 0.0001) at 6 months. West has shown that due to the enormous benefits of quitting smoking, increases of even 1% in 6‐month continuous abstinence rates lead to 3 years of life gained for every 100 smokers treated 23. The 11.8% calculated benefit is therefore of substantial clinical importance and cost‐beneficial.

Net benefit still remained at the end of the study. Over all medications, one in five trial participants in the intervention arm was a sustained non‐smoker at the 12‐month end‐point, and benefit from medication at 12 months accrued to 8.2 of 100 participants in the treatment arms. Previous research has shown that quit rates continue to decline after the first year in the context of RCTs. A meta‐analysis of long‐term results from NRT trials showed that quit rates declined by 30% between 1 year and an average of 4.3 years 11. This suggests that our estimated net benefit of medication of 8.2% at the end of 1 year would decline further to less than 6% by 4 years. Because there may be some small continuing decline in quit rates beyond 4 years, 6% probably reflects the upper limit of long‐term medication benefit to smokers under ideal RCT conditions. Under real‐world conditions, the benefit of these medications is likely to be lower than in the context of trials, due to poorer adherence to treatment protocol and fewer patient/provider interactions in real‐world settings than in RCTs 36, 37, 38.

The decay of benefit found in this paper during the first year of treatment is important at both individual and population levels. At the individual level, providers would be well served by realistic expectations when prescribing medications: clinicians could plan from the outset to continue to intervene over time. Identification of the decline in benefit over time is also useful at the population level: it contributes to understanding of the lack of clear population‐level impact of smoking cessation medications in societies 21, 36, 37, 39, 40, 41, 42, 43 and highlights the importance of relapse prevention at both clinical and societal levels.

Limitations

We did not assess smoking cessation medications which were not FDA first‐line approved medications, or non‐medical interventions.

Although we intended originally to analyse changes in benefit at time‐points longer than 1 year, sufficient data were unavailable. For the same reason, we were unable to analyse changes in benefit from very early (< 3 months) time‐points.

Although this review included a large number of studies, we were unable to use all the studies included in the three original Cochrane Reviews. Twenty‐six per cent of NRT trials were included, as were 20% of bupropion trials and 25% of varenicline trials. Exclusions were due primarily to our exclusive focus upon approved first‐line cessation medications (two of the three Cochrane Reviews 6, 7 also included other medications), the need for information on both 6‐ and 12‐month end‐points (Cochrane did not require information at multiple end‐points), our requirement that there be a comparable non‐medicated control group, which stemmed from our intent to estimate the full effect of medication at different time‐points, and our requirement that the report be peer‐reviewed, contain sufficient information for the meta‐analysis and report on at least 95% of randomized participants. Five potentially eligible trials were excluded due to language of publication. On the basis of publications about those five studies [Cochrane tables and English language publications], we do not believe that their inclusion would have altered our findings.

We were unable to assess fully the extent to which differential attrition between treatment arms may have affected results, because attrition information was not available for most studies. We used the standard assumption that those lost to follow‐up were continuing smokers in calculating ITT quit rates. Most investigators make this assumption, as they assume that the great majority of those not continuing to participate have in fact lost interest in quitting. (If this were not the case, our estimated quit rates would underestimate the true quit rates.) The few papers which we identified that did report attrition rates between 6 and 12 months show that attrition during this period was small, suggesting that the difference between attrition in intervention and control groups was minimal for that period 44, 45, 46. Therefore, the impact on our primary result—that the difference between quit rates between intervention arms decreases over time—should be negligible.

We did not compare results from different cessation medications directly, due to the fact that few head‐to‐head comparisons were available.

Length of treatment with medication and type of behavioural counseling may contribute to length of cessation success. We did not evaluate these or other potential modifiers.

Implications

The common practice of providing RRs or ORs at single or composite time‐points to describe effectiveness of smoking cessation medications 6, 7, 8, 12, 13, 26 does not inform providers and policymakers fully about the benefits of smoking cessation medications at the individual and population levels.

Clinicians and policymakers should be aware that the public health benefits of smoking cessation medications are considerably smaller than those indicated by the risk difference at 6 months and diminish over time, most intervention group participants are smokers at 1 year and only a small proportion of intervention arm participants benefit in the long term from the medications. Nevertheless, even these small effects confer important long‐term benefits of public health significance.

Researchers and/or reviewers should collect and report on quit rates at various time‐points, including long‐term end‐points of at least 1 year and preferably much longer; report on RDs and quit rates in addition to RRs; strive to minimize attrition rates among study participants, and report numbers of participants followed‐up at every time‐point.

Statistical software used for meta‐analysis should provide straightforward procedures for calculating intervention group‐specific, appropriately weighted RDs and quit rates for each time‐point.

Improved smoking cessation strategies, with supportive environments for promotion of cessation and prevention of relapse, are needed urgently to help millions of smokers quit smoking permanently.

Declaration of interests

None.

Table S1 Characteristics of included studies.

Click here for additional data file.

Figure S1 Flow‐chart for identification of studies

Click here for additional data file.

Table S2 Reasons for exclusion of studies.

Click here for additional data file.

Figure S2 Funnel plots for meta‐analyses of interaction terms (time by intervention arm), overall and for individual medications, for sustained abstinence quit rates.

Click here for additional data file.

Table S3 Point prevalence quit rates (QRs), risk differences (RDs), relative risks (RRs), and odds ratios (ORs) by time, overall and for individual medications.

Click here for additional data file.

Table S4 Changes over time in point prevalence quit rates: risk differences (RDs), relative risks (RRs), and odds ratios (ORs), overall and for individual medications.

Click here for additional data file.

Table S5 Results from sensitivity analysis using biochemically validated data only.

Click here for additional data file.

Statistical Supplement Variance of interaction terms.

Click here for additional data file.