Smoking cessation treatment and risk of depression, suicide, and self harm in the Clinical Practice Research Datalink: prospective cohort study.

OBJECTIVE: To compare the risk of suicide, self harm, and depression in patients prescribed varenicline or bupropion with those prescribed nicotine replacement therapy.
DESIGN: Prospective cohort study within the Clinical Practice Research Datalink.
SETTING: 349 general practices in England. PARTICIPANTS: 119,546 men and women aged 18 years and over who used a smoking cessation product between 1 September 2006 and 31 October 2011. There were 81,545 users of nicotine replacement products (68.2% of all users of smoking cessation medicines), 6741 bupropion (5.6%), and 31,260 varenicline (26.2%) users. MAIN OUTCOME MEASURES: Outcomes were treated depression and fatal and non-fatal self harm within three months of the first smoking cessation prescription, determined from linkage with mortality data from the Office for National Statistics (for suicide) and Hospital Episode Statistics data (for hospital admissions relating to non-fatal self harm). Hazard ratios or risk differences were estimated using Cox multivariable regression models, propensity score matching, and instrumental variable analysis using physicians' prescribing preferences as an instrument. Sensitivity analyses were performed for outcomes at six and nine months.
RESULTS: We detected 92 cases of fatal and non-fatal self harm (326.5 events per 100,000 person years) and 1094 primary care records of treated depression (6963.3 per 100,000 person years). Cox regression analyses showed no evidence that patients prescribed varenicline had higher risks of fatal or non-fatal self harm (hazard ratio 0.88, 95% confidence interval 0.52 to 1.49) or treated depression (0.75, 0.65 to 0.87) compared with those prescribed nicotine replacement therapy. There was no evidence that patients prescribed bupropion had a higher risk of fatal or non-fatal self harm (0.83, 0.30 to 2.31) or of treated depression (0.63, 0.46 to 0.87) compared with patients prescribed nicotine replacement therapy. Similar findings were obtained using propensity score methods and instrumental variable analyses.
CONCLUSIONS: There is no evidence of an increased risk of suicidal behaviour in patients prescribed varenicline or bupropion compared with those prescribed nicotine replacement therapy. These findings should be reassuring for users and prescribers of smoking cessation medicines.

Introduction

Smoking is a major cause of premature mortality and preventable morbidity in the United Kingdom and worldwide.1 2 Treatments such as varenicline, bupropion, and nicotine replacement therapy, along with advice and referral to smoking cessation services, are recommended by the National Institute for Health and Care Excellence as options for smokers who want to quit smoking.3 Since its UK launch in 2006, varenicline (Champix in the UK, Chantix in the United States) has been widely used; there were about one million community prescriptions of varenicline in England in 2011.4 5 Bupropion, marketed as Zyban, is the other main product used in the UK for smoking cessation that does not contain nicotine. It is also used to treat depressive illnesses in some countries, but is not licensed for this indication in the UK.3 6

Since their launch, there have been concerns from spontaneous reporting systems that varenicline and bupropion could increase the risk of fatal and non-fatal self harm; these have resulted in safety warnings by regulatory agencies, including the Medicines and Healthcare Products Regulatory Agency (MHRA) in the UK and the Food and Drug Administration in the United States.7 8 Possible biological mechanisms for the increased suicide risk with varenicline might be through its action as a partial nicotinic agonist, because nicotine receptors could influence impulsivity and aggression.9 Furthermore, smoking may be used as self treatment for depression; consequently, smoking cessation could lead to depression, which increases the risk of suicide.10 Concerns about the safety of varenicline are still ongoing; Pfizer has settled the majority of lawsuits brought against varenicline in the US.11

Because suicide is a rare event, randomised controlled trials and meta-analyses of trials usually lack statistical power to investigate safety signals of this outcome.12 Large databases that record general practitioner prescribing and clinical outcomes—such as the UK’s Clinical Practice Research Datalink (CPRD), formerly known as the General Practice Research Database (GPRD)—provide an alternative source of information in the investigation of safety issues for rare outcomes.

However, results from observational studies could be compromised by factors such as incomplete recording of outcomes and confounding by indication. Confounding by indication occurs when the characteristics of patients prescribed a treatment systematically differ from those prescribed the control agent. For example, smokers have a higher risk of mental illness and suicide than non-smokers.13 14 This increased risk could account for the higher numbers of suicides that have been observed with those prescribed agents for smoking cessation.15 A basic approach to overcome confounding by indication is to compare adverse event rates between drugs used to treat the same condition—that is, to compare the risk of fatal and non-fatal self harm in people prescribed varenicline with those prescribed other smoking cessation products, such as bupropion and nicotine replacement therapy. However, if there are subtle differences in the baseline characteristics of patients prescribed each of these drugs, for instance, if certain pharmacological treatments are less likely to be prescribed to those considered to be at a higher risk of suicide or self harm, then the results from observational studies can still be biased.

In an earlier study, we reported on the risk of fatal and non-fatal suicidal behaviour among people prescribed varenicline versus those prescribed other smoking cessation medicines; the hazard ratio for self harm among people prescribed varenicline was 1.12 (95% confidence interval 0.67 to 1.88) compared with those prescribed nicotine replacement therapy.16 However, this study had several limitations: there was limited study power, because only 10 973 people had been prescribed varenicline in a 20 month period and there was evidence of an underestimation of suicide deaths (the ratio of non-fatal self harm was 80:1, whereas it is 20-30:1 in the general population).17 Furthermore, adjusting for a range of confounders led to a reversal in the direction of the hazard ratio, suggesting that unmeasured confounders further suppressed an increased risk in relation to varenicline. Other approaches for tackling confounding by indication include the use of propensity score methods and instrumental variable analysis.18 19 20

The aim of this study was to estimate the risk of suicide related outcomes derived from linked mortality data from the Office for National Statistics (ONS) and Hospital Episode Statistics (HES) in patients prescribed varenicline or bupropion compared with those prescribed nicotine replacement therapy in the CPRD. We used conventional and novel methods to assess the possible effect of confounding by indication.

Methods

Source data and population

The CPRD is one of the largest primary care databases in the world and contains electronic medical records from roughly 5.4 million active patients, representing about 8.5% of the UK population (www.cprd.com). During a consultation, information on a patient’s diagnosis or symptoms can be recorded using written text or Read codes (which are linked to specific phrases of text). At the time of the present study, about 50% of CPRD general practices in England had been linked to ONS mortality data and HES data (www.hesonline.nhs.uk).21

We identified all patients aged 18 years and over in CPRD practices linked with ONS and HES data, who were prescribed varenicline, bupropion, and nicotine replacement therapy from 1 September 2006 to 31 October 2011 (varenicline was first licensed for use in December 2006). We chose 31 October 2011 as the end date because it was the final day for which both HES and ONS linkage data were available, and 18 years as the minimum age because smoking cessation medicines are only licensed for use in adults.

We restricted our analyses to patients with records classified as “acceptable” by the CPRD from practices designated as “up to standard.” Acceptable records were classified as patients with no breaks in their records, and information on their year of birth, first registration date, and sex. Practices that were up to standard had continuous recording of data, including accurate recording of when patients may have been transferred out of the practice. We excluded patients who registered with their practice within 365 days of their first recorded prescription, to allow for comprehensive assessment of baseline data and possible confounders and to have enough previous data to define first time use of smoking cessation products. Main cohort entry was defined by the date of treatment initiation of a smoking cessation product in our study period with no previous documentation of a prescription for a smoking cessation medicine in the previous 12 months. Patients were categorised into the three exposure groups based on the smoking cessation product they were first prescribed in the study period. We excluded patient episodes that involved multiple prescriptions with the same drug or combined drug treatments in the follow-up period, after treatment initiation with a specific smoking cessation drug; therefore, all included patients were unique.

We also carried out a sensitivity analysis for a group of true “first time” users to reduce potential exposure misclassification. Cohort entry for this subset of “first time” users was defined by the date of their first documented prescription of a smoking cessation medicine in the study period with no previous record of use of a smoking cessation drug in the database. Power calculations indicated that we needed 80% power at a 5% level of statistical significance to detect a 1.5-fold increase in risk of self harm with varenicline compared with other products. The study had more power to examine the risk of depression.

Exposures, outcomes, and possible confounding factors

We limited the main analysis to patients who initiated treatment with a smoking cessation medicine from linked practices within the study period. Patients were excluded if they had used a smoking cessation product 12 months before 1 September 2006. Nicotine replacement therapy was used as the comparison group for varenicline and bupropion. Because bupropion is only licensed for use as a smoking cessation medicine in the UK, we assumed that all prescriptions were for this indication only. We made this assumption because 98.8% of Read codes recorded on the day of bupropion treatment in our database were for smoking cessation.

The primary outcomes were incident episodes of depression as measured by the date that antidepressant treatment was initiated (treated depression), and by fatal and non-fatal self harm (as measured by death from suicide in the ONS mortality database) and hospital admission for self harm (as recorded in the HES database). We used the following ICD-10 (international classification of diseases, 10th revision) codes: X60-X84 (intentional self harm) and Y10-Y34 (event of undetermined intent) excluding Y33.9 where verdict was still pending. Undetermined deaths were included because most of these deaths are probable suicides.22 In contrast to the previous analysis, we did not use Read codes to identify our primary outcomes because we have recently shown in a validation study that Read codes do not accurately detect suicide and could also under-report non-fatal self harm.23 Although there is no validated measure of depression in the CPRD, the use of initiation of antidepressant treatment as an outcome could indicate the occurrence of a more severe depressive episode than one that did not need the use of antidepressants. However, to allow comparison with the previous cohort study,16 we also examined depression and fatal and non-fatal self harm defined on the basis of Read codes in the complete cohort of UK CPRD practices—that is, we included practices without HES and ONS linkage. Exploratory analyses were performed for all cause mortality; date of death is well recorded in the CPRD.24

We investigated the following potential confounders:

Sex

Age (using age categories 18-20, 21-30, 31-40, 41-50, 51-60, and >60 years)

Previous psychiatric illness or consultation

Previous use of psychotropic drugs such as hypnotics, antipsychotics, anxiolytics, lithium, and antidepressants

Previous self harm

Socioeconomic position (using Index of Multiple Deprivation score for area of residence)

Drug and alcohol misuse

Major chronic illness using the Charlson index25 (diabetes, coronary heart disease, cancer, arthritis)

Number of general practice consultations in the year before the prescription as a means of assessing healthcare consulting behaviour

Whether exposure to the drug occurred before or after 2008, when there was substantial negative media publicity regarding varenicline

Year of first prescription and previous use of a smoking cessation product.

Read code and drug code lists for psychiatric illness, psychotic illness, self harm, major chronic diseases, hypnotics, antipsychotics, antidepressants, and drug and alcohol misuse are available on request from the authors.

The dataset was almost complete with respect to sex and year of birth, and Index of Multiple Deprivation score was available for 99.3% of patients. We dealt with missing data on confounders by excluding patients with no record of sex or year of birth, or by coding presence or absence of the confounder as a binary variable—that is, a record of alcohol misuse was coded as 1—whereas no record of alcohol misuse was coded as 0.

We compared the baseline characteristics and use of drug treatments in patients from practices linked to HES and ONS data with patients from practices without linkage in England.

Follow-up

Follow-up began on the day of the first prescription of the smoking cessation drug, and ended with the earliest of our primary outcome (censoring from the study due to death, transfer out of the practice, end of the study period, or three months after the date of first prescription). The drug that was used initially determined exposure status for the entire follow-up period. Sensitivity analyses were also conducted for outcomes that occurred at six and nine months after the date of the first prescription.

Data analysis

Stata version 12 (Statacorp) was used to perform all the statistical analyses.

Conventional analysis

We used models of Cox proportional hazards regression to examine associations of the smoking cessation drugs with the outcomes of interest and report hazard ratios and 95% confidence intervals. We checked the proportional hazards assumption using estat phtest in Stata. In the basic models, we controlled for age, sex, and year of treatment; in the fully adjusted models, we adjusted for all of the potential confounding variables listed above, fitted as categorical variables to the models. We also investigated any differences in the association of smoking cessation products with outcomes by sex, age, psychiatric history, treatment before or after 2008, and past treatment with psychotropic drugs using the likelihood ratio test for interaction. To investigate possible clustering by general practice, we refitted the basic model for the primary outcomes using fixed effects Poisson regression methods, with the general practice as the unit of clustering (using the Stata command xtmepoisson). Cox regression analyses were also carried out for fatal and non-fatal self harm (defined by Read codes) at three, six, and nine months in the entire UK CPRD cohort.

Propensity score analysis

In a second analysis, we used propensity scores to construct a sample of patients prescribed the smoking cessation medicines with balanced risk factors. We constructed a propensity score using all of the previously listed potential confounders, and used logistic regression to calculate the probability of receiving treatment for each patient. We used single nearest neighbour matching with no replacement (a single participant could not be selected multiple times) and no caliper (the closest match was selected, even if not a very close match), to match patients in the varenicline cohort to those receiving nicotine replacement therapy (using Stata command psmatch2).26 We dropped patients from our analysis without overlapping propensity scores between the varenicline group and the group receiving nicotine replacement therapy. Methods were repeated for patients prescribed bupropion compared with those prescribed nicotine replacement therapy. We estimated hazard ratios using Cox regression, including the propensity score as a covariate.

Instrumental variable analysis

We were concerned about the possibility of bias in our conventional analyses owing to residual confounding (that is, there may have been differences between patients prescribed different smoking cessation products that had not been accounted for by conventional methods). Instrumental variable analysis can provide unbiased estimates of the effects of treatments in the presence of residual confounding.27 A valid instrument needs to be associated with prescriptions, must not directly affect the outcome except through its effect on the treatment, and cannot be associated with confounding factors.27 28

Previous studies have shown that physicians’ prescribing preferences can be valid instruments for drug prescribing.29 30 31 We could not directly measure physicians’ preferences. Therefore, the type of smoking cessation product most recently prescribed by the physicians was used as an indication of their current preferences for a smoking cessation product relative to alternative treatments. For example, if the instrument was based on the most recent prior prescription, and the most recent prescription was for varenicline, then for the next patient beginning treatment, the physician was classified as a “varenicline prescriber.” We used an instrument based on indicator variables derived from the seven most recent prescriptions. We estimated structural mean models using generalised method of moments to efficiently estimate—given the moment conditions—the effects of prescriptions on the outcome.32

We estimated standard errors that were robust to a general form of heteroskedasticity and clustering by physician, and reported partial F statistics for the association of instruments and the exposure (using the Stata command ivreg2 and option gmm2).33 Under the no effect modification (NEM) assumption in the structural mean models—that is, the treatment effect is not modified by the value of the instrument—the instrumental variable analysis identifies the average effect of the treatment on treated patients.32 We tested for any difference between the instrumental variable models and the conventional regression models by using a Hausman test, and reported P values for the test for over-identifying restrictions when multiple instruments were used (Hansen’s J test).34 35 The null hypothesis of the Hansen’s J test is that the instruments are valid—that is, they do not have a direct effect on the outcome, are not correlated with unobserved confounders, and satisfy the NEM assumption. Methods for using instrumental variables to estimate hazard ratios are not well developed; therefore, we reported the risk difference in outcomes at three, six, and nine months.

Results

Conventional analysis

In the study period, 216 605 patients in the CPRD were prescribed a course of a smoking cessation product; 119 546 patients were from 349 English practices with linked ONS mortality and HES admission data. There were 81 545 patients prescribed nicotine replacement therapy (68.2%), 6741 for bupropion (5.6%) and 31 260 for varenicline (26.2%) in the linked practices (table 1). There were 97 059 patients from English practices without linkage. Patients in the linked practices were similar to those from practices without linkage, although patients in the linked practices had lower levels of alcohol misuse (6.4% (n=7619) v 8.2% (n=7917)), use of hypnotics (21.8% (n=26 085) v 23.4% (n=22 701)) and anxiolytics (20.8% (n=24 854) v 24.6% (n=23 881)), as well as higher levels of previous psychiatric illness (46.2% (n=55 265) v 45.0% (n=43 658)). Patients in linked practices were also less likely to be prescribed nicotine replacement therapy (68.2% (n=81 545) v 70.4% (n=68 326)) and more likely to be prescribed bupropion (5.6% (n=6741) v 3.9% (n=3771)) or varenicline (26.2% (n=31 260) v 25.7% (n=24 962)) than those from unlinked practices.

Table 1

 Baseline characteristics of patients prescribed nicotine replacement therapy, bupropion, and varenicline. Data are number (%) of patients unless otherwise specified

Variable	Nicotine replacement therapy (n=81 545)	Bupropion (n=6741)	Varenicline (n=31 260)	Total (n=119 546)
Female sex	44 300 (54.3)	3430 (50.9)	15 727 (50.3)	63 457 (53.1)
Median age (years)	45 (25)	43 (18)	44 (19)	45 (23)
Alcohol misuse	5789 (7.1)	294 (4.4)	1536 (4.9)	7619 (6.4)
Hypnotics use (previous)	18 785 (23.0)	1284 (19.1)	6016 (19.3)	26 085 (21.8)
Antipsychotic use (previous)	17 154 (21.0)	1050 (15.6)	4937 (15.8)	23 141 (19.4)
Antidepressant use (previous)	38 905 (47.7)	2826 (41.9)	12 836 (41.1)	54 567 (45.7)
Previous self harm	8519 (10.5)	547 (8.1)	2769 (8.9)	11 835 (10)
Previous smoking cessation	32 315 (39.6)	3229 (47.9)	12 149 (38.9)	47 693 (39.9)
Median No (interquartile range) of clinical visits in year before treatment	19 (23)	13 (17)	14 (19)	17 (22)
Exposed to treatment before Jan 2008	33 396 (41.0)	3965 (58.8)	5352 (17.1)	42 713 (35.7)
Index of Multiple Deprivation score
Missing	558 (0.7)	55 (0.8)	258 (0.8)	871 (0.7)
Least deprived fifth of patients	10 930 (13.4)	1042 (15.5)	4538 (14.5)	16 510 (13.8)
Most deprived fifth of patients	19 379 (23.8)	1226 (18.2)	7044 (22.5)	27 649 (23.1)
Previous anxiolytic	17 851 (21.9)	1298 (19.3)	5705 (18.3)	24 854 (20.8)
Previous lithium	610 (0.8)	20 (0.3)	56 (0.2)	686 (0.6)
Previous drug misuse	2422 (3.0)	127 (1.9)	565 (1.8)	3114 (2.6)
Previous psychiatric illness	39 529 (48.5)	2886 (42.8)	12 850 (41.1)	55 265 (46.2)
Previous chronic disease	31 016 (38.0)	1998 (29.6)	10 026 (32.1)	43 040 (36.0)

We followed 115 327 patients for the entire follow-up period, and censored 4219 owing to death (for our primary outcomes) and transfer out of the practice. Total exposure time for fatal and non-fatal self harm at three months was 28 181 person years (19 196 for nicotine replacement therapy, 1622 for bupropion, 7363 for varenicline) with median follow-up time of 91.3 days. Table 1 shows the baseline characteristics for patients prescribed each smoking cessation product in linked English practices. Patients prescribed varenicline had similar characteristics to those prescribed bupropion, but patients prescribed nicotine replacement therapy were more likely than those prescribed the other two drugs to be female and to have a previous history of chronic disease, alcohol misuse, drug misuse, and psychiatric illness including self harm. Patients prescribed nicotine replacement therapy were also more likely to have previously used hypnotics, antipsychotics, and antidepressants than those prescribed the other smoking cessation products. There was no evidence that the proportional hazards assumption was violated in basic and adjusted models in relation to fatal and non-fatal self harm, using linked HES and ONS data and depression measured by start of antidepressant therapy and Read codes.

We identified 92 cases of suicide and non-fatal self harm (including six suicides in the nicotine replacement therapy group and two in the varenicline group) at three months of follow-up after the date of treatment initiation with a smoking cessation product. Overall incidence of suicide and non-fatal self harm was 326.5 (95% confidence interval 263.2 to 400.4) per 100 000 person years. Incidence of suicide was 27.4 per 100 000 person years (11.8 to 54.0) in this cohort. The incidence of suicide and non-fatal self harm per 100 000 person years was 359.5 (279.7 to 454.9) in patients prescribed nicotine replacement therapy, 258.0 (155.4 to 403.0) in those prescribed varenicline, and 246.6 (67.2 to 631.4) in those prescribed bupropion.

Table 2 shows the risks of suicide, non-fatal self harm, treated depression, and all cause mortality at three months in users of varenicline, bupropion, and nicotine replacement therapy. We used a composite outcome of fatal and non-fatal self harm, owing to the small numbers of incident suicides (n=8) in the cohort. In the basic model (that is, adjusted for age, sex and year of treatment), hazard ratios showed a lower risk of treated depression (hazard ratio 0.69, 95% confidence interval 0.60 to 0.80) and all cause mortality (0.37, 0.26 to 0.54) in patients prescribed varenicline than in those prescribed nicotine replacement therapy. The results for fatal and non-fatal self harm included the null (0.70, 0.41 to 1.18). Findings were similar for bupropion; compared with those prescribed nicotine replacement therapy, patients prescribed bupropion had a lower risk of suicide and self harm (hazard ratio 0.62, 95% confidence interval 0.22 to 1.70), treated depression (0.56, 0.41 to 0.77), and all cause mortality (0.31, 0.13 to 0.74) at three months. All of our results were attenuated towards null in the fully adjusted models.

Table 2

 Risks of suicide and non-fatal self harm, treated depression, and all cause mortality at three months in patients prescribed varenicline, bupropion, and nicotine replacement therapy (NRT)

Smoking cessation product	Total person time (person years)	No of events/No of patients prescribed product	Hazard ratio (95% CI)
			Basic model*	Fully adjusted model†
Main cohort: treatment initiators
Fatal and non-fatal self harm
NRT	19 196	69/78 407	1	1
Bupropion	1622	4/6568	0.62 (0.22 to 1.70)	0.83 (0.30 to 2.31)
Varenicline	7363	19/30 352	0.70 (0.41 to 1.18)	0.88 (0.52 to 1.49)
Treated depression‡
NRT	10 315	799/42 475	1	1
Bupropion	961	40/3910	0.56 (0.41 to 0.77)	0.63 (0.46 to 0.87)
Varenicline	4435	255/18 386	0.69 (0.60 to 0.80)	0.75 (0.65 to 0.87)
All cause mortality
NRT	19 947	292/81 496	1	1
Bupropion	1665	5/6740	0.31 (0.13 to 0.74)	0.39 (0.16 to 0.95)
Varenicline	7575	33/31 227	0.37 (0.26 to 0.54)	0.44 (0.30 to 0.63)
Secondary cohort: first time users
Fatal and non-fatal self harm
NRT	11 565	41/47 376	1	1
Bupropion	846	2/3427	0.64 (0.15 to 2.66)	0.87 (0.21 to 3.66)
Varenicline	4495	10/18 591	0.57 (0.28 to 1.17)	0.74 (0.36 to 1.52)
Treated depression‡
NRT	6887	529/28 415	1	1
Bupropion	554	22/2255	0.56 (0.37 to 0.86)	0.63 (0.41 to 0.96)
Varenicline	2971	179/12 346	0.72 (0.60 to 0.86)	0.77 (0.65 to 0.92)
All cause mortality
NRT	12 006	186/49 195	1	1
Bupropion	867	2/3512	0.23 (0.06 to 0.92)	0.29 (0.07 to 1.19)
Varenicline	4614	15/19 084	0.28 (0.16 to 0.48)	0.32 (0.19 to 0.55)

Hazard ratios calculated using Cox proportional hazards regression model.

*Basic model includes age, sex, and year of first prescription.

†Fully adjusted model includes sex; age; previous psychiatric illness or consultation; previous use of psychotropic drugs such as hypnotics, antipsychotics, and antidepressants; previous self harm; socioeconomic position; major chronic illness; number of general practice consultations in the year before the prescription; exposure to the drug before or after 2008; year of first prescription; and previous use of a smoking cessation product.

‡Restricted to those with no previous antidepressant use.

Findings in the subset of first time users were similar to those for the main cohort of treatment initiators (table 2). There was no evidence for a difference in the association of smoking cessation products with suicide and self harm by age (P=0.30 for interaction), sex (P=0.64 for interaction), history of psychiatric illness (P=0.57 for interaction), or timing of prescribing before or after 2008 (P=0.11 for interaction).

Sensitivity analyses at six and nine months resulted in similar findings to those obtained at three months (web appendix A). When we included the entire CPRD cohort and used Read codes to determine fatal and non-fatal self harm, the results were consistent with the main results for patients prescribed varenicline (hazard ratio 0.65, 95% confidence interval 0.45 to 0.93) and bupropion (0.80, 0.41 to 1.58), compared with nicotine replacement therapy. Sensitivity analyses with depression identified by Read codes were also consistent with the main results for patients prescribed varenicline (0.65, 0.58 to 0.73) and bupropion (0.77, 0.62 to 0.94), compared with nicotine replacement therapy. There was no evidence of clustering by practice (that is, between practice variation in the outcome rate) for suicide, self harm, and all cause mortality for varenicline or bupropion compared with nicotine replacement therapy (P=1.00, indicating no clustering by practice). However, there was some evidence of clustering by practice for depression as indexed by the first antidepressant prescription (P=0.02), with attenuation of the incidence rate ratios. For treated depression, the incidence rate ratio for the cluster adjusted model was 0.71 (95% confidence interval 0.42 to 1.2) for varenicline and 0.65 (0.24 to 1.8) for bupropion, compared with nicotine replacement therapy.

Propensity score analysis

For the propensity score analysis, 616 (0.5%) patients were dropped in the analysis comparing bupropion with nicotine replacement therapy; 9125 (7.6%) patients were dropped in the varenicline versus NRT analysis because the propensity scores did not overlap. Web appendix B shows the differences in confounding factors between the treated and untreated groups before and after matching on the propensity score. We saw no differences in the findings when using propensity score methods instead of the traditional multivariable Cox analyses (web appendix C).

Instrumental variable analysis

Web appendix D shows that the instrumental variables (physicians’ prior prescriptions and the indicators for the seven previous prescriptions) were less associated with baseline characteristics than the actual prescriptions (table 1). The patients’ actual prescriptions were associated with all covariates (P<0.001) whereas only two factors—previous smoking cessation treatment and index of multiple deprivation—were associated with the seven previous prescriptions (web appendix E). We tested whether physicians’ prior prescriptions (as proxies for physicians’ prescribing preferences) were valid instruments. There were no weak instruments because the physicians’ previous prescriptions were strongly associated with their patients’ actual prescriptions (all F tests >40; web appendix F).36

We found no evidence using instrumental variable analyses that varenicline or bupropion was associated with a higher risk of suicide and self harm, depression, or all cause mortality compared with nicotine replacement therapy (web appendix F). For most of the analyses, the 95% confidence intervals for the risk differences included the null value of 0 and the Hausman test suggested no differences between the conventional regression analysis and the instrumental variable regression. Exceptions included suicide and self harm at three and nine months and all cause mortality at six months in the bupropion group compared with nicotine replacement therapy, where the instrumental variable analyses showed a reduced risk (web appendix F). There was little evidence from the Hansen’s J test for rejection of the over-identifying restrictions, and therefore no indication that the instruments were not valid.

Discussion

Main findings

We found no evidence of an increased risk of fatal or non-fatal self harm or depression (as measured by antidepressant prescriptions) at three months in individuals prescribed varenicline or bupropion compared with those prescribed nicotine replacement therapy, using three different analytical approaches (Cox multivariable regression, propensity score methods and instrumental variable analyses). In the conventional analyses, varenicline and bupropion users had a reduced risk of treated depression and all cause mortality compared with users of nicotine replacement therapy. The instrumental variable analysis suggests that the lower risk of all cause mortality seen in those prescribed varenicline is probably due to residual confounding. However, the instrumental variable results were too imprecise to draw any inferences for bupropion. Sensitivity analyses at six and nine months produced similar findings, as did analyses that were limited to first time users (that is, patients with no records of previous prescriptions for smoking cessation products in the database).

Strengths and limitations

This study had several strengths. Firstly, we used data from the CPRD, one of the largest primary care databases in the world; the large cohort was amenable to the investigation of rarer outcomes. Secondly, the population of the CPRD is representative of the UK population as a whole, and thus findings are generalisable to the UK population.37 Thirdly, we benefitted from new CPRD linkages to ONS mortality statistics and HES data, which resulted in better ascertainment of the suicide and self harm outcomes.23 Patients from linked practices are also representative of the entire CPRD population.38 The incidence of suicide in the present study cohort was 27.4 per 100 000, which is about three times the rate obtained from the CPRD population used in our validation study (9.4 per 100 000 person years)23 and accords with findings from other cohort and case control studies that have shown a twofold to threefold higher risk of suicide in smokers versus non-smokers.13 14 Reassuringly, we obtained similar findings when we repeated the analyses using the methodology in our previous study, which was based on Read code ascertainment of outcomes.16

Fourthly, we used three different methods to assess the effect of confounding by indication, a major limitation of observational studies. The instrumental variable analysis is a novel aspect of the study design because the use of an effective instrument attempts to mimic a randomisation process and should result in the equal distribution of measured and unmeasured confounders among the different treatment groups.19 29 Lastly, this study had increased power, with about three times as many varenicline prescriptions since our previous analysis.

Some limitations remained. This study was based on the recording of prescriptions in primary care; we had no information on products bought over the counter (of most relevance for nicotine replacement products, because the other smoking cessation drugs require prescriptions) or those received from smoking cessation clinics in the UK’s health service. We did not look at repeat prescriptions or adherence in this dataset, and there was no way of determining whether individuals actually used the drugs that were prescribed or when they started taking them. We were unable to exclude patients with previous exposure to nicotine replacement therapy obtained over the counter, however, we also analysed a sample of patients with first time prescriptions of each of the smoking cessation products (that is, by excluding those with any previous record of a smoking cessation product in the database). These findings were similar to those from the main analysis.

Another possible limitation was the influence of smoking abstinence on the primary outcomes of the study. Varenicline has been shown to be more effective than the other products in achieving smoking abstinence in the short term.39 Therefore, the existence of a causal association between smoking abstinence and suicide or self harm may compromise the interpretation of our results. However, since a drug’s effectiveness and effect on risk factor levels and disease processes—which could, in turn, cause adverse effects—is an integral part of any analysis of adverse drug reactions, its mechanism of action is less important than the balance of risks and benefits.

We were also unable to use a validated measure of depression; therefore, the initiation of antidepressant therapy was used as a proxy. In a previous CPRD study, about 80% of patients diagnosed with depression received an antidepressant prescription in their first year of diagnosis.40 However, antidepressants are also used to treat anxiety and sleep disorders, thus antidepressant prescribing is not a pure measure of incident depression. Lastly, although the conventional regression showed a lower all cause mortality in patients prescribed varenicline compared with nicotine replacement therapy, we believe that this association was likely caused by residual confounding, because this finding was not supported by the instrumental variable analysis.

Comparison with other studies

For psychiatric outcomes, in addition to our previous observational study,16 the safety of varenicline has been examined using prescription event monitoring in the UK, postmarketing surveillance data in New Zealand, and several US studies, including an adverse event reporting database in veterans with post traumatic stress disorder and members of the Military Health System.15 41 42 43 44 In the study monitoring prescription events, varenicline was associated with two cases of attempted suicide and 29 cases of depression in a cohort of 2682 patients; 42 the suicide attempts only occurred in patients with a history of mental illness. The New Zealand study reported one case of suicide and five cases of non-fatal self harm in their cohort of 3415 patients prescribed varenicline; depression was reported in 3% of patients.41 In the retrospective study of veterans with post traumatic stress disorder, suicidal ideation and other serious adverse events occurred in 8% of patients.44 Unlike our study, none of these three studies included a comparison group, and therefore the findings should be interpreted more cautiously. Moore and colleagues reported an increased risk of reported depression and suicidal behaviour for varenicline compared with nicotine replacement using data from the Food and Drug Administration’s adverse event reporting database.15 However, spontaneous reporting data are susceptible to the effects of stimulated reporting, owing to negative media publicity about drugs and do not provide evidence of causal associations.45

Our findings were consistent with the results of the Military Health System study, which found similar rates of neuropsychiatric hospitalisations in new users of varenicline compared with new users of nicotine replacement therapy.43 A Cochrane review of the use of nicotine receptor agonists for smoking cessation (which included varenicline, cytosine, and dianicline) showed that the main side effect of varenicline use was nausea; however, the study was unable to rule out possible links with serious psychiatric events.46

Conclusions and implications for practice and future research

We used three methods to assess the effect of confounding by indication (a problem inherent to observational studies), and consistently found no evidence of an increased risk of depression or suicidal behaviour in users of varenicline or bupropion compared with users of nicotine replacement products. Although larger studies with fewer assumptions may add more conclusive evidence, these findings should provide some reassurance for users and prescribers of smoking cessation products, because the health benefits of smoking cessation are well documented.

Varenicline is recommended by the National Institute for Health and Care Excellence as a treatment option for smoking cessation in the UK, but has been linked with spontaneous reports of psychiatric adverse events such as depression and suicide

The US Food and Drug Administration requires boxed warnings concerning the risk of serious neuropsychiatric adverse effects on the product labelling of varenicline and bupropion, and Pfizer has settled the majority of lawsuits brought against varenicline in the US

An observational study found no clear association between either drug and the risk of fatal and non-fatal self harm, compared with nicotine replacement therapy; however, it had limited study power and probable under-recording of self harm outcomes

This study used linked mortality data from the Office for National Statistics (for suicides) and Hospital Episode Statistics (for non-fatal self harm data)

Instrumental variable analyses were used in addition to conventional methods (multivariable regression and propensity score matching) to investigate the effect of confounding by indication

Compared with nicotine replacement therapy, there was no clear evidence that varenicline or bupropion increased risks of treated depression or fatal or non-fatal self harm in any analysis