Are take-home naloxone programmes effective? Systematic review utilizing application of the Bradford Hill criteria.

BACKGROUND AND AIMS: Fatal outcome of opioid overdose, once detected, is preventable through timely administration of the antidote naloxone. Take-home naloxone provision directly to opioid users for emergency use has been implemented recently in more than 15 countries worldwide, albeit mainly as pilot schemes and without formal evaluation. This systematic review assesses the effectiveness of take-home naloxone, with two specific aims: (1) to study the impact of take-home naloxone distribution on overdose-related mortality; and (2) to assess the safety of take-home naloxone in terms of adverse events.
METHODS: PubMed, MEDLINE and PsychINFO were searched for English-language peer-reviewed publications (randomized or observational trials) using the Boolean search query: (opioid OR opiate) AND overdose AND prevention. Evidence was evaluated using the nine Bradford Hill criteria for causation, devised to assess a potential causal relationship between public health interventions and clinical outcomes when only observational data are available.
RESULTS: A total of 1397 records (1164 after removal of duplicates) were retrieved, with 22 observational studies meeting eligibility criteria. Due to variability in size and quality of the included studies, meta-analysis was dismissed in favour of narrative synthesis. From eligible studies, we found take-home naloxone met all nine Bradford Hill criteria. The additional five World Health Organization criteria were all either met partially (two) or fully (three). Even with take-home naloxone administration, fatal outcome was reported in one in 123 overdose cases (0.8%; 95% confidence interval = 0.4, 1.2).
CONCLUSIONS: Take-home naloxone programmes are found to reduce overdose mortality among programme participants and in the community and have a low rate of adverse events.

Introduction

Opioid overdose represents a major cause of premature death 1 and accounts for the majority of deaths among injection drug users (IDUs) world‐wide 2. Opioid overdose deaths are preventable through timely administration of naloxone, a potent mu‐opiate antagonist that rapidly reverses opiate‐induced respiratory depression.

In 2014, the World Health Organization (WHO) launched guidelines on the community management of opioid overdose 3, recommending that ‘people likely to witness an opioid overdose should have access to naloxone and be instructed in its administration’ (p. x).

The community‐based provision of naloxone rescue kits to opioid users (‘take‐home naloxone’, THN) was first proposed in the 1990s 4. THN programmes typically involve training opioid users and/or their family members or peers in overdose risk awareness, overdose emergency management and naloxone administration 5. During the past 15 years, THN programmes have been implemented in Europe, North America, Asia and Australia 1. However, the vast majority of evaluations have been pilot schemes with uncontrolled study designs.

The evaluation of THN programmes is challenging: randomized controlled trials (RCTs) are often considered the gold standard of scientific study of clinical impact, but conducting such trials in this context would often be unethical and fraught with methodological difficulties, given the infrequency and unpredictability of overdose.

Critics of THN programmes argue that the existing observational data are not strong enough to infer causation from naloxone provision to the reduction of overdose deaths 6, 7. A counter‐argument may be that similar reservations initially blocked other harm reduction strategies, including needle exchange programmes and opioid substitution therapy 8 that are now evidence‐based practice 9 (and would still be absent if the precautionary principle had been strictly applied).

A clearer understanding of the potential benefits and risks of THN provision is essential. If concerns are valid they need to be identified and considered in context, but mere assertions of hypothetical disadvantages must not prohibit access to a life‐saving medication. A previous systematic review 10 found that participation in THN programmes led to improved overdose‐related knowledge as well as appropriate use and administration of naloxone, but the impact on overdose mortality was not assessed.

Our goal in this review is to assess the effectiveness of THN programmes by following a well‐recognized process (i.e. Bradford Hill criteria) rigorously to evaluate the data within eligible studies, addressing the following two aims: (1) to describe the impact of THN provision on overdose‐related mortality in opioid users; and (2) to assess the safety of THN provision by quantifying adverse events associated with naloxone administration.

Methods

A systematic literature search was performed following PRISMA (Preferred Reporting Items for Systematic Reviews and Meta‐Analyses) guidance (see Fig. 1 for PRISMA flow diagram and Supporting information, Appendix S1 for search protocol and excluded studies).

Figure 1

Preferred Reporting Items for Systematic Reviews and Meta‐Analyses (PRISMA) flow diagram of study selection process

Identification of eligible studies

Electronic databases were searched to identify relevant peer‐reviewed papers published between January 1946 and June (third week) 2015. Replicating the search strategy reported by Clark et al. 10, the following Boolean search query was used: (opioid OR opiate) AND overdose AND prevention.

Electronic database searching generated 1397 records: 150 on Medline, 772 on PsycInfo (both via OVID) and 475 on PubMed. Five studies 11, 12, 13, 14, 15 were added after a manual search of the reference lists of recent literature reviews 10, 16, 17.

Original quantitative (or mixed‐method) studies of randomized or observational trials of THN programmes that trained opioid users in overdose prevention AND reported on overdose outcomes were included into the study. Several exclusion criteria were applied: reporting on buprenorphine/naloxone; case reports; not reporting primary research data; not reporting on heroin/opioid users, naloxone or overdose.

Under supervision of the senior investigator, the first author extracted data using the STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) checklist 18, assessed study eligibility and conducted quality appraisal using an eight‐item scale by Jinks et al. 19, which had been applied previously by Clark et al. 10 (see Table 4).

Table 4

Included studies: follow‐up rate, study design and quality rating.

Study	Location	n	FU	FU %	FU type	Design	Score
Bennett 2011	Pittsburg	426	89	21%	Non‐systematic	Pre–post	5
Bennet 2012	Wales	525	28	5%	Systematic	Pre–post	6
Dettmer 2001a	Jersey	101	NR	NR	Non‐systematic	Case series	4
Dettmer 2001a	Berlin	124	40	32%	Non‐systematic	Case series	4
Doe‐Simkins 2009	Boston	385	278	72%	Non‐systematic	Pre–post	5
Dwyer 2015	Boston	415	51	12%	Systematic	Pre–post	6
Enteen 2010	San Francisco	1942	310	16%	Non‐systematic	Pre–post	6
Galea 2006	New York	25	22	88%	Systematic	Pre–post	7
Lankenau 2013	Los Angeles	30	NA	NA	NA	Cross‐sectional	6
Leece 2013	Toronto	209	NR	NR	Non‐systematic	Case series	5
Lopez‐Gaston 2009	Birmingham & London	70	46	65%	systematic	Pre–post	7
Markham Piper 2008	New York	122	NR	NR	Non‐systematic	Pre–post	6
Maxwell 2006	Chicago	1120	NR	NR	Non‐systematic	Case series	4
McAuley 2010	Lanarkshire	41	17	89%	Systematic	Pre–post	7
Rowe 2015	San Francisco	2500	613	25%	Non‐systematic	Pre–post	7
Seal 2005	San Francisco	24	24	100%	Systematic	Pre–post	5
Strang 2008	England	239	186	78%	Systematic	Pre–post	7
Tobin 2009	Baltimore	250	85	34%	Systematic	Pre–post	6
Tzemis 2014	British Columbia	692	NA	NA	NA	Cross‐sectional	6
Wagner 2009	Los Angeles	66	47	71%	Systematic	Pre–post	7
Walley 2013 20	Massachusetts	2912	212	7%	Non‐systematic	ITS	7
Walley 2013 33	Massachusetts	1553	286	18%	Non‐systematic	Pre–post	6
Yokell 2011	Rhode Island	120	10	8%	Non‐systematic	Pre–post	5

FU: number of follow‐up participants; FU%: FU participants as percentage of study sample; ITS: interrupted time–series analysis; NA: not applicable; NR: not reported; score: summary quality score based on eight‐point scale by Jinks et al. 19, modified from Clark et al. 10

Multi‐site study with two samples (Jersey, Berlin).

All 22 studies that met the inclusion criteria were entered into the analysis. Among these, one was an interrupted time–series analysis that provided quasi‐experimental data. Sixteen were pre–post studies (nine with systematic follow‐up), three were case series and two were cross‐sectional. None of the studies involved randomization to the intervention (i.e. THN distribution), although two studies were controlled 12, 20. Of the 22 included studies, 15 were carried out in the United States, two in Canada, four in the United Kingdom and one in the United Kingdom and Germany (multi‐site). Sample sizes varied from a minimum of 24 to a maximum of 2912 (median: n = 203).

Analysis

There was large variability in the size and quality of the THN intervention studies identified: for example, many were merely descriptive reports which, while valuable communications to other practitioners, were nevertheless lacking study design or analytical rigour. Moreover, while nine studies involved systematic follow‐up, they were not considered necessarily representative of the majority of included studies due to small sample sizes. As a consequence, narrative synthesis was chosen as the more appropriate method of analysis in lieu of meta‐analysis.

In this context, the evidence was evaluated using the Bradford Hill criteria 21, a set of nine criteria (see Table 1) devised in 1965 by British epidemiologist and statistician Sir Austin Bradford Hill to assess causality when only correlational data are available: (1) strength of association, (2) consistency, (3) specificity, (4) temporality, (5) dose–response relationship, (6) plausibility, (7) coherence, (8) experimental evidence and (9) analogy. The Bradford Hill criteria are considered a standard tool to assess the impact of broad‐based public health interventions where it is not ethically feasible or operationally impractical to conduct RCTs.

Table 1

Bradford Hill criteria: definition and application to take‐home naloxone.

Criterion	Definition	Take‐home naloxone (THN)
Strength of association	The stronger the association between the exposure to a treatment and the clinical outcome, the less likely it is influenced by an external variable	How strong is the association between THN and overdose (OD) reversal?
Temporality	A cause‐and‐effect hypothesis can only find empirical support if the presumed cause precedes the effect in time	Did the distribution of THN precede a reduction in OD deaths?
Consistency	The credibility of a finding increases if different investigators can replicate it across different locations and under different circumstances	Have there been multiple observations of OD reversals as a result of THN provision?
Biological plausibility	There is stronger support for causality if there is a likely biological or pharmacological mechanism that can explain the association between exposure to a treatment and the outcome	Is it biologically plausible that a reduction in OD deaths occurs when THN is available?
Coherence	Causality between a treatment and outcome is supported when the association is coherent with current knowledge of the disease. Vice versa, conflicting or lack of supporting evidence would count against coherence	Are there documented examples of opioid OD mortality declining without THN availability? If so, does this empirical evidence conflict with the assumed association between THN and OD prevention?
Specificity	Causality can be established when one intervention leads to one specific outcome	Does THN have the unique effect of reversing opioid ODs?
Dose–response relationship	If a dose–response relationship can be observed for the cause‐and‐effect hypothesis, increased exposure to treatment will proportionally impact the clinical outcome	Does increased THN supply go hand‐in‐hand with more OD reversals?
Experimental evidence	If experimental manipulation of the exposure–outcome association impacts the outcome, (semi)experimental evidence is given. This delivers the strongest support for causation	Is there (semi)experimental evidence to support the hypothesized impact of THN on OD mortality?
Analogy	If a treatment/exposure factor similar to A leads to a clinical outcome similar to B, then this analogy counts as evidence in support of our hypothesis that A causes B	Is there a treatment similar to THN that leads to an outcome similar to OD reversal?

The Bradford Hill criteria have been applied valuably in a WHO ‘Evidence for Action’ report 22 on the effectiveness of needle‐exchange interventions in reducing HIV among IDUs. The WHO report also considered evidence according to five additional criteria relating to feasibility and implementation (see Table 2), which we include as supplementary analysis: (10) cost‐effectiveness; (11) absence of negative consequences; (12) feasibility of implementation, expansion and coverage; (13) unanticipated benefits; and (14) special populations.

Table 2

Additional feasibility and implementation criteria and application to take‐home naloxone.

Criterion	Take‐home naloxone (THN)
Cost‐effectiveness	Is THN for lay overdose reversal cost‐effective compared to treatment as usual (no intervention)?
Absence of negative consequences	Does the distribution of THN to users bear the risk of adverse events?
Feasibility of implementation, expansion, and coverage	Is it feasible to introduce THN distribution in diverse settings, including resource‐poor settings, and scale up implementation?
Unanticipated benefits	Does the distribution of THN to users lead to unanticipated benefits?
Special populations	How successful are THN programmes in reaching special populations that have been identified as particularly ‘at‐risk’ opioid users?

Where summary outcome measures (e.g. number of naloxone administrations, overdose reversals, adverse events) were calculated across studies, we sought to avoid (partial) duplication of samples by including only the study with the largest participant sample per project 20, 23 for THN projects that had produced more than one published study (i.e. Boston/Massachusetts, Los Angeles, New York, San Francisco). Vice versa, if the time‐periods covered by multiple studies from the same project could be distinguished clearly and did not overlap, all project evaluations entered analysis 24, 25, 26, 27. All summary statistics are pooled, unweighted estimates from the referenced studies. The number of overdose reversals is used as proxy for the impact of THN provision on opioid overdose mortality (aim 1), as a ratio of one fatal overdose in every 20 overdose events has been described in the literature 28, and it is impossible to ascertain for each overdose event whether, in the absence of intervention, the outcome would have been fatal or whether respiratory function would have recovered.

Results

We now present the findings from application of the nine original Bradford Hill criteria 21, followed by consideration of the extra five criteria added in the WHO report 22, 29.

Consideration according to the original Bradford Hill criteria

Strength of association

In 21 of the 22 studies, pre‐provision of naloxone was followed by use of the naloxone to reverse opioid overdose. After exclusion of four studies that possibly contained duplicate samples 30, 31, 32, 33, a total of 2336 THN administrations were found across 17 studies (see Table 3). Due the binary outcome (survival/death), the number of successful overdose reversals can be estimated by deducting the number of deaths from the number of THN administrations. By deducting the 20 confirmed deaths (1 + 1 + 2 + 6 + 10) where overdose victims did not recover following naloxone administration 12, 24, 25, 34, 35, we obtain an upper estimate of 2316 successful overdose reversals. If the four deaths where it was unclear if naloxone had been administered 23 and 63 cases (8 + 36 + 14 + 5) of naloxone administration with ‘unknown outcome’ 23, 24, 27, 34 are also counted towards fatalities following naloxone administration, a conservative, lower estimate of 2249 successful overdose reversals emerges. In the only study where THN provision did not lead to overdose reversals 11, nine of 46 programme participants witnessed a total of 16 overdoses at 6‐month follow‐up, but none administered naloxone to the overdose victims. The main reason for non‐administration was that participants did not have their naloxone supply available.

Table 3

Included studies: naloxone kits distributed and used, overdose reversals and adverse events.

Study	n	THN kits distributed	THN kits used (%)	Deaths	OD reversal after THN e	Unknown outcomes	Adverse reactions
Bennett 2011	426	426	249 (58%)	2	≥ 96%	8	NR
Bennet 2012	525	NR	28 (NR)	1	96%		NR
Dettmer 2001f	101	101	5 (5%)	0	100%		Withdrawal (NR)
Dettmer 2001f	124	124	29 (23%)	0	100%		Withdrawal (10)
Doe‐Simkins 2009d	385	385	74 (19%)	0	100%		Withdrawal (2)
Dwyer 2015d	415	56	6 (11%)	0	100%		NR
Enteen 2010	1942	2962	399 (13%)	6	≥ 89%	36	Vomiting (50), agitation (36), seizures (3)
Galea 2006	25	25	10 (40%)	1 a	100%	1a	None
Lankenau 2013d	30	30	15 (50%)	0	≥ 97%	1	NR
Leece 2013	209	209	17 (8%)	0	100%		None
Lopez‐Gaston 2009	70	70	0 (0%)	1a	NA		NA
Markham Piper 2008	122	122	82 (67%)	0	≥ 83%	14	NR
Maxwell 2006	1120	3500	319 (9%)	1 c	99%		Seizures (1), vomiting (1)
McAuley 2010	41	19	2 (11%)	1 a	100%		NR
Rowe 2015	2500	2500	702 (28%)	10	99%		NR
Seal 2005	24	24	15 (63%)	0	100%		NR
Strang 2008	239	239	1 (5%)	1 a	100%		Withdrawal
Tobin 2009	250	250	22 (9%)	0	100%		NR
Tzemis 2014	692	836	85 (10%)	0	100%		Withdrawal (55), agitation (9)
Wagner 2009	66	66	28 (42%)	4 b	NR	5	Agitation (5), vomiting (1)
Walley 2013 20	2912	2912	327 (11%)	0	100%		NR
Walley 2013 33, d	1553	1553	92 (6%)	0	100%		NR
Yokell 2011	120	120	5 (4%)	0	100%		NR

Naloxone not administered;

unclear if naloxone administered;

non‐opioids present; NA: not applicable; NR: not reported; OD = overdose; THN: take‐home naloxone;

not included in summary measures to avoid (partial) duplication of samples;

where applicable, unknown outcomes were counted towards unsuccessful THN administrations (as indicated by the ≥ symbol);

Multi‐site study with two samples: Jersey (n=101) and Berlin (n=124).

2316 overdose (OD) reversals = 2336 THN administrations minus 20 deaths (see Table 3).

2249 overdose (OD) reversals = 2336 THN administrations minus 20 deaths minus four unclear cases minus 63 cases with unknown outcome.

In summary, there is a strong association between THN programmes and overdose survival, as evidenced by at least 2249 successful overdose reversals [96.3%; 95% confidence interval (CI) = 95.5, 97.1] among 2336 THN administrations.

Temporality

In 21 of the 22 studies, training in overdose prevention and THN provision preceded overdose reversals. Two of these studies provide clear evidence in support of the temporality criterion. Supportive evidence comes from descriptive accounts of early THN distribution in Chicago and surrounding Cooks County 35: after a 135% increase in local overdose deaths from 1996 to 2000, the introduction of THN in 2001 led to reduction in fatal overdoses by 20% in 2001, 8% in 2002 and 6% in 2003 (compared to past‐year rate). While these data are indicative of a temporal sequence between THN introduction and reduced overdose mortality, no definite conclusion can be drawn, as the lack of control group means that other causes may have contributed to decreasing overdose mortality rates.

Stronger evidence comes from Walley et al. 20 who conducted an evaluation of a state‐funded THN programme in Massachusetts. Between 2006 and 2009, the Massachusetts Department of Public Health used a phased roll‐out to introduce THN in 19 communities, enrolling 2912 individuals in total. To evaluate the impact of THN, Walley et al. used an interrupted time–series analysis, where each community served as its own geographic control and communities without concurrent THN availability served as time control. For all 19 participating communities, overdose mortality rates in the time‐periods before and after THN implementation were compared. Overdose mortality rates were reduced significantly in communities where THN was implemented, compared to pre‐implementation rates and to communities without THN.

Consistency

Overdose reversals by means of THN have been documented in the selected studies by independent investigators under different circumstances in at least 15 different cities, states and countries: in Canada (Toronto and British Columbia), the United States (Baltimore, Boston/Massachusetts, Chicago, Los Angeles, San Francisco, New York, Pittsburgh, Rhode Island), the United Kingdom (England, Jersey, Scotland, Wales) and Germany (Berlin). Overdose reversals by THN have also been documented repeatedly in New York 26, 27 and San Francisco 14, 24, 25. In conclusion, there is substantial support for the consistency criterion.

Biological plausibility

This criterion addresses the therapeutic effect of naloxone. Naloxone is a pure opioid antagonist that binds to the μ‐opioid receptor and blocks competing agonists, such as heroin 36. All but one study 11 reported on THN administration in cases of suspected opioid overdoses, and the pharmacological effects of naloxone led to at least 2249 overdose reversals. In conclusion, there is strong empirical support to the biological plausibility criterion.

Coherence

Declining overdose rates in the absence of THN have been reported in the literature. The Australian heroin drought constitutes a prominent example, where between 2001 and 2002 overdose‐related mortality rates dropped in conjunction with a shortage in illicit heroin imports. THN could not have accounted for the decline in mortality, as it was introduced in Australia only in 2011 37, 38. However, the Australian example does not conflict with the presumed effect of THN on reduced overdose mortality. The cause‐and‐effect interpretation of our data is consistent with current understanding of the mechanisms of opioid overdose, and the 21 studies which reported overdose reversals provide strong support for the coherence criterion.

Specificity

The specificity criterion relates to efficacy of the intervention (the same as biological plausibility), rather than population‐wide effectiveness. THN exclusively reverses opioid‐induced overdoses, as illustrated by the following two cases: in the Dettmer et al. study 39, naloxone had zero effect when administered to a person suffering from cocaine intoxication. The Chicago Recovery Alliance reported one fatality after naloxone administration 35 where naloxone failed to revive an overdose victim with non‐opioids in their system. The mooted benefit from naloxone is specific to opioid overdose. In practice, THN may be primarily beneficial for the reversal of overdoses from heroin and other short‐acting opioids. (All 22 studies reported primarily on heroin overdoses, and one study specified that the long‐acting opioid methadone was involved in less than 5% of overdose reversals [33].) Overall, the evidence constitutes strong support for this criterion.

Dose–response relationship

Researchers estimate that THN distribution can only achieve maximum impact on overdose reduction if a certain volume of THN kits is available in the community. Among the 22 studies, only Walley et al. 20 assessed the impact of varying degrees of THN availability on overdose mortality by splitting the 19 participating communities into three groups based on volume of THN distribution: zero implementation, low implementation (1–100 programme enrolments per 100 000 inhabitants) and high implementation (>100 enrolments). Both low and high implementers had significantly reduced overdose mortality rates compared to communities without implementation, and there was a significant implementation dose‐relationship with overdose death rates, with greatest effect with greatest implementation.

To summarize, there is only this limited empirical evidence for a dose‐related impact of THN availability, and hence this criterion is only partially fulfilled.

Experimental evidence

While none of the 22 studies deliver experimental evidence, the interrupted time–series analysis by Walley et al. 20 provides quasi‐experimental evidence in support of causation. Importantly, even communities with low‐level THN implementation of THN (1–100 participants, see above) saw a reduction in overdose mortality, compared to communities without THN distribution. Interrupted time–series analysis is considered to be the strongest quasi‐experimental research design 40. The results of the study by Walley et al. 20 thus provide preliminary support for the experimental evidence criterion.

Analogy

THN is analogous to naloxone treatment for the same clinical indication in emergency medical care, and also to the prescription of other emergency medications (typically antidotes for overdose or poisoning) for peer administration: THN has been compared to the provision of adrenaline injection kits (e.g. EpiPen) to individuals with severe allergic reactions for family members to administer in the event of anaphylactic shock 15 or the provision of glucagon for insulin overdose 35. Similarly, THN has been likened to pre‐placement of defibrillators and cardiopulmonary resuscitation (CPR) training for lay people likely to witness cardiac arrest 41. For all these emergency interventions, timely delivery is crucial. We conclude that the analogy criterion is fulfilled.

Consideration according to additional feasibility and implementation criteria

Cost‐effectiveness

Separate modelling data from both the United States and Russia conclude that THN is cost‐effective even under conservative circumstances, i.e. when the cost of naloxone increases and the rate of observed overdoses decreases 42, 43. Bearing in mind the potential limitation that both studies were conducted by the same authors, there is consistent evidence for the cost‐effectiveness of THN.

Absence of negative consequences

In five of the 17 studies that did not contain duplicate samples, 20 overdose victims did not survive naloxone administration 12, 24, 25, 34, 35. In addition, Wagner et al. 23 reported four deaths where it was unclear if naloxone had been administered. Based on these observations, the following fatality rates emerge: 20 confirmed deaths per 2336 naloxone administrations (0.9%; 95% CI = 0.5, 1.2) or 24 deaths per 2336 naloxone administrations (1.0%; 95% CI = 0.6, 1.4) if we include the four fatalities where it was unclear if naloxone had been administered. If we limit the study selection to the nine papers with systematic follow‐up, a similar ratio of one confirmed death per 123 naloxone administrations (0.8%; 95% CI = 0.4, 1.2) was observed.

In six 15, 23, 24, 35, 39, 44 of the 17 studies, several adverse reactions were reported in conjunction with a total of 2336 naloxone administrations: at least 65 instances of withdrawal symptoms (2.8%), 52 cases of vomiting (2.2%), 50 cases of agitation (2.1%) and four seizures (0.1%).

In conclusion, THN programmes have a low rate of adverse events. Where adverse reactions occurred, these were most frequently symptoms of opioid withdrawal (including nausea/vomiting, agitation).

Feasibility of implementation, expansion and coverage

The 22 studies document THN implementation in a variety of settings across 16 geographical locations, and naloxone usage rates between 5 and 63% are reported. San Francisco is an example of rapid expansion, as the volume of THN kits distributed increased from 24 in 2001 to 2962 kits during the 6‐year period between 2003 and 2009 (i.e. approximately 494 kits/year) 24, and to 2500 kits from 2010 to 2013 (i.e. approximately 833 kits/year) 25. Outside the 22 studies included in this review, implementation in resource‐poor settings has been achieved in Kyrgyzstan and Tajikistan, with reported naloxone usage rates of 47 and 78%, respectively 45. These studies suggest that THN schemes are capable of implementation across a wide range of settings and cultures.

Unanticipated benefits

Four of the 22 studies reported unanticipated benefits. In THN programmes in California, 25% of participants in San Francisco entered treatment within 6‐month follow‐up 14 and 53% of participants in Los Angeles reported decreased drug use at 3‐month follow‐up 23. Similarly, Maxwell et al. reported anecdotal evidence of increased willingness among THN recipients to be tested for HIV and hepatitis C virus (HCV) 35. Strang et al. 15 found a secondary training effect: within a 3‐month follow‐up period, 28% of THN recipients had trained a family member or peer.

Special populations

THN provision has been implemented successfully in programmes targeting special populations with high risk of overdose: detox patients 11, 33, homeless users 23, 24, 25, 27, 46, methadone patients 33 and prison inmates 12. The Massachusetts THN programme 20 also enrolled attendees of HIV education centers, and a Los Angeles‐based programme recruited more than 50% HCV‐positive patients. Both represent particularly vulnerable groups due to their comorbid health issues and risk of blood‐borne virus transmission by needle‐sharing. From the perspective of implementation, THN schemes can be delivered to populations in special need.

Summary of findings

Empirical evidence from the 22 studies reporting on THN interventions for opioid users meets all nine Bradford Hill original criteria. Among these, Sir Austin Bradford Hill considered the experimental evidence criterion to deliver the strongest support for causation 21, but only quasi‐experimental evidence from one study 20 is available here. The robustness of empirical support ranges from one study per criterion (dose–response, experimental evidence) to 21 studies per criterion (strength of association, coherence) (see Supporting information, Appendix S1). With regard to the five additional criteria assessing feasibility and implementation, THN fulfils fully or partially all five criteria. It is found to be cost‐effective, and existing projects were able to access and train high‐risk populations that led to 2336 layperson naloxone administrations (aim 1) with a low rate of adverse effects (aim 2).

Discussion

Application of the Bradford Hill criteria to the current evidence base on THN supports the causation hypothesis. While the evidence is sometimes based on only one or two studies, we nevertheless conclude that this constitutes support for all nine criteria. THN provision reduced fatal outcome of overdose among programme participants themselves, among fellow opioid users and in the wider community, as evidenced by public vital statistics records 14, 20. Alternative explanations for this observation are unlikely: in control communities that did not implement THN, opioid overdose mortality was significantly higher 20. The risk associated with THN programmes is relatively low, especially when the life‐threatening nature of the emergency situation is borne in mind: in studies with systematic follow‐up, one death was reported among 123 overdose victims who were administered THN. Moreover, there is no empirical evidence to support the concern that THN programmes might encourage heroin use. Two studies reported decreased drug use among THN programme participants at follow‐up 14, 23, whereas a more recent study found no overall change in the frequency of heroin use across THN recipients 47.

This is the first published application of the Bradford Hill criteria to assess the international evidence base on THN. Our findings extend and substantiate the 2014 WHO Guidelines as well as the results of the previous systematic reviews by Clark et al. 10 and the European Monitoring Centre for Drugs and Drug Addiction (EMCDDA) 17. Clark et al. (2014) cautiously concluded: ‘participation [in THN programs] is associated with overdose reversals’ (p. 162), but avoided statements on the effectiveness of THN, whereas the EMCDDA stated: ‘there is evidence that educational and training interventions with provision of take‐home naloxone decrease overdose‐related mortality’ (p. 11).

There are potential limitations to this analysis, which need to be borne in mind. Selection bias may have affected the internal validity of the data included. Among 19 studies with pre–post and case series designs, 10 relied on unsystematic follow‐up to capture overdose events and naloxone usage, relying upon spontaneous follow‐up, with THN programme participants asked typically to report back on naloxone usage when collecting a naloxone refill. This raises scientific analytical doubt about data quality and interpretations: first, across these 10 studies, fewer than a quarter (22.9%; i.e. 1973 of 8602) of THN recipients returned for refills after THN use, and information on the majority of participants was consequently lost. Secondly, it is possible that users with positive naloxone experiences (e.g. successful overdose reversals) may be more likely to return for a refill of their THN kit and complete a follow‐up survey, whereas those with negative naloxone experiences may not be captured in the follow‐up. The lack of systematic follow‐up in the majority of studies is reflected in the wide range of follow‐up rates attained across all studies (min. 5%, max. 100%). High levels of dropout can reduce the external validity and generalizability of results. A further source of potential bias lies in the fact that, for 21 of the 22 studies, there was an exclusive reliance on self‐report data for overdose outcomes. Only the interrupted time–series analysis by Walley et al. 20 included a public database of vital statistics to calculate overdose fatality rates. A further limitation concerns the fact that the experimental evidence and dose–response criteria hinge on data from the Walley et al. 20 study. More well‐conducted studies are needed to confirm these results and assess their applicability to other regions internationally, in particular low‐ and middle‐income countries. Moreover, the findings from the studies do not inform which distribution model of overdose education and THN distribution is preferable. Future studies could evaluate the impact of programme components formally by providing THN to all subjects and randomizing subjects into different training conditions (e.g. ‘overdose education’ versus ‘overdose education + CPR training’).

Despite these methodological limitations, positive reports of overdose reversals following THN distribution were reported across 21 studies, regardless of type of follow‐up (systematic versus unsystematic) or data source (self‐report versus objective data), suggesting that the finding is indeed robust and not an artefact of methodological flaws.

To control for potential publication bias, we additionally searched the grey literature for documents reporting on THN initiatives that are not published in the peer‐reviewed journal domain. While this search was probably not exhaustive, the data reported in the grey literature are broadly consistent with the results of the studies included in our systematic review. For instance, in the Scottish National Naloxone Programme, in 2012 and 2013 the percentage of opioid‐related deaths occurring within 4 weeks of prison release (5.5 and 4.7%) was almost half that of the pooled 2006–10 baseline indicator (9.8%), suggesting that distribution of naloxone kits on release may reduce the risk of fatal overdose among (former) prisoners with history of opioid use 48.

With regard to clinical implications, it needs to be emphasized that the vast majority of studies included in this review reported on heroin overdoses. Consequently, the generalizability of our findings to overdoses from long‐acting opioids is unclear. Even when methadone patients were recruited specifically into a THN programme 33, more than 90% of witnessed (and reversed) overdoses were heroin‐induced. The results of this review on the effectiveness of THN are thus limited to impact on heroin overdoses, and the effectiveness of the intervention for overdoses from long‐acting opioids (e.g. methadone or many prescription opioids) needs to be explored in future research.

To conclude, application of the Bradford Hill criteria to the current evidence base from non‐randomized studies finds that THN programmes have led to improved survival rates among programme participants and reduced heroin overdose mortality rates in the community (aim 1) and are accompanied by only a low rate of adverse events (aim 2). In the absence of RCTs, we conclude that THN distribution to at‐risk users should be introduced as standard of care for the community‐based prevention of heroin overdose deaths.

Declaration of interests

R.M. has no interests to declare, except that R.M. and J.S. declare that King's College London (employer for both R.M. and J.S.) has registered intellectual property on a novel buccal naloxone formulation with which J.S. and R.M. are involved. J.S. declares that he is a researcher and clinician who has worked with a range of types of treatment and rehabilitation service‐providers. He has also worked with a range of governmental and non‐governmental organizations and with pharmaceutical companies to seek to identify new or improved treatments (including naloxone products), and from whom he and his employer (King's College London) have received research funding, honoraria, travel costs and/or consultancy payments. J.S. has also been named in a patent registration by a Pharma company as inventor of a further new naloxone formulation. For a fuller account of J.S.'s interests, see his personal web‐page for King's College London at: http://www.kcl.ac.uk/ioppn/depts/addictions/people/hod.aspx. J.S. is supported by the National Institute for Health Research (NIHR) Biomedical Research Centre for Mental Health at South London and Maudsley NHS Foundation Trust and King's College London.

Appendix S1: Search protocol

Supporting info item

Click here for additional data file.