Literature DB >> 23038795

Opiate substitution treatment and HIV transmission in people who inject drugs: systematic review and meta-analysis.

Georgie J MacArthur1, Silvia Minozzi, Natasha Martin, Peter Vickerman, Sherry Deren, Julie Bruneau, Louisa Degenhardt, Matthew Hickman.   

Abstract

OBJECTIVE: To quantify the effect of opiate substitution treatment in relation to HIV transmission among people who inject drugs.
DESIGN: Systematic review and meta-analysis of prospective published and unpublished observational studies. DATA SOURCES: Search of Medline, Embase, PsychINFO, and the Cochrane Library from the earliest year to 2011 without language restriction. REVIEW
METHODS: We selected studies that directly assessed the impact of opiate substitution treatment in relation to incidence of HIV and studies that assessed incidence of HIV in people who inject drugs and that might have collected data regarding exposure to opiate substitution treatment but not have reported it. Authors of these studies were contacted. Data were extracted by two reviewers and pooled in a meta-analysis with a random effects model.
RESULTS: Twelve published studies that examined the impact of opiate substitution treatment on HIV transmission met criteria for inclusion, and unpublished data were obtained from three additional studies. All included studies examined methadone maintenance treatment. Data from nine of these studies could be pooled, including 819 incident HIV infections over 23,608 person years of follow-up. Opiate substitution treatment was associated with a 54% reduction in risk of HIV infection among people who inject drugs (rate ratio 0.46, 95% confidence interval 0.32 to 0.67; P<0.001). There was evidence of heterogeneity between studies (I(2)=60%, χ(2)=20.12, P=0.010), which could not be explained by geographical region, site of recruitment, or the provision of incentives. There was weak evidence for greater benefit associated with longer duration of exposure to opiate substitution treatment.
CONCLUSION: Opiate substitution treatment provided as maintenance therapy is associated with a reduction in the risk of HIV infection among people who inject drugs. These findings, however, could reflect comparatively high levels of motivation to change behaviour and reduce injecting risk behaviour among people who inject drugs who are receiving opiate substitution treatment.

Entities:  

Mesh:

Substances:

Year:  2012        PMID: 23038795      PMCID: PMC3489107          DOI: 10.1136/bmj.e5945

Source DB:  PubMed          Journal:  BMJ        ISSN: 0959-8138


Introduction

Use of injected drugs is a major risk factor for the acquisition and transmission of HIV, and about 5-10% of HIV infections are attributable to injecting drug use worldwide.1 Transmission of HIV between people who inject drugs is predominantly a result of the sharing of contaminated injecting equipment, but also sexual transmission, both of which are influenced by wider structural and environmental factors such as housing, patterns of drug use, commercial sex work, and the availability and nature of interventions aimed at reducing harm.2 3 4 Estimates of the prevalence of HIV among people who inject drugs are over 40% in many parts of the world, and around three million (range 0.8-6.6 million) of the 16 million (range 11-21 million) people who inject drugs worldwide could be HIV positive.5 6 Although HIV incidence has fallen or remains low among people who inject drugs in North America, Australasia, and parts of Western Europe,3 7 8 9 injecting drug use has driven recent outbreaks in Europe,10 as well as epidemics in cities in the United States, United Kingdom, Thailand, Russia, India, China, Vietnam, and Estonia,4 11 12 13 14 and HIV incidence is increasing in parts of Russia, Eastern Europe, and Asia, where the burden of HIV among people who inject drugs is comparatively higher.5 6 15 Methadone, an opioid agonist, and buprenorphine, a partial agonist, are the main pharmacological drugs prescribed for people dependent on opiates, and they were included in the list of World Health Organization (WHO) essential medicines in 2005. They are provided in oral, film, or sublingual form and are often prescribed as opiate substitution treatments for people with established opiate dependence.16 In addition to reducing craving for, and use of, heroin or other illicit opiates, there is good evidence to suggest that opiate substitution treatment reduces drug related mortality, offending, and injection risk behaviour, and that it improves adherence to antiretroviral treatment for those with HIV.17 18 19 20 21 22 Methadone and buprenorphine can also be prescribed in tapering doses for detoxification. Opiate substitution treatment has been implemented in 70 countries, but remains unavailable in 66, and in several countries detoxification or residential rehabilitation is the primary mode of treatment.2 23 Although coverage of opiate substitution treatment is being expanded in some countries, such as Ukraine, India, and China,24 25 26 intervention at a regional and national level remains poor in many countries, including those with a high burden of HIV among people who inject drugs.23 Previous reviews have generally concluded that reductions in the frequency of injection and sharing of injecting equipment that are associated with exposure to opiate substitution treatment translate into reductions in cases of HIV infection.21 The conclusions of these reviews, including a recent Cochrane systematic review, however, have been based on a small number of primary studies with limited sample size.19 21 27 Critically, no pooled analysis has been carried out to date, and as such, there remains no quantitative estimate of the effect of opiate substitution treatment in relation to HIV transmission. We carried out a systematic review and meta-analysis of published and unpublished observational studies to quantify the association between opiate substitution treatment and risk of HIV transmission among people who inject drugs.

Methods

Primary and secondary objectives

Our primary objective was to assess the impact of opiate substitution treatment in relation to HIV incidence among people who inject drugs. Secondary research objectives were to examine the effect of variables including mode of treatment (such as methadone or buprenorphine; continuous or interrupted treatment; detoxification treatment); duration of treatment; geographical region; study setting (such as drug treatment clinic or other setting); and characteristics of participants (such as age, sex).

Search strategies

We carried out two separate systematic searches to identify relevant studies. The first search identified studies that directly examined the impact of opiate substitution treatment in relation to HIV incidence. We searched Medline, Embase, and PsycINFO from the earliest possible year to October 2011 using the Ovid platform with no limit to language and searched the Cochrane Library up to 2011. Reference lists of robust literature reviews were assessed to identify relevant studies. Briefly, terms included those relating to HIV infection or transmission; intravenous substance use, misuse, addiction, or dependence; and substitution, maintenance, methadone or buprenorphine treatment (table A in appendix 1). The search included exploded MESH terms and text words to enhance retrieval of relevant studies. Secondly, we searched Medline, Embase, and PsycINFO up to May 2011 to identify prospective cohort studies that reported HIV incidence in people who inject drugs. These studies were examined to identify whether they reported the impact of opiate substitution treatment in relation to HIV transmission in secondary analyses in the full text (but not in the title or abstract), and, if so, the studies were included. Authors of studies of HIV incidence in people who inject drugs that did not report opiate substitution treatment as a covariate were contacted in case data regarding exposure to had been collected but not yet published. The search strategy used similar terms to the first search but was limited to longitudinal or cohort studies (table A in appendix 1).

Study selection

After export of all identified studies to Reference Manager 12 and removal of duplicates, two reviewers screened titles and abstracts and disagreements were resolved by discussion. Two reviewers screened full text copies of relevant articles to determine whether they met eligibility criteria for inclusion and suitability for inclusion in the meta-analysis or for contact of study authors. Full text papers in languages other than English were translated by individuals fluent in those languages or, for one paper, by Google translate.

Data extraction and analysis

Two authors used piloted forms to extract data from studies, including details of studies (such as year of publication, country, study design); study characteristics (such as inclusion criteria, site of recruitment or study); intervention (such as methadone or buprenorphine maintenance treatment; continuous versus interrupted treatment; detoxification treatment only); and intervention effect (seroconversions, follow-up period, incidence rates, or unadjusted and adjusted measures of effect, including incidence rate ratio or hazard ratio, and confounders used in adjusted estimates).

Eligibility criteria

We included studies in this review if they were randomised controlled trials, prospective cohort studies, or case-control studies that directly examined the impact of opiate substitution treatment in relation to HIV incidence in people who inject drugs; or if they prospectively examined HIV incidence in people who inject drugs but did not report data regarding the impact of opiate substitution treatment. We included studies that assessed the impact of continuous or interrupted opiate substitution treatment; current, recent, or previous opiate substitution treatment; or opiate substitution treatment at baseline. We excluded cross sectional or serial cross sectional studies and those studies identifying the outcome from retrospective analysis of routine medical records to identify outcome or exposure to opiate substitution treatment (in the latter case they were considered subject to selection bias because of different motivations and characteristics of individuals undergoing voluntary testing). We also excluded studies carried out in prisons. Studies were included only if data relating to opiate substitution treatment were reported in opiate injectors. We excluded studies that reported fewer than two seroconversions during follow-up to ensure that estimates generated were sufficiently precise. Participants of the included studies were people who inject opiates with no restriction around age, sex, ethnic group, or socioeconomic group. Duplicate papers from the same cohort study were grouped, and studies with the largest number of seroconversions or that reported adjusted and unadjusted analyses, or both, were selected.

Assessment of risk of bias

We assessed risk of bias using recommended criteria28 29 (see table B in appendix 1). Studies were judged to be at low, high, or unclear risk of bias on the basis of what was reported in the study for each of these domains. Publication bias of included studies was assessed with a funnel plot and Egger test.30 31

Summary measures and synthesis of results

We used Poisson regression to obtain incidence rate ratios and 95% confidence intervals from raw data in unpublished datasets provided (A Judd, 2012, and S Deren, 2012) and where information regarding seroconversion and person years of follow-up was provided in published studies. When only effect estimates were reported, we directly extracted unadjusted and adjusted estimates if they were reported as odds ratio, incidence rate ratio, or hazard ratio with 95% confidence intervals. As the incidence of HIV in included studies was low (<10/100 person years), effect estimates that were reported as odds ratios were extracted directly as it was assumed that the odds ratio was a reasonable approximate of the relative risk.32 Studies that reported data relating to exposure to opiate substitution treatment and HIV seroconversion were pooled for meta-analysis if data were provided regarding HIV seroconversions, person years of follow-up, and exposure to opiate substitution treatment; or an effect estimate of the impact of opiate substitution treatment in relation to HIV incidence with 95% confidence intervals. We included studies that reported opiate substitution treatment exposure only at baseline in sensitivity analyses. We excluded studies that examined methadone maintenance treatment compared with methadone detoxification treatment from the primary meta-analysis but included them in separate subgroup analyses. As we expected heterogeneity between studies, we used a random effects meta-analysis for the primary analyses, allowing for heterogeneity between and within studies. Adjusted and unadjusted effect estimates were pooled in separate meta-analyses. We examined heterogeneity with the I2 statistic and χ2test and explored reasons for heterogeneity using univariable random effects meta-regression (using the “metareg” command)33 to compare subgroups by geographical region of study; the provision of monetary incentives; site of recruitment; the examined duration of exposure to opiate substitution treatment; and the percentage of female participants or individuals from ethnic minorities. Sensitivity analyses were conducted to include studies that assessed the impact of exposure to methadone maintenance treatment at baseline only; the impact of inclusion only of studies at lower risk of bias (that is, scoring as low risk across three or fewer domains with the risk of bias tool); including only studies that measured an incidence rate ratio; and the impact of excluding those that did not adjust for confounders. All analyses were carried out with STATA version 11.

Results

Figures 1 and 2 show the number of studies identified, reviewed, and selected and the reasons for exclusion. The first search enabled the identification of seven eligible studies, four of which included data that could be included in the quantitative synthesis (fig 1). Three studies were excluded on the basis that no HIV seroconversions were identified in either treatment arm.34 35 36

Fig 1 Selection of studies directly investigating impact of opiate substitution treatment on HIV transmission

Fig 2 Selection of prospective cohort studies relating to HIV incidence in people who inject drugs. Of seven identified, two were included in sensitivity or subgroup analyses only

Fig 1 Selection of studies directly investigating impact of opiate substitution treatment on HIV transmission Fig 2 Selection of prospective cohort studies relating to HIV incidence in people who inject drugs. Of seven identified, two were included in sensitivity or subgroup analyses only The second search (fig 2) enabled identification and inclusion of 10 eligible papers that included data directly relating to the impact of exposure to opiate substitution treatment in relation to HIV transmission, of which five were newly identified studies not identified in the first search.8 11 37 38 39 In addition, we identified 16 eligible prospective studies that measured HIV incidence among people who inject drugs and contacted authors of these articles. Of these, unpublished data were obtained from cohort studies in London, UK (A Judd, personal communication, 2012); Montreal, Canada (J Bruneau, personal communication, 2012); and a dual site study of Puerto Rican injecting drug users in Bayamon, Puerto Rico, and East Harlem, New York City (S Deren, personal communication, 2012). Overall, eight studies were included from this search, seven of which could be included in the quantitative synthesis (fig 2). In the second search (fig 2), we excluded one study because no HIV seroconversions occurred among participants,40 and two studies that constructed a retrospective cohort based on clinical records of voluntary testing for hepatitis C virus and HIV.41 42 Seventy one studies were duplicates from the same cohort (particularly the Amsterdam Cohort Study; Vancouver Injection Drug User Study; and AIDS Linked to Intravenous Experience (ALIVE) cohort study). We therefore included 12 published studies8 11 17 37 38 39 43 44 45 46 47 48 and the three unpublished studies, comprising 1016 incident HIV infections and over 26 738 person years of follow-up.

Study characteristics

Table 1 summarises the characteristics of the 15 included studies undertaken in the US (including unpublished data from S Deren, 2012),8 39 43 44 46 Canada (including unpublished data from J Bruneau, 2012),37 UK (unpublished data from A Judd, 2012), the Netherlands,45 Austria,47 Italy,48 Thailand,11 17 Puerto Rico (unpublished data from S Deren), and China.38 All of the included studies reported the impact of methadone maintenance treatment. The included studies were variable in terms of sample size (range 80-2546), method of recruitment (drug treatment clinic (six studies); community settings and outreach (six studies); drug treatment clinics and community settings (three studies); duration of follow-up (range 1-20 years), and year of publication (1992-2009).
Table 1

 Characteristics of included studies of opiate substitution treatment (OST) and impact on HIV transmission

StudyFollow-upStudy typeRecruitment site/methodInclusion criteriaSample size*HIV incidence (95% CI)†InterventionImpact of treatment on HIV seroconversionConfounders included in adjusted analyses
Effect measureEffect estimate (95% CI)
Studies included in meta-analysis
Bruneau, 2012 (unpublished)‡ Montreal Canada1992-2008CohortStreet, word of mouth, community programmesAge >18; injected for past 6 months13742.57 (2.0 to 3.2)MMT in previous 6 months (yes/no)HR0.2 (0.0 to 0.7), adjusted 0.3 (0.1 to 1.3)Age, sex, housing, cocaine use, heroin use, sharing, booting, having sex with person known to be HIV positive, period of recruitment
Chitwood, 1995, Miami, US391988-91Nested case-controlDrug treatment programme; streetCases: participants testing positive for HIV infection 1988-1991. Control: participants testing negative for HIV infection within 2 week period before date of first positive test of case972.1Methadone treatment in year before seroconversion or last negative test date (yes/no)OR0.3 (0.1 to 0.9), adjusted 0.7 (0.2 to 2.8)Age, sex, ethnicity
Deren, 2012 (unpublished), Bayamon, Puerto Rico1998-2002CohortTargeted sampling based on ethnographic mappingAge ≥18, injected drugs in previous 30 days, use of heroin in previous 48 hours, self identification as Puerto Rican1433.7 (2.3 to 6.1)MMT at baseline (yes/no)IRR0.2 (0.0 to 3.5)
Deren, 2012 (unpublished), New York City, USAs aboveAs aboveAs above1540.83 (0.3 to 2.2)MMT at baseline (yes/no)IRR0.3 (0.0 to 2.4)
Judd, 2012 (unpublished), London, UK2001-02CohortCommunity, such as needle and syringe exchange programmes or street settingsAge <30; injecting ≤6 years; injected previous 4 weeks; address for follow-up2722.9OST for ≥6 months in last year (yes/no)IRR0.8 (0.2 to 3.2), adjusted 0.8 (0.2 to 3.2)Crack injection, homelessness
Kerr, 2006, Vancouver, Canada371996-2002CohortStreet outreach; self referralInjected drugs in past month; resident in Vancouver region10133.2 (2.6 to 3.7)Methadone treatment in previous 6 months (yes/no)RH0.7 (0.4 to 1.1), adjusted 0.8 (0.5 to 1.4)Sex, aboriginal ethnicity, frequent cocaine injection, frequent heroin injection, requiring help injecting, binge drug use, having HIV positive sexual partner‡
Metzger, 1993, Philadelphia441989-91CohortOutpatient methadone programme; community outreachMMT: methadone programme participation; no MMT: >18 years; history of intravenous opiate use (≥3/week); no drug treatment in previous 10 months1853.0 in treatment; 10.7 out of treatmentMMT over 18 month period (yes/no)OR0.1 (0.0 to 0.5), adjusted 0.2 (0.1 to 0.6)Age, sex, ethnicity, needle sharing
Nelson, 2002, Baltimore, US81988-98CohortCommunity outreachΑge ≥18; free from AIDS at baseline; injected drugs in past 10 years18463.1 (2.8 to 3.5)MMT in previous 6 mo (yes/no)IRR0.6 (0.3 to 0.9)
Suntharasamai, 2009, Bangkok, Thailand171999-2003Nested cohort in RCTDrug treatment clinics and communityHIV negative; history of injected drug use in previous year; age 20-60; informed consent25463.4 (3.0 to 3.9)MMT in previous 6 months (yes/no)HR0.8 (0.6 to 1.0), adjusted 0.60 (0.4 to 0.8)Incarceration, injecting drugs, injecting daily, sharing injecting equipment, participation in methadone detoxification programme§
Van den Berg, 2007 Amsterdam, Netherlands451985-2005CohortDrug treatment clinics and self referralHistory of ever injecting drugs; negative for HIV and/or HCV negative at study entry or at start of injecting drug use7101.65Any dose of MMT daily in previous 6mo (yes/no)IRR0.4 (0.2 to 0.5)
Vanichseni, 2001, Bangkok, Thailand111995-98CohortDrug treatment clinicHistory of injected drug use; age 18-50; attendance at Bangkok Metropolitan Administration drug treatment clinics; not known to be HIV seropositive11245.8 (4.8 to 6.8)MMT or detoxification in previous 4 months (yes/no)IRR0.8 (0.5 to 1.3)
Williams, 1992, New Haven, US461992-98CohortMethadone maintenance programmeHistory of injected drug use; past/current client of MMT programme; age ≥18 and one of three additional criteria**982.8Continuous MMT v interrupted MMT during follow-upIRR0.2 (0.0 to 1.3)
Studies not included in meta-analysis
Moss, 1994 San Francisco, US431985-90CohortMethadone maintenance/ detoxification programmes; hospitalHIV negative heterosexual people who inject drugs tested at least twice6811.9≤12 lifetime months in MMT (yes/no)HR5.1, adjusted 4.0Age, sex, ethnicity
Ruan, 2007, Xichang City, China382002-05CohortCommunity outreach and peer referralHIV seronegative; age ≥18; injected of drugs in previous 3 months; willing to provide informed consent2292.3 (1.1 to 3.5)MMT in previous 6 months (yes/no)MMT: 0 sc No MMT: incidence 2.9/100 person years
Serpelloni, 1994, Verona, Italy481985-91Nested case-controlDrug dependency unitHIV seronegative injecting drug users80No of cycles of MMT; daily dose; time out of treatment (previous 12 months)See footnote††
Zangerle, 1992, Innsbruck, Austria471985-90CohortDrug dependency or AIDS clinicInjecting drug users attending drug dependency or AIDS clinic of University of Innsbruck1025.8MMT for ≥2 months duration (yes/no)MMT: 0/43 sc over mean 1.6 year; No MMT: 10/59 sc over mean 1.76 years (incidence 9.6/100 person years)

sc=seroconversions; MMT=methadone maintenance treatment; IRR=incidence rate ratio; RH=relative hazard; HR=hazard ratio; OR=odds ratio.

*Size of sample of HIV negative participants in analysis of impact of opiate substitution treatment on HIV incidence.

†As rate per 100 person years. Incidence relates to entire study period when such an estimate was provided in study.

‡Effect estimate adjusted for all variables significantly associated with time to HIV infection in univariate analyses at P≤0.1, as listed. Variables refer to preceding 6 months

§Effect estimate adjusted for all variables significantly associated with incident HIV infection at P≤0.1 in bivariate analysis, as listed.Variables refer to preceding 6 months.

¶Study participants with “no dependence” were excluded from analysis to calculate this estimate from reported data.

**Additional criteria for study entry included: had participated in 1982-83 study of hepatitis and had identifiable serum sample from that study in serum bank of researchers; had been in methadone programme consistently since 1982; or entered methadone programme between October 1985, and July 1988. Entry into 1982-83 hepatitis study required mild increases in transaminase activity without serological evidence of active hepatitis B infection at time of study (1982-83).

††Lower dose and more time out of treatment associated with increased risk of HIV infection in univariate but not multivariate analyses. Number of cycles of MMT not significantly associated with risk of HIV infection.

Characteristics of included studies of opiate substitution treatment (OST) and impact on HIV transmission sc=seroconversions; MMT=methadone maintenance treatment; IRR=incidence rate ratio; RH=relative hazard; HR=hazard ratio; OR=odds ratio. *Size of sample of HIV negative participants in analysis of impact of opiate substitution treatment on HIV incidence. †As rate per 100 person years. Incidence relates to entire study period when such an estimate was provided in study. ‡Effect estimate adjusted for all variables significantly associated with time to HIV infection in univariate analyses at P≤0.1, as listed. Variables refer to preceding 6 months §Effect estimate adjusted for all variables significantly associated with incident HIV infection at P≤0.1 in bivariate analysis, as listed.Variables refer to preceding 6 months. ¶Study participants with “no dependence” were excluded from analysis to calculate this estimate from reported data. **Additional criteria for study entry included: had participated in 1982-83 study of hepatitis and had identifiable serum sample from that study in serum bank of researchers; had been in methadone programme consistently since 1982; or entered methadone programme between October 1985, and July 1988. Entry into 1982-83 hepatitis study required mild increases in transaminase activity without serological evidence of active hepatitis B infection at time of study (1982-83). ††Lower dose and more time out of treatment associated with increased risk of HIV infection in univariate but not multivariate analyses. Number of cycles of MMT not significantly associated with risk of HIV infection. Most studies reported the impact of methadone maintenance treatment as one of a range of factors assessed in relation to the risk of HIV infection and most reported an associated lower risk of HIV infection (unpublished data from S Deren and J Bruneau, 2012).8 38 39 45 47 48 Four studies could not be included in the meta-analyses because of the comparisons made or the nature of the data presented (table 1).38 43 47 48 Among these studies, Moss and colleagues reported a lower incidence or risk of HIV among people who inject drugs with more than 12 lifetime months of methadone maintenance treatment compared with those with less than 12 months, and a lower seroconversion rate among those recruited from methadone maintenance treatment programmes compared with those recruited from detoxification programmes or hospital.43 Serpelloni and colleagues reported a lower risk of seroconversion associated with methadone dose and cycles of treatment in univariate but not multivariate analyses.48 Two studies38 47 reported HIV seroconversions among people who inject drugs out of treatment but not among those receiving treatment over follow-up (table 1). All of the included studies were judged to be at high risk of selection bias and low risk of performance bias and most were at low risk of ascertainment bias. A few studies, however, were at high risk of attrition bias (table 2 and table C in appendix 1) and few adjusted for confounders.
Table 2

 Risk of bias in included studies assessed with criteria drawn from Newcastle-Ottawa scale and EPOC group, adapted for assessment of randomised controlled trials, case-control trials, and prospective observational studies according to criteria recommended by Cochrane Drugs and Alcohol Review Group28 29

StudySelection biasPerformance bias*Detection bias†Attrition bias‡Other biasAscertainment bias¶
Random sequence generationAllocation concealmentComparability of cohorts§Selection of non-exposedContamination
Bruneau et al, unpublishedHighHighLowLowHighLowUnclearUnclearLow
Chitwood, 199539HighHighLowLowLowUnclearLowUnclearLow
Deren et al, unpublishedHighHighLowLowHighLowLowUnclearLow
Judd et al, unpublishedHighHighLowLowHighLowUnclearUnclearLow
Kerr, 200637HighHighLowLowLowLowLowLowLow
Metzger, 199344HighHighLowHighLowHighLowUnclearLow
Moss, 199443HighHighLowLowHighHighHighHighHigh
Nelson, 20028HighHighLowLowHighHighLowUnclearLow
Ruan, 200738HighHighLowLowHighHighUnclearUnclearLow
Serpelloni, 199448HighHighLowLowUnclearLowLowUnclearUnclear
Suntharasamai, 200917HighHighLowLowLowLowLowLowLow
Van den Berg, 200745HighHighLowLowHighHighLowUnclearLow
Vanichseni, 200111HighHighLowLowUnclearHighLowUnclearLow
Williams, 199246HighHighLowUnclearHighHighLowUnclearUnclear
Zangerle, 199247HighHighLowLowHighHighLowUnclearLow

*Outcome unlikely to be biased by lacking of blinding.

†Objective outcomes, record linkage.

‡Incomplete outcome data.

§Studies deemed at high risk if there was no matching or no adjustment for most important confounding factors.

¶Ascertainment of exposure.

Risk of bias in included studies assessed with criteria drawn from Newcastle-Ottawa scale and EPOC group, adapted for assessment of randomised controlled trials, case-control trials, and prospective observational studies according to criteria recommended by Cochrane Drugs and Alcohol Review Group28 29 *Outcome unlikely to be biased by lacking of blinding. †Objective outcomes, record linkage. ‡Incomplete outcome data. §Studies deemed at high risk if there was no matching or no adjustment for most important confounding factors. ¶Ascertainment of exposure.

Primary meta-analysis

Of the 15 included studies, we were able to pool data from nine to assess the impact of opiate substitution treatment in relation to HIV transmission (unpublished data from A Judd and J Bruneau, 2012),8 17 37 39 44 45 46 (two additional studies (unpublished data from S Deren, 2012, and Vanichseni and colleagues11) were included only in sensitivity or subgroup analyses). The nine studies predominantly included men (range among studies 61-93%), with median age ranging from 26 to 39, and the proportion of participants from an ethnic minority group ranging from 15% to 92%. The sample included 819 incident HIV infections over 23 608 person years of follow-up. Random effects meta-analysis showed strong evidence that opiate substitution treatment was associated with a 54% reduction in the risk of HIV infection (rate ratio 0.46, 95% confidence interval 0.32 to 0.67; P<0.001), though there was heterogeneity between studies (I2=60%, χ2=20.12; P=0.010) (fig 3). Inclusion of unpublished data regarding the impact of methadone maintenance treatment at baseline (S Deren, 2012) gave a similar estimate of effect (0.45, 0.32 to 0.65; P<0.001; test for heterogeneity I2=52.5%, χ2=21.06; P=0.021; fig A in appendix 2).

Fig 3 Meta-analysis of studies showing impact of opiate substitution treatment in relation to HIV transmission in people who inject drugs among all pooled studies and studies reporting only adjusted effect estimates

Fig 3 Meta-analysis of studies showing impact of opiate substitution treatment in relation to HIV transmission in people who inject drugs among all pooled studies and studies reporting only adjusted effect estimates Pooling of a subset of six studies that adjusted for confounders (including unpublished data from S Deren and J Bruneau, 2012) (see table 1) 17 37 39 44 reduced the sample to 450 incident HIV infections during over 10 064 person years of follow-up and suggested that methadone maintenance treatment was associated with a 40% reduction in risk of incident HIV infection (rate ratio 0.60, 0.42 to 0.85; P=0.004; test for heterogeneity I2=23%, χ2=6.49; P=0.26; fig 3). Furthermore, meta-analysis of a subset of five studies that excluded those at higher risk of bias (including unpublished data from J Bruneau, 2012)17 37 49 also showed effectiveness of opiate substitution treatment (0.61, 0.41 to 0.91; P=0.016 test for heterogeneity: I2=38%, χ2=6.50; P=0.165); while pooling of a subset of studies for which effect estimates were expressed only as incidence rate ratio gave a similar effect (0.53, 0.34 to 0.84; P=0.006; I2=64%, χ2=11.0; P=0.027; data not shown). As HIV incidence rates varied substantially between the sites (from less than one to more than five cases per 100 person years), we have reported the rate reduction, rather than an absolute measure of effect (the risk difference), which would not be generalisable to other sites.

Metaregression analyses

Based on univariable metaregression analyses, we found no evidence that the effectiveness of opiate substitution treatment varied according to geographical region or by site of recruitment (P=0.47 and P=0.59, respectively) (figs 4 and 5; table D in appendix 1). We also found no evidence to suggest that the effect of opiate substitution treatment differed by the provision of incentives to participants (P=0.70; fig B in appendix 2 (top panel)). Though there was weak evidence that longer duration of exposure to opiate substitution treatment could be associated with greater benefit (P=0.18; fig B in appendix 2 (bottom panel)), the confidence intervals around the summary effect estimates were wide and few studies assessed exposure over time periods greater than six months. Lastly, our analyses did not support a differential impact by the proportion of female participants or proportion of participants from ethnic minorities (table D in appendix 1).

Fig 4 Impact of opiate substitution treatment in relation to HIV incidence among people who inject drugs by geographical region

Fig 5 Impact of opiate substitution treatment in relation to HIV incidence among people who inject drugs by site of recruitment of study participants

Fig 4 Impact of opiate substitution treatment in relation to HIV incidence among people who inject drugs by geographical region Fig 5 Impact of opiate substitution treatment in relation to HIV incidence among people who inject drugs by site of recruitment of study participants

Methadone detoxification treatment

Four studies reported the impact of methadone detoxification treatment, three of which examined detoxification (in the preceding six months) compared with no treatment (unpublished data from J Bruneau, 2012)8 17 and one of which examined 45 day methadone detoxification compared with methadone maintenance treatment in the preceding four months.11 Together the four studies included 687 incident HIV infections in 20 616 person years of follow-up. HIV incidence was reported to be higher among those undergoing detoxification treatment (table 3), and our pooled analysis showed no evidence that detoxification was associated with a decreased risk of HIV infection compared with either no treatment or methadone maintenance treatment (relative risk 1.54, 1.05 to 2.26; P=0.026; n=4 studies; fig 6). The effect was similar when we pooled studies that compared detoxification with no treatment only (1.66, 1.04 to 2.66; P=0.035; n=3 studies).
Table 3

 Data regarding HIV incidence and estimate of effect of methadone detoxification treatment in relation to HIV transmission among people who inject drugs

Exposure variableHIV seroconversions/person yearsHIV incidence per 100 py (95% CI)Rate ratio (95% CI)
Vanichseni et al 20011145 day detoxification in past 4 months; methadone maintenance treatmentDetox: 106/1730.6; MMT: 24/488.36.1 (5.0 to 7.4); 4.9 (3.1 to 7.3)1.2 (0.7 to 1.8)
Nelson et al 20028Detoxification in past 6 months (yes/no)Yes: 33/778; No: 243/8033Yes: 4.24*; No: 3.031.4 (0.97 to 2.02)
Suntharasamai et al 20091745-day methadone detoxification in past 6 months (yes/no)Yes: 89/1666; No: 121/5130Yes: 5.3 (4.3-6.6); No: 2.4 (2.0-2.8)2.4 (1.8 to 3.3)
Bruneau, 2012 (unpublished)Methadone detoxification in past 6 months (yes/no)NANA0.73 (0.18 to 2.99)

NA=not available.

*95% confidence intervals were not reported.

Fig 6 Meta-analysis of included studies showing impact of detoxification treatment on incident HIV infection among people who inject drugs compared with either no treatment or methadone maintenance treatment

Data regarding HIV incidence and estimate of effect of methadone detoxification treatment in relation to HIV transmission among people who inject drugs NA=not available. *95% confidence intervals were not reported. Fig 6 Meta-analysis of included studies showing impact of detoxification treatment on incident HIV infection among people who inject drugs compared with either no treatment or methadone maintenance treatment

Publication bias

We did not identify studies of small sample size that reported negative effects of opiate substitution treatment in relation to HIV transmission in the published literature, although data were obtained from one small unpublished study. There was weak evidence of publication bias (Egger’s bias coefficient −1.82; P=0.052) (fig C in appendix 2).

Discussion

Principal findings

There is evidence from published and unpublished observational studies that opiate substitution treatment is associated with an average 54% reduction in the risk of new HIV infection among people who inject drugs. There is weak evidence to suggest that greater benefit might be associated with longer measured duration of exposure to opiate substitution treatment. All of the eligible studies examined the impact of methadone maintenance treatment, indicating that there are few data regarding the impact of buprenorphine or other forms of non-methadone opiate substitution treatment in relation to HIV transmission. We found no evidence that methadone detoxification is associated with a reduction in the risk of HIV transmission.

Strengths and weaknesses of the study

To our knowledge this is the first study that synthesises the available evidence and generates a quantitative estimate of the impact of opiate substitution treatment on incidence of HIV. A previous Cochrane systematic review concluded that “available data are limited but it appears that the reductions in risk behaviour related to drug use do translate into actual reductions in cases of HIV infection.”21 Our review identified an additional 11 studies compared with this review and provided a quantitative assessment, including data from an additional 924 HIV seroconversions in over 25 660 additional person years of follow-up. As such, our study extends and strengthens this conclusion, providing the most comprehensive quantitative measure to date of the association between opiate substitution treatment and risk of incident HIV infection. This was achieved partly by identifying studies that measured HIV incidence among people who inject drugs and that reported the impact of opiate substitution treatment in secondary analyses (and hence did not report the data in the title or abstract), and also by identifying studies that might have collected data relating to opiate substitution treatment but not yet have published the analyses. Three of 16 authors contacted were able to provide unpublished data for inclusion in our study, and nine of the 13 other studies were ineligible for inclusion (because opiate substitution treatment was unavailable when the study was conducted, data regarding exposure to opiate substitution treatment were not collected, all participants received treatment, or the participants were mostly stimulant injectors), while four authors did not respond (table E in appendix 1). We consider it unlikely that obtaining additional data from this small number of additional potential studies would affect our results.

Limitations

Nevertheless, our review has several limitations. All of the studies included were observational studies subject to bias, particularly selection and attrition bias. Randomised controlled trials to assess effectiveness of opiate substitution treatment in relation to HIV transmission are no longer ethical, however, given the range of benefits of this treatment,17 19 20 21 22 so meta-analysis of observational analyses, as conducted here, is required. Nonetheless, the extent to which the studies were representative of all people who inject drugs and are receiving opiate substitution treatment is unclear. The proportion of participants who stopped injecting during opiate substitution treatment might have varied between cohorts. In addition, it is possible that cohorts might under-represent short term injectors and those who have stopped injecting or individuals who have considerably reduced the frequency of injection during opiate substitution treatment. For example, such individuals might be under-sampled in studies of injectors recruited in the community at needle exchanges or other venues for active injectors,50 and they might be at decreased risk of HIV infection.11 45 51 If this is the case, our synthesis would underestimate the reduction in risk of HIV infection associated with opiate substitution treatment. Equally, individuals that enter treatment might be more motivated and more likely to change behaviour, thereby reducing injecting frequency or the sharing of equipment, or both, which might overestimate the effect of opiate substitution treatment on risk of HIV infection. Our finding regarding methadone detoxification treatment might also reflect selection bias if individuals who enter detoxification are less likely to permanently reduce injecting drug use compared with those entering opiate substitution treatment. In some countries, detoxification treatment might be compulsory or be a requirement before entry to opiate substitution treatment (as in Thailand, where opiate substitution treatment is provided only after several unsuccessful attempts at 45 day methadone detoxification).17 Outcomes of treatment might therefore differ among those who self initiate treatment with higher motivation to change compared with those for whom treatment is compulsory. Additionally, high rates of relapse have been reported after detoxification,52 53 54 which might put these individuals at greater risk of HIV infection. Therefore, if individuals with less motivation to reduce injecting drug use and higher relapse rates were more likely to receive methadone detoxification, the potential impact of detoxification treatment could be underestimated. We could not compare the association between type of opiate substitution treatment and HIV transmission as studies on non-methadone treatment, such as buprenorphine maintenance treatment, did not meet eligibility criteria (see table F in appendix 1). Although this limits generalisability of our findings, systematic reviews report that several other treatment outcomes—such as retention—are found to be similar for buprenorphine and methadone.55 Most studies also used self reported binary measures of methadone maintenance treatment, which precluded us from exploring the relation between methadone dose and risk of HIV infection. Evidence suggests that doses of at least 60 mg are required with an extended duration of treatment,45 48 50 and lower doses could be associated with intermittent injecting during treatment. Despite this possibility, we found strong evidence of an association between opiate substitution treatment and reduced risk of HIV seroconversion, suggesting that the observed associations might be conservative estimates of the true association between active engagement with opiate substitution treatment and HIV transmission. The control of confounders was limited and inconsistent between studies, and in those studies that did incorporate confounders (unpublished data from A Judd and J Bruneau, 2012)17 37 39 the intervention effect of opiate substitution treatment was diluted, although still consistent with a strong protective effect. Although we identified heterogeneity between studies, in meta-regression analyses, we found no evidence that this was explained by geographical region, site of recruitment, or the provision of incentives, although there was weak evidence to suggest that there could be greater benefit associated with longer recorded duration of treatment. Published studies provided insufficient data for exploration of further differences in study design and reasons for heterogeneity. We also cannot discount the possibility that part of the impact of opiate substitution treatment is attributable to the provision of additional interventions such as attendance at needle and syringe exchange programmes, psychosocial interventions, practical support, or supervised injection facilities, which might additionally reduce the risk of injecting if they are combined with opiate substitution treatment.19 56 Individual characteristics such as motivation, social networks, and the extent of engagement in HIV risk behaviours could also play a role.2 Unfortunately the extent to which additional harm reduction services were accessed by participants receiving methadone maintenance treatment could not be deciphered from the data presented in individual studies; and the use of additional drug treatment or other support services was included in multivariable analyses in only one study, which adjusted for detoxification treatment.17 Finally, our search strategy targeted opiate substitution treatment, and our findings on opioid detoxification therefore need to be treated cautiously as other forms of detoxification were not included. The risks and benefits of detoxification should be examined further in future studies, though our findings are consistent with several studies reporting high rates of HIV infection among people exposed to detoxification treatment and in countries where maintenance treatment is unavailable.23 57 58 59 60 61

Implications

The search strategy we used enabled us to identify several relevant articles not included in previous narrative or systematic reviews. In this way, our study suggests that that more widespread registration of such cohort studies and/or publication of characteristics of cohort studies and the variables measured62 could play a role in strengthening future systematic reviews while also limiting publication bias. Our findings further support and highlight the importance of opiate substitution treatment in the prevention of HIV among people who inject (opiate) drugs. The incidence of HIV in people who inject drugs continues to rise in many parts of the world5 6 15 and HIV infection in such people has been shown to increase the probability of death almost sixfold (range 3.5-10.1) before long term cessation of injection, in the absence of antiretroviral treatment.63 There is good evidence showing that opiate substitution treatment is associated with reductions in the frequency of injecting, the sharing of injection equipment, and drug related HIV risk scores,19 21 and it is likely that the association between treatment and reduced risk of HIV infection is mediated by these changes in behaviour. Involvement in such treatment, as part of a package of interventions, might also increase engagement with health services and access to care and services focused on HIV prevention. Opiate substitution treatment for people who inject drugs and have HIV improves adherence and the virological response to antiretroviral treatment, which might therefore reduce the likelihood of onward transmission.19 64 65 Despite this, the coverage of opiate substitution treatment is variable worldwide, with recent estimates indicating that only 6-12% of people who inject drugs receive it.23 Critically, provision of this treatment is minimal in many of the countries with highest prevalence of HIV in people who inject drugs,66 67 and treatment for opiate dependence with methadone remains illegal in Russia.58 Increased coverage in many of these countries,15 24 25 however, is a welcome advance. Most studies included in our review examined the impact of opiate substitution treatment alone in relation to HIV transmission and only one study examined opiate substitution treatment alone and in combination with needle and syringe exchange programmes.45 Emerging evidence indicates that adequate coverage with opiate substitution treatment and needle and syringe exchange programmes reduces the risk of transmission of HIV (and hepatitis C virus) to a greater extent than either intervention alone,45 50 68 and modelling studies suggest additional benefits in reducing HIV incidence if people who inject drugs who have HIV receive antiretroviral therapy (ART) at the same time as opiate substitution treatment and needle and syringe exchange programmes.69 Further studies are needed to assess the impact of “comprehensive,” or combinations of, interventions including opiate substitution treatment, in a range of settings,19 including prison.70 71 Notably, we identified few studies carried out in low or middle income countries, and further evidence of the effectiveness of harm reduction approaches is required in these settings.

Conclusions

HIV/AIDS account for nearly a fifth of the burden of disease among people who use illicit drugs,72 and increases in HIV incidence have recently been reported among people who inject drugs in several different countries across geographical regions. Our study provides strong quantitative evidence of an association between opiate substitution treatment and reduced risk of HIV transmission among people who inject drugs. These data further support studies showing a range of benefits of opiate substitution treatment, and support calls for the global increase of harm reduction interventions to reduce the transmission of HIV between people who inject drugs and between people who inject drugs and the wider community. Opiate substitution treatment is effective for heroin and other opioid dependence and might reduce HIV transmission among people who inject drugs, primarily by reducing the frequency of unsafe injections Pooling of published and unpublished observational studies showed that opiate substitution treatment is associated with a substantial reduction in risk of HIV infection among people who inject drugs Though we did not find evidence of an association between detoxification and risk of HIV infection, the findings might reflect comparatively high levels of motivation to change behaviour among individuals exposed to opiate substitution treatment Findings could also reflect the additional benefit of other interventions provided alongside such treatment, such as needle and syringe exchange programmes, psychosocial interventions, practical support, or supervised injection facilities
  64 in total

1.  Feasibility of buprenorphine maintenance therapy programs in the Ukraine: first promising treatment outcomes.

Authors:  Michael Schaub; Emilis Subata; Victor Chtenguelov; Gundo Weiler; Ambros Uchtenhagen
Journal:  Eur Addict Res       Date:  2009-05-07       Impact factor: 3.015

2.  Associations between availability and coverage of HIV-prevention measures and subsequent incidence of diagnosed HIV infection among injection drug users.

Authors:  Lucas Wiessing; Giedrius Likatavicius; Danica Klempová; Dagmar Hedrich; Anthony Nardone; Paul Griffiths
Journal:  Am J Public Health       Date:  2009-04-16       Impact factor: 9.308

Review 3.  A situation update on HIV epidemics among people who inject drugs and national responses in South-East Asia Region.

Authors:  Mukta Sharma; Edna Oppenheimer; Tobi Saidel; Virginia Loo; Renu Garg
Journal:  AIDS       Date:  2009-07-17       Impact factor: 4.177

Review 4.  Global epidemiology of injecting drug use and HIV among people who inject drugs: a systematic review.

Authors:  Bradley M Mathers; Louisa Degenhardt; Benjamin Phillips; Lucas Wiessing; Matthew Hickman; Steffanie A Strathdee; Alex Wodak; Samiran Panda; Mark Tyndall; Abdalla Toufik; Richard P Mattick
Journal:  Lancet       Date:  2008-09-23       Impact factor: 79.321

Review 5.  HIV prevention, treatment, and care services for people who inject drugs: a systematic review of global, regional, and national coverage.

Authors:  Bradley M Mathers; Louisa Degenhardt; Hammad Ali; Lucas Wiessing; Matthew Hickman; Richard P Mattick; Bronwyn Myers; Atul Ambekar; Steffanie A Strathdee
Journal:  Lancet       Date:  2010-02-26       Impact factor: 79.321

6.  Factors associated with incarceration and incident human immunodeficiency virus (HIV) infection among injection drug users participating in an HIV vaccine trial in Bangkok, Thailand, 1999-2003.

Authors:  Pravan Suntharasamai; Michael Martin; Suphak Vanichseni; Frits van Griensven; Philip A Mock; Punnee Pitisuttithum; Jordan W Tappero; Udomsak Sangkum; Dwip Kitayaporn; Marc Gurwith; Kachit Choopanya
Journal:  Addiction       Date:  2009-02       Impact factor: 6.526

Review 7.  Does opioid substitution treatment in prisons reduce injecting-related HIV risk behaviours? A systematic review.

Authors:  Sarah Larney
Journal:  Addiction       Date:  2010-02       Impact factor: 6.526

8.  HIV among injecting drug users in Europe: increasing trends in the East.

Authors:  L Wiessing; M J van de Laar; M C Donoghoe; B Guarita; D Klempová; P Griffiths
Journal:  Euro Surveill       Date:  2008-12-11

9.  The impact of methadone or buprenorphine treatment and ongoing injection on highly active antiretroviral therapy (HAART) adherence: evidence from the MANIF2000 cohort study.

Authors:  Perrine Roux; M Patrizia Carrieri; Virgine Villes; Pierre Dellamonica; Isabelle Poizot-Martin; Isabelle Ravaux; Bruno Spire
Journal:  Addiction       Date:  2008-09-04       Impact factor: 6.526

10.  Can needle and syringe programmes and opiate substitution therapy achieve substantial reductions in hepatitis C virus prevalence? Model projections for different epidemic settings.

Authors:  Peter Vickerman; Natasha Martin; Katy Turner; Matthew Hickman
Journal:  Addiction       Date:  2012-07-12       Impact factor: 6.526

View more
  181 in total

1.  Buprenorphine Initiation and Linkage to Outpatient Buprenorphine do not Reduce Frequency of Injection Opiate Use Following Hospitalization.

Authors:  Phoebe A Cushman; Jane M Liebschutz; Bradley J Anderson; Merredith R Moreau; Michael D Stein
Journal:  J Subst Abuse Treat       Date:  2016-06-11

2.  Rationale and design of an integrated bio-behavioral approach to improve adherence to pre-exposure prophylaxis and HIV risk reduction among opioid-dependent people who use drugs: The CHRP-BB study.

Authors:  Roman Shrestha; Frederick L Altice; Brian Sibilio; Jude Ssenyonjo; Michael M Copenhaver
Journal:  Contemp Clin Trials       Date:  2019-06-20       Impact factor: 2.226

3.  Low levels of initiation, engagement, and retention in substance use disorder treatment including pharmacotherapy among HIV-infected and uninfected veterans.

Authors:  Kevin L Kraemer; Kathleen A McGinnis; David A Fiellin; Melissa Skanderson; Adam J Gordon; Jonathan Robbins; Susan Zickmund; Kendall Bryant; P Todd Korthuis
Journal:  J Subst Abuse Treat       Date:  2019-05-07

4.  "In their perception we are addicts": social vulnerabilities and sources of support for men released from drug treatment centers in Vietnam.

Authors:  Cecilia Tomori; Vivian F Go; Le Nhan Tuan; Nguyen Mai Huong; Nguyen Thanh Binh; Carla E Zelaya; David D Celentano; Do Tuan Dat; Vu Minh Quan
Journal:  Int J Drug Policy       Date:  2014-04-28

5.  The OPTIMA study, buprenorphine/naloxone and methadone models of care for the treatment of prescription opioid use disorder: Study design and rationale.

Authors:  M Eugenia Socias; Keith Ahamad; Bernard Le Foll; Ron Lim; Julie Bruneau; Benedikt Fischer; T Cameron Wild; Evan Wood; Didier Jutras-Aswad
Journal:  Contemp Clin Trials       Date:  2018-04-05       Impact factor: 2.226

Review 6.  Advancing global health and strengthening the HIV response in the era of the Sustainable Development Goals: the International AIDS Society-Lancet Commission.

Authors:  Linda-Gail Bekker; George Alleyne; Stefan Baral; Javier Cepeda; Demetre Daskalakis; David Dowdy; Mark Dybul; Serge Eholie; Kene Esom; Geoff Garnett; Anna Grimsrud; James Hakim; Diane Havlir; Michael T Isbell; Leigh Johnson; Adeeba Kamarulzaman; Parastu Kasaie; Michel Kazatchkine; Nduku Kilonzo; Michael Klag; Marina Klein; Sharon R Lewin; Chewe Luo; Keletso Makofane; Natasha K Martin; Kenneth Mayer; Gregorio Millett; Ntobeko Ntusi; Loyce Pace; Carey Pike; Peter Piot; Anton Pozniak; Thomas C Quinn; Jurgen Rockstroh; Jirair Ratevosian; Owen Ryan; Serra Sippel; Bruno Spire; Agnes Soucat; Ann Starrs; Steffanie A Strathdee; Nicholas Thomson; Stefano Vella; Mauro Schechter; Peter Vickerman; Brian Weir; Chris Beyrer
Journal:  Lancet       Date:  2018-07-20       Impact factor: 79.321

7.  Brief video intervention to improve attitudes throughout medications for opioid use disorder in a correctional setting.

Authors:  Jeffrey A Lam; Hye In Sarah Lee; Ashley Q Truong; Alexandria Macmadu; Jennifer G Clarke; Josiah Rich; Brad Brockmann
Journal:  J Subst Abuse Treat       Date:  2019-06-10

8.  Supervised Injection Facility Utilization Patterns: A Prospective Cohort Study in Vancouver, Canada.

Authors:  Mary Clare Kennedy; David C Klassen; Huiru Dong; M-J S Milloy; Kanna Hayashi; Thomas H Kerr
Journal:  Am J Prev Med       Date:  2019-08-01       Impact factor: 5.043

9.  HIV seroconversion and risk factors among drug users receiving methadone maintenance treatment in China: A qualitative study.

Authors:  Changhe Wang; Cynthia X Shi; Bo Zhang; Hong Chen; Hua Wang; Nanci Zhang; Keming Rou; Xiaobin Cao; Wei Luo; Zunyou Wu
Journal:  Drug Alcohol Depend       Date:  2016-08-21       Impact factor: 4.492

10.  Emergence of methadone as a street drug in St. Petersburg, Russia.

Authors:  Robert Heimer; Aleksandra Lyubimova; Russell Barbour; Olga S Levina
Journal:  Int J Drug Policy       Date:  2015-10-19
View more

北京卡尤迪生物科技股份有限公司 © 2022-2023.