Can We Predict Which Adolescents Will Engage in Outpatient Substance Abuse Treatment?

Understanding which adolescents remain in substance abuse treatment may facilitate the development of effective strategies for enhancing engagement and retention. Using clinical service data from a large naturalistic sample of adolescents, we examined whether client characteristics predict retention in outpatient alcohol and other drug (AOD) treatment. χ2 tests and multinomial logistic regressions were conducted to examine relationships between sociodemographic and substance use variables and clinic attendance. Client characteristics of ethnicity, sex, age, living situation (whether living with or away from family), and substance use severity were associated with retention in community AOD treatment. Pacific Island youth, females, 13- to 15-year olds, clients living with family, and clients with more severe substance use were generally more likely to be "engaged" with the service (ie, attended 4+ sessions) than their European and Maori, male, 16- to 19-year-old, and living away from home counterparts. These findings may inform more targeted engagement strategies in the future.

Introduction

A key challenge for youth alcohol and other drug (AOD) services is getting young people to remain in treatment.[1] Young people, as do adults, seldom recognize their substance abuse as problematic[2,3] and even when they do, they usually encounter barriers to accessing services.[4,5]

Once engaged in services, treatment for young people is effective,[6,7] and the longer that they remain engaged, the more likely they are to make significant positive changes.[8,9] Furthermore, the initiation of treatment can change young people’s perception of their problems and previously resistant young people will begin to participate and engage more fully.[10,11] Thus, treatment engagement and retention is a key consideration for services[12] and a range of mechanisms thought to enhance effective youth engagement have been examined.[13] Understanding which young people are more likely to remain in treatment can help services adapt and develop targeted strategies geared toward reluctant attenders.

Current research in the area of treatment engagement and retention is difficult to consolidate or compare in light of the variable treatment settings and methodologies used across studies. In general, research shows that failure to complete treatment is high and fixed client characteristics are not a good predictor of which young people will remain in treatment.[14,15]

That said, most published studies have sample sizes of less than 200 clients and are conducted in residential settings or in 12-step abstinence-based programs. Findings from these studies are wide ranging but seldom repeated. For example, one study showed that characteristics such as comorbidity, family substance use disorder, high severity of substance use, lower education, and younger age were associated with low completion rates in residential treatment.[16] Another reported that previous trauma may predict early treatment drop out.[17] A further study found that the use of illicit drugs in addition to alcohol and marijuana, having less deviant peers, fewer emotional problems, and positive counselor regard were associated with longer treatment retention, and that pressure to enter treatment was unrelated to treatment duration.[18] Yet, another examination of young people in a range of abstinence and 12-step programmes[14] found that young people’s socioeconomic status had no impact on treatment initiation, treatment retention, or long-term abstinence from alcohol or drugs.

Outpatient treatments are generally more cost-efficient than residential settings but just as effective.[7] The few studies of retention in outpatient settings have found that ethnicity has some impact on engagement; one large study suggested that social factors such as lack of insurance coverage and a lack of specifically targeted cultural services contributed to reduced treatment retention in black and Hispanic young people,[19] whereas another study of an outpatient service found that ethnicity, legal charges and drug court involvement, cannabis dependence, and conduct disorder were positively associated with treatment retention.[20]

In this study, we aimed to examine factors associated with retention of adolescents attending an outpatient AOD treatment service to determine whether fixed characteristics can predict engagement. The study was conducted in a free community treatment service with a harm minimization approach, where attendance is voluntary. As such, it offers an alternative perspective and may contribute to understanding predictors of retention in an area of research that is dominated by studies in residential or abstinence-based services.

Methods

Participants

The sample was a retrospective, naturalistic cross section of young people (age: 13-19 years) attending an outpatient AOD service called “Altered High” in Auckland, New Zealand (NZ), from 2010 to 2014 inclusive. The “Altered High” service is a free, multidisciplinary and mainstream service, forming part of the government-provided health services for the area. “Altered High” has a harm reduction focus and provides a range of individual and family-based interventions aimed at minimizing substance use and related harm, with abstinence often a treatment goal. Typically, clinicians provide motivational interviewing for young people who are precontemplative or have minor problems and cognitive behavioral therapy (CBT) informed interventions in association with family work for those with more severe AOD problems. It has an emphasis on engagement; in practice, this means that barriers to entry into the service are minimal and youth with a wide range of substance use severity and concerns are accepted. The service stresses its independence from referring agencies and young people attend voluntarily; those who are pressured from family or youth justice organizations to attend are able to leave should they wish. Following assessment, brief (1-3 sessions) or longer interventions can be delivered as indicated. Deidentified demographic and clinical data from all young people who completed at least one face-to-face appointment that included the usual service assessment requirements (including the Substances and Choices Scale [SACS] and discharge treatment information [DTI] described below) were included. Participants of “other” ethnicity were excluded from the analysis as they were a heterogeneous group (n = 152) and may have introduced a confound if combined with one of the major ethnicity categories.

Measures

In addition to a clinical assessment interview, young people who attend the service also complete the SACS. The SACS is a youth AOD screening and outcome measurement instrument with very good reliability and validity.[21] The young person’s score out of 20 is an indication of the severity of his or her substance use. More information about the SACS is available at www.sacsinfo.com. Clinicians are also required to complete a DTI form for all young people who attend for an assessment. This is a service-specific outcome measure designed to record clients’ demographic information, treatment attributes, and treatment outcomes. It is completed by clinicians at the point of discharge and consists of 31 multichoice questions for which clinicians are only able to select 1 answer. The questions cover areas such as living situation, number of appointments attended, and discharge arrangements. The DTI has not been formally validated but has been used for more than 8 years for internal reporting on service performance.

Data collection

Data from the measures described above were obtained directly from the service electronic record keeping system in spreadsheet form. The data were collated, deidentified, checked, and cleaned using Microsoft Excel software. Data collection spanned 5 years, and because minor changes to 4 of the DTI instrument items had occurred over that time (ie, changed wording of items, added items), the data sets were standardized into one consistent version (however, the changed items were not applicable to our analyses here). Following data standardization, a random sample of 50 client data sets was reviewed by another investigator to check for data consistency with inter-rater reliability of greater than 95%. The data set was complete (no missing data) as the electronic record requires every item in the forms to be filled. Clients with more than 1 episode across the duration of the sample were treated as separate cases. The study was conducted as an audit or related activity according to the NZ National Ethics Advisory Committee guidelines for observational studies.[22]

Statistical analysis

Statistical analyses were performed using SPSS software (version 22). The dependent variable (number of appointments attended) was categorical, falling within one of several ranges (1, 2-3, and 4+ appointments), thus χ2 tests and ordinal (followed by multinomial) logistic regressions were conducted to examine the research questions. χ2 tests were conducted to assess the bivariate relationships between the baseline substance use severity and sociodemographic predictor variables and the clinic attendance dependent variable. A multivariate ordinal regression was then conducted to assess predictors of attendance. Age (13- to 15-year olds, 16- to 19-year olds), sex (male, female), ethnicity (European, Maori, Pacific Island), living situation (with family, away from family), and substance use severity at assessment (low—SACS score of 1-5, high—SACS score of 6+) were entered as predictors in the model. The proportional odds assumption was violated in the ordinal regression; therefore, a multivariate multinomial regression was conducted. The reference category, “engagers” (attended between 2 and 3 sessions), was compared against “assessed” (attended 1 session only) and “engaged” (attended 4+ sessions). There was no serious multicollinearity as indicated by the tolerance and variance inflation factor. The significance (α) level was adjusted down to .01 for the multiple χ2 tests to help protect against type 1 error.

Results

A total of 2180 clients, aged 13 to 19 years, of European, Maori, or Pacific Island ethnicity attended the service at least once (regarding their own substance use) and completed the SACS. Of these clients, 438 (20.1%) attended once only (“assessed” group), 873 (40.0%) attended 2 or 3 sessions (“engagers” group), and 869 (39.9%) attended 4+ sessions (“engaged” group). Descriptive statistics for sex, ethnicity, age, living situation, substance use severity at assessment, and clinic attendance are reported in Table 1.

Table 1.

Descriptive statistics for total sample and by attendance.

Variable	Total sample (N = 2180)	Assessed (n = 438)	Engager (n = 873)	Engaged (n = 869)	χ2
Age, y					6.01[^]
13-15	586	120 (20.5%)	211 (36.0%)	255 (43.5%)
16-19	1594	318 (19.9%)	662 (41.5%)	614 (38.5%)
Gender					24.81***
Female	555	102 (18.4%)	183 (33.0%)	270 (48.6%)
Male	1625	336 (20.7%)	690 (42.5%)	599 (36.9%)
Ethnicity					13.93**
Pacific Island	324	54 (16.7%)	124 (38.3%)	146 (45.1%)
Maori	606	142 (23.4%)	218 (36.0%)	246 (40.6%)
European	1250	242 (19.4%)	431 (42.5%)	477 (38.2%)
Living situation					67.36***
Away from family	449	152 (33.9%)	142 (31.6%)	155 (34.5%)
With family	1731	286 (16.5%)	731 (42.2%)	714 (41.2%)
Substance use severity					29.67***
Low severity (SACS < 6)	946	217 (22.9%)	413 (43.7%)	316 (33.4%)
High severity (6+)	1234	221 (17.9%)	460 (37.3%)	553 (44.8%)

Values are counts and, in parentheses, percentages. Assessed = attended 1 session only; engagers = attended 2 or 3 sessions; engaged = attended 4+ sessions.

P = .05; **P ≤ .01; ***P < .001.

χ2 tests revealed significant associations between attendance and ethnicity, χ2(4) = 13.93, P = .010; sex, χ2(2) = 24.81, P < .001; living situation, χ2(2) = 67.36, P < .001; and substance use severity at assessment, χ2(2) = 29.67, P < .001. Of note, the association between attendance and age, χ2(2) = 6.01, P = .050, was significant at a .05 α level, but not at the α-adjusted .01 level. Pacific Island young people were more likely to be “engaged” (clients who attended 4+ sessions; 45.1%) than their Maori (40.6%) or European (38.2%) counterparts. Similarly, 48.6% of women versus 36.9% of men, 43.5% of 13- to 15-year olds versus 38.5% of 16- to 19-year olds, and 41.2% of those living with family compared with 34.5% of those not with family were in the “engaged” group. More severe substance use problems were also associated with engagement; 44.8% with an SACS score of 6 or higher were in the “engaged” group compared with 33.4% with an SACS score less than 6. Of note, 33.9% of clients living away from family versus 16.5% of those living with family attended once only.

There were 4 significant 2-way or 3-way interactions in the multivariate multinomial regression that included age, sex, ethnicity, living situation, and substance use severity as predictors of attendance. The final model was significant, χ2(34) = 186.56, P < .001, Nagelkerke R2 = .093. Nagelkerke R2, a commonly used pseudo R2 in logistic regression, is an approximate of the variation explained in the dependent variable (ie, the final model explained 9.3% of the variation in attendance). Given that 3 of the 4 interactions included the age variable, a follow-up multinomial regression was conducted for each age category (13- to 15-year olds, 16- to 19-year olds).

Table 2 reports the results of the multinomial regression for the 13- to 15-year-old sample. There was a significant main effect for living situation, χ2(2) = 39.70, P < .001, and a borderline significant effect for substance use severity, χ2(2) = 8.47, P = .014. Of note, the main effect for sex, χ2(2) = 7.37, P = .03, was significant at a .05 α level, but not at the α-adjusted .01 level. Clients who were living away from family were 4.5 times more likely than those living with family to have attended only once rather than 2 or 3 times (ie, be in the “assessed” rather than the “engagers” group). Female clients were 1.7 times more likely than male clients to have attended 4+ times rather than 2 to 3 times (ie, be in the “engaged” rather than the “engagers” group), and clients with higher substance use severity were 1.6 times more likely than those with low severity to have attended 4+ times rather than 2 to 3 times (the adjusted odds ratio [AOR] = 0.63 in Table 2 because the reference category is “higher severity”).

Table 2.

Predictors of clinic attendance—13- to 15-year olds.

Predictor	Adjusted OR (95% CI)Ref. cat. = engagers (attended 2 or 3 sessions)
	Assessed (attended once)	Engaged (attended 4+)
Ethnicity (ref.: European)
Pacific Island	0.63 (0.29-1.36)	0.80 (0.46-1.38)
Maori	1.14 (0.68-1.89)	0.86 (0.57-1.31)
Gender (ref.: male)
Female	1.16 (0.69-1.93)	1.70 (1.14-2.52)**
Living situation (ref.: with family)
Away from family	4.52 (2.56-8.00)***	0.89 (0.50-1.59)
Substance use severity (ref.: higher)
Lower severity	1.13 (0.71-1.81)	0.63 (0.44-0.92)*

Abbreviations: CI, confidence interval; OR, odds ratio; ref., reference category.

The reference category is engagers (attended 2 or 3 sessions). The final model was significant: χ2(10) = 63.95, P < .001, Nagelkerke R2 = .12.

P < .05; **P < .01; ***P < .001.

The multinomial regression for the 16- to 19-year-old sample included 2 significant 2-way interactions involving the ethnicity, sex, and living situation predictor variables. There was also a significant main effect for substance use severity, χ2(2) = 22.98, P < .001. Clients with higher substance use severity were 1.5 times more likely than those with low severity to have attended 4+ times rather than 2 to 3 times. The final model was significant, χ2(18) = 106.19, P < .001, Nagelkerke R2 = .07. Given that both significant interactions in the model included ethnicity, a follow-up multinomial regression was conducted for each ethnic group (European, Maori, Pacific Island).

Table 3 reports the results of the multinomial regressions for the 16- to 19-year-old sample by ethnicity. For the European 16- to 19-year-old model, there were significant main effects for sex, χ2(2) = 16.58, P < .001; living situation, χ2(2) = 15.18, P = .001; and substance use severity, χ2(2) = 27.83, P < .001. Clients who were living away from family were 2.4 times more likely than those living with family to have attended only once rather than 2 or 3 times, and clients with lower substance use severity were 1.6 times more likely than those with high severity to have attended once only rather than 2 to 3 times. Clients living away from family were 1.5 times more likely, clients with higher substance use severity were 1.7 times more likely (AOR = 0.60 in Table 3 because the reference category is “higher severity”), and female clients were 1.9 times more likely to have attended 4+ times, rather than 2 to 3 times.

Table 3.

Predictors of clinic attendance—16- to 19-year olds, by ethnicity.

Predictor	Adjusted OR (95% CI)Ref. cat. = engagers (attended 2 or 3 sessions)
	Assessed (attended once)	Engaged (attended 4+)
European
Gender (ref.: male)
Female	1.01 (0.65-1.58)	1.91 (1.36-2.68)***
Living Situation (ref.: with family)
Away from family	2.38 (1.54-3.67)***	1.52 (1.03-2.25)*
Substance use severity (ref.: higher)
Lower severity	1.56 (1.10-2.21)*	0.60 (0.45-0.81)**
Maori
Gender (ref.: male)
Female	1.13 (0.59-2.16)	1.38 (0.81-2.35)
Living situation (ref.: with family)
Away from family	3.07 (1.73-5.46)***	1.22 (0.73-2.03)
Substance use severity (ref.: higher)
Lower severity	0.97 (0.55-1.71)	0.84 (0.53-1.34)
Pacific Island
Gender (ref.: male)
Female	1.74 (0.68-4.47)	0.69 (0.30-1.62)
Living situation (ref.: with family)
Away from family	1.32 (0.58-2.97)	0.28 (0.13-0.61)**
Substance use severity (ref.: higher)
Lower severity	0.70 (0.32-1.55)	0.61 (0.34-1.10)

Abbreviations: CI, confidence interval; OR, odds ratio; ref., reference category.

The reference category is engagers (attended 2 or 3 sessions). The final models were significant for European, χ2(6) = 63.17, P < .001, Nagelkerke R2 = .07; for Maori, χ2(6) = 18.49, P = .005, Nagelkerke R2 = .05; and for Pacific Island, χ2(6) = 20.69, P = .002, Nagelkerke R2 = .09.

P < .05; **P < .01; ***P < .001.

For the Maori 16- to 19-year-old model, there was a significant main effect for living situation, χ2(2) = 16.32, P < .001. Clients who were living away from family were 3.1 times more likely than those living with family to have attended only once rather than 2 or 3 times.

For the Pacific Island 16- to 19-year-old model, there was a significant main effect for living situation, χ2(2) = 15.63, P < .001. Clients who were living with family were 3.6 times more likely than those living away from family to have attended 4+ sessions rather than 2 or 3 sessions (AOR = 0.28 in Table 3 because the reference category is “living with family”).

Discussion

Previous youth retention research has focused predominantly on residential or abstinence-based services. This study adds to the literature using a large naturalistic sample to examine factors associated with adolescent attendance to outpatient AOD treatment with a harm minimization approach. In contrast to previous studies,[15,20,23] we found that client characteristics of age, sex, ethnicity, living situation (whether living with or away from family), and substance use severity were associated with continued attendance at our community AOD treatment service. Fixed client characteristics are usually described as variables that are unchangeable and specific to each individual and in previous studies have included attributes such as age, sex, ethnicity, substance use and criminal history, and previous mental health[23-26]

A greater percentage of females (49%) than males (37%) attended the service for 4 or more sessions. Furthermore, female sex predicted attending 4 or more sessions (except for the 16- to 19-year-old Pacific and Maori clients). Previous studies in youth populations have not found significant sex differences in this area, and the increased power from our large sample size may explain our positive findings. A recent study examining retention in private residential adult patients (mean age: 36 years) with co-occurring disorders reported a similar (though smaller) difference.[27]

Males made up over three quarters of our sample, which is expected; the preponderance of young men in services is usually on account of criminal justice involvement. However, young women usually present to substance use services with more severe substance abuse and psychosocial problems in general.[28-30] Because males make up most of the new referrals into the service, one might expect the service to have adopted practices that are more focused around the engagement of boys. Despite this, we found better engagement and retention in females, perhaps because the girls presenting to “Altered High” have more severe problems. Our findings support this in that substance abuse severity was associated with increased retention. Another explanation, which has not been studied in-depth, is that counseling-based treatments (as delivered in our service) may be less suited to boys than girls. However, studies of treatment modalities that might be expected to have more appeal to boys, such as adventure therapy, also tend to find better outcomes in girls.[31] A better understanding of the reasons for poor retention of males in AOD treatment is needed to assist in designing services and treatment approaches that remedy this sex difference.

Ethnicity was also associated with retention in that a greater percentage of Pacific Island clients (45%) were in the engaged group than Maori (41%) and European (38%) clients. Although the differences are small, they are interesting in that the “Altered High” service is mainstream and does not specifically use culturally based treatments (although it does use support from associated cultural services). Specific culturally oriented Pacific Island and Maori AOD services are available for young people in Auckland if they elect to attend; however, many request treatment in the mainstream service. Our findings of better retention in Maori and Pacific young people compared with their NZ European counterparts may, therefore, be related to sample bias given that Maori and Pacific Island young people are able to make a choice of which service they would prefer. Similar results were found in another NZ study, with Maori and Pacific Island ethnicities enhancing group retention.[20] A study of retention and satisfaction with treatment in a US adolescent population accessing community-based treatment also found either no differences or better engagement in ethnic minorities.[32] Duncan and Ahmed[33] suggest that organizational factors and race are important predictors of client retention in treatment and have implications in improving program retention.

In our sample, 44% of those 15 years and younger remained in treatment for 4 or more sessions compared with 39% of the 16- to 19-year olds. One possible reason for this difference is that younger adolescents are more likely to be living at home with parents or caregivers, which we found to be a much stronger predictor of treatment retention. Nevertheless, the difference is fairly small and is probably not of clinical significance in terms of informing future service design. Other studies have not shown an age-related difference in retention in treatment in youth populations.

Living at home also protected against early treatment drop out. Young people living with family or caregivers were more likely to attend 4 or more sessions (41%) than those living away (35%), and only 17% of those living with family attended once only (compared with 34% of clients living away from family). Except for 16- to 19-year-old Pacific clients, living at home was a significant predictor of attending 2 or 3 sessions versus just 1 session. Interestingly, in the 16- to 19-year-old Pacific Island clients, living with family was predictive of attending 4 or more sessions versus 2 or 3 sessions; however, for 16- to 19-year-old Europeans, the opposite was found—those living away from family were more likely to attend 4 or more sessions than those living with family. These results are somewhat contradictory and may be due to a number of factors. For example, different cultural family values could mean that Pacific families have more influence on their older adolescents (in terms of encouraging them to continue attending treatment) compared with European families where independence and autonomy may be emphasized more. In addition, Pacific families have higher levels of engagement in organized religion and are often more disapproving of even low levels of substance use, which can impact on engagement in complex ways, dissuading some from treatment (because of confidentiality concerns) but supporting engagement in others.[34]

Family support and involvement in treatment has been shown to be a key factor in youth engagement[13] and positive AOD treatment outcomes.[12] Our results suggest that it may be useful to identify which young people are not living at home at the time of referral so that more active efforts at engaging and retaining them in treatment can be implemented from their first contact with the service. This would of course include trying to involve their family in some way[35] (however, if this was not possible, other strategies such as assistance with transport[12] and other practical support practices could be considered.[36] A varying approach may be required depending on the ethnicity of the client and their family, including offering specific culturally oriented services, if available.

Substance abuse severity was associated with treatment retention. The SACS is a good measure of substance abuse severity[21] and, using the cutoff recommended, we found that 45% of youth with an SACS score of 6 or higher attended 4 or more sessions compared with only 33% of youth with lower scores. Similarly, clients with low substance use severity were more likely to drop out after 1 session (23%) compared with those with more severe problems (18%). These results indicate that the flexibility of the “Altered High” treatment (allowing for briefer interventions in those with mild problems) is appropriate, given that the clients who are expected to require more intensive treatment are also the clients who are more likely to remain engaged. Follow-up analyses revealed that for 16- to 19-year-old Europeans and 13- to 15-year olds of all ethnicities, higher severity predicted attendance at 4 or more sessions. However, Maori and Pacific Island 16- to 19-year olds did not show differences in engagement related to the severity of their substance use, which suggests that factors other than the level and impact of substance use are more important drivers of engagement or drop out in these population groups.

As with all research, this study has some specific limitations. The DTI questionnaire is a clinician completed instrument and bias may be introduced through different clinicians completing the questionnaire in varying ways. However, this potential for bias is mitigated by the fact that more than 15 different clinicians are employed in the service (with some turnover over the years). Significant systematic bias is, therefore, unlikely. In contrast, the SACS is self-completed and has good validity and reliability. Another limitation was that the dependent variable, “number of sessions,” was recorded in the DTI as an ordinal rather than continuous variable, which limited the statistical analyses. Despite this, we still found statistically significant predictors of attendance and retention. Finally, our study is limited by its retrospective design, and clear outcomes using recognized measures of engagement were not possible. It is naturalistic, based on data from an established and active service, which makes it more difficult to achieve a precise research design; however, the real-world nature of the study is, in many respects, also a strength.

Conclusions

Youth engagement in substance use treatment is an imprecise construct and difficult to measure,[1] and a more nuanced understanding is recommended given its substantial impact on treatment outcomes. Contrary to many other studies, we found that certain fixed client characteristics had an impact on engagement, at least in an outpatient AOD service with a harm reduction framework. Services, via audit and internal monitoring, should seek to understand and respond to their different groups of clients and target engagement strategies accordingly. In particular, attention should be paid to the client’s living situation, substance use severity, and ethnicity.