The Montreal Cognitive Assessment as a predictor of dropout from residential substance use disorder treatment.

BACKGROUND: Cognitive function is a challenge for many SUD patients, and residential SUD treatment is cognitively demanding. Treatment retention is a predictor for success in SUD treatment, and the literature links low cognitive function to increased dropout rates. In our study we investigate cognitive function and dropout in a residential SUD treatment setting, also accounting for psychological distress.
METHODS: We screened a cohort (N = 142) of inpatients for cognitive function (MoCA®) and psychological distress (SCL-10) and calculated the relative risk for dropping out if over versus under the respective cut-off values (<26 and >1.85), and sex, and age-group (<23 years). We also employed a logistic regression with dropout as outcome and MoCA- and SCL-10 scores, and age and days before testing as input.
RESULTS: Dropout risk was higher (RR = 1.70) if scoring below MoCA cut-off, and for those younger than 23 years (RR = 2.36). The other variables did not influence dropout risk. MoCA raw scores, age, and SCL-10 were associated with dropout (p < .05); with lower symptoms of psychological distress predicting increased dropout. The interaction between MoCA and SCL-10 scores was not significant (p = .26).
CONCLUSIONS: SUD patients should routinely be screened for cognitive impairment, as it predicts dropout. Screenings should be ensued by appropriate adaptations to treatment and further assessment. The MoCA is a useful screening tool for this, independent of psychological distress. Future studies should replicate our findings, investigate specific interventions, and establish SUD population norms for the MoCA.

Introduction

Adverse neuropsychological effects from substance use disorder (SUD) are both drug-specific and generalized across drugs, and symptoms of cognitive impairment is a common expression {Fernández-Serrano et al., 2011 #4}. Estimates of the prevalence of cognitive deficits among alcohol and drug abusers that seek treatment vary in studies from about 30 to 80% (Bates et al., 2002, 2006; Copersino et al., 2009; Manning et al., 2017). Cognitive impairment has a high impact on quality of life and occupational functioning and restricts SUD patient's benefit from therapy, which subsequently affects the course of rehabilitation and the level of community integration among patients with SUD after treatment (Fernández-Serrano et al., 2011).

Mild cognitive impairment (MCI) refers to measurable subclinical symptoms of cognitive deficiency beyond the normal age-related decline, but do not interfere notably with activities of daily life (Gauthier et al., 2006). The onset of cognitive deficits is slow and gradual, which could mask patients' deteriorating functioning (Horner et al., 1999). Patients in the transitional phase between healthy ageing and dementia will typically score within the range of MCI on tests of cognitive functioning (Petersen et al., 2001). MCI may be difficult to diagnose for patients with SUD as many confounders are present. Evidence indicates that patients' self-reports of cognitive functioning do not give an accurate picture of neurocognitive status, and closely associated with emotional distress than reflecting the level of actual cognitive functioning (Shelton and Parsons, 1987).

Treatment retention is a significant factor for successful treatment outcome for SUD patients (Dalsbø et al., 2010; De Leon and Jainchill, 1986), and the literature suggests that lowered cognitive functions are an important predictor for dropping out of SUD treatment (Brorson et al., 2013). Traditionally, research into dropout and retention has focused on the CMRS factors (Circumstances, Motivation, Readiness, and Suitability). Subjective psychological features within the CMRS framework, typically denial of problem severity, a shortage of motivation and impulsivity may, in fact, stem from specific neuropsychological insufficiencies (Brorson et al., 2013; Hagen et al., 2017).

Studies further show that therapists cannot identify SUD patients' cognitive status with satisfactory rates without appropriate tools (Fals-Stewart et al., 1997). Therapists tend to overrate the cognitive abilities of their patients, and the mechanism behind this is complicated, but it is suggested that the stigma associated with cognitive impairment plays a role (Crisp et al., 2000). Underreporting of cognitive impairment may negatively influence the individual adaptation of the treatment.

There is indeed evidence indicating that cognitive impairment may gradually improve spontaneously after sobriety (Hagen et al., 2017). However, if a patient drops out or otherwise prematurely terminates treatment, spontaneous recovery may come too late to add to the benefit of treatment. Therapy in early phases of a SUD treatment trajectory may occur at a time when a person has the most pronounced cognitive impairments and is the least able to benefit from it (Vocci, 2008).

To alleviate clinicians' blind spots, self-report incoherence, and bypass the stigma that might arise from pointing out individual patients already suspected of sub-normal cognitive functions, mandatory screening of all patients are warranted. Knowledge of the status of cognitive function should play a role in the tailoring, individually timing and adjustment of the treatment, as should be a goal throughout the treatment process. However, the application of neuropsychological assessments is often cost-prohibitive for SUD treatment services, with access to professional resources like neuropsychologists and other specialized clinicians often being scarce. Therefore, a readily available screening measure that can be administered in ambulatory-, out-patient-, day- and inpatient settings by various professional categories is needed.

The Montreal Cognitive Assesment (MoCA®) is a short, easily administered screening measure that has shown good discriminative ability (acceptable sensitivity and specificity) for the detection of symptoms of MCI in many conditions. Studies have also shown promising results for MoCA as a time-efficient screening tool for MCI in SUD patients (Copersino et al., 2009; Ridley et al., 2017). MoCA was initially developed for the detection of MCI in the geriatric population, but has since been validated and applied as a useful screening measure in a variety of other populations and patient groups, including SUD (Bergly and Sømhovd, 2018; Bruijnen et al., 2016; Copersino et al., 2009; Ridley et al., 2017; Vogel et al., 2015).

The primary hypothesis of this study is that lowered cognitive function in the range of mild cognitive impairment defined as a MoCA score below 26 points predicts dropout in our cohort of SUD patients. We also test whether psychological distress influence the association between cognitive function and dropout, as is shown in studies with other patient populations (Aharonovich et al., 2006, 2003; Fals-Stewart et al., 1997), and also control for the influence of age, gender and number of days between submission and screening.

Materials and methods

Treatment organisation and trajectory

The Tyrili foundation in Norway is an NGO that runs seven interdisciplinary inpatient treatment and rehabilitation centres for patients with SUD. The foundation also runs three prison embedded units and has an in-house detox unit in one of the clinics in Oslo. The detox unit ensures prompt curbing of minor episodes of substance use during an inpatient stay. The inpatient treatment mode averages at nine months enclosed in a treatment trajectory of ∼21 months, which includes a mandatory six months ambulant treatment before and a voluntary six months after an inpatient stay. Patients may have less severe co-occurring psychiatric diagnoses but are usually not eligible if they have severe mental conditions in an active phase and if they have severe personality disorders (typically antisocial, dissocial or narcissistic). Applicants that are currently under conviction for sexual offences are also barred.

Participants

The 142 participants in this study comprise all consenting inpatients in the 2016 cohort in Tyrili except the prison-embedded units. The project's main exclusion criteria were having had recent treatment in Tyrili, which we defined as any length of inpatient treatment in Tyrili during the preceding year, or having had more than two inpatient treatments in Tyrili during the preceding five years.

Following the study protocol, the time of testing should be within 7–30 days after entering the inpatient treatment. The mean time since the last intake of drugs was 28 days. We excluded cases if testing had happened later than 30 days after submission from analyses, which rendered a study sample of N = 129.

All participants had a SUD as their principal diagnosis, and about 60 per cent had an additional psychiatric diagnosis. The bulk of the SUD diagnoses were opioid use (28%), other stimulants (16.3%), and cannabinoids use (12.4%). Among the participants that had one or more co-occurring psychiatric diagnosis, ADHD was by far the most frequent (18.5%), with recurring depression (6.2%) as the next most frequent, reactions to severe stress (3.8), and specific personality disorders.

The age range of the participants in the study sample was between 18 and 54 with a mean age of 32 years (SD = 7.6). There were 25 women (19.9%), with a mean age of 30 years (SD = 9.6). The 102 men had a mean age of 32 years (SD = 7.1).

Measures and variables

Cognitive function

We screened all participants for mild cognitive impairment with the MoCA® (Nasreddine et al., 2005). MoCA is a short screening instrument for cognitive function that is scored in integers up to 30 points on seven dimensions: Visuospatial/Executive (up to 5 points), Naming (up to 3 points), Memory (up to 5 points), Attention (up to 6 points), Language (up to 3 points), Abstraction (up to 2 points), and Orientation (up to 6 points). One adjustment point is given if the responder has formal education less than 13 years.

At a sum-score cut-off value of 26 MoCA has excellent sensitivity and acceptable specificity to identify reduced cognitive function in the range of mild cognitive impairment (Nasreddine et al., 2005). It is superior to the previous ‘gold-standard' for the screening of cognitive function, the Mini-Mental State Examination (MMSE) (Folstein et al., 1975; Hoops et al., 2009).

Completing a MoCA typically takes between 15 and 30 minutes, and most personnel categories can administer it. MoCA is freely available for clinical and research applications (Gallant K., 2015).

Psychological distress

To screen for psychological distress, we used the Hopkins Symptom Check List 10-items version (SCL-10). The SCL-10 is a self-rating scale with ten items representing aspects of the affective disorder symptomatology. SCL-10 has good psychometric properties (Strand et al., 2003), and was derived from the Symptom Checklist—90 (Derogatis et al., 1973). The SCL-10 comprise the ten items that accounted for the most significant variance in the SCL-90 scores as initially rendered in a factor analysis by Hoffmann and Overall (1978). The questions in the version we used are scored from 1 (not at all) to 4 (very much), rendering a sum-score range of 10–40. For defining a clinically relevant level of psychological distress, we used a recommended sum-score mean of 1.85 for both women and men (Strand et al., 2003).

Substance use during treatment

Albeit substance use is unwanted while in treatment, it is indeed common. Generally, minor relapses are accepted, and expected, as a symptom of the SUD. If a patient leaves one of the Tyrili foundation's treatment facilities, they can return, under the same referral, providing they do so within 21 days. The Tyrili foundation runs an in-house detox unit for three-four days detoxes after minor episodes of substance use to facilitate swift returns to treatment after minor relapses.

Dropout definition

Coding of treatment termination in the participants' medical records adheres to the Norwegian health authorities' codex that defines the six termination categories. The categories are: a) treatment termination after an agreement [between the patient and the treatment institution] in accordance with the treatment plan, b) treatment termination by the treatment institution without agreement with the patient, c) treatment termination initiated by the patient (with given notice), d) treatment termination due to a patient not showing up (without notice), e) transferral to another health institution based on regulations for the use of force or f) death.

In our study, we define a patient as dropped out if b), c), or d) were registered as the reason for termination. If a) the participant is in the retained group. If a participant were transferred to other facilities or died, we removed their data. No participants deceased during the project period. In the case of d), when a patient does not return or showing up after a leave or other absence from the treatment facility, regulations require a 21 days lag before formally terminating the stay. If the patient comes back within the 21 days and is otherwise eligible for treatment, she can continue the treatment without a new referral.

Statistics

We present descriptive statistics of the MoCA score, the SCL-10 score, and the time to testing. For the principal analysis, we present relative risk estimates of dichotomized variables. Risk of dropping out of treatment if scoring below MoCA® threshold (indicating the presence of cognitive impairment in the MCI range), scoring above the SCL-10 threshold (indicating psychological distress), having had detox during the treatment (indicating at least one episode of substance use), being male, or being under 23 years. Young SUD patients under 23 years are priority in the Norwegian specialist healthcare, which is the reason for the particular cut-off. In cross tables with low cell counts (age), we did additional Fisher Exact testing.

Finally, we assess a logistic regression with MoCA, and SCL-10 raw-scores entered into a common model with age and days before testing as well as the interaction between MoCA and SCL-10. Box-plots are subsequently presented to visualise the MoCA and SCL-10 scores for the dropout and the retained group respectively.

Ethics, approvals and data access

The Norwegian Centre for Research Data approved the study. An internal committee in the foundation also assessed and approved the study protocol. All participants submitted an informed consent.

Data in the study will be made available from mid-2019 when the overarching study is completed. Data will be accessible from the Norwegian Centre for Research Data (http://www.nsd.uib.no/nsd/english/index.html).

Results

Descriptives and preliminary analyses

The overall mean MoCA score was 26.6 (SD = 2.87), with a range between 16 and 31. According to the Grubbs tables (Grubbs, 1969), the lowest value of 16 was not an outlier in the respective group. The mean sum-score for the SCL-10 score was 1.9 (SD = .57). The mean number of days until screening was 18.2 (SD = 7.33). Table 1 presents descriptive statistics for the dropout group and the retained group separately. For the dropout group, the mean MoCA raw score was 25.6 (SD = 3.33) and mean sum-score for the SCL-10 score was 1.7 (SD = .53). The retained group had a mean MoCA raw score of 27.1 (SD = 2.53), and a mean sum-score for the SCL-10 score was 2.0 (SD = .56). We found no association between the age of participants and their MoCA scores (t = −1.79, p = .08), nor differences between sexes (X2 = .03, p = .87).

Table 1

Descriptive statistics for the groups.

		Minimum	Maximum	Mean	Std. deviation
Retained group (n = 88)	Age	20	54	33.1	7.77
	MoCA score	19	31	27.1	2.53
	SCL-10 score	.80	3.4	2.0	.56
	Days before testing	7	30	17.8	7.28
Dropout group (n = 41)	Age	18	41	28.0	5.97
	MoCA score	16	30	25.6	3.33
	SCL-10 score	1.0	2.9	1.7	.53
	Days before testing	7	30	19.2	7.28

Risk ratios

Table 2, summarizes the risk ratios, with a confidence interval, for dropout providing scores over versus under threshold values on the MoCA and the SCL-10, being under 23 years of age, being male, and having had detox during treatment. Those having a MoCA score <26 had an approximate 70 per cent increase in the risk of dropping out. Participants under 23 years of age had a 136 per cent increased risk. Considering the low cell counts in the lower age category, we ran an additional Fisher's exact test confirming the significantly increased risk for this group (Χ2 = 7.43, p = .02). Neither SCL-10 scores ≥1.85, indicating psychological distress, nor having had detox during treatment indicated a higher dropout risk.

Table 2

Relative Risk for Dropout by Cognitive Function, psychological Distress, Age Group, Sex, and Substance Use during Treatment (N = 129).

	Retained	Dropout	RR	95% CI
MoCA© score ≥ 26	67	24
MoCA© score < 26	21	17	1.70*	1.04 to 2.78
SCL-10 mean score < 1.85	36	24
SCL-10 mean score ≥ 1.85	52	17	0.62	0.37 to 1.03
Age ≥ 23 years	84	33
Age < 23 years	4	8	2.36**	1.44 to 3.87
Female	18	9
Male	70	32	0.94	0.51 to 1.72
No substance use	58	30
Substance use	30	11	0.79	0.44 to 1.41

Note: * statistical significance at α .05; ** statistical significance at α.001.

Multivariate logistic model

Table 3 sums up the multivariate logistic model of the raw scores from the MoCA, SCL-10, continuous age, and the number of days after submission before testing as predictors of dropout. The equation also include the interaction between MoCA and SCL-10. In this equation, lower MoCA scores (worse cognitive functioning), lower SCL-10 scores (less psychologically distressing symptoms), and lower age did all significantly predict dropout. The number of days between submission and testing did not predict dropout.

Table 3

Logistic regression analysis of dropout as predicted by MoCA score, age, SCL-10 score, and days before testing.

	Beta	S. E.	Sig.	Odds ratio (95% CI)
MoCA score	−.16	.07	.03	.85 (.74–.97)
Age	−.10	.04	.004	.90 (.84–.97)
SCL-10 score	−.12	.04	.004	.89 (.82–.96)
Time before testing	.02	.30	.60	1.02 (.96–1.1)
MoCA by SCL-10	−.02	.01	.26	.99 (.96–.1.0)

The interaction term between MoCA score and the SCL-10 score did not predict dropout significantly.

The box-plot in Fig. 1 shows MoCA raw scores for the retained and the dropout group respectively. For the retained group, the 25th percentile lower boundary of the box is 26 (indicated by the horizontal line in the plot), indicating that ∼75 per cent of the retained group scored above the threshold. In the dropout group, a considerable part of the box lies below the threshold. The height of the box and the extension of the lower whisker suggests a larger spread in the dropout group, particularly for the lower scores within the 25th percentile. The spread is, however, influenced by a very low score of 16 (Z = 2.89), but value is not identified as an outlier (following the Grubbs tables for extreme studentized outliers (Grubbs, 1969)).

Figure 1

Box-plots of MoCA scores in the retained and dropout groups.

Fig. 2 plots SCL-10 raw scores for the respective groups so that higher scores indicate more psychological distress. The retained group has a visibly wider spread of scores, spanning from 8 to 35 compared to 10 to 30 in the dropout group. In accordance with the results from the logistic regression it is also visible that the dropout group reports lower psychological distress on the SCL-10.

Figure 2

Box-plots of SCL-10 scores in the retained and dropout groups.

Discussion

The main finding in this study was that participants who scored below the MoCA cut-off, indicating symptoms in the range of mild cognitive impairment, had a statistically significant higher risk of dropping out compared to those with normal cognitive functioning. This result supports and strengthens previous findings showing that the association between reduced cognitive functioning and negative input on treatment adherence (Bates et al., 2006) influences attendance, and predicts dropout in both outpatient and inpatient treatment trajectories (Aharonovich et al., 2006, 2003; Brorson et al., 2013; Guthrie and Elliott, 1980).

The dropout rate in the study was 31%, with a definition of dropout that is based on the Norwegian health authority's definitions and length of absence. Due to public health policies, a patient can effectively suspend their treatment for up to three weeks, without being registered as a dropout. Albeit reflecting a liberal attitude towards using drugs during treatment, concerning analyses, this definition renders conservative dropout rates. A broader definition of dropout would include patients dropping out for many practical reasons not connected to the treatment or their cognitive functioning.

Also in the retained group the prevalence (23%) of MoCA scores below 26 is high compared to the general population. Conclusions of a direct link between dropout on an individual level cannot be made. A screening tool like MoCA should not be a freestanding dropout-risk assessment tool. Nevertheless, considering the importance of retaining patients in treatment both for treatment benefit and the potentially lethal consequences of dropout from an overdose, the blanket screening of cognitive function in SUD patients with a cost effective measure may be crucial.

Our second aim was to assess if psychological distress, defined by the SCL-10, influenced the association between cognitive function and dropout. As a stand-alone variable, there was no significant association between the SCL-10 scores and dropout. However, interestingly, in a multivariate logistic model, higher SCL-10 scores (more symptoms) were associated with less dropout. The logistic equation included an interaction term for MoCA scores by SCL-10 scores, which rendered as not significant. This could be a statistical artifact due to low sample size. However, the small effect sizes indicate that the interaction has low clinical relevance. The lack of interaction suggests that the dropout patients with lower SCL-scores, is not the same group as those with lowered cognitive function; thereby bolstering the risk analysis that the SCL-10 does not predict dropout and does not moderate the association. The lack of interaction also does not indicate a negative correlation between cognitive function and psychological distress. Symptoms of depression, which are very common among patients submitted to SUD treatment, has been linked to higher dropout rates (Andersson et al., 2018), and has been qualitatively identified as a primary reason given for dropping out (Nordheim et al., 2018). Notwithstanding, other studies find no association (López-Goñi et al., 2012; Preti et al., 2015). An extensive literature review (Brorson et al., 2013) revealed only a handful of studies investigating mood and anxiety as predictors for dropout in SUD patients that concluded with an increased risk. Most were inconclusive, which may reflect the clinical notion that some SUD patients rapidly feel better, and therefore terminates the treatment prematurely, while others feel worse without drugs, and terminates on that ground.

We also found that younger patients had significant 135 per cent increased dropout risk compared to older patients. Many studies investigates dropout as a function of age, and that young age is predicting dropout is reported in several studies, but neither this is a concluded issue (Brorson et al., 2013). In our risk analysis, we dichotomised young age as under 23 years. We chose that cut-off as the health care system in Norway guarantees shorter processing time from referral to submission for this age group when the diagnosis is a SUD. Our results, therefore, prompt awareness of the higher risk for dropout for this group, suggesting that a high focus on swift re-referrals or resubmissions to treatment. In our data, age had no main effect on the cognitive function status.

Finally, sex and substance use while in treatment, the latter indicated by at least one detox stay during treatment, did not significantly increase dropout risk. This result also adds to an unresolved literature (Brorson et al., 2013).

Clinical implications and suggestions

Treatment retention is a central predictor of treatment outcome (De Leon and Jainchill, 1986). Residential SUD treatment is very cognitively demanding, and if the cognitive skills required exceeds the individual patient's function level that may prevent patients from benefitting from the treatment, and cement a sense of not being ‘smart’ enough, even for drug rehabilitation. Individual adaptations to treatment is a goal. MoCA seems like a promising screening tool to include in first step assessments of SUD patients entering residential treatment guiding individual adjustments. Studies assessing specific interventions concerning cognitive screenings is mandated. Further, to reflect also the spontaneous improvement during treatment, larger scale studies to work out specific reference values for the SUD population is warranted.

To make screening feasible in clinics, care, and treatment organizations, with scarce availability to specialized personnel, it should be administrable by more personnel groups. Again, the MoCA appears promising.

Further studies delineating patients that potentially leaves treatment because of a high level of distress making retention difficult, from those who conversely experience a rapidly declining distress level and leave treatment prematurely in the illusion it means they are recovered.

Conclusion

Considering the importance of retaining patients in treatment for treatment benefit and the potentially lethal consequences of dropout from an overdose, prediction of dropout in SUD patients during treatment is crucial.

Our results suggest that the MoCA is a good candidate to meet the need for precise screening measures, and may predict dropout.

First, the prediction of dropout in this study using a cost and time useful screening instrument should make it feasible to get screening routines in place for most treatment facilities.

Secondly, to make obligatory screening feasible in clinics, care, and treatment organisations, with scarce availability to specialised personnel, screenings should also be administrable by several personnel groups.

The screening results should subsequently incite action to alleviate the dropout risk by general and individual adaptations to treatments as to lower dropout risk.

The results of our study should be replicated and future studies should further validate the MoCA in the SUD population. Studies assessing interventions subsequent of cognitive screenings are needed and large-scale studies to work out norm cut-off reference values for the SUD population is warranted to reflect spontaneous improvement during treatment.

Declarations

Author contribution statement

Mikael Sømhovd, Tone Bergly: Conceived and designed the experiments; Analyzed and interpreted the data; Wrote the paper.

Egon Hagen, Espen Ajo Arnevik: Analyzed and interpreted the data; Wrote the paper.

Funding statement

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Competing interest statement

The authors declare no conflict of interest.

Additional information

No additional information is available for this paper.