Factors associated with treatment uptake, completion, and subsequent symptom improvement in a national digital mental health service.

Digital mental health services (DMHS) have proven effectiveness and play an important role within the broader mental health system by reducing barriers to evidence-based care. However, improved understanding of the factors associated with successful treatment uptake, treatment completion and positive clinical outcomes will facilitate efforts to maximise outcomes. Previous studies have demonstrated that patient age is positively associated, and initial symptom severity negatively associated with treatment uptake and treatment completion rates in both DMHS and other mental health services. The current study sought to extend these findings by examining the effect of other patient characteristics, in particular, self-reported psychosocial difficulties, using data from a large-scale national DMHS. Using a prospective uncontrolled observational cohort study design, we collected self-reported demographic, psychosocial and clinical data from 15,882 patients who accessed the MindSpot Clinic, Australia, between 1 January and 31 December 2019. Using a series of univariate regression models and multivariate classification algorithms we found that older age, higher educational attainment, and being in a relationship were all positively associated with uptake, completion and significant symptom improvement, while higher initial symptom severity was negatively associated with those outcomes. In addition, self-reported psychosocial difficulties had a significant negative impact on uptake, completion, and symptom improvement. Consistent with previous literature, the presence of these characteristics in isolation or in combination have a significant impact on treatment uptake, completion, and symptomatic improvement. Individual and multiple psychosocial difficulties are associated with reduced capacity to participate in treatment and hence an increased treatment burden. Identifying patients with lower capacity to complete treatment, modifications to treatments and the provision of supports to reduce treatment burden may promote greater engagement and completion of treatments offered by digital mental health services. Crown

Introduction

Mental and substance use disorders account for a significant portion of the worldwide disease burden (Vos et al., 2017), with fewer than half of the people with these conditions seeking or receiving evidence-based treatment (Wang et al., 2007; Whiteford et al., 2014). In traditional face-to-face mental health services (MHS) there are several known barriers to access, including stigma, cost, preference to self-manage problems, low perceived need, and availability of services (Mojtabai et al., 2011; Andrade et al., 2014). The past three decades have witnessed a significant growth of psychological treatments delivered over the internet (Andersson et al., 2019) in services sometimes described as digital mental health services (DHMS). DMHS take a variety of forms, from standalone self-help apps to services that provide mental health information, assessment and treatment, delivered by mental health professionals (Andersson et al., 2019). The clinical outcomes of psychological interventions such as internet-delivered cognitive behavioural therapy (iCBT) have been shown to be equivalent to high quality face-to-face psychological interventions (Carlbring et al., 2018). DMHS already provide treatment as part of routine care in several countries (Titov et al., 2019a; Titov et al., 2018) (Hadjistavropoulos et al., 2021; Nordgreen et al., 2019; Titov et al., 2019b; Ruwaard et al., 2012; Johansson et al., 2019; Staples et al., 2021).

Although DMHS are generally provided free or at low cost to consumers, and overcome barriers to traditional service provision such as distance, availability, and stigma (Andersson and Titov, 2014), the uptake and completion of treatments offered by DMHS varies considerably. A recent systematic review and meta-analysis of iCBT for anxiety and depression delivered in routine care settings (k = 19) showed that, on average, 73.0% of all who were included went on to start an intervention, and 61.3% completed treatment, with significant variability noted across studies (95% CI: 16.6–80.8%) (Etzelmueller et al., 2020). However, an important issue affecting such estimates relates to the patients' or consumers' main purpose of engagement. For example, a recent analysis of patients' reasons for using a national DMHS in Australia found that while only 22.6% of 121,652 patients who completed an assessment began treatment, only 25.9% stated at the outset that they intended to use the service to access digital treatment, whereas 67.0% indicated they only wanted an assessment or information(Titov et al., 2020) emphasising the role of DMHS in also providing education about mental health, assessments, and supporting patients to self-manage their conditions.

There is limited information about the patient-related factors that determine treatment uptake within DMHS. Using an online survey, Moskalenko et al. (2020) reported that participants with lower self-stigma of help seeking, higher perceived need for mental health treatment, lower access to care and lower computer anxiety showed significantly greater interest in engaging in iCBT. Conversely, participants with financial concerns, ‘life chaos’ (defined as numerous commitments or unstable living arrangements), or a greater preference for autonomy did not report an interest in engaging in iCBT (Moskalenko et al., 2020).

Similar to treatment uptake, there is variability in rates of treatment completion, based in part on how treatment completion is defined. A review of data collected from nine studies found a median completion rate of 69% for iCBT programs targeting depression (Andrews et al., 2016). Another meta-analysis of 40 studies with a total of 7313 participants found an overall completion rate of 43%, with completion rates increasing with greater levels of support: 26% without support, 62% administrative support only, and 72% for therapeutic support (Richards and Richardson, 2012). A more recent meta-analysis of therapist guided iCBT for anxiety and depression (Etzelmueller et al., 2020) reported an overall completion rate of 61.3%. By way of comparison, treatment dropout in face-to-face psychological treatment is also highly variable, and is influenced by a range of factors, including treatment orientation, manualisation and setting. A large meta-analysis of 669 studies with almost 84,000 participants (Swift and Greenberg, 2012) found the average weighted dropout (non-completion) rate across all studies was 19.7% (i.e., 80.3% completion). The dropout rate varied from 18.4% for studies of CBT, and differed between manualised versus non-manualised interventions (18.5% versus 28.3%), as well as between research clinics (17.3%) and university-based clinics (30.4%) (Swift and Greenberg, 2012). This study further highlighted the significance of how treatment completion is defined. Dropout rates differed according to the metric used to define treatment completion. Indeed, dropout was highest when determined by therapist judgment (37.6%) and lowest when determined by completion of a set number of sessions (18.3%) or a treatment protocol (18.4%) (Swift and Greenberg, 2012).

The patient characteristics known to be most strongly associated with DMHS treatment completion are limited to demographic characteristics and initial symptoms. One study found older patient age and lower initial symptom severity as measured by the K10 to be significant predictors of treatment course completion, but no other demographic variables, including gender, education levels, relationship status or employment status were associated with completion or drop out (Edmonds et al., 2018). Another recent study found that lower educational level significantly predicted dropout from therapist guided iCBT (Schmidt et al., 2019). Predictors of dropout from face-to-face services include younger age and lower education, with varying effects of gender, race, marital status and employment, none of which were significant in the large meta-analysis (Swift and Greenberg, 2012). When examining reasons for iCBT treatment dropout, Proudfoot et al. (2003) and Waller and Gilbody (2009) reported that participants who dropped out cited unhappiness with treatment allocation, physical illness, lack of time, work commitments, family problems, moving house, and perceived lack of treatment progress. Similarly, in a review of their five iCBT anxiety programs, Klein and colleagues (Klein et al., 2011) found that the top three participant-reported barriers for completion were time pressures, lack of motivation and apprehension about the content. Together, this literature highlights the important role that contextual psychosocial stressors (stressful life events) may play in the likelihood of completing treatment.

The aim of the current study was to perform a detailed examination of patient factors associated with service pathway engagement and outcomes using a large sample of patients who accessed a national DMHS. In addition to examining the effect of demographic and clinical characteristics on treatment uptake, treatment completion and subsequent clinical improvement, we further aimed to examine the effect of a range of self-reported psychosocial stressors, and whether these factors were similar, or independent in these groups.

Methods

Design and participants

This study was designed as a prospective uncontrolled observational cohort study and is reported according to STROBE guidelines (Erik von Elm et al., 2007). It includes all patients who registered for assessment or treatment between 1st January 2019 and 31st December 2019. As a naturalistic study, we aimed to follow the observed service flow through the pathway during that period, which resulted in three pathway groups and one outcome group. The ‘assessment’ group comprised of all who completed the online assessment to enter the service. The ‘uptake’ group is a subset of the ‘assessment’ group who chose to begin a treatment course. The ‘completion’ group is a subset of the ‘uptake’ group who completed the treatment, defined by here when at least the fourth of the five lessons were clicked through to completion of the material. The fifth lesson focussed primarily on relapse prevention. The treatment outcome group known as the ‘improvement’ group is also a subset of the treatment uptake group who experience a 50% reduction in symptoms, as described below.

Ethical approval for the collection and use of the data was obtained from the Macquarie University Human Research Ethics Committee (5201200912) and registered on the Australian and New Zealand Clinical Trials Registry (ACTRN12613000407796). All included patients consented to their non-identifiable, aggregated data being used for research purposes.

Procedure

MindSpot is an Australia-wide DMHS that has provided services to more than 160,000 Australian adults and enrolled more than 30,000 people in one of its 8-week iCBT-based treatment courses provided as part of routine care (Titov et al., 2020; Titov et al., 2017). Patient characteristics and treatment outcomes have been reported elsewhere (Titov et al., 2020). People register with MindSpot by creating an account and completing an online or telephone-administered screening assessment. Participants provide demographic and service use information, and answer questions about symptoms, safety risk and current psychosocial difficulties. People who complete the assessment are invited to discuss their results and treatment options with a therapist by telephone. Depending on preference and suitability, participants are able to enroll into one of seven online treatment courses, unless they are considered ineligible for digital treatment because their clinical presentation indicates the need for comprehensive or urgent face-to-face assessment. Four of the seven courses are ‘transdiagnostic’ in that they each target depression and anxiety using similar content, adapted to suit different patient characteristics. Given their content similarities, these four courses (Mood Mechanic for 18 to 25 year olds (Dear et al., 2018), Wellbeing for 26–65 year olds (Dear et al., 2016), Wellbeing Plus for 60+ (Staples et al., 2016) and the Indigenous Wellbeing course (Titov et al., 2019c)) were included in the current analysis.

Measures

Standardised and validated symptom questionnaires were administered to patients at assessment and throughout treatment and included the Kessler Psychological Distress 10-Item Scale (K10), Generalized Anxiety Disorder 7-item Scale (GAD-7) and the Patient Health Questionnaire −9 (PHQ-9). The PHQ-9 consists of nine items measuring symptoms of major depressive disorder according to criteria of the Diagnostic and Statistical Manual of Mental Disorders, 4th edition (Kroenke et al., 2001). Scores range from 0 to 27, with a score of 10 or more considered indicative of a diagnosis of depression. The GAD-7 consists of seven items and is sensitive to the presence of generalized anxiety disorder, social phobia, and panic disorder (Spitzer et al., 2006). Scores range from 0 to 21, with a score of 8 or more indicating the probable presence of an anxiety disorder. The Kessler Psychological Distress 10-Item Scale (K−10) scores range from 10 to 50 and scores of 21 or more associated with the presence of anxiety and depressive disorders (Kessler et al., 2002) Patients are administered the K10, PHQ-9 and GAD-7 at the screening assessment and post-treatment. We chose a single symptom outcome measure (K10) given the included courses were transdiagnostic and it measures both depressive and anxiety symptoms. Symptom improvement was defined as a 50% or more reduction in K10 score between assessment and treatment completion. Additionally, we also examined the rates of participants with 50% change within different bands of baseline severity. This symptom change criterion was used as it most closely approximated the mean symptom improvement which has been reported in the larger national sample (Titov et al., 2020). Regarding psychosocial difficulties, patients were asked “are you having significant difficulties with any of the following?” and indicated (endorsed) at least one of the 14 options listed in Table 1, which included an option for ‘none of these’. Culminative difficulties were simply the total number of difficulties indicated.

Table 1

Baseline demographic and clinical information for the three pathway groups (assessment, uptake and completion).

		Assessment	Treatment uptake	Treatment completion
Age	Mean (SD)	33.54 (13)	40.82 (14.3)	42.8 (14.4)
Age groups	18–24	4076 (34.2%)	611 (15.4%)	281 (12%)
	25–35	3499 (29.4%)	1056 (26.7%)	572 (24.4%)
	35–45	2127 (17.9%)	880 (22.2%)	526 (22.5%)
	45–55	1289 (10.8%)	690 (17.4%)	437 (18.7%)
	55–65	658 (5.5%)	483 (12.2%)	364 (15.5%)
	65+	258 (2.2%)	237 (6%)	161 (6.9%)
Gender	Female	8953 (75.8%)	2716 (68.6%)	1596 (68.4%)
	Male	2865 (24.2%)	1241 (31.4%)	738 (31.6%)
Locality	Missing	110 (0.9%)	31 (0.8%)	21 (0.9%)
	Capital city or surrounding suburbs	7220 (60.6%)	2206 (55.7%)	1304 (55.7%)
	Other urban region	2434 (20.4%)	805 (20.4%)	485 (20.7%)
	Rural or remote region	2143 (18%)	915 (23.1%)	531 (22.7%)
Born in Australia	No	2443 (21.1%)	879 (22.2%)	514 (22.5%)
	Yes	9134 (78.9%)	3078 (77.8%)	1769 (77.5%)
Aboriginal and/or Torres Strait Islander	No	8643 (95.5%)	3853 (97.4%)	1716 (97.9%)
	Yes	406 (4.5%)	104 (2.6%)	36 (2.1%)
Employment Status	Other (unemployed, student, retired)	4739 (40.1%)	1531 (38.7%)	842 (36.2%)
	Employed (fulltime, parttime, casual)	7079 (59.9%)	2426 (61.3%)	1483 (63.8%)
Education	Other	7292 (61.8%)	2142 (54.1%)	1143 (49.2%)
	University degree	4513 (38.2%)	1815 (45.9%)	1181 (50.8%)
Relationship Status	Married Defacto	4063 (34.1%)	1805 (45.6%)	1174 (50.1%)
	Missing	117 (1%)	26 (0.7%)	19 (0.8%)
	Never married	6375 (53.5%)	1457 (36.8%)	772 (33%)
	Separated	1251 (10.5%)	606 (15.3%)	339 (14.5%)
	Widowed	101 (0.8%)	63 (1.6%)	37 (1.6%)
Baseline K-10	Mean (SD)	21.93 (7.5)	20.09 (7.1)	19.2 (7)
Baseline K10 Severity Categories	Mild (10–24)	1669 (14%)	714 (18%)	586 (25%)
	Moderate (25–29)	2094 (17.6%)	919 (23.2%)	489 (20.9%)
	Severe (30+)	8144 (68.4%)	2324 (58.7%)	1266 (54.1%)
Baseline PHQ9	Mean (SD)	14.94 (6.1)	11.92 (6)	12.9 (5.8)
Baseline PHQ9 Severity Categories	Mild (0–9)	2462 (20.7%)	1061 (26.8%)	721 (30.8%)
	Moderate (10–14)	3127 (26.3%)	1162 (29.4%)	704 (30.1%)
	Severe (15+)	6318 (53.1%)	1734 (43.8%)	916 (39.1%)
Baseline GAD7	Mean (SD)	12.58 (5.2)	11.95 (5.1)	11.5 (5.2)
Baseline GAD7 Severity Categories	Mild (0–9)	3528 (29.6%)	1328 (33.6%)	848 (36.2%)
	Moderate (10–14)	3703 (31.1%)	1282 (32.4%)	773 (33%)
	Severe (15+)	4676 (39.3%)	1347 (34%)	720 (30.8%)
Total Number of Difficulties Endorsed	0	1824 (15.3%)	694 (17.5%)	472 (20.2%)
	1	2099 (17.6%)	874 (22.1%)	571 (24.4%)
	2	2577 (21.6%)	859 (21.7%)	543 (23.2%)
	3	2301 (19.3%)	709 (17.9%)	380 (16.2%)
	4	1584 (13.3%)	421 (10.6%)	199 (8.5%)
	5	857 (7.2%)	238 (6%)	106 (4.5%)
	6+	665 (5.6%)	162 (4.1%)	70 (3%)
Psychosocial Difficulties indicated	Relationship difficulties	7232 (60.7%)	1779 (51.7%)	1153 (49.2%)
	Parenting difficulties	1645 (13.8%)	464 (13.5%)	279 (11.9%)
	Vocational difficulties	6301 (52.9%)	1654 (48%)	1063 (45.4%)
	Physical difficulties	4915 (41.3%)	1385 (40.2%)	909 (38.8%)
	Financial difficulties	3684 (30.9%)	867 (25.2%)	504 (21.5%)
	Housing difficulties	1423 (12%)	250 (7.3%)	141 (6%)
	Alcohol difficulties	1162 (9.8%)	297 (8.6%)	183 (7.8%)
	Drug difficulties	877 (7.4%)	124 (3.6%)	63 (2.7%)
	Religion difficulties	500 (4.2%)	114 (3.3%)	68 (2.9%)
	Cultural difficulties	304 (2.6%)	61 (1.8%)	37 (1.6%)
	Sexual difficulties	508 (4.3%)	94 (2.7%)	61 (2.6%)
	Grief difficulties	476 (4%)	127 (3.7%)	80 (3.4%)
	Carer difficulties	138 (1.2%)	57 (1.7%)	30 (1.3%)
	No difficulties	1607 (13.5%)	560 (16.3%)	412 (17.6%)

Significance set at p<0.05.

Statistical analysis

Analysis and identification of participant characteristics that influence the rates of treatment uptake, treatment completion and symptom improvement, were explored in three steps. In the first step, a series of univariate logistic regressions models were conducted, aiming to detail the association of each variable (predictor) as a single (univariate) model against each of the outcome groups. These models employed a binomial distribution with a log link function to determine and test the rate (% proportion) of each event across the different groups. Power analyses were determined using a power analysis package that uses a binomial distribution and a sequence of Bernoulli trials.

In the second step, a classification algorithm was employed to develop a multivariate model for each of the uptake, completion and symptom improvement rates. These models specified exhaustive Chi-square automatic interaction detection classification algorithms (CHAID), which is data mining algorithm with the ability to identify subgroups that are characterised with more than one variable (higher order interactions), and without parametric assumptions (Bi et al., 2019; Aafjes-van Doorn et al., 2021). This multivariate analysis aims to group variables that characterize a more complex pattern of combined (multivariate profile) prediction against each of the groups, with variables that don't feature in the multivariate model redacted to emphasis the importance and lesser importance of some variables over others. The model evaluates the list of possible predictors and selects a combined, parsimonious model from a larger list of alternatives (25 considered predictors, 54 subgroups, and multi-fold higher order interactions). The multivariate classification tree employed the decision tree procedure to classify cases into groups or predict values of a dependent (target) variable based on values of independent (predictor) variables. The assessment of predictor robustness was analyzed using the Categorical Regression Regularization package within SPSS.

In the third step, in line with STROBE guidelines (Erik von Elm et al., 2007), sensitivity analyses were conducted to test the robustness of the multivariate models, and evaluate problems such as overfit and poor result generalization. In this step, the multivariate analyses of step two were re-tested with the inclusion of regularization parameter (elastic net). These regularized models aim to evaluate and identify the most robust and generalizable predictors from the list of alternatives (Zhang et al., 2018).

Missing cases in all three samples were not imputed to avoid uncertain, and potentially artificial, influences on the testing of large and specific models related to the natural clinical flow. Statistical analyses were performed using SPSS version 27. Type I error was set at 0.05, to balance the possibility of type I within the multiple contrasts conducted with the need to detect possible marginal trends within more nuanced subgroups (as well as higher order interactions). Statistical power was determined at 0.8.

Results

Sample characteristics

Within the 2019 period of recruitment, 15,882 patient participants completed a screening assessment, forming the total potential ‘uptake sample’ (Fig. 1). Of the participants that completed an assessment, 3444 (21.69%) started treatment (uptake). Post treatment data was available for 2103 (60.8%) participants. Statistical power analyses demonstrated that the sample was adequately powered to detect nuanced subgroup differences associated with uptake (min ORgroup∆ > 1.06), completion (min ORgroup∆ > 1.045) and improvement events (min ORgroup∆ > 1.093) with power levels set at 80% and an alpha of 0.05.

Fig. 1

Participant flow from assessment through to treatment completion.

The sample characteristics are presented in Table 1. The mean age of participants for the entire sample was 33.5 years. More females than males completed an assessment (75.8% vs 24.2%), with the ratio of males increasing in the treatment uptake (68.6% vs 31.4%) and treatment completed (68.4% vs 31.6%) groups. Most of the assessment group lived in capital cities or surrounds (60.6%), were single or never married (53.5%) and did not have a university-level education (61.8%). The majority scored in the severe range on the K10 (68.4%), PHQ9 (53.1%) and GAD7 (39.3%). Only 13.5% of the assessment group reported having no psychosocial difficulties.

Univariate analyses

Results from the series of univariate logistic regressions models, testing the joint association of each predictor to the rate of uptake, completion and symptom improvement, are presented in Table 2 under “Univariate models”. Table 2 shows that when compared to the average sample uptake of 21.6%, rates of uptake vary significantly within and across variables. Age, as an example, shows that those aged 18–24 years have a 10.9% rate of uptake compared to those aged 65 years and older who have an uptake of 40.1%, the variation in the group being statistically significant (p < .000). Similar large variations are also observed for employment status (11.7% student vs 22.6% employed vs 29.7% retired/disability/other, p < .000) and relationship status (16.1% single/never married vs 27.6% married vs 30.5% widowed, p < .000). Variance in the average rate of treatment completion (68%) and improvement (45.8%) were also observed across similar variables, including age, initial symptom severity, marital status and total number of psychosocial difficulties faced with significance levels highlighted in Table 2.

Table 2

Univariate and multivariate estimates of uptake, completion, and significant improvement.

			Univariate models			Multivariate models
		Uptake %	Completion %	Improvement %	Uptake %	Completion %	Improvement %
	Total estimate (no covariates)	21.6 (21.1 to 22.3)	68 (66.4 to 69.5)	45.8 (43.7 to 47.9)	21.7% (21.5 to 21.8)	63.4% (63.2 to 63.6)	42.2% (42.1 to 42.4)
Age groups	18–24	10.9 (10.1 to 11.9)	54.8 (50.4 to 59)	42 (36 to 48.3)	10.9% (10.8 to 11)	53.8% (53.6 to 54)	–
	25–35	20.6 (19.4 to 21.8)	61 (57.8 to 64.1)	46.2 (41.9 to 50.6)	20.6% (20.4 to 20.7)	59.1% (58.7 to 59.5)	–
	35–45	25.6 (24 to 27.2)	68.3 (64.9 to 71.5)	45.4 (40.9 to 49.9)	25.6% (25.4 to 25.8)	67.5% (67.3 to 67.7)	–
	45–55	29.8 (27.8 to 31.9)	73.9 (70.3 to 77.3)	45.1 (40.3 to 50)	29.8% (29.6 to 30.1)	73.7% (73.3 to 74.1)	–
	55–65	37.9 (35.1 to 40.7)	84.1 (80.3 to 87.2)	44.7 (39.5 to 50.1)	38.5% (38.2 to 38.8)	82.5% (82.1 to 82.9)	–
	65+	40.1 (35.9 to 44.5)	80.9 (74.8 to 85.8)	56 (48 to 63.7)	38.8% (38.3 to 39.3)	83.5% (82.9 to 84.1)	–
	Test of differences (p value)	0.000	0.000	0.162
Gender	Female	20.4 (19.7 to 21.1)	67.1 (65.2 to 68.9)	46.8 (44.2 to 49.4)	20.9% (20.7 to 21.1)	62.7% (62.5 to 62.9)	–
	Male	25.7 (24.4 to 27.1)	69.9 (67 to 72.6)	43.8 (40.1 to 47.6)	24% (23.7 to 24.3)	65.6% (65.2 to 66)	–
	Test of differences (p value)	0.000	0.105	0.205
Locality	Capital city or surrounding suburbs	20.3 (19.5 to 21.1)	68.1 (66 to 70.2)	46.4 (43.6 to 49.3)	21.2% (21 to 21.4)	–	–
	Other urban region	21.6 (20.2 to 23.1)	69.2 (65.7 to 72.5)	44.6 (39.9 to 49.4)	21.5% (21.2 to 21.8)	–	–
	Rural or remote region	26.1 (24.5 to 27.6)	66.5 (63.2 to 69.7)	45.4 (41.1 to 49.8)	23.4% (23 to 23.8)	–	–
	Test of differences (p value)	0.000	0.742	0.916		–	–
Born in Australia	No	22.5 (21.1 to 23.9)	69.3 (65.9 to 72.5)	51.6 (46.9 to 56.2)	–	–	–
	Yes	21.6 (20.8 to 22.3)	67.5 (65.7 to 69.3)	44.8 (42.3 to 47.2)	–	–	–
	Test of differences (p value)	0.269	0.367	0.011
Aboriginal and/or Torres Strait Islander	No	21.9 (21.2 to 22.7)	67.8 (66 to 69.6)	45.1 (42.6 to 47.5)	21.6% (21.4 to 21.7)	–	–
	Yes	14.4 (11.6 to 17.9)	51.4 (39.9 to 62.9)	39.5 (25.4 to 55.6)	16.5% (15.6 to 17.3)	–	–
	Test of differences (p value)	0.000	0.005	0.495
Employment	Full or part time employment	22.6 (21.8 to 23.5)	68.9 (66.9 to 70.8)	48.7 (46 to 51.4)	–	–	–
	Unemployed	20.2 (18.3 to 22.2)	56 (50.6 to 61.2)	33.2 (26.7 to 40.3)	–	–	–
	Student	11.7 (10.4 to 13)	59.6 (53.8 to 65.2)	39.6 (32.1 to 47.7)	–	–	–
	Other (retired, disability, etc)	29.7 (27.8 to 31.6)	74.2 (70.7 to 77.4)	45.4 (40.8 to 50)	–	–	–
	Test of differences (p value)	0.000	0.000	0.000
Education	Other	19 (18.2 to 19.8)	63.9 (61.7 to 66.1)	45.2 (42.2 to 48.2)	19.6% (19.4 to 19.8)	61.6% (61.4 to 61.8)	–
	University degree	25.9 (24.8 to 27)	72.2 (70 to 74.3)	46.6 (43.6 to 49.7)	24.7% (24.5 to 24.9)	66% (65.8 to 66.2)	–
	Test of differences (p value)	0.000	0.000	0.495
Relationship Status	Married/ Defacto	27.6 (26.5 to 28.8)	72.3 (70.1 to 74.4)	48.4 (45.3 to 51.4)	30.8% (30.6 to 31)	–	–
	Never married/ Single	16 (15.3 to 16.9)	61.4 (58.6 to 64)	41.4 (37.7 to 45.1)	18.8% (18.6 to 19)	–	–
	Separated	26.3 (24.4 to 28.4)	69.2 (65 to 73.1)	45.2 (39.8 to 50.8)	31.8% (31.4 to 32.2)	–	–
	Widowed	30.5 (23.9 to 37.9)	74 (60.2 to 84.3)	65 (49.2 to 78.1)	39.1% (37.3 to 40.9)	–	–
	Test of differences (p value)	0.000	0.000	0.005
K10 severity	Mild (10–24)	26.9 (25.2 to 28.8)	76 (72.6 to 79.2)	50.7 (46.1 to 55.2)	26.4% (26.1 to 26.8)	72.2% (71.8 to 72.6)	–
	Moderate (25–29)	27.4 (25.8 to 29)	70.9 (67.8 to 73.9)	52.3 (48 to 56.6)	25.4% (25.1 to 25.7)	66.1% (65.7 to 66.5)	–
	Severe (30+)	18.8 (18.1 to 19.6)	64.1 (62 to 66.2)	40.8 (38 to 43.7)	19.5% (19.3 to 19.7)	60.7% (60.5 to 60.9)	–
	Test of differences (p value)	0.000	0.000	0.000
PHQ9 severity	Mild (0–9)	26.9 (25.2 to 28.8)	75.5 (72.7 to 78.1)	53.8 (49.9 to 57.6)	26% (25.7 to 26.3)	–	53.8% (53.8 to 53.8)
	Moderate (10–14)	27.4 (25.8 to 29)	67.3 (64.4 to 70.1)	46.8 (42.9 to 50.7)	23% (22.7 to 23.3)	–	45.4% (45.3 to 45.6)
	Severe (15+)	18.8 (18.1 to 19.6)	63.5 (61 to 65.9)	38.8 (35.5 to 42.2)	19.1% (18.9 to 19.3)	–	38% (37.9 to 38.2)
	Test of differences (p value)	0.000	0.000	0.000
GAD7 severity	Mild (0–9)	26.9 (25.2 to 28.8)	71.9 (69.2 to 74.4)	49.3 (45.8 to 52.8)	–	–	–
	Moderate (10–14)	27.4 (25.8 to 29)	68.5 (65.7 to 71.1)	45.3 (41.7 to 49)	–	–	–
	Severe (15+)	18.8 (18.1 to 19.6)	63.4 (60.6 to 66.2)	42.1 (38.3 to 45.9)	–	–	–
	Test of differences (p value)	0.000	0.000	0.025
Total Number of Psychosocial Difficulties Endorsed (from list below)	0	25.3 (23.6 to 27)	74.1 (70.6 to 77.4)	45.9 (41.3 to 50.7)	24.1% (23.7 to 24.5)	70.7% (70.5 to 70.9)	45.2% (44.8 to 45.6)
	1	25.8 (24.2 to 27.4)	74.4 (71.2 to 77.4)	51.3 (46.9 to 55.5)	24.4% (24 to 24.8)	72% (71.8 to 72.2)	47.6% (47.4 to 47.8)
	2	22.4 (21 to 23.8)	70.6 (67.3 to 73.7)	47.4 (43 to 51.9)	22.5% (22.1 to 22.8)	65.6% (65.2 to 66)	46.3% (46.1 to 46.5)
	3	19.9 (18.6 to 21.4)	63.2 (59.3 to 67)	46.8 (41.6 to 52.1)	20.2% (19.9 to 20.5)	58.3% (57.9 to 58.7)	45.4% (45.2 to 45.6)
	4	17.4 (15.8 to 19.1)	57 (51.8 to 62.1)	35.8 (29.1 to 43.1)	18.6% (18.2 to 18.9)	53.8% (53.4 to 54.2)	36.5% (36.1 to 36.9)
	5	18.2 (16.1 to 20.6)	53 (46.1 to 59.8)	33.3 (24.9 to 43)	19.1% (18.6 to 19.6)	53.4% (52.8 to 54)	35.1% (34.7 to 35.5)
	6	14.6 (12.4 to 17.2)	57.9 (48.9 to 66.3)	29.3 (19.1 to 42.2)	17.5% (17 to 18.1)	56.4% (55.6 to 57.2)	33.7% (33.3 to 34.1)
	Test of differences (p value)	0.000	0.000	0.000
Relationship difficulties	Not indicated (0)	25.4 (24.4 to 26.5)	71.5 (69.3 to 73.6)	47.5 (44.6 to 50.5)	–	–	–
	Indicated (1)	19.1 (18.3 to 19.9)	64.7 (62.4 to 66.9)	44 (40.9 to 47)	–	–	–
	Test of differences (p value)	0.519	0.000	0.102
Parenting difficulties	Not indicated (0)	21.8 (21.1 to 22.5)	69.2 (67.5 to 70.9)	45.5 (43.2 to 47.8)	–	–	–
	Indicated (1)	21.2 (19.5 to 22.9)	59.9 (55.4 to 64.3)	48 (42 to 54.2)	–	–	–
	Test of differences (p value)	0.519	0.000	0.440
Vocational difficulties	Not indicated (0)	23.4 (22.5 to 24.4)	71.5 (69.3 to 73.5)	47.2 (44.3 to 50.1)	23.7% (23.5 to 24)	–	–
	Indicated (1)	20.1 (19.2 to 21)	64.2 (61.9 to 66.5)	44.1 (40.9 to 47.3)	19.8% (19.6 to 20)	–	–
	Test of differences (p value)	0.000	0.000	0.152
Physical difficulties	Not indicated (0)	22.1 (21.3 to 22.9)	69.5 (67.5 to 71.5)	48.8 (46.1 to 51.6)	–	–	–
	Indicated (1)	21.1 (20.1 to 22.1)	65.6 (63.1 to 68.1)	41.1 (37.8 to 44.5)	–	–	–
	Test of differences (p value)	0.135	0.016	0.001
Financial difficulties	Not indicated (0)	23.1 (22.4 to 23.9)	71.3 (69.5 to 73)	47.5 (45.1 to 49.9)	22.7% (22.5 to 22.9)	66.2% (66 to 66.4)	–
	Indicated (1)	18.3 (17.2 to 19.4)	58 (54.7 to 61.3)	39.7 (35.3 to 44.3)	19.4% (19.1 to 19.6)	56.8% (56.4 to 57.2)	–
	Test of differences (p value)	0.000	0.000	0.003
Housing difficulties	Not indicated (0)	22.6 (21.9 to 23.3)	68.9 (67.3 to 70.5)	46.2 (44 to 48.4)	–	–	–
	Indicated (1)	14.3 (12.7 to 16)	56 (49.8 to 62)	40.4 (32.5 to 48.9)	–	–	–
	Test of differences (p value)	0.000	0.000	0.196
Alcohol difficulties	Not indicated (0)	21.9 (21.2 to 22.6)	68.6 (66.9 to 70.2)	46.2 (43.9 to 48.4)	–	–	–
	Indicated (1)	19.6 (17.7 to 21.7)	61.6 (56 to 67)	41.5 (34.2 to 49.1)	–	–	–
	Test of differences (p value)	0.038	0.014	0.247
Drug difficulties	Not indicated (0)	22.4 (21.7 to 23.1)	68.6 (67 to 70.1)	46.1 (44 to 48.3)	–	–	–
	Indicated (1)	11.9 (10.1 to 14)	51.6 (42.9 to 60.3)	32.7 (21.7 to 46.1)	–	–	–
	Test of differences (p value)	0.000	0.000	0.052
Religion difficulties	Not indicated (0)	21.9 (21.2 to 22.5)	68.2 (66.6 to 69.8)	45.9 (43.7 to 48)	–	–	–
	Indicated (1)	17.8 (15 to 20.9)	60.5 (51.3 to 69.1)	43.8 (32.2 to 56)	–	–	–
	Test of differences (p value)	0.014	0.084	0.739
Cultural difficulties	Not indicated (0)	21.8 (21.2 to 22.5)	68.1 (66.5 to 69.6)	45.9 (43.8 to 48)	–	–	–
	Indicated (1)	16.2 (12.8 to 20.3)	62.3 (49.6 to 73.5)	40 (25.3 to 56.7)	–	–	–
	Test of differences (p value)	0.010	0.339	0.489
Sexual difficulties	Not indicated (0)	22 (21.3 to 22.6)	68.1 (66.5 to 69.6)	46.1 (44 to 48.3)	–	–	–
	Indicated (1)	15 (12.4 to 18.1)	63.8 (53.7 to 72.9)	31.9 (20.2 to 46.4)	–	–	–
	Test of differences (p value)	0.000	0.383	0.057
Grief difficulties	Not indicated (0)	21.8 (21.1 to 22.4)	68.2 (66.6 to 69.7)	46.3 (44.1 to 48.4)	–	–	–
	Indicated (1)	20 (17 to 23.3)	63 (54.3 to 70.9)	28.8 (18.7 to 41.6)	–	–	–
	Test of differences (p value)	0.284	0.221	0.009
Carer difficulties	Not indicated (0)	21.6 (21 to 22.3)	68.2 (66.6 to 69.7)	46 (43.9 to 48.2)	–	–	–
	Indicated (1)	27.5 (21.9 to 34)	54.4 (41.5 to 66.8)	28 (14 to 48.2)	–	–	–
	Test of differences (p value)	0.041	0.029	0.080

Some variables also show differential effects across uptake, completion and improvement. Gender for instance shows a significant effect for treatment uptake, but no significant effect on treatment completion or symptom improvement, suggesting that even though fewer males than females complete an assessment (24.2%vs 75.8%, Table 1), males are more likely to start treatment than females (25.7% vs 20.4%, p < .001), and once started are equally likely to complete (67.1% vs 69.9%, p = .105) and equally gain significant symptomatic improvement (46.8% vs 43.8%, p = .205). Education status shows a similar pattern, where those with a university degree are more likely to start (19% vs 25.9%, p < .001) and complete treatment (63.9% vs 72.2%, p < .001), but having a degree does not affect symptom improvement (45.2% vs 46.6%, p = .495). Some variables show significant within-variable differences across uptake, completion and improvement. For example, treatment completion rates decreased as psychosocial problems increased (74.1% to 53%, p < .001).

Multivariate analyses

In the second step, the multivariate analyses of uptake, completion and improvement outcomes were conducted, with the results collated in Table 2, under the column “multivariate models”. The multivariate analyses of uptake probability identified a combined list of patient features, including age, gender, locality, Aboriginal and Torres Strait Islander Status, educational attainment, relationship status, PHQ-9 symptoms and total number of endorsed psychosocial difficulties. From the list of possible predictors, age, gender, education, locality, Aboriginal and Torres Strait Islander status, relationship status, PHQ9 score, K10 score, total number of difficulties, vocational difficulties and financial difficulties appeared to most clearly delineate the likelihood of taking up treatment. The association of each of the listed predictors to the rate of uptake can be evaluated both in the table, as a total effect, as well as graphically, as a classification tree “node”, describing subgroups that are characterised by more than one predictor. A visualization of the uptake classification algorithm is presented in the supplementary material, Fig. A.

The multivariate analyses of completion probability identified a similar joint list of patient features, including age, gender, education, K10 score and total number of endorsed psychosocial difficulties, and financial difficulties as the strongest variables that delineate the likelihood of completing treatment. A visualization of the treatment completion classification algorithm is presented in the supplementary material, Fig. B.

The multivariate analyses of symptom improvement events, as defined by a greater than 50% reduction in K10 scores, resulted in a more parsimonious model where lower initial depressive symptom severity (PHQ9) and fewer endorsed psychosocial difficulties were the greatest predictors of significant symptomatic improvement events. A visualization of the symptom improvement classification algorithm is presented in the supplementary material, Fig. C.

Similarities across the models for uptake, completion and symptomatic improvement resulted in high and significant shared probability co-variance. For example, the correlation between the identified model uptake probability and completion probability was moderate to high, with r = 0.724 (95%CI 0.716 to 0.731). The correlation between the model's uptake and significant symptom improvement probability estimates was also significant (r = 0.317, 95%CI 0.303 to 0.331), as was the correlation between completion and significant symptom improvement (r = 0.365, 95%CI 0.351 to 0.378). These results suggest that the kinds of patient characteristics that influence treatment uptake are strongly implicated in both treatment completion and symptom improvement, with only minor exceptions.

Culminative multivariate model

The cumulative influence of variables that significantly moderated the rate of uptake, completion and improvement can also be visualized in a multivariate model using a classification algorithm to illustrate how some variables can increase or decrease the rate of all three outcomes in the same direction. For example, variables such as initial symptom severity, relationship status, employment and total number of psychosocial difficulties demonstrate either culminative decreases or increases to uptake, completion, and improvement in the same direction, leading to an aggregate difference of up to 45% from the average estimate. These cumulative trends can be visualized in Fig. 2, illustrating the size, and common direction of the impact on these three outcomes, highlighting the consistency of certain participant characteristics across outcomes. For example, those with less than two psychosocial difficulties show a culminative increased likelihood of treatment across all uptake, completion and improvement groups, while those with more than three psychosocial difficulties show a decreased culminative likelihood across all treatment uptake, completion and improvement groups.

Fig. 2

A CHAID (chi-square automatic interaction detection classification algorithm) diagram that identifies the higher-order interactions across subgroups of variables that are characterised by their culminative impact on all three variables in the same direction.

Sensitivity analyses

In a third step, sensitivity analyses of the multivariate models were retested with the inclusion of a regularization parameter. Results from these sensitivity models demonstrated that the predictors list for uptake, completion and symptom improvement can be reduced into a shorter and more robust list without losing predictive accuracy. The multivariate models resulted in a moderate rate of sensitivity (true-positive) for the prediction of uptake (67.1%) adherence (72.6%) and improvement (60.0%). Less rates of specificity (true negatives) prediction accuracy were observed for uptake (57.4%), completion (50%) and improvement (55%). The overall diagnostic ability of these multivariate models, conveyed with the cumulative area under the curve metric, was fair for uptake (66.1% [65.1 to 67.1]) and completion (66.0% [64.1 to 67.9]) but poor for improvement (58.7% [56.3 to 66.1]). An optimal (robust) model of treatment uptake was identified with a Ridge penalty parameter of 0.90, and a Lasso parameter of 0.40 by only including age, relationship status and baseline symptom (K-10) severity as predictors. For treatment completion, an optimal (robust) model was identified with a Ridge penalty parameter of 0.30, and a Lasso parameter of 0.36 by including the predictors of age, total number of psychosocial difficulties and initial PHQ9 score. Lastly, the optimal (robust) model of symptom improvement rates was identified with a Ridge penalty parameter of 0.70, and a Lasso parameter of 0.24 by including initial PHQ-9 and K-10 scores, and total number of psychosocial difficulties. These results seem congruent with the classification trees visualized in Figs. A-C, identifying age as the primary (strongest) predictor node for the classification of uptake and completion, and (lower) baseline symptom severity as the primary (strongest) predictor of symptomatic improvement.

Discussion

The current study sought to examine the unique patient characteristics associated with treatment uptake and treatment completion, and their association with symptomatic improvement. Importantly, our use of a large routine care sample provided sufficient power to analyse a range of predictors simultaneously, as well as testing the relative impact of each model on uptake, completion and improvement in the context of a typical service flow. While some of the factors identified in the current study such as age, initial symptom severity, relationship status, educational attainment have been identified in previous studies, the effect of multiple self-reported psychosocial difficulties has not been studied. Those difficulties were found to be strongly associated with uptake, completion and subsequent improvement. As with previous findings, psychosocial difficulties on their own have limited impact on each of these variables. However, the presence of multiple co-existing difficulties was negatively associated with all three outcomes. As the number of psychosocial problems reported by an individual increase, the likelihood of starting treatment, completing treatment and obtaining significant symptom improvement significantly decreases. In our large community sample, just under half reported more than two concurrent psychosocial difficulties, suggesting a large proportion of prospective patients who might benefit from treatment, but may find it challenging progress. Further, each of the implicated factors were shown to interact in a culminative manner across the treatment journey, such that the combination of age, relationship status, educational attainment, baseline symptom severity and psychosocial difficulties act as the strongest cumulative predictors of uptake, completion and improvement. Nevertheless, it is also important to note that none of the factors identified were so dominant as to suggest that certain groups would not take up, complete or benefit in their entirety. For example, focussing on psychosocial difficulties, those with the most psychosocial difficulties (compared with those without any difficulties) were still fairly likely to complete an assessment (17.5% vs. 24.1%), and complete (56.4% vs. 70.7%) and benefit from treatment (33.7% vs. 45.2%).

A possible mechanism for the observed relationship between these culminative demographic, clinical and psychosocial difficulties and lower uptake, completion and symptom improvement is the construct of ‘treatment burden’ (Heckman et al., 2015; Mair and May, 2014; Tran et al., 2015; Rock and Cross, 2020). Most commonly studied in chronic disease management, treatment burden broadly considers the amount of patient effort required to successfully engage in a particular treatment, and the patients' capacity to meet this burden, which is in addition to the burden attributable to the disease itself. In their workload-capacity model, Heckman et al. suggest that a higher ratio of workload over capacity can lead to ‘treatment fatigue’ which can ultimately lead to poor treatment adherence and associated negative outcomes (Heckman et al., 2015). Any psychological or behavioural treatment, whether online or face-to-face, places certain demands on patients over and above the burden of the disorder itself and can be disproportionally taxing for some patients with limited or reduced capacity to complete the established requirements of treatment. Examples of treatment demands include completing numerous or lengthy questionnaires, reading material, setting time aside to practice skills, completing out of session homework tasks such as exposure or cognitive challenging, and other behaviours or activities recommended in order to achieve maximum therapeutic benefit. While these treatment demands are fixed, usually based on established evidence guidelines or treatment protocols that include set amount to read, set homework assignments, set frequency of appointments and lessons over a set period of time, the ability of individual patients to meet these demands will naturally vary with their personal capacities, and the number of psychosocial difficulties they face. The main factors identified in this study: age, educational level, initial symptom severity, coexisting psychosocial difficulties, in one way or another could represent increased burden or reduced capacity which may reduce the likelihood of successful treatment uptake, completion and subsequent improvement. Young people, for example, may experience proportionally increased disease and lifestyle risk factor burden (Mokdad et al., 2016; Patel et al., 2007) as well as lower maturity and organisational skills, while initial symptom severity may indicate greater disorder-related burden. In addition to the ‘treatment burden’ factors found in this study, other ‘burden’ and ‘capacity’ factors such as roles in the family, activities required to manage symptoms, self-efficacy, motivation for change, emotional states, cognitive processes, social supports, and others may all work in combination to influence treatment uptake and completion behaviour.

It is also possible that some patients with a large number of psychosocial difficulties do not take up or complete treatment because they find the content not suitable or relevant for directly addressing their psychosocial needs. For instance, their distress might be more maintained by ongoing social or environmental situations, and less by maladaptive cognitive or behavioural patterns that are the primary focus of CBT. In those cases, early identification, and referral to appropriate supports (e.g., financial counselling, accommodation services, domestic violence services, etc) may be more appropriate than a CBT intervention.

Further research might explore a broader range of burden, capacity and maintenance factors and the nature of their relationship with uptake and completion, which may allow the development of targeted interventions that might assist individual patients to develop the capacity to participate and complete treatment if appropriate, such as a broader range of social or other supports. Further research might also explore ways of reducing treatment burden and offering supportive technologies that make treatment easier to access and complete, such as ‘effort-optimized interventions’ (Baumel and Muench, 2021). These strategies can be offered to patients prospectively from the point at which they access the DMHS, given the ability to identify subgroups of patients who are less likely to take up and complete treatment and experience symptom improvement, based on characteristics they report at assessment. While many behavioural and mental health apps are designed to simplify treatment and minimise treatment burden, none have yet proven to be able to reduce that burden while maintaining treatment outcome.

Limitations and strengths

This study made use of a large community-based sample of patients receiving digital interventions as part of routine care and is the largest study of its kind to date. The context of routine care offers the strength that the findings are arguably more applicable to community service provision. Weaknesses include limited control of variables such as the ‘dose’ of treatment received and the degree of therapist contact. Another perceived weakness may have been our approach to handling missing cases. As we aimed to show the natural participant flow and drop out at different stages of treatment, imputing missing cases to the analysis would have detracted from the stated aims of the paper. Additionally, the treatment completion group represented 13% of the assessment group. Nevertheless, sample numbers were sufficiently high, and the statistical methods used were powerful enough to detect differences in this relatively smaller sample. Also, we set the rate of symptom improvement at 50%, which is comparatively high, and which may have underestimated the degree of broader symptom improvement for patients dealing with psychosocial difficulties. Another strength of the study was the examination of variables across both uptake and completion, and symptom improvement together enabling us to identify and examine those factors which are common across the treatment journey.

Conclusions

DMHS are increasingly becoming a part of mental health systems and providing services to people with mental ill health who may otherwise be unable to access psychological assessment and treatment services. Several patient characteristics including age, initial symptom severity, relationship status, educational attainment, and culminative number of self-reported psychosocial difficulties may all contribute to increased burden or decreased capacity and in turn, negatively affect treatment uptake, completion, and subsequent improvement. Future research should further explore the effect of interventions and treatment modifications that improve uptake and treatment completion particularly targeting those affected by treatment burden and related challenges.

Funding

Investigator-initiated.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.