Internet-delivered treatment for older adults with anxiety and depression: implementation of the Wellbeing Plus Course in routine clinical care and comparison with research trial outcomes.

BACKGROUND: The Wellbeing Plus Course is an internet-delivered psychological intervention for older adults with anxiety or depression. AIMS: To compare the effectiveness of the Wellbeing Plus Course in a public health setting (clinic group) with its efficacy in a randomised controlled trial (research group).
METHOD: Participants (n=949) were Australian adults aged 60 and above. Primary outcome measures were the Patient Health Questionnaire (PHQ-9) and Generalized Anxiety Disorder scale (GAD-7).
RESULTS: Initial symptom severity was higher in the clinic group and course completion was lower. Both groups showed significant symptom reductions at post-treatment and were satisfied with the treatment. Results were maintained at 3-month follow-up. Within-group symptom changes were comparable between settings; there were no between-group differences on primary outcomes or satisfaction.
CONCLUSIONS: The Wellbeing Plus Course is as effective and acceptable in routine clinical care, as it is in controlled research trials. DECLARATION OF INTEREST: N.T. and B.F.D developed the Wellbeing Plus Course but derived no financial benefit from it. COPYRIGHT AND USAGE: © The Royal College of Psychiatrists 2016. This is an open access article distributed under the terms of the Creative Commons Non-Commercial, No Derivatives (CC BY-NC-ND) license.

RCT Entities:   Population Interventions Outcomes

Symptoms of depression and anxiety are common in older adults and are associated with increased disability and decreased quality of life,; however, the proportion of this population accessing treatment is low, and therefore represents a significant public health problem., One way to increase treatment uptake is to provide evidence-based cognitive behavioural therapy via the internet (iCBT)., Research trials conducted by the eCentreClinic, an online research unit at Macquarie University Australia, have developed and evaluated iCBT for older adults. These trials have demonstrated that the Wellbeing Plus Course, a transdiagnostic intervention designed for adults aged 60 years and above with symptoms of anxiety or low mood, is both efficacious and acceptable.– Although the research outcomes are encouraging, and some studies have demonstrated successful translation of online treatment for adults in general,– other studies have reported less successful implementation,– and there is little published evidence of the real-world effectiveness of internet-delivered therapy specifically for adults aged 60 years and above. Implementation data are critical when translating treatments from highly controlled research settings to routine clinical settings, as lower levels of compliance and more complicated or severe presentations may be expected.,

As part of the Australian Government’s e-mental health strategy, the MindSpot Clinic was funded to provide online treatment to Australian adults with anxiety and depression, including iCBT specifically designed for adults aged 60 and above. Between January 2013 and June 2015, the Wellbeing Plus Course was provided to over 500 people. This paper compares outcomes from the MindSpot Clinic (‘clinic group’) with a group consisting of participants from the most recent randomised controlled trial of the course conducted by the eCentreClinic (‘research group’). Based on previous published research,, it was expected that initial symptom severity would be higher in the clinic group, compared with the research group where more stringent inclusion criteria were used. It was hypothesised that the clinic group would show significant improvements in symptoms comparable to the results obtained in the research trial.

Method

Participants

Participants in the clinic group were consecutive patients starting the Wellbeing Plus Course at MindSpot from 1 January 2013 to 30 June 2015. A total of 516 participants were eligible for analysis. Patients were eligible if they met the following criteria: (1) identified as an Australian resident eligible for publicly funded health services, (2) were aged 60 years and above, (3) reported a principal complaint of anxiety or depression and (4) agreed for their de-identified data to be reported. Individuals who were acutely suicidal, preferred face-to-face services, were currently participating in CBT or had clinical presentations (e.g. untreated and disabling psychotic symptoms, or severe symptoms across four or more types of anxiety or depressive disorders) deemed to require face-to-face assessment were referred to suitable alternative services. Participants self-referred through the MindSpot Clinic website (www.mindspot.org.au) and completed an online assessment, providing demographic details and completing standardised self-report symptom questionnaires. These questionnaires were used in conjunction with assessment by the clinicians to determine the likely benefits of treatment for individual applicants.

Participants forming the research group (n=433) were enrolled in a randomised controlled trial conducted by the eCentreClinic to test the effect of providing varying levels of clinician contact prior to and during the Wellbeing Plus Course. Participants were recruited online via the eCentreClinic website (www.ecentreclinic.org). Consistent with the clinic group, participants in the research group were Australian residents aged 60 years and above who reported a principal complaint of anxiety or depression. In contrast to the clinic group, patients were excluded from the research trial if they had a total score ≥24 on the 9-item Patient Health Questionnaire (PHQ-9) or a score of 3 on the question regarding suicidal ideation. Participant flow is summarised in Fig. 1. Approval to conduct the study was provided by the Human Research Ethics Committee at Macquarie University.

Fig. 1

Participant flow.

Procedure

The Wellbeing Plus Course is a transdiagnostic iCBT intervention designed to treat core symptoms of anxiety and depression in adults aged 60 years and above. The treatment consists of five lessons and homework assignments, made available over an 8-week period. Each lesson is presented as a series of slides that include didactic text and case-enhanced learning examples, photos and images, which illustrate the principles of CBT, together with supplementary material on related topics. The case-enhanced stories detail the experiences of older adults recovering from symptoms of anxiety and depression. The content of the course lessons and related procedures have been described previously.

Clinicians in the research unit were registered clinical psychologists with doctoral-level qualifications and several years of experience in internet-delivered treatments. Clinicians at the MindSpot Clinic were mainly nationally registered psychologists (n=15; 75%), three were provisional psychologists in training, one was an indigenous mental health worker and one was a counsellor. All had previous clinical experience of 1–15 years. Clinicians in both the research unit and the MindSpot Clinic received specific training in the principles of online psychological assessment and treatment, as well as detailed course instructions. Supervision and ongoing training was provided in both settings. For the clinic group, clinicians attempted to contact all the participants each week via telephone or email to provide guidance in completing the course, although the level of contact varied widely. For the research group, participants were originally allocated to one of three levels of clinician contact. As the published results from this trial showed that the level of clinician contact did not affect outcomes, the participants were analysed as one group for the purposes of this study.

Outcome measures

Two primary measures were used to assess the presence and severity of symptoms: the first, the Patient Health Questionnaire 9-item scale (PHQ-9), was used to measure symptoms of depression. Scores on the PHQ-9 range from 0 to 27, with higher scores indicating more severe symptoms. A score ≥10 on the PHQ-9 has been identified as the threshold for identifying clinically significant depression, and a score ≥20 indicates symptoms of severe depression. A score ≥24 on the PHQ-9 can be used to define very severe symptoms of depression, and patients scoring ≥24 were excluded from the research group (but not the clinic group). The PHQ-9 criterion originally applied to the research group has good internal consistency and is sensitive to change.

The second, the Generalized Anxiety Disorder 7-item scale (GAD-7), was used to measure symptoms of anxiety. Scores on the GAD-7 range from 0 to 21, with higher scores indicating more severe symptoms. A score ≥8 has been identified as the threshold for identifying clinically significant anxiety, and a score ≥15 indicates severe symptoms. The GAD-7 also has good internal consistency and good convergent and divergent validity with other anxiety and disability scales., The PHQ-9 and GAD-7 were administered at assessment, weekly throughout the course, post-treatment and at 3-month follow-up.

The Kessler 10-item scale (K-10) was used as a secondary measure to assess general psychological distress. Scores range from 10 to 50, with higher scores indicating more severe distress. The K-10 was administered at assessment, mid-treatment, post-treatment and follow-up. Treatment satisfaction and acceptability were assessed at post-treatment by two questions requiring a ‘yes’ or ‘no’ response: (1) ‘Would you feel confident in recommending this course?’ and (2) ‘Was it worth your time doing the course?’

Statistical analyses

Group differences in demographics, clinician time and satisfaction were assessed by general linear models. Treatment outcomes over time on the PHQ-9, GAD-7 and K-10 were analysed with generalised estimating equation (GEE) models. Consistent with the principles of intention-to-treat analyses, separate GEE models utilising random intercepts were employed to impute missing data. An unstructured working correlation matrix and maximum likelihood estimation were used, and a gamma distribution with a log link response scale was specified to address positive skewness in the dependent variable distributions. Outcome measures were analysed at assessment, post-treatment and at 3-month follow-up, with assessment scores included as a covariate.

To assess clinical significance, Cohen’s d effect sizes and 95% confidence intervals were calculated for within-group and between-groups effects, based on the estimated marginal means derived from the GEE models. Average percentage improvements across time were also calculated. To provide data about negative outcomes, the proportion of participants reporting deteriorations ≥30% on the primary measures as well as symptoms in the clinical ranges at post-treatment (i.e. ≥10 for the PHQ-9 and ≥8 for the GAD-7) were calculated.

Results

Demographic characteristics, course engagement and clinician contact time

Demographic characteristics are included in Table 1. The mean age of both groups was 65.7 years (clinic group: s.d. 5.1; range 60–88; research group: s.d. 4.8; range 60–93). There were no significant differences between the clinic and research groups in age, gender, marital status, education or vocational status (P’s>0.05). There was a significant difference between groups in psychotropic medication use and initial symptom severity (Table 1). At assessment, 42% of the clinic group reported taking medication for anxiety or depression, compared with 33% of the research group. In addition, 14% and 25% of the clinic group reported severe symptoms of depression or anxiety, respectively (≥20 on the PHQ-9 or ≥15 on the GAD-7), compared with 4% and 19% of the research group (P’s>0.05). All other participants scored in the mild range or above on either the PHQ-9 or GAD-7. Distributions of PHQ-9 and GAD-7 scores according to the established severity categories for each measure are shown in Fig. 2.

Table 1

Demographic characteristics and symptom and treatment histories

	Clinic group (n=516)	Research group (n=433)	Statistical comparison between groups
Age
Mean (s.d.)	65.7 (5.1)	65.7 (4.8)	F1,947=0.0, P=0.993
Range	60–88	60–93
Gender
Female	59% (302/516)	64% (278/433)	Wald’s χ² =3.2, P=0.074
Marital status
Single/never married	12% (62/516)	4% (19/433)	Wald’s χ² =0.1, P=0.820
Married/de facto	56% (290/516)	68% (293/433)
Separated/divorced/widowed/other	26% (133/516)	28% (121/433)
No answer	6% (31/516)	0% (0/433)
Education
High school or less	21% (110/516)	20% (85/433)	Wald’s χ² =0.0, P=0.970
Certificate/diploma/university	75% (389/516)	80% (348/433)
No answer	3% (17/516)	0% (0/433)
Vocational statusa
Retired	50% (257/516)	53% (229/433)	Wald’s χ² =1.0, P=0.318
Employed full-time or part-time	31% (159/516)	33% (143/433)
Other/no answer	19% (100/516)	14% (61/433)
Treatment history
Seen mental health professional in previous 12 months	34% (176/516)	34% (149/433)	Wald’s χ² =0.0, P=0.922
Taking psychotropic medication	42% (216/503)	33% (145/433)	Wald’s χ² =7.0, P<0.01
Symptom severity at assessmentb
Severe symptoms of depression (PHQ-9 ≥20)	14% (70/516)	4% (19/433)	Wald’s χ² =21.1, P<0.001
Severe symptoms of anxiety (GAD-7 ≥15)	25% (127/516)	19% (81/433)	Wald’s χ² =4.8, P<0.05

Categories of vocational status were not mutually exclusive for the research group data. Participants choosing more than one category were placed in the ‘other’ category for the purposes of the current comparison.

Symptom severity thresholds were defined by the original psychometric papers.,

Fig. 2

Distributions of PHQ-9 and GAD-7 scores by category.

There were statistically significant between-group differences in adherence and attrition (Fig. 1). The clinic group completed a mean of 4.4 lessons (s.d. 1.2) compared with a mean of 4.7 lessons (s.d. 1.1) in the research group (P<0.001). Post-treatment and follow-up questionnaire completion rates were also lower in the clinic group (81% and 50%, respectively) compared with the research group (97% and 96%; P’s<0.001), and course withdrawal rates were higher (13% compared with <1%; P<0.001).

Clinician contact comprised making and receiving phone calls, and reading and responding to secure emails. There was a significant difference between groups in clinician contact time (F=520.6, P<0.001). The clinic group had more contact with a clinician throughout the course, with a mean total clinician time per participant of 129 min (s.d. 76.3), compared with 36 min (s.d. 40.0) for the research group.

Treatment outcomes

PHQ-9 scores at assessment were higher for the clinic group compared with the research group (Wald’s χ²=1066.4, P<0.001). There was a significant main effect over time on the PHQ-9 (Wald’s χ²=2117.1, P<0.001), with both groups showing a significant decrease in scores from assessment to post-treatment and from assessment to 3-month follow-up (P’s<0.001). There were no group (Wald’s χ²=0.4, P=0.549) or interaction effects (Wald’s χ²=2.4, P=0.304).

Initial scores on the GAD-7 were higher for the clinic group compared with the research group (Wald’s χ²=1045.1, P<0.001). There was a significant main effect over time (Wald’s χ²=1963.7, P<0.001), with both groups showing a significant decrease in GAD-7 scores from assessment to post-treatment, from assessment to 3-month follow-up and from post-treatment to follow-up (P’s<0.001). There were no group (Wald’s χ²=0.1, P=0.737) or interaction effects (Wald’s χ²=3.5, P=0.179).

Initial scores on the K-10 were higher for the clinic group compared with the research group (Wald’s χ²=3015.2, P<0.001). There was a significant main effect of time (Wald’s χ²=12566.0, P<0.001), with both groups showing a significant decrease in symptoms from assessment to post-treatment, assessment to 3-month follow-up and from post-treatment to follow-up (P’s<0.001). There was no main group effect (Wald’s χ²=1.9, P=0.164), but there was an interaction effect (Wald’s χ²=6.6, P=0.037). Pairwise comparisons indicated that the improvement in K-10 scores at 3-month follow-up was greater for the clinic group than the research group (P=0.020).

Clinical significance

Effect sizes based on estimated marginal means are given in Table 2. Within-group effect sizes ranged from 1.4 to 1.7 for the clinic group and 1.2 to 1.6 for the research group. Between-group effect sizes were ≤0.3. Percentage changes are also shown in Table 2. Percentage improvements (i.e. reduction in symptoms) for the clinic group ranged from 57% to 62%. For the research group, percentage improvements ranged from 55% to 60%.

Table 2

Means and effect sizes (Cohen’s d) for the primary and secondary outcome measures

	n	Estimated means			Percentage change from assessment		Within-group effect sizes from assessment		Between-group effect sizes
		Assessment	Post-treatment	3-month follow-up	To post-treatment	To 3-month follow-up	To post-treatment	To 3-month follow-up	Post-treatment	3-month follow-up
PHQ-9
Clinic group	516	12.7 (6.0)	5.3 (4.1)	5.3 (3.8)	58% [55–61]	58% [56–61]	1.4 [1.3–1.6]	1.5 [1.3–1.6]	0.2 [0.1–0.3]	0.3 [0.1–0.4]
Research group	433	10.5 (4.8)	4.5 (3.7)	4.4 (3.3)	57% [54–61]	58% [55–61]	1.4 [1.3–1.6]	1.5 [1.3–1.6]
GAD-7
Clinic group	516	10.5 (5.0)	4.5 (3.5)	4.0 (2.8)	57% [54–60]	62% [59–64]	1.4 [1.3–1.5]	1.6 [1.5–1.7]	0.0 [−0.2–0.1]	0.0 [−0.2–0.1]
Research group	433	10.2 (4.4)	4.6 (3.9)	4.1 (3.3)	55% [51–58]	60% [57–63]	1.4 [1.2–1.5]	1.6 [1.4–1.7]
K-10
Clinic group	516	28.1 (6.9)	19.1 (6.2)	17.8 (5.0)			1.4 [1.2–1.5]	1.7 [1.6–1.9]	0.2 [0.1–0.4]	0.3 [0.1–0.4]
Research group	433	25.0 (6.5)	17.6 (6.0)	16.5 (5.5)			1.2 [1.0–1.3]	1.4 [1.3–1.6]

Note: Standard deviations are shown in round parentheses for the means and 95% confidence intervals are shown in square parentheses for percentage changes and effect sizes.

PHQ-9, Patient Health Questionnaire 9-item scale; GAD-7, Generalized Anxiety Disorder 7-item scale; K-10, Kessler 10-item scale.

On both the PHQ-9 and GAD-7, 1.2% (6/516) of participants in the clinic group showed a deterioration in scores of ≥30% and also scored above the clinical cut-off at post-treatment. For the research group, the deterioration rate was 1.4% (6/427) for both the PHQ-9 and GAD-7.

Treatment satisfaction

Of the participants who completed the evaluation questions, 97.3% (389/400) of the clinic group and 93.5% (361/386) of the research group reported that they would recommend the course to a friend. Similarly, 98.7% (390/395) of the clinic group and 94.3% (363/385) of the research group reported that the course was worth their time. There were no differences between groups in treatment satisfaction (P’s>0.05).

Discussion

This study evaluated the effectiveness of the Wellbeing Plus Course, an iCBT intervention designed for older adults with symptoms of anxiety and depression, following its implementation at a national online mental health service. The clinic group was compared with a research group, comprising participants from a randomised controlled trial. At assessment, psychotropic medication use was higher for the clinic group, as were symptoms of depression and anxiety, though for both groups, mean scores on the PHQ-9 and GAD-7 were in the moderate range.

Course completion was lower and clinician contact times were higher for the clinic group; however, both groups showed significant reductions in symptoms of anxiety and depression at post-treatment and 3-month follow-up and reported high levels of satisfaction with the intervention. There were no differences between the groups on the primary outcomes or in terms of satisfaction.

The results of this study indicate that iCBT for older adults with anxiety and depression can be successfully translated from the research environment into a public health setting, highlighting the considerable potential of iCBT to improve treatment access for older adults. Importantly, as well as being effective and acceptable, deterioration rates were low (<1.5%). The results are consistent with several previous studies showing the effectiveness of iCBT in routine clinical care for adults in general, and older adults in particular. The results are also consistent with the significant symptom improvements observed in older adults completing earlier versions of the course.,,,, Furthermore, the differences in clinician contact times observed in this and previous studies, suggest that there is scope for improving the efficiency of routine clinical care.

There are some limitations to this study. First, it is based on self-reported data from treatment-seeking individuals, and the absence of a formal diagnosis means that some care is needed in generalising the results. Second, in both the clinic group and the research group, the participants were drawn from the ‘young elderly’, with a mean age of around 66 years. It would be of interest to replicate the results in an older group, although previous research in this population suggests that age cohorts (60–64 years, 65–74 years and 75 years and above) respond similarly to the intervention. However, age-associated neurological disorders could be expected to affect symptom presentation in a more elderly sample, and the potential impact of age-related cognitive impairment in an elderly sample would be an important consideration in future studies. It is also important to note that participants with very severe symptoms of depression and thoughts of suicide or self-harm were referred to other services. Therefore, we cannot extend the current results to this group. Another limitation is the relatively short 3-month follow-up period, although similar studies have found that the gains made following iCBT are still evident 24 months after treatment., The short follow-up period also limits the ability to determine the cost-effectiveness of the intervention, although again the economic analyses after longer-term follow-up in similar studies have been favourable.

Although the study’s design can partially inform about the effectiveness of interventions in real-world conditions, there are limitations associated with the lack of randomisation of participants and the lack of formal matching of participants between conditions. These limitations are partly mediated by sample size; however, the use of alternative designs (e.g. pragmatic randomised controlled trial) and sampling methods (e.g. the use of stepwise-matching of participants) could address these issues in future trials.

It is also important to recognise the specialist nature of the MindSpot Clinic itself, which includes psychologists, psychiatrists, nurses and indigenous health workers trained in iCBT, a system of careful supervision of clinicians and a unique and well-supported software platform capable of delivering online assessment and treatment to more than 15 000 people nationally per year. This may limit the generalisability of the results to service providers that provide comparable levels of infrastructure, specialisation or support.– Notwithstanding this, a version of the service model developed at the MindSpot Clinic has been successfully implemented in Saskatchewan, Canada, highlighting the real-world potential of iCBT for people from a range of age groups and localities.

The growing availability of online mental health services, combined with the increasing use of the internet by older adults and an ageing population, highlights the need for real-world data from this age group. Overall, this study confirms that the effectiveness and acceptability of iCBT for older adults can be maintained when delivered as part of routine clinical care. Such studies play an essential role in informing researchers, health service providers, policy makers and consumers.