User characteristics and outcomes from a national digital mental health service: an observational study of registrants of the Australian MindSpot Clinic.

Background: Interest is growing in digital and telehealth delivery of mental health services, but data are scarce on outcomes in routine care. The federally funded Australian MindSpot Clinic provides online and telephone psychological assessment and treatment services to Australian adults. We aimed to summarise demographic characteristics and treatment outcomes of patients registered with MindSpot over the first 7 years of clinic operation.
Methods: We used an observational design to review all patients who registered for assessment with the MindSpot Clinic between Jan 1, 2013, and Dec 31, 2019. We descriptively analysed the demographics, service preferences, and baseline symptoms of patients. Among patients enrolled in a digital treatment course, we evaluated scales of depression (Patient Health Questionnaire-9 [PHQ-9]) and anxiety (Generalized Anxiety Disorder 7-Item Scale [GAD-7]), as primary measures of treatment outcome, from the screening assessment to post-treatment and a 3 month follow-up. The Kessler Psychological Distress 10-Item Plus Scale was also used to assess changes in general distress and disability, and course satisfaction was measured post-treatment. Outcomes: A total of 121 652 screening assessments were started, of which 96 018 (78·9%) were completed. The mean age of patients was 35·7 years (SD 13·8) and 88 702 (72·9%) were women. Based on available assessment data, 36 866 (34·5%) of 106 811 participants had never previously spoken to a health professional about their symptoms, and most people self-reported symptoms of anxiety (88 879 [81·9%] of 108 494) or depression (78 803 [72·6%] of 108 494), either alone or in combination, at baseline. 21 745 patients started treatment in a therapist-guided online course, of whom 14 503 (66·7%) completed treatment (≥four of five lessons). Key trends in service use included an increase in the proportion of people using MindSpot primarily for assessment and information, from 52·6% in 2013 to 66·7% in 2019, while the proportion primarily seeking online treatment decreased, from 42·6% in 2013 to 26·7% in 2019. Effect sizes and percentage changes were large for estimated mean scores on the PHQ-9 and GAD-7 from assessment to post-treatment (PHQ-9, Cohen's d effect size 1·40 [95% CI 1·37-1·43]; and GAD-7, 1·45 [1·42-1·47]) and the 3 month follow-up (PHQ-9, 1·36 [1·34-1·38]; and GAD-7, 1·42 [1·40-1·44]); proportions of patients with reliable symptom deterioration (score increase of ≥6 points [PHQ-9] or ≥5 points [GAD-7]) were low post-treatment (of 13 058 respondents, 184 [1·4%] had symptom deterioration on the PHQ-9 and 282 [2·2%] on the GAD-7); and patient satisfaction rates were high (12 452 [96·6%] of 12 895 respondents would recommend the course and 12 433 [96·7%] of 12 860 reported the course worthwhile). We also observed small improvements in disability following treatment as measured by days out of role. Interpretation: Our findings indicate improvement in psychological symptoms and positive reception among patients receiving online mental health treatment. These results support the addition of digital services such as MindSpot as a component in contemporary national mental health systems. Funding: None.

Introduction

Mental disorders and substance use disorders are major contributors to the burden of disease in Australia and worldwide,2, 3 with only a minority of those affected seeking or receiving evidence-based treatments.4, 5 Barriers to care include stigma, cost, and availability of services. The COVID-19 pandemic has created additional challenges, as many traditional mental health providers stopped providing face-to-face service. As a result, interest is increasing in the digital delivery of psychological services. Digital mental health services (DMHS) remotely deliver mental health information, assessments, and treatment, via the internet, telephone, or other digital channels. DMHS are already part of routine care in several countries, operating either as stand-alone services or in conjunction with traditional face-to-face care.8, 9 For example, the Improving Access to Psychological Therapies (IAPT) service of the National Health Service (England) provides both face-to-face and digital services to patients with anxiety or depression; a stepped-care approach that allows patients to move from low-intensity intervention (such as guided self-help) to high-intensity intervention (traditional face-to-face therapy). Stepped care is not a common feature of stand-alone DMHS, in which patients often report being unwilling or unable to access traditional face-to-face therapy.

Evidence before this study

Digital mental health services (DMHS) remotely deliver mental health information, assessments, and treatment services, via the internet, telephone, or other digital channels. We searched PubMed on Jun 12, 2020 (repeated Sept 30, 2020), with the terms (English [Language]) AND ((Internet [Title/Abstract]) AND (mental health services [Title/Abstract])), without date restrictions. We identified more than 250 published reports of internet-delivered structured psychological treatments mostly for anxiety, depression, and somatic disorders. With some exceptions, most trials have shown large and clinically significant reductions in target symptoms. DMHS are now delivered as part of routine care in several countries, including Australia, Canada, Denmark, the Netherlands, Norway, and Sweden. Successful implementation is associated with high standards of clinical and organisational governance, and robust systems for training and supervision of therapists. Health-care providers in other countries have now either developed or are exploring strategies for introducing DMHS into health-care frameworks. However, information is scarce on usability, real world effectiveness, patient demographics, and the reasons patients choose to use DMHS.

Added value of this study

This study describes the characteristics and treatment outcomes of a large sample of consecutive users (n=121 652) of the national Australian DMHS, MindSpot Clinic, from data collected over its first 7 years of operations. We provide information about the demographic characteristics, service preferences, symptoms, and treatment outcomes for people using this particular model of digital service. We found that clinic users represented a broad cross-section of the Australian population, and used MindSpot for a variety of reasons, with most seeking a confidential assessment rather than treatment. We also found that people who engaged in treatment achieved significant reductions in symptoms, which were sustained 3 months after treatment completion. Importantly, these findings confirm the role of DMHS in providing evidence-based assessment and treatment to large numbers of people, many of whom are not accessing other services.

Implications of all the available evidence

The present findings contribute to the evidence base for DMHS in reducing barriers to care, and confirm the utility of DMHS as an important component of contemporary mental health systems.

In this paper, we report outcomes from the Australian MindSpot Clinic, which by volume of patients, is one of the world's largest publicly funded DMHS. The MindSpot project was launched in December, 2012, and is funded by the Australian Department of Health as part of the Australian Government's e-Mental Health Strategy. MindSpot provides information about symptoms and local mental health services, brief psychological assessments, and therapist-guided treatments delivered via the internet and telephone to adults with symptoms of anxiety, depression, or chronic pain. We have previously reported results from 12 months and 30 months of operations, characteristics of service users during the COVID-19 pandemic, and treatment outcomes for specific populations, including Aboriginal and Torres Strait Islander (Indigenous) people and people born overseas. In this paper, we aimed to provide a summary of demographic characteristics and treatment outcomes for patients registered with MindSpot over its first 7 years of operation, including service use and symptom severity, and examined trends in these characteristics over time.

Methods

Design and participants

This study was designed as an observational study and is reported according to STROBE guidelines. We evaluated all patients who registered for assessment or treatment with the MindSpot Clinic between Jan 1, 2013, and Dec 31, 2019. Ethical approval for the collection and use of patient data was obtained from the Macquarie University Human Research Ethics Committee (Macquarie University, Sydney, NSW, Australia; approval number 5201200912) and registered on the Australian and New Zealand Clinical Trials Registry, ACTRN12613000407796. MindSpot is funded by the Australian Government as a project and recruitment is ongoing as patients continue to access the service.

As MindSpot is funded by the Australian Department of Health, patients seeking assessment or treatment must complete an online registration questionnaire and meet the following eligibility criteria: Australian resident eligible for publicly funded health services (ie, Medicare-funded services); aged 18 years or older; and self-reported principal complaint of anxiety, depression, or chronic pain. Patients are also provided with the terms of use explaining that non-identifiable, aggregated data could be used for reporting and service evaluation purposes. Patients are required to consent to the terms of use, either online or by telephone, before proceeding with assessment and treatment.

Clinical model

The MindSpot clinical model was developed in collaboration with patients, referrers, researchers, and the requirements of the Australian Department of Health. Clinic services are provided at no cost to patients. The MindSpot website provides information about symptoms and management of anxiety, depression, and chronic pain, as well as information about MindSpot. It includes online versions of several symptom screening questionnaires that can be used anonymously by all visitors to the website (ie, without registering for a MindSpot service), and which generate advice about self-management. Information on the website is available to the general public in Australia and overseas, and is viewed by more than 500 000 unique visitors each year. People can also telephone and email the MindSpot Clinic for information about symptoms and advice about services. Users report learning about MindSpot via social media, website searches, family and friends, and referrals from other health services. MindSpot is promoted via online advertising.

People register with MindSpot by creating an account and completing a screening assessment, online or by telephone. The screening assessment includes questions on demographic and service use information, and symptoms and current stressors. Participants are also asked about suicidal thoughts and plans. Those who disclose suicidal plans or intent and who can subsequently be contacted by telephone are administered a structured risk assessment aligned with the New South Wales Government best practice guidelines, and safety plans are developed for all users to assist them to stay safe while seeking treatment or in the event of an increase in symptoms during treatment. Those unable to be contacted are referred to local police for a welfare check. People who continue to express suicidal intent are referred to local mental health services or emergency services, depending on the urgency of the situation. However, patients with suicidal thoughts can also continue to access MindSpot services if they agree to a safety plan. MindSpot operates under comprehensive internal and external oversight and reporting that includes clinical, organisational, and information technology governance frameworks. The clinical governance frameworks align with Australian national standards for mental health services and include policies, systems, and protocols for identifying patients or others at risk, their management, clinical escalation in the event of increased risk, and training and supervision of staff.

People who do not complete an assessment are sent information about managing symptoms, contact details for crisis services, and are invited to contact MindSpot. People who complete the assessment are invited to discuss their results with a therapist by telephone (appendix p 1), who provides tailored advice over the appointment of approximately 25 min. An assessment report that identifies clinically significant symptoms and includes information about how to access mental health services (including those offered by MindSpot) or other services, is sent by the therapist to the patient and, if requested by the patient, to a nominated health professional, usually a general practitioner. Information is also provided about evidence-based techniques for self-managing symptoms. Participants who complete an assessment and elect for a MindSpot digital treatment course are then enrolled, unless they are considered ineligible for digital treatment by the therapist because their clinical presentation suggests the need for comprehensive or urgent face-to-face assessment. Those cases are supported to access specialist services.

MindSpot delivers seven digital treatment courses, which were developed and validated in a series of randomised controlled trials at the Macquarie University online research clinic, the eCentreClinic. Four of the treatment courses are based on transdiagnostic principles recognising that people often experience symptoms of anxiety and depressive disorders simultaneously, and that similar psychological skills are used to treat these symptoms. The four transdiagnostic courses offered by the MindSpot Clinic are Mood Mechanic (for individuals aged 18–25 years), the Wellbeing Course (26–65 years), Wellbeing Plus (>65 years), and the Indigenous Wellbeing Course (for Aboriginal and Torres Strait Islander people).14, 19, 20, 21, 22 These four interventions comprise evidence-based psychological treatment components, including psychoeducation about mediators and moderators of symptoms, cognitive therapy, behavioural activation, graded exposure, sleep training, communication and interpersonal skills, problem solving, and relapse prevention.19, 20 MindSpot also offers disorder-specific courses for obsessive compulsive disorder, post-traumatic stress disorder, and chronic pain. Patients can choose a treatment course based on symptoms and demographic characteristics, and via telephone consultation with a MindSpot therapist.

All courses consist of five lessons delivered over 8 weeks. Each lesson comprises a series of slides that presents the principles of psychological treatment for the target symptoms via text and images, based on an instructional design that accommodates both didactic and case-based learning. Course completion is defined as completion of four or more lessons. Courses are delivered online with regular support initiated from the therapist once a week, either via telephone, secure email, or both. The therapist is also available at any time throughout the course. The approximate amount of therapist time per patient per course ranges from around 1·5 h to 3 h. Therapist time includes all contact with patients, preparation time for each patient including reading and responding to messages, and administration and supervision time during treatment and during follow-up. Course materials are available online, although around 10% of people elect to receive materials via a printed workbook, sent by postal mail. In addition to the therapist-delivered treatment courses, a 6 month trial of telephone-based counselling was conducted in 2018, and a self-guided version of the Wellbeing Course was introduced in 2019, the results of which will be reported elsewhere.

Treatment outcome measures

Standardised and validated symptom questionnaires are administered to patients at the screening assessment and throughout treatment. For the purposes of this study, treatment outcomes on the Patient Health Questionnaire-9 (PHQ-9), Generalized Anxiety Disorder 7-Item Scale (GAD-7), and Kessler Psychological Distress 10-Item Plus Scale (K-10+) were analysed as treatment outcomes.

The PHQ-9 consists of nine items measuring symptoms of major depressive disorder according to criteria of the Diagnostic and Statistical Manual of Mental Disorders, 5th edition. Scores range from 0 to 27, with a score of 10 or more indicating a diagnosis of depression. The GAD-7 consists of seven items and is sensitive to the presence of generalised anxiety disorder, social phobia, and panic disorder. Scores range from 0 to 21, with a score of 8 or more indicating the probable presence of an anxiety disorder. The K-10+ was used as a secondary outcome measure to assess general psychological distress and disability. The first ten items comprise the Kessler Psychological Distress 10-Item Scale (K-10), with scores ranging from 10 to 50 and scores of 21 or more associated with the presence of anxiety and depressive disorders. The K-10+ contains four additional questions used to assess the functional effect of the psychological distress. In the current analysis, we used two of the additional questions to assess the number of full and part days a person had been out of role (unable to do usual duties and activities) in the past month. We also report the quantifiable K-10 score. Patients are administered the PHQ-9 and GAD-7 at the screening assessment, once a week during treatment (days 1, 8, 15, 22, 29, 36, and 43), post-treatment (day 50), and at a 3 month follow-up (day 162). Patients complete the K-10+ at the screening assessment, the start of treatment (day 1), midtreatment (day 22), post-treatment (day 50), and the 3 month follow-up (day 162). Patients also complete a satisfaction questionnaire post-treatment. The satisfaction questions we report on are: “Would you recommend this course to others?” and “Was it worth your time doing this course?” All questionnaires are delivered online and patients have 3 weeks to complete the post-treatment and follow-up questionnaires before they are considered closed.

Statistical analysis

We did descriptive analyses of demographics, service preferences, and baseline symptoms for the total sample and for each year. For categorical variables, χ2 analyses of linear-by-linear associations were used to examine trends with time. ANOVA was used to examine the significance of changes to continuous variables with time. χ2 values represent changes in categorical variables over time, and F-values from ANOVA represent significant differences in dependent variables, with years as the independent variable. Generalised estimating equation (GEE) models with Wald's χ2 as the test for significance were used to examine changes in symptom measures from assessment to post-treatment and the 3 month follow-up. Consistent with the principles of intention-to-treat analyses, we imputed missing data for all patients starting treatment, using separate GEE models that assumed data were missing at random, and adjusted for baseline symptoms and lesson completion. An unstructured working correlation matrix and maximum likelihood estimation were used, and gamma distribution with a log link response scale was specified to address positive skewness in dependent variable distributions.

We calculated the clinical significance of change in PHQ-9, GAD-7, and K-10 measures using percentage change in symptoms from baseline and within-group Cohen's d effect sizes, based on the estimated marginal means derived from GEE modelling at the screening assessment, post-treatment, and the 3 month follow-up. Reliable recovery was calculated as the proportion of patients whose scores were higher than the clinical cutoffs of primary measures (PHQ-9 ≥10 or GAD-7 ≥8) at assessment and lower than the cutoffs post-treatment, with evidence of reliable change. Reliable change was defined as a change of at least 6 points on the PHQ-9 and at least 5 points on the GAD-7.9, 31 Reliable deterioration in patients who completed treatment was defined as a score increase of at least 6 points on the PHQ-9 and at least 5 points on the GAD-7 post-treatment. Data were analysed with SPSS (version 26.0). A significance level of 0·05 was used for all tests, with the Bonferroni correction applied for multiple comparisons.

Role of the funding source

There was no funding source for this study.

Results

During the first 7 years of clinic operation from Jan 1, 2013, to Dec 31, 2019, a total of 121 652 online screening assessments were started, of which 96 018 (78·9%) were completed (figure ). The number of people starting an assessment at MindSpot increased consistently from 2013 to 2016, and subsequently plateaued at around 20 000 per annum as directed by funding contracts. A breakdown of completed assessments by year is available in the appendix (p 2).

Figure

Patient flow through the MindSpot Clinic

Course completion is defined as four or more lessons. Post-treatment completion is defined as completion of lessons and post-treatment symptom and satisfaction questionnaires. *6 month trial of telephone-based counselling (2018), and a self-guided version of the Wellbeing Course (introduced in 2019), the results of which will be reported elsewhere.

Demographic characteristics of the total sample and by year, representing all those who started the screening assessment, are shown in table 1 . The mean age of patients in the total sample was 35·7 years (SD 13·8) and 88 702 (72·9%) were women. During the 7 years of clinic operation, small but significant changes were observed in the age, sex, Indigenous status, employment, education, and marital status of people initiating an assessment. For age, we observed a slight increase in the proportion of people aged 18–35 years with time, and less change in the proportion aged 55 years and older (appendix p 3). The proportion of women fluctuated between 71·6% and 74·3%, and the proportion of people married decreased from 42·9% in 2013 to 37·0% in 2019. The proportion of patients born in Australia remained around 78·0%, while the proportion identifying as Aboriginal or Torres Strait Islander increased from 2·1% in 2013 to 4·4% in 2019. We observed some change in employment status over time, particularly in the proportion of students (11·1% in 2013 to 16·0% in 2019), and a concurrent increase in the proportion of people with a university degree (38·3% to 40·8%). The proportion of patients living outside capital cities remained relatively stable (38·7% for the total sample of 2013–18). Proportions of patients from each state and territory are shown in the appendix (p 4). Almost a third of respondents were from New South Wales.

Table 1

Demographic characteristics

			Total	2013	2014	2015	2016	2017	2018	2019	Analysis of change over time
Assessments started			121 652	8929	14 729	16 917	20 672	20 461	19 660	20 284	..
Assessments completed			96 018 (78·9%)	7172 (81·4%)	11 600 (78·8%)	13 973 (82·6%)	15 801 (76·4%)	15 644 (76·5%)	15 685 (79·8%)	16 143 (79·6%)	χ2=7·3, p=0·0073
Assessments started
	Age
		Mean age, years (SD; range)	35·7 (13·8; 18–96)	36·4 (13·0; 18–86)	37·3 (13·8; 18–94)	35·2 (13·0; 18–95)	36·4 (14·5; 18–92)	34·6 (13·9; 18–96)	35·0 (13·6; 18–95)	35·4 (13·8; 18–93)	F=83·5, p<0·0001
		18–34 years	66 802 (54·9%)	4587 (51·4%)	7115 (48·3%)	9476 (56·0%)	10 980 (53·1%)	11 951 (58·4%)	11 386 (57·9%)	11 307 (55·7%)	..
		35–54 years	40 605 (33·4%)	3361 (37·6%)	5703 (38·7%)	5801 (34·3%)	6731 (32·6%)	6243 (30·5%)	6137 (31·2%)	6629 (32·7%)	..
		≥55 years	14 245 (11·7%)	981 (11·0%)	1911 (13·0%)	1640 (9·7%)	2961 (14·3%)	2267 (11·1%)	2137 (10·9%)	2348 (11·6%)	χ2=157·6, p<0·0001
	Sex
		Male	32 214 (26·5%)	2501 (28·0%)	4127 (28·0%)	4481 (26·5%)	5661 (27·4%)	5280 (25·8%)	4914 (25·0%)	5250 (25·9%)	..
		Female	88 702 (72·9%)	6401 (71·7%)	10 541 (71·6%)	12 345 (73·0%)	14 892 (72·0%)	15 030 (73·5%)	14 609 (74·3%)	14 884 (73·4%)	..
		Other	736 (0·6%)	27 (0·3%)	61 (0·4%)	91 (0·5%)	119 (0·6%)	151 (0·7%)	137 (0·7%)	150 (0·7%)	χ2=59·6, p<0·0001
	Locality*†
		Capital city or surrounding region	49 548/80 849 (61·3%)	..	..	5001/7865 (63·6%)	10 456/17 443 (59·9%)	11 192/18 032 (62·1%)	11 428/18 523 (61·7%)	11 471/18 986 (60·4%)	..
		Urban region	16 038/80 849 (19·8%)	..	..	1382/7865 (17·6%)	3349/17 443 (19·2%)	3605/18 032 (20·0%)	3816/18 523 (20·6%)	3886/18 986 (20·5%)	..
		Rural or remote region	15 263/80 849 (18·9%)	..	..	1482/7865 (18·8%)	3638/17 443 (20·9%)	3235/18 032 (17·9%)	3279/18 523 (17·7%)	3629/18 986 (19·1%)	χ2=0·1, p=0·77
	Country of birth*‡
		Born in Australia	79 974/102 576 (78·0%)	2360/3005 (78·5%)	9509/12 572 (75·6%)	11 755/14 841 (79·2%)	13 666/17 377 (78·6%)	13 976/17 908 (78·0%)	14 161/18 269 (77·5%)	14 547/18 604 (78·2%)	..
		Born overseas	22 602/102 576 (22·0%)	645/3005 (21·5%)	3063/12 572 (24·4%)	3086/14 841 (20·8%)	3711/17 377 (21·4%)	3932/17 908 (22·0%)	4108/18 269 (22·5%)	4057/18 604 (21·8%)	χ2=2·0, p=0·16
	Indigenous status*‡§
		Aboriginal or Torres Strait Islander	29 08/79 390 (3·7%)	50/2345 (2·1%)	201/9389 (2·1%)	352/11 619 (3·0%)	545/13 634 (4·0%)	580/13 923 (4·2%)	545/14 048 (3·9%)	635/14 432 (4·4%)	..
		None	76 482/79 390 (96·3%)	2295/2345 (97·9%)	9188/9389 (97·9%)	11 267/11 619 (97·0%)	13 089/13 634 (96·0%)	13 343/13 923 (95·8%)	13 503/14 048 (96·1%)	13 797/14 432 (95·6%)	χ2= 98·7, p<0·0001
	Employment*
		Paid employment	61 701/108 864 (56·7%)	4414/7960 (55·5%)	7337/12 859 (57·1%)	8590/15 059 (57·0%)	9523/17 346 (54·9%)	9952/18 063 (55·1%)	10 535/18 562 (56·8%)	11 350/19 015 (59·7%)	..
		Student	16 729/108 864 (15·4%)	883/7960 (11·1%)	1525/12 859 (11·9%)	2265/15 059 (15·0%)	2372/17 346 (13·7%)	3177/18 063 (17·6%)	3472/18 562 (18·7%)	3035/19 015 (16·0%)	..
		Unemployed	12 863/108 864 (11·8%)	918/7960 (11·5%)	1597/12 859 (12·4%)	1869/15 059 (12·4%)	2324/17 346 (13·4%)	2163/18 063 (12·0%)	2024/18 562 (10·9%)	1968/19 015 (10·3%)	..
		Other (retired, home duties, disability)	17 571/108 864 (16·1%)	1745/7960 (21·9%)	2400/12 859 (18·7%)	2335/15 059 (15·5%)	3127/17 346 (18·0%)	2771/18 063 (15·3%)	2531/18 562 (13·6%)	2662/19 015 (14·0%)	χ2=162·7, p<0·0001
	Education*
		University degree	41 820/108 316 (38·6%)	3026/7900 (38·3%)	4869/12 606 (38·6%)	6047/14 873 (40·7%)	6290/17 328 (36·3%)	6614/18 062 (36·6%)	7209/18 528 (38·9%)	7765/19 019 (40·8%)	..
		Other	66 496/108 316 (61·4%)	4874/7900 (61·7%)	7737/12 606 (61·4%)	8826/14 873 (59·3%)	11 038/17 328 (63·7%)	11 448/18 062 (63·4%)	11 319/18 528 (61·1%)	11 254/19 019 (59·2%)	χ2=6·8, p=0·0092
	Relationship status*
		Married (de-facto or registered)	41 317/108 649 (38·0%)	3417/7959 (42·9%)	5438/12 761 (42·6%)	5960/15 025 (39·7%)	6656/17 346 (38·4%)	6391/18 064 (35·4%)	6430/18 518 (34·7%)	7025/18 976 (37·0%)	..
		Other	67  332/108 649 (62·0%)	4542/7959 (57·1%)	7323/12 761 (57·4%)	9065/15 025 (60·3%)	10 690/17 346 (61·6%)	11 673/18 064 (64·6%)	12 088/18 518 (65·3%)	11 951/18 976 (63·0%)	χ2=264·3, p<0·0001

Data were missing from incomplete assessments; available data are shown in each cell.

Question introduced July, 2015.

Question introduced Sept, 2013.

Only patients who answered yes to being born in Australia were asked about Indigenous status, and not all patients provided an answer on Indigenous status.

Reported psychological symptoms and stressors at the time of the screening assessment by year are shown in table 2 . In 7 years of clinic operation, significant fluctuations were observed in symptoms. Mean scores at assessment (baseline) on the PHQ-9 decreased from 15·6 (SD 6·1) in 2013 to 14·5 (6·2) in 2019, with a concurrent decrease in the proportion of people self-reporting current difficulties with depression during that period. Mean baseline scores on the GAD-7 remained close to the mean for the whole period (12·5 [5·2]; with the exception of 12·9 in the first year), although the proportion reporting anxiety or worry increased over the 7 years. Mean baseline K-10 scores decreased slightly, from 32·2 (7·5) in 2013 to 31·3 (7·6) in 2019. Based on a series of questions specifically about suicidal thoughts, intentions, and plans, the proportion of people reporting thoughts relating to suicide fluctuated between 29·9% and 34·7%, while the proportion reporting both suicidal thoughts and current intent or a plan increased from 2·4% in 2013 to 3·5% in 2019. Significant changes over time were also observed in reported psychosocial stressors. The proportion of people reporting relationship difficulties increased, while the proportions of people reporting vocational, physical health, or financial difficulties decreased (table 2).

Table 2

Psychological symptoms at assessment

	Total	2013	2014	2015	2016	2017	2018	2019	Analysis of change over time
Baseline symptom scores
Patient Health Questionnaire-9	14·9 (6·2)	15·6 (6·1)	15·4 (6·2)	15·1 (6·2)	14·9 (6·3)	15·0 (6·1)	14·9 (6·2)	14·5 (6·2)	F=33·4, p<0·0001
Generalized Anxiety Disorder 7-Item Scale	12·5 (5·2)	12·9 (5·0)	12·5 (5·1)	12·6 (5·2)	12·5 (5·2)	12·4 (5·2)	12·4 (5·2)	12·4 (5·2)	F=5·2, p<0·0001
Kessler Psychological Distress 10-Item Scale	31·8 (7·5)	32·2 (7·5)	32·0 (7·5)	31·9 (7·4)	31·7 (7·5)	31·9 (7·4)	31·8 (7·5)	31·3 (7·6)	F=19·3, p<0·0001
Functional effect of symptoms (Kessler Psychological Distress 10-Item Plus Scale)*
Whole days out of role in past month	5·6 (7·4)	..	..	..	..	5·7 (7·5)	5·6 (7·4)	5·4 (7·4)	F=3·9, p=0·019
Part days out of role in past month	8·9 (8·3)	..	..	..	..	9·0 (8·2)	9·0 (8·3)	8·9 (8·3)	F=0·6, p=0·536
Self-reported symptoms
Depression or low mood	78 803/108 494 (72·6%)	6120/8036 (76·2%)	9869/12 916 (76·4%)	10 998/15 161 (72·5%)	12 513/17 443 (71·7%)	12 929/17 823 (72·5%)	13 096/18 273 (71·7%)	13 278/18 842 (70·5%)	χ2=153·2, p<0·0001
Depression >12 months†	50 088/69 288 (72·3%)	1470/1880 (78·2%)	7067/8952 (78·9%)	6856/9074 (75·6%)	8946/12 044 (74·3%)	9333/12 950 (72·1%)	7791/11 118 (70·1%)	8625/13 270 (65·0%)	χ2=682·5, p<0·0001
Anxiety or worry	88 879/108 494 (81·9%)	6333/8036 (78·8%)	9958/12 916 (77·1%)	12 151/15 161 (80·1%)	14 340/17 443 (82·2%)	14 469/17 823 (81·2%)	15 337/18 273 (83·9%)	16 291/18 842 (86·5%)	χ2=524·6, p<0·0001
Anxiety >12 months†	58 182/78 679 (73·9%)	1416/1837 (77·1%)	6917/9054 (76·4%)	7671/10 273 (74·7%)	10 322/13 876 (74·4)	10 898/14 494 (75·2%)	9524/12 897 (73·8%)	11 434/16 248 (70·4%)	χ2=114·0, p<0·0001
Suicidal thoughts or plans‡
Suicidal thoughts	29 504/91 919 (32·1%)	1369/4487 (30·5%)	2542/8220 (30·9%)	4042/13 499 (29·9%)	4923/15 767 (31·2%)	5510/16 040 (34·4%)	5794/16 706 (34·7%)	5324/17 200 (31·0%)	χ2=28·2, p<0·0001
Current plan	3549/91 919 (3·9%)	108/4487 (2·4%)	231/8220 (2·8%)	440/13 499 (3·3%)	641/15 767 (4·1%)	752/16 040 (4·7%)	776/16 706 (4·6%)	601/17 200 (3·5%)	χ2=40·1, p<0·0001
Psychosocial stressors
Relationships	62 503/106 247 (58·8%)	4081/8043 (50·7%)	6523/12 983 (50·2%)	8122/15 240 (53·3%)	10 447/16 023 (65·2%)	11 237/17 564 (64·0%)	11 104/18 027 (61·6%)	10 989/18 367 (59·8%)	χ2=612·9, p<0·0001
Vocational	61 549/106 247 (57·9%)	5321/8043 (66·2%)	8517/12 983 (65·6%)	9591/15 240 (62·9%)	9163/16 023 (57·2%)	9760/17 564 (55·6%)	9559/18 027 (53·0%)	9683/18 367 (52·7%)	χ2=1073·4, p<0·0001
Physical health	47 537/106 247 (44·7%)	3914/8043 (48·7%)	6519/12 983 (50·2%)	7129/15 240 (46·8%)	7423/16 023 (46·3%)	7430/17 564 (42·3%)	7438/18 027 (41·3%)	7684/18 367 (41·8%)	χ2=381·5, p<0·0001
Finances	40 074/106 247 (37·7%)	3846/8043 (47·8%)	6961/12 983 (53·6%)	6451/15 240 (42·3%)	5826/16 023 (36·4%)	5971/17 564 (34·0%)	5504/18 027 (30·5%)	5515/18 367 (30·0%)	χ2=1738·6, p<0·0001

Data are mean (SD) or n/N (%), where numerators are the number of positive responses and denominators are the number of patients who provided an answer to that question.

Questions introduced July, 2017.

Questions regarding the duration of symptoms (anxiety or depression >12 months) were introduced Sept, 2013; for both, the denominator is the number of people who reported current relevant symptoms and provided a response to the duration question.

Missing data for the last quarter of 2014 due to system changes.

Service use and preferences by year are reported in table 3 . Significant changes were observed in the main reported purpose of using MindSpot among patients during the first 7 years of clinic operation. From 2013 to 2019, the proportion of people using MindSpot primarily for assessment and information increased from 52·6% in 2013 to 66·7% in 2019, while the proportion primarily seeking online treatment decreased, from 42·6% in 2013 to 26·7% in 2019. Table 3 also reports the reasons participants gave for using an online service rather than a face-to-face service. Since introduction of the question in 2015, around a third of respondents consistently reported convenience and (absence of) cost, and another third reported privacy and anonymity as their main reason. Over 7 years, 27·7% to 37·6% patients reported that they had never previously seen a mental health professional. 45·9% to 48·7% patients reported speaking to a general practitioner about their mental health (accounting for no data in 2013–14 and 2018). The proportion of patients reporting current treatment with psychotropic medication decreased from 32·0% in 2013 to 27·7% in 2019.

Table 3

Mental health service preferences and use at assessment

	Total	2013	2014	2015	2016	2017	2018	2019	Analysis of change over time
Main purpose of using MindSpot*
Assessment and information	65 042/97 127 (67·0%)	3547/6748 (52·6%)	7177/11 456 (62·6%)	8726/13 533 (64·5%)	10 878/15 814 (68·8%)	8109/11 133 (72·8%)	13 661/19 049 (71·7%)	12 944/19 394 (66·7%)	..
Treatment	25 165/97 127 (25·9%)	2877/6748 (42·6%)	3768/11 456 (32·9%)	3966/13 533 (29·3%)	3904/15 814 (24·7%)	1986/11 133) (17·8%)	3488/19 049 (18·3%)	5176/19 394 (26·7%)	..
Other	6920/97 127 (7·1%)	324/6748 (4·8%)	511/11 456 (4·5%)	841/13 533 (6·2%)	1032/15 814 (6·5%)	1038/11 133 (9·3%)	1900/19 049 (10·0%)	1274/19 394 (6·6%)	χ2=139·9, p<0·0001
Main reason for choosing MindSpot over traditional services†
Privacy and anonymity	28 533/86 128 (33·1%)	..	..	4512/14 341 (31·5%)	5329/17 094 (31·2%)	6272/17 693 (35·4%)	6467/18 255 (35·4%)	5953/18 745 (31·8%)	..
Convenience and cost	29 038/86 128 (33·7%)	..	..	5158/14 341 (36·0%)	5559/17 094 (32·5%)	5600/17 693 (31·7%)	6066/18 255 (33·2%)	6655/18 745 (35·5%)	..
Other	28 557/86 128 (33·2%)	..	..	4671/14 341 (32·6%)	6206/17 094 (36·3%)	5821/17 693 (32·9%)	5722/18 255 (31·3%)	6137/18 745 (32·7%)	χ2=0·6, p=0·44
Treatment history
Current psychotropic medication	29 515/108 031 (27·3%)	2468/7722 (32·0%)	3801/12 826 (29·6%)	4125/15 102 (27·3%)	4907/17 443 (28·1%)	4458/17 823 (25·0%)	4544/18 273 (24·9%)	5212/18 842 (27·7%)	χ2=103·7, p<0·0001
Never seen a health professional	36 866/106 811 (34·5%)	2972/8038 (37·0%)	4460/12 916 (34·5%)	3728/13 476 (27·7%)	6105/17 443 (35·0%)	6695/17 823 (37·6%)	6678/18 273 (36·5%)	6228/18 842 (33·1%)	χ2=7·2, p=0·0069
Speaks to general practitioner about mental health‡	27 129/57 176 (47·4%)	..	..	4486/9454 (47·5%)	7513/15 756 (47·7%)	7245/15 769 (45·9%)	..	7885/16 197 (48·7%)	χ2=4·4, p=0·035

Data are n/N (%), where numerators are the number of positive responses and denominators are the number of patients who provided an answer to that question.

Missing data from July to Dec, 2017, due to system changes.

Data available from July, 2015.

Question introduced April, 2015; missing data for 2018 due to system changes.

23 952 patients enrolled in treatment in a therapist-guided online course. Due to the age range of people enrolling in treatment, most (17 533 [73·2%]) elected to access the Wellbeing Course (table 1, figure). 21 745 (90·8%) enrolled patients started treatment, of whom 14 503 (66·7%) completed treatment (ie, ≥4 lessons; figure). Patient satisfaction was high. Among respondents to the post-treatment satisfaction questionnaire, 12 452 (96·6%) of 12 895 reported that they would recommend the course to others and 12 433 (96·7%) of 12 860 reported that participating in the course was worth their time (appendix p 5).

Treatment outcomes are shown in table 4 . Effect sizes and percentage changes were large for estimated mean scores on the PHQ-9 and GAD-7 from assessment to post-treatment (PHQ-9, Cohen's d effect size 1·40 [95% CI 1·37–1·43]; and GAD-7, 1·45 [1·42–1·47]) and the 3 month follow-up (PHQ-9, 1·36 [1·34–1·38]; and GAD-7, 1·42 [1·40–1·44]). Similar results were observed for K-10 scores. These results indicated marked improvement in clinical symptoms. We also identified a small but clinically significant improvement in the number of whole and part days out of role. In primary measures, the proportions of patients with reliable recovery were high (5419 [59·5%] of 9105 patients with clinically significant symptoms at assessment on the PHQ-9 and 5862 [59·8%] of 9810 on the GAD-7) and the proportions with reliable deterioration were low (184 [1·4%] of 13 058 patients who completed the post-treatment questionnaire on PHQ-9 and 282 [2·2%] of 13 058 on the GAD-7) post-treatment.

Table 4

Treatment outcomes

		Estimated mean*			Cohen's d effect size from assessment		Percentage change in estimated mean*from assessment		Reliable treatment outcome†
		Assessment	Post-treatment	3 month follow-up	To post-treatment	To 3 month follow-up	To post-treatment	To 3 month follow-up	Deterioration	Recovery
Depression (Patient Health Questionnaire-9; n=21 745)		13·6 (5·9)	6·5 (4·2)	6·5 (4·4)	1·40 (1·37–1·43)	1·36 (1·34–1·38)	52·2% (51·6–52·8)	52·2% (51·5–52·9)	184/13 058 (1·4%)	5419/9105 (59·5%)
Anxiety (Generalized Anxiety Disorder 7-Item Scale; n=21 745)		12·0 (5·0)	5·7 (3·6)	5·7 (3·8)	1·45 (1·42–1·47)	1·42 (1·40–1·44)	52·5% (52·1–52·9)	52·5% (52·1–52·9)	282/13 058 (2·2%)	5862/9810 (59·8%)
Distress (Kessler Psychological Distress 10-Item Scale; n=21 745)		30·1 (6·9)	20·8 (6·2)	20·5 (6·4)	1·42 (1·40–1·44)	1·44 (1·42–1·46)	46·3% (45·9–46·7)‡	47·8% (47·3–48·2)‡	..	..
Functional effect (Kessler Psychological Distress 10-Item Plus Scale; n=5120)
	Whole days out of role	5·4 (7·5)	3·5 (6·2)	3·3 (6·4)	0·28 (0·25–0·31)	0·30 (0·28–0·32)	35·2% (33·7–36·7)	38·9% (37·2–40·6)	..	..
	Part days out of role	9·3 (8·4)	6·0 (7·2)	5·8 (7·4)	0·42 (0·39–0·45)	0·44 (0·42–0·46)	35·5% (34·4–36·6)	37·6% (36·6–38·7)	..	..

Data are mean (SD), % change (95% CI), effect size (95% CI), or n/N (%); for reliable recovery, numerators are number of patients meeting thresholds for reliable recovery and denominators are number of patients with clinically significant symptoms at assessment and complete post-treatment questionnaires, and for reliable deterioration, numerators are number of patients meeting threshold for reliable deterioration and denominators are number of patients who completed the post-treatment questionnaires.

Estimated marginal means derived from generalised estimating equation modelling.

Reliable deterioration and reliable recovery calculated from actual scores post-treatment.

After transformation to 0–40 scale for calculation of percentage change from scores of 10–50.

GEE analyses showed significant overall symptom reductions in PHQ-9 (Wald's χ2=29 432·8, p<0·0001), GAD-7 (Wald's χ2=27 731·1, p<0·0001), and K-10 (Wald's χ2=33 261·8, p<0·0001). Pairwise comparisons showed that scores on all measures decreased significantly from assessment to post-treatment and from assessment to follow-up (all p<0·0001). Analyses of the clinical significance of treatment outcomes by year revealed consistent results, with symptom reductions post-treatment for all years on all measures (appendix pp 5–7).

Discussion

This study described the demographic characteristics, service preferences, and symptoms of more than 120 000 users of a national DMHS, collected during 7 years of clinic operations. Users of the service represented a broad cross-section of the Australian population, many of whom were seeking a confidential assessment rather than treatment. Those who did engage in treatment achieved significant reductions in symptoms that were sustained for up to 3 months. The results confirm the efficacy and efficiency of MindSpot in providing evidence-based assessment and treatment to large numbers of people, many of whom are not accessing other services. Our findings contribute to the evidence in support of DMHS within contemporary mental health systems.

Consistent with reports before 2018,11, 12 MindSpot has continued to serve a broad and geographically dispersed cross-section of the Australian population. Some changes in demographics and symptoms have occurred with time, including an increase in the proportion of young adult users, an increase in the proportion identifying as Aboriginal or Torres Strait Islander, and an increase in people reporting anxiety.

A key observation was the increase in proportions of people reporting a primary purpose for contacting MindSpot was to receive an assessment rather than treatment. Many patients reported to therapists that a confidential assessment was the only intervention required at the time of consultation. This finding suggests that a discussion with a therapist about the nature of symptoms and treatment options is valued by many people, and can serve as a brief therapeutic intervention in itself. The data also raise important questions about engagement and attrition in digital and traditional mental health services, and whether all patients accessing a service can be assumed to be treatment-seeking. These results confirm our view that DMHS should align with patient-centred models of care, and offer a range of services, including education, assessment, triage, support to access urgent help for people in crisis, and referral, as well as providing evidence-based treatment.

With regard to treatment outcomes, the overall magnitude of clinical improvements obtained across the MindSpot treatment courses remain consistently high, with greater than 50% symptom reductions in anxiety and depression post-treatment, which were sustained for up to 3 months. Outcomes compare favourably with benchmarks relating to substantial clinical improvement, low rates of deterioration, and high patient satisfaction with DMHS in other countries, including when offered in primary care, and via other initiatives for large-scale implementation of psychological treatment in Australia and the UK.

This study has several limitations. We report on characteristics and outcomes of patients registering for assessment or treatment with MindSpot, which restricted our sample to a small proportion of visitors to the MindSpot website (>500 000 per year), and limits the generalisability of our results. We also acknowledge the issue of missing responses, which is a limitation of many studies, particularly those reporting outcomes obtained in routine care, in which patients are receiving a service rather than participating under controlled trial conditions. The absence of a control group also means that we are unable to account for natural remission or the effect of missing data. However, this limitation was mitigated by the weekly collection of symptom scores during treatment and by conservative statistical modelling, and we found no indication of systematic bias in trends over time due to missing data. A post-hoc analysis did find some evidence that young patients and those with severe symptoms are not necessarily continuing to or completing treatment (appendix p 8), which might affect the generalisability of our results.

Generally, we found that several key demographic factors, such as the proportion of people born overseas, distribution by states and territories, Indigenous status (Aboriginal and Torres Strait Islander), and proportion living in rural or remote regions, closely matched national statistics. However, we acknowledge that other factors might be under-represented or over-represented in our sample. For example, the proportion of men contacting MindSpot was always less than 30%, an under-representation consistent with reports that men are less likely to seek help for anxiety and depression from traditional mental health services, despite having higher rates of suicide than women.35, 36 The question of how to engage men in both traditional mental health services and DMHS remains important and might require new service models.

Despite these limitations, our results show that a high-volume digital mental health service can be successfully implemented as part of routine care. The main strengths of this study are the analyses of comprehensive data on a large consecutive sample, combined with the regular measurement of symptoms to monitor treatment effects. Furthermore, treatment results over 7 years match those reported in earlier papers, confirming the robust nature of the digitised clinic procedures and clinical effects.

As of 2020, MindSpot has been operating for more than 7 years. In that time, the delivery of health care, including some forms of mental health care, via digital technology has become increasingly acceptable. Services such as MindSpot have shown that digital delivery of care increases accessibility and convenience for patients and can reduce other barriers to care such as stigma. Other key learnings from MindSpot are that DMHS could have an important role in contemporary mental health care, not only by providing treatment, but also by providing information and assessment services to diverse groups of people that often under-utilise traditional health services, including Indigenous Australians, and people living in rural and remote regions.8, 14 We maintain that DMHS are not a panacea and should not replace existing services, but instead can complement those services by reducing barriers and delivering evidence-based care to large numbers of patients in an efficient and cost-effective way.8, 37 People who do not respond to DMHS can then be supported to seek more intensive treatment, consistent with a stepped-care approach.

An important feature of DMHS is the potential for systematic measurement of progress via treatment and outcomes, which is rarely implemented with existing service models. By providing services to large numbers of people and routinely collecting and reporting data about user characteristics and clinical outcomes, DMHS are not only providing valuable benchmarking data, but are having a growing influence on the planning of mental health systems across an increasing number of countries. The routine collection and reporting of user data, with the exception of the UK's IAPT model, is not typical of publicly funded psychological services. Thus, such reporting by DMHS is not only increasing understanding among policy makers on the relative strengths and limitations of different service models, but is likely to lead to increased expectations from funders and policy makers for similar reporting from traditional services. In the long term, this influence might lead to policy and funding decisions based more on evidence than traditional practice, but in the short term this will require change in the culture and operations of services that do not routinely collect or report these kind of data.

Developing, delivering, and evaluating DMHS is challenging, requiring complex procedures and ongoing evaluation in the context of ever-changing technology and a rapidly evolving governance and regulatory environment. Despite the challenges, we no longer need to question whether DMHS will become part of the framework of mental health services. The new question is, how will this integration occur and how do we best integrate DMHS with existing face-to-face services? Based on the preferences of many patients for more easily accessible, confidential mental health care, we believe that a need will be ongoing for stand-alone services, which provide the option of assessment and treatment and are not always linked with an existing provider. Ideally, existing mental health services should receive support to deliver both face-to-face and digital mental health care, and we strongly recommend engagement during the development and implementation of these services with patients and other stakeholders, including policy makers and funders, to ensure that services are not only effective but also acceptable. Mental health professionals could then be trained and equipped to use digital tools with their own clients, to improve both quality of care and collection of treatment outcome data. Without such training and support, patients are unlikely to receive consistently high-quality care, and funders are unlikely to receive data on clinical outcomes to guide service or programme improvements.

The MindSpot project has become one of the leading providers globally of DMHS as part of routine care and has delivered mental health services with proven effectiveness to a large number of Australians in its first 7 years. The consistency of results provides support for the adoption of this model of care within the national mental health system, particularly in the present context of increased consumer acceptance of digital and telephone health-care services. Nonetheless, we maintain that the role of DMHS is to provide consumers and referrers with an additional choice of service model.

Data sharing

Access to de-identified data might be provided on reasonable request. Requests are subject to the establishment of appropriate data governance, and the approval of an independent and recognised human research ethics committee. Requests must be made in writing to Dr Lauren Staples, MindSpot Clinic, Macquarie University, Sydney, NSW 2109, Australia (lauren.staples@mq.edu.au).