A novel multi-component online intervention to improve the mental health of university students: Randomised controlled trial of the Uni Virtual Clinic.

BACKGROUND: Of the millions of students enrolled in university, up to 50% will experience a mental disorder. Many of these students do not seek help, and for those who do, university-based services are often over-burdened. Anonymous, evidence-based, online interventions can improve students' access to mental health support. The Uni Virtual Clinic (UVC) is a transdiagnostic online mental health program designed specifically for university students. This paper reports on a randomised controlled trial examining the effectiveness of the UVC in a sample of Australian university students.
METHODS: University students with elevated psychological distress (K10>15; n = 200) were randomised to the UVC intervention or a waitlist control condition for a period of 6 weeks. Baseline, post-intervention, and 3-month follow-up surveys assessed depression, anxiety, self-efficacy, quality of life, adherence, and satisfaction with the UVC intervention.
RESULTS: Mixed models analysis demonstrated that use of the UVC was associated with small significant reductions in social anxiety and small improvements in academic self-efficacy. The program was not effective in reducing symptoms of depression, anxiety, or psychological distress compared to a control group. The majority of participants in the intervention condition who were retained at follow-up engaged with the program, and most of these participants reported satisfaction with the UVC. DISCUSSION: The results suggest that multi-component online interventions such as the UVC have utility in a university environment. Future trials of the UVC should examine the impact of guidance and/or tailoring on treatment efficacy, and the potential role of the UVC in a stepped care model incorporating on-campus services.

Introduction

Undertaking higher education is associated with increased risk of developing a mental disorder among young people. The majority of students commence university study during late adolescence and early adulthood; a time when the first onset of mental disorders peaks (de Girolamo et al, 2012; Kessler et al., 2007). In addition, transition to university may provoke unique stressors that place tertiary students at additional risk of developing a mental disorder, over and above that which they already face as young adults. These include moving away from familial support networks, pressure to succeed, instability in social relationships, balancing study with other demands, and financial stress (Cleary et al., 2011; Deasy et al., 2014; Eisenberg et al., 2011; Farrer et al., 2016). In particular, factors related to social functioning (social skills, relationships, social interaction, loneliness, and anxiety) and academic performance (grades, pressure to succeed, time management) are consistently reported by students and clinicians as among the top concerns students face while at university (Farrer et al., 2016; Perez-Rojas et al., 2017; Poyrazli, 2015; Ryan et al., 2010).

Accordingly, approximately 30–50% of university students internationally meet criteria for a mental disorder (Auerbach et al., 2018; Blanco et al., 2008; Eisenberg et al., 2011; Said et al., 2013; Stallman, 2010). Moreover, university students may be at greater risk than their community-based counterparts, with research demonstrating that the prevalence of severe psychological distress is significantly higher in tertiary students (19–48%) compared to their age-matched peers (3–11%) (Leahy et al., 2010; Stallman, 2010). Data from university counselling services further corroborate this evidence, with counselling services reporting yearly increases of up to 15% in demand for treatment, leading to increased waiting lists and long intervals between appointments (Broglia et al., 2017).

Untreated mental disorders in young people have severe consequences, including disability (Andrews and Titov, 2007), suicide (Mortier et al., 2018), and reduced quality of life, even for those experiencing sub-clinical symptoms (Evans et al., 2007). Poor mental health in university students also leads to poorer interpersonal relationships, reduced academic performance, lower engagement in campus activities, and greater risk of dropping out of university (Bruffaerts et al., 2018; Salzer, 2011).

Despite the severity of the negative outcomes associated with untreated mental disorders in students, very few receive appropriate mental health care (Eisenberg et al., 2011). Less than half of students with a mental health problem seek professional help for their symptoms, with help seeking rates as low as 5% for drug and alcohol use disorders (Blanco et al., 2008; Wynaden et al., 2013). The most frequently reported barriers to help seeking by university students include stigma, concerns about confidentiality, lack of time, and high treatment costs (Givens and Tjia, 2002; Mowbray et al., 2006; Yorgason et al., 2008).

Given the barriers to help seeking reported by students, online self-help interventions may be highly suited to the university student population (Levin et al., 2018). Online interventions are easily accessible, can be utilised in private, are cost-effective, and typically require less time than face-to-face appointments (Andersson, 2018). Young people frequently report using the internet for help with their mental health (Burns et al., 2010). One study reported that 72% of young people aged 18–25 years believed that websites were helpful for managing mental health problems (Oh et al., 2009), and almost half of the young people in another study stated that they had used the internet to access information about mental disorders (Burns et al., 2010).

Additionally, several systematic reviews and meta-analyses have indicated that online interventions targeting mental health problems are effective for university students (Conley et al., 2016; Davies et al., 2014; Farrer et al., 2013; Harrer et al., 2019). A range of online programs and apps exist for supporting student mental health, targeting either specific disorders or clinical issues (e.g., test anxiety, stress, procrastination) (Benton et al., 2016; Harrer et al., 2018), or providing a particular therapeutic approach (e.g., mindfulness, acceptance and commitment therapy) (Galante et al., 2018; Levin et al., 2016).

However, co-morbidity between mental disorders can impede help seeking and treatment efficacy (Mojtabai et al., 2014; Krawczyk et al., 2017), suggesting that transdiagnostic or tailored interventions that are capable of targeting multiple disorders and related issues at once may have more clinical utility and may be more appealing to treatment seekers. Transdiagnostic interventions provide therapeutic content that is capable of targeting the common mechanisms that underlie multiple disorders, whereas interventions focused on tailoring can allow users to select from a range of different content depending on their individual symptom profile or treatment preferences (Andersson and Titov, 2014). One transdiagnostic online program evaluated in a pilot study in the UK (Personality and Living of University Students) demonstrated reductions in depression and anxiety among students at high risk of developing a mental disorder (Musiat et al., 2014). This program offered modules designed to be completed sequentially, in any order. However, internet users, particularly young people and students, may have a preference for open, simultaneous access to all components of a program in order to rapidly access the content most relevant to them (Stallman and Kavanagh, 2018). An Australian online program for university students, thedesk, was designed with this principle in mind, and has been found to be effective in reducing distress in university students (Stallman et al., 2018). Rather than focusing on clinical symptomatology, thedesk focuses primarily on student wellbeing and normalising students’ experiences of stress at university. In recognition of the potential need for a clinically-focused, fully open-access, and transdiagnostic online intervention for university student mental health, The Uni Virtual Clinic was developed using a comprehensive participatory design process involving students and other university stakeholders (described further below in the Methods section).

This paper reports on a 2-arm randomised controlled trial of the Uni Virtual Clinic (UVC), a multi-component, transdiagnostic intervention that incorporates multiple tools and therapeutic approaches to help university students manage mental health problems and related issues. The program has a clinical focus, targeting major groups of mental disorders that commonly affect university students, and also addresses a range of developmental and contextual issues that impact on student mental health (e.g., relationship issues, financial stress, perfectionism and procrastination, coping with study). The program was designed specifically to provide open access to a suite of evidence-based resources tailored to the specific clinical needs and preferences of university students. In this trial we compared an active treatment condition involving unguided use of the UVC for 6 weeks, with a wait-list control condition. The purpose of this trial was to examine feasibility and the effectiveness of the intervention on depression symptoms, anxiety symptoms, and a range of other symptom outcomes, as well as satisfaction with and usage of the intervention.

Material and methods

Participants

Trial participants were undergraduate and postgraduate students from mid-sized university in a major capital city in Australia. The university had approximately 22,000 students enrolled at the time of the trial in 2017, and has a large proportion of postgraduate and international students. Recruitment took place between July and September 2017. Multiple strategies were used to recruit students into the trial. Students were provided with information about the trial in person at university events, via posts to university-affiliated groups on social media sites, and through media channels such as radio and a student-run newspaper. Recruitment materials targeted students feeling ‘stressed, down, or overwhelmed’. Deans of residential halls and academic colleges were approached to send email invitations to students. Student services and organisations, such as the university student's association, postgraduate and research student's association, the university counselling service, the university psychology clinic, and the international students department were contacted via email and/or Facebook to request assistance in distributing information about the trial to students. Persons involved in distributing recruitment materials were not asked to target any specific groups of students.

Procedure

Students interested in participating in the trial were provided with a link to a Qualtrics survey containing study information and a question to obtain informed consent. To be eligible for the trial, students were required to be currently enrolled at the university, aged 18 years or older, and score above 15 on the Kessler Psychological Distress Scale (K10), which indicates clinically significant distress (Kessler et al., 2002). Eligible participants were required to provide an e-mail address and mobile phone number, and were e-mailed a link to the baseline survey. Participants who completed the baseline survey were randomised to either the UVC intervention condition or a wait-list control condition for a period of 6 weeks. A researcher who was independent of the trial generated a sequence of random integers between the values of 1 and 2 at https://www.random.org/, and manually allocated participants to the trial conditions according to this sequence. Randomisation was not conducted in blocks or stratified. The trial researchers were blinded to condition allocations.

During the 6 week intervention period, participants allocated to the intervention condition received a weekly e-mail and a text message via mobile phone encouraging them to engage with the UVC. The content of the weekly text message was brief and generic (e.g., “Hi, have you had a look at the UVC this week? Cheers! The UVC team”), whereas the weekly e-mail suggested specific resources that the participant could access, depending on the time of semester when the e-mail was sent (e.g., in the week before a major exam period, reminder e-mails for that week suggested that participants visit exam anxiety resources in the UVC).

All participants received an e-mail containing a link to the post-intervention survey 6 weeks after they completed the baseline survey, and a link to the follow-up survey was sent 3 months after the post-intervention survey. Participants received two e-mail reminders, one mobile text message reminder, and one phone call reminder to complete the post-intervention and follow-up surveys. Participants who contacted the research team in distress were followed-up by the primary author (LF), a registered psychologist, who provided support and referral to relevant health, counselling, and crisis support services. The trial was registered at the Australian New Zealand Clinical Trials Registry (ACTRN12617000915358) and ethical approval was obtained from the ANU Human Research Ethics Committee (protocol 2017/217).

Intervention condition: the Uni Virtual Clinic

The Uni Virtual Clinic (UVC) is a comprehensive online mental health program that was developed at the Centre for Mental Health Research, ANU. The UVC was developed by the primary author and her team using a multi-phase participatory design process (Gulliver et al., 2016). Students, university teaching and administrative staff, and student service providers were involved in several phases of research and development involving qualitative and quantitative data collection to identify problem prevalence, user needs and preferences for treatment, and to establish a service model for the program (Chan et al., 2016; Farrer et al., 2016,2015a2015b; Gulliver et al., 2015). A prototype of the UVC was developed and subjected to multiple rounds of iterative user testing and feedback, and a leadership group comprised of 10-15 students provided input into the graphic design and other elements of the intervention.

The UVC targets four major groups of mental disorders that commonly affect university students: mood disorders (i.e., major depression, bipolar disorder); anxiety and trauma-related disorders (i.e., generalised anxiety disorder, social anxiety disorder, obsessive compulsive disorder, post-traumatic stress disorder, specific phobia, panic disorder, agoraphobia); substance use disorders (i.e., alcohol, smoking, and other drugs); and eating disorders (i.e., anorexia, bulimia, binge eating disorder). The UVC also contains resources targeting unique issues experienced by university students that impact on their mental health including: insomnia; suicide and self-harm; financial issues; loneliness/social isolation; relationship issues; homesickness; adjustment to university (with a specific focus on international students); grief and loss; career and life after university; perfectionism; stress; physical health (nutrition, exercise), time management and procrastination, disability, living arrangements; sexual and gender identity; and exam anxiety. The UVC delivers online information via tailored factsheets, brief screening tools that provide feedback about symptom severity and normative data, and psychotherapeutic modules (e.g., cognitive behaviour therapy, mindfulness) targeting the clinical and related issues listed above (see Fig. 1). The screening tools assess symptoms of mental disorders (e.g. depression, OCD, panic disorder) and related issues affecting university students (e.g. perfectionism, exam anxiety). These measures were chosen based on their brevity, psychometric properties, and previous use in university student/young adult populations. Following completion of these screening tools, participants are provided feedback about the severity of their symptoms, any change in symptom severity since they last completed the tool, and normative data about the severity of their symptoms in relation to other young people their age (where available). Users can freely access all content within the intervention; and if preferred, the program also contains tools designed to guide students to the most appropriate content based on their needs and level of mental health literacy (see Fig. 2).

Fig. 1

Screenshots of psychoeducation and self-help hubs.

Fig. 2

Homepage of the UVC.

Participants randomised to the UVC condition were provided with access to the program for 6 weeks. They had unrestricted and unguided access to the program, and were instructed to use the program however they wished during the intervention period.

Control condition: wait-list

Participants allocated to the control condition did not receive any program content or contact with researchers during the 6 week intervention period. Participants were not restricted in their use of usual services or support during this period. After the final reminder email for the 3 month follow-up survey was sent, control participants were provided access to the UVC for a period of 6 weeks. Control participants could access the UVC regardless of whether they completed the survey or not.

Measures

At baseline, the following demographic information was collected: age, gender, ethnicity, living situation, financial stress, relationship status, employment status, study discipline and year of degree, study load, international or domestic student status, academic performance, and engagement with student life. The following were assessed at baseline, post-intervention, and 3 month follow-up: depression symptoms, generalised anxiety symptoms, social anxiety symptoms, psychological distress, quality of life, self-efficacy, academic self-efficacy, and help seeking behaviour. At post-intervention, intervention use and satisfaction were assessed.

Depressions symptoms were assessed using the PHQ-9 (range 0–27, 0–4: no symptoms, 5–9: mild symptoms, 10–14: moderate symptoms, 15–27: severe symptoms) (Kroenke et al., 2001), and generalised anxiety symptoms were assessed using the GAD-7 (range 0–21, 0–4: no symptoms, 5–9: mild symptoms, 10–14: moderate symptoms, 15–21: severe symptoms) (Spitzer et al., 2006). Both scales have robust psychometric properties in general population samples, and are sensitive to detecting change (Kroenke et al., 2010; Lowe et al., 2008). Symptoms of social anxiety were assessed using the SOPHS (Batterham et al., 2017), a brief five-item screening scale with symptom severity scores ranging from 0 to 20 (0–4: no symptoms, 5–7: mild symptoms, 8–15: moderate symptoms, 16–20: severe symptoms). The SOPHS has been validated in population and clinical samples (Batterham et al., 2017). Psychological distress was assessed using the K10 (range 10–50, 10–15: no symptoms, 16–21: moderate symptoms, 22–29: high symptoms, 30–50: very high symptoms) (Kessler et al., 2002). The K10 has good internal consistency (Cronbach's alpha coefficients 0.92–0.93), and correlates well with other symptom and diagnostic measures of psychological illness (Andrews and Slade, 2001; Kessler et al., 2002, 2003). Quality of life was assessed using the EURO-HIS 8, an eight-item measure designed to assess the psychological, physical, social, and environmental aspects of quality of life (range 0–32) (Schmidt et al., 2006). The EURO-HIS 8 has satisfactory discriminant validity and acceptable internal consistency (da Rocha et al., 2012). Self-efficacy was assessed using the General Self Efficacy Scale (GSE-10), which contains 10 items designed to measure perceived coping with daily hassles and adaptation to the experience of stressful life events (range 0–30) (Schwarzer and Jerusalem, 1995). The GSE-10 has been validated in 31 countries and languages, has acceptable internal consistency, and good discriminant and concurrent validity (Luszczynska et al., 2005; Scholz et al., 2002). Academic self-efficacy (i.e., confidence in one's ability to successfully complete university-related tasks, such as performing well in exams or participating in lecture/tutorial discussions) was assessed using the Study and Social subscales of the College Self-Efficacy Inventory (CSEI; 15 items, 5-point Likert scale ranging from totally unconfident to totally confident, range 0–60) (Solberg et al., 1993). The CSEI study subscale measures self-efficacy related to completing tasks associated with university study (e.g. “How confident do you feel to keep up to date with your university work?”). The CSEI social subscale measures self-efficacy related to social interactions at university (e.g. “How confident do you feel to talk to your lecturers/tutors?). The CSEI has been found to measure three distinct constructs of academic self-efficacy, supporting the independent use of CSEI subscales (Gore et al., 2006). The study and social subscales of the CSEI have reasonable internal consistency (α = 0.79–0.86), the social and study subscales has been found to be negatively correlated with social anxiety and academic worry/concern, and positively correlated with grade point average (Gore et al., 2006; Barry and Finney, 2007). Help seeking from formal and informal sources was measured using the Actual Help Seeking Questionnaire (Wilson et al., 2005). Participants were asked whether they had sought help from a range of formal (i.e., GP, psychologist, counsellor, helpline) and informal (i.e., friends, family, partner) help sources in the past month. The AHSQ has been found to adequately differentiate help-seeking behaviour for different problems and help sources (Wilson et al., 2005).

Adherence to the intervention was assessed by self-reported number of logins, self-reported amount of time spent using the intervention, devices used to access the intervention, and reasons for accessing the program. Satisfaction with the intervention was assessed by asking participants how satisfied they were with the intervention, what they liked and did not like about the intervention (open-ended), whether they would recommend the intervention to other students, whether they could find what they were looking for when using the intervention, and how useful they found the intervention.

Statistical analyses

Data were analysed using SPSS release 25 for Windows (IBM Corp, 2017). The primary and secondary outcome variables were analysed on an intention to treat (ITT) basis using mixed models repeated measures ANOVA, with measurement occasion as a within groups factor and trial condition as a between groups factor (Verbeke and Molenberghs, 2000). Within-person variation was modelled using an unstructured covariance matrix. Effect sizes (Cohen's d) were calculated by dividing the mean difference between scores by the pooled standard deviation (Cohen, 1988). Logistic and linear regressions were used to examine differences between conditions on all baseline variables, and logistic regression was used to examine the impact of trial condition on help seeking behaviour at post-intervention and 3 month follow-up.

Power

The initial target sample size was 100 participants (50 per condition), which enabled detection of an effect of d = 0.6 between conditions with 80% power and α = 0.05 (assuming an attrition rate of 10%). However, demand for participation in the trial was high, and a sample size of 200 was achieved. This enabled the detection of a more modest effect (e.g. d = 0.4) and the ability to account for potentially higher rates of attrition from the trial (e.g. 30% instead of 10%).

Results

Sample characteristics and response rates

Table 1 presents the demographic and clinical characteristics of the sample at baseline. The sample was predominantly female, and participants were on average 22 years of age (SD = 4.1). Most participants were domestic students living either on-campus or in shared accommodation off-campus. The majority of students were in some form of paid employment, and over 70% of the sample indicated that they experienced a level of financial stress. Students were predominantly studying full-time at the undergraduate level, and were studying degrees from a range of different disciplines. On average, the sample reported high rates of psychological distress, moderate to severe depression symptoms, moderate symptoms of generalised anxiety, and mild to moderate symptoms of social anxiety at baseline. There were no significant differences between participants randomised to the intervention or control conditions on any baseline demographic or symptom variables.

Table 1

Demographic and clinical characteristics of the sample at baseline.

Categorical variables	Intervention (n = 102)	Control (n = 98)	Total (n = 200)	χ2	p
	n (%)	n (%)	n (%)
Gender				0.14	0.94
Female	78 (76.5)	77 (78.6)	155 (77.5)
Male	18 (17.5)	16 (16.3)	34 (17.0)
Other	6 (5.9)	5 (5.1)	11 (5.5)
Ethnicity				2.40	0.79
Caucasian/European	63 (61.8)	66 (67.3)	129 (64.5)
Asian/Indian	29 (28.4)	27 (27.6)	56 (28.0)
Aboriginal/Torres Strait Islander/Pacific Islander	1 (1.0)	1 (1.0)	2 (1.0)
Latino/South American	2 (2.0)	1 (1.0)	3 (1.5)
African	1 (1.0)	1 (1.0)	2 (1.0)
Other	6 (5.9)	2 (2.0)	8 (4.0)
Current living situation				4.32	0.51
On-campus housing	43 (42.2)	42 (42.9)	85 (42.5)
With parents	16 (15.7)	15 (15.3)	31 (15.5)
With partner/children	12 (11.8)	6 (6.1)	18 (9.0)
With others off campus	28 (27.5)	31 (31.6)	59 (29.5)
Alone	3 (2.9)	2 (2.0)	5 (2.5)
No fixed housing	0 (0.0)	2 (2.0)	2 (1.0)
Level of financial stress				5.19	0.16
No financial stress	26 (25.5)	30 (30.6)	56 (28.0)
Occasional financial stress	51 (50.0)	34 (34.7)	85 (42.5)
Frequent financial stress	17 (16.7)	25 (25.5)	42 (21.0)
Constant financial stress	8 (7.8)	9 (9.2)	17 (8.5)
Hrs per week in paid employment				6.76	0.24
None	40 (39.2)	34 (34.7)	74 (37.0)
1–10 h	26 (25.5)	30 (30.6)	56 (28.0)
10.1–20 h	23 (22.5)	23 (23.5)	46 (23.0)
20.1–30 h	7 (6.9)	8 (8.2)	15 (7.5)
30.1–40 h	1 (1.0)	3 (3.1)	4 (2.0)
> 40 h	5 (4.9)	0 (0.0)	5 (2.5)
Relationship status				3.42	0.18
Single	63 (61.8)	60 (61.2)	123 (61.5)
In relationship – not living together	22 (21.6)	29 (29.6)	51 (25.5)
In relationship – living together	17 (16.7)	9 (9.2)	26 (13.0)
Discipline of degree studied (n = 180)				8.55	0.20
Health/Medicine	13 (14.9)	7 (7.5)	20 (11.1)
Science	24 (27.6)	18 (19.4)	42 (23.3)
Engineering	3 (3.4)	4 (4.3)	7 (3.9)
Business/Economics	5 (5.7)	12 (12.9)	17 (9.4)
International relations	12 (13.8)	14 (15.1)	26 (14.4)
Humanities	13 (14.9)	23 (24.7)	36 (20.0)
Law	17 (19.5)	15 (16.1)	32 (17.8)
Year of degree				2.61	0.46
First-year undergraduate	19 (18.6)	26 (26.5)	45 (22.5)
Later-year undergraduate	56 (54.9)	51 (52.0)	107 (53.5)
Honours	4 (3.9)	5 (5.1)	9 (4.5)
Postgraduate	23 (22.5)	16 (16.3)	39 (19.5)
Study load				2.31	0.13
Part time	14 (13.7)	7 (7.1)	21 (10.5)
Full time	88 (86.3)	91 (92.9)	179 (89.5)
Student status				0.10	0.75
International	21 (20.6)	22 (22.4)	43 (21.5)
Domestic	81 (79.4)	76 (77.6)	157 (78.5)
Average mark/grade achieved last semester (n = 196)				4.49	0.34
High Distinction	19 (19.2)	26 (26.8)	45 (23.0)
Distinction	35 (35.4)	31 (32.0)	66 (33.7)
Credit	34 (34.3)	28 (28.9)	62 (31.6)
Pass	9 (9.1)	6 (6.2)	15 (7.7)
Fail	2 (2.0)	6 (6.2)	8 (4.1)
Engagement with university life				1.66	0.80
Not at all (only attend classes)	30 (29.4)	26 (26.5)	56 (28.0)
Somewhat	26 (25.5)	32 (32.7)	58 (29.0)
Moderate (participate in some university-based activities)	28 (27.5)	22 (22.4)	50 (25.0)
High	9 (8.8)	8 (8.2)	17 (8.5)
Extremely high (involved in student leadership activities within university)	9 (8.8)	10 (10.2)	19 (9.5)
Continuous variables	M (SD)	M (SD)	M (SD)	t	p
Age (years)	22.2 (4.13)	21.9 (5.52)	22.1 (4.86)	0.33	0.74
Psychological distress (K10)	29.5 (7.84)	28.6 (8.03)	29.0 (7.93)	0.86	0.39
Depression (PHQ-9)	14.6 (5.47)	14.6 (5.53)	14.6 (5.49)	0.28	0.98
Generalised anxiety (GAD-7)	11.2 (5.00)	10.8 (5.05)	11.0 (5.02)	0.63	0.53
Social anxiety (SOPHS)	7.6 (4.75)	7.2 (5.07)	7.4 (4.90)	0.54	0.59
Quality of life (EUROHIS-8)	16.2 (5.24)	15.8 (5.30)	16.0 (5.26)	0.62	0.54
Self-efficacy (GSE-10)	16.3 (5.28)	16.3 (5.00)	16.3 (5.13)	–0.08	0.94
Academic self-efficacy	29.1 (9.87)	31.1 (11.05)	30.1 (10.49)	–1.36	0.18

The flow of participants through the trial is shown in Fig. 3. A total of 840 students clicked on the study invitation and were screened for eligibility. Of these, 259 (30.8%) consented and were eligible for the trial and 200 completed the baseline questionnaire and were randomised. The majority (n = 144, 72%) of participants completed the post-intervention questionnaire, and 47.5% (n = 95) completed the 3 month follow-up questionnaire. Missingness (failure to complete a post or follow-up survey) varied between trial conditions. Participants in the intervention condition were three times more likely to be missing at post intervention (OR = 3.31, p <0.001) and 2 times more likely to be missing at 3 month follow-up (OR = 2.55, p <0.001) than participants the control condition.

Fig. 3

CONSORT diagram depicting the flow of participants through the trial.

Primary outcomes (depression and anxiety symptoms)

Table 2 shows the estimated marginal means for the intervention and control groups at each measurement occasion in the study, and the results of the mixed models analysis. Participants in both conditions showed reductions in depression symptoms (F = 43.02, p <0.001) and anxiety symptoms (F = 25.49, p <0.001) over time, however, the occasion by condition interactions for these outcomes were non-significant, indicating no differences between the conditions on these symptom measures over time.

Table 2

Estimated marginal means for intervention and control groups at each measurement occasion, and results of mixed models analysis.

	Intervention (UVC) condition			Control condition			Occasion x Condition
	Baseline	Post	3 months	Baseline	Post	3 months
	M (SE)	M (SE)	M (SE)	M (SE)	M (SE)	M (SE)	F	p
Depression	14.6 (0.54)	11.9 (0.77)	9.2 (0.92)	14.5 (0.56)	12.9 (0.70)	10.1 (0.78)	0.81	0.45
Anxiety	11.2 (0.50)	9.5 (0.63)	7.5 (0.73)	10.8 (0.51)	10.2 (0.57)	7.8 (0.61)	1.17	0.32
Distress	29.5 (0.79)	26.6 (1.06)	22.2 (1.33)	28.6 (0.80)	27.8 (0.97)	24.0 (1.07)	2.42	0.09
Social anxiety	7.6 (0.49)	6.6 (0.56)	4.7 (0.60)	7.2 (0.50)	7.0 (0.52)	6.1 (0.50)	4.06	0.02
QOL	16.3 (0.52)	16.7 (0.63)	18.4 (0.80)	15.7 (0.53)	16.4 (0.59)	17.4 (0.67)	0.30	0.74
Self-efficacy	16.3 (0.51)	16.9 (0.66)	17.0 (0.74)	16.3 (0.52)	16.5 (0.60)	17.3 (0.63)	0.60	0.55
Academic self-efficacy	29.2 (1.04)	32.6 (1.36)	35.9 (1.39)	31.1 (1.06)	31.0 (1.26)	33.5 (1.20)	5.13	0.007

Between-group effect sizes at post-intervention were small for both depression (d = -0.16) and anxiety (d = -0.16). Similarly, small effect sizes were observed between conditions at 3 month follow-up for depression (d = -0.12) and anxiety (d = 0.15).

Secondary outcomes (social anxiety, distress, quality of life, self-efficacy, and academic self-efficacy)

Significant occasion by condition interactions were found for social anxiety symptoms and academic self-efficacy. Participants in the intervention condition showed significant reductions in social anxiety symptoms and significant improvement in academic self-efficacy over time compared to participants in the control condition. No significant occasion by condition interactions were found for distress, quality of life, or general self-efficacy. However, participants in both conditions showed significant reductions in distress (F = 29.60, p <0.001) and significant improvements in quality of life (F = 9.74, p <0.001) over time.

Between group effect-sizes were small at post-intervention for social anxiety (d = -0.03) and academic self-efficacy (d = 0.10), and small to moderate at 3 month follow-up (social anxiety: d = -0.17, academic self-efficacy: d = 0.60). Similarly, effect sizes for distress, quality of life, and self-efficacy were small between groups at post-intervention (d = -0.05, d = 0.09, d = -0.01, respectively) and at 3 month follow-up (d = 0.03, d = 0.19, d = -0.21, respectively).

Help seeking

Trial condition did not significantly predict likelihood of help seeking from any of the formal or informal sources of help at post-intervention or 3 month follow-up, except for seeking help from a friend. Participants allocated to the control condition were more likely to report seeking help from a friend at post-intervention (OR = 2.46, p < .05) and 3 month follow-up (OR = 3.34, p < .001) than participants in the intervention condition.

Intervention adherence and usage

Of those in the UVC condition who returned a post-intervention survey, 75.8% (n = 47) accessed the UVC during the intervention period at least once. Most of these participants logged in to the program around once per week (n = 30, 63.8%), and 42.6% (n = 20) spent between 5 and 15 minutes using the program per visit. The most common device used to access the program was a laptop computer (n = 29, 61.7%). Participants were asked what usually prompted them to use the program, and over half of the participants (n = 25, 53.2%) reported that they accessed the program when they were reminded to (e.g. via text or e-mail), followed by 38.3% (n = 18) who indicated that they accessed the program when they felt down or anxious.

Satisfaction

Most participants who returned a post-intervention survey reported being satisfied with the intervention; 61.7% (n = 29) of participants were either ‘very satisfied’ or ‘somewhat satisfied’ following use of the UVC. 80.9% (n = 38) reported that they would be willing to recommend the UVC to other students, 66% (n = 31) reported that they found the program ‘useful’, and 59.6% (n = 28) reported that they learned either ‘a lot’ or ‘a fair bit’ from the program. Most participants (n = 39, 83.0%) reported that they found the program ‘easy’ or ‘very easy’ to understand, and 63.8% (n = 30) reported that they found it ‘easy’ or ‘very easy’ to find what they were looking for on their first visit to the program. In terms of specific features of the program, 32.6% of participants (n = 15) reported that they liked the interactive symptoms quizzes the most, followed by the problem solver tool on the homepage (n = 7, 15.2%), and the self-help modules (n = 6, 13.0%). Open-ended questions were used to ask participants what they liked and disliked about the program. Regarding what participants liked about the program, comments focused on the look and feel of the program (“It's a really nice looking site, very inviting and welcoming”), program content and usability (“[the program]…covers a wide variety of areas”, “Mindfulness videos were great”, “I really liked how easy it was to access and that there was a large variety of tools”), and the utility of the program for prompting professional help seeking and providing an alternative to face-to-face services (“[the UVC]… did help when making a counselling appointment and waiting for a month wouldn't have worked for me”, “the clinic really helped me identify what I was going through…and then I went and got help”). In terms of what participants liked least about the program, some participants reported that the program was either “not detailed enough” or was “too complex” and contained “too much information”. Other comments focused on the difficulty of completing self-help interventions when experiencing symptoms of a mental health problem (“One major flaw, the idea of actively logging in, taking quizzes and exercises doesn't work on days when you are the most vulnerable. If someone is not even in the state to get out of bed, how will they get up and take an online quiz?”). Finally, some participants reported that they program was difficult to navigate (“sometimes I would find something I liked and then [would] have trouble finding it again”).

Discussion

This trial explored the feasibility and effectiveness of a multi-component intervention targeting mental disorders and related issues in university students. The trial found no evidence for the effectiveness of the Uni Virtual Clinic in reducing symptoms of depression, anxiety or psychological distress, or improving quality of life or general self-efficacy, compared to a no-treatment control group. This finding is inconsistent with other studies conducted with university student populations that have found significant small to moderate effects for depression, anxiety, and stress (Harrer et al., 2019). The findings of the current study could be due to several factors, including inadequate treatment dosage, lack of guidance in delivery of the intervention, and considerable variation in the type of intervention content to which participants were exposed. Less than half of participants in the UVC condition logged in to the program, and of those who did, the majority did so relatively infrequently (e.g., once per week) and spent less than 30 minutes per visit. Although research has demonstrated preference in the community for brief online interventions (Batterham and Calear, 2017), it is possible that participants in this trial spent an insufficient amount of time engaging with the intervention to derive therapeutic benefit. It is also possible that due to the size and scope of the intervention, participants spent more time browsing and exploring the program superficially than engaging meaningfully with therapeutically active content. On a related note, due to the fully ‘open access’ design of the intervention, and the lack of guidance provided to participants about which content they should access, each participant in the UVC group was essentially exposed to a ‘different’ intervention. We assume that engagement with the intervention content was driven by participants’ clinical needs, preferences, and personal interests, making it difficult with a limited sample size to determine the impact of any of the specific elements of the program. In contrast, interventions involving sequential modules (i.e. Personality and Living of University Students; Musiat et al., 2014) have been found to be effective in reducing symptoms of depression and anxiety, as they may be more effective in ensuring that users engage sufficiently with the intervention content. It may be useful in future trials to test the effect of providing tailored guidance on which aspects of the intervention content are most suitable to the individual, based on their symptoms and preferences.

Participants in both the intervention and the control groups improved over time on most outcomes. Given that participants in the control group received no program content or researcher contact during the intervention period, they may have sought out and accessed online therapeutic content independently, motivated by their initial interest in being involved in a research study for an online mental health intervention. Participants in the control group were also more likely than intervention group participants to have sought help from friends at post-intervention, and this may have played a role in alleviating symptoms during the intervention period for this group.

The UVC was efficacious in reducing symptoms of social anxiety disorder among participants, although with a small effect size. Improvement in social anxiety has been demonstrated recently in two studies of online interventions targeting university students (Kahlke et al., 2019; McCall et al., 2018). It may be that the key features of social anxiety disorder that prevent traditional help seeking (e.g. fear of negative evaluation and embarrassment) are effectively mitigated by anonymous online interventions, making them particularly appealing and potentially more engaging and effective for people experiencing social anxiety. The program was also effective in improving academic self-efficacy. This may reflect the salience of this issue among students and their specific motivation for change in this area. Self-efficacy as a construct may also be more easily shifted by brief, low-intensity interventions than clinical symptoms that may be more entrenched. The improvement observed in academic self-efficacy as opposed to general self-efficacy is a possible reflection of the intervention content being highly tailored to the specific experiences of university students. A comprehensive co-design process was used during the development of the UVC to ensure that the content of the program was as relevant to the needs and experiences of university students as possible (Gulliver et al., 2016).

Encouragingly, most participants in the intervention condition who returned a post-intervention survey reported that they were satisfied with their use of the UVC, and rated their ability to effectively navigate and understand the program particularly highly. These elements of user experience are critical for fostering initial and ongoing engagement with online interventions, which are inherently prone to high rates of attrition. Although research engaging end-users in design and development can be time-consuming and resource intensive, it allows developers of interventions to consider and effectively design for the needs of users and their environments (van der Velden and Mortberg, 2014). However, as was reflected in the open-ended feedback, it can be difficult to harmonise conflicting priorities among end-users, resulting in mixed feedback about some elements of the program (e.g. the program being ‘not detailed enough’ and also containing ‘too much information’).

Despite emphasis on the importance of optimising online interventions for mobile devices, most participants accessed the UVC on their laptop computers. Preference for accessing online mental health programs via computers has been demonstrated previously (Batterham and Calear, 2017). In this study, it may reflect the amount of time that students already spend on their computer engaged in study-related tasks. It may also reflect the reluctance of users to engage with such a large and complex intervention via their mobile device. Despite the intervention being designed for ease of use on mobile devices, it was not formally optimised for this modality, and user preferences for access via a laptop may have resulted from an inability to effectively engage with the content through their mobile device.

Limitations

Data regarding adherence (amount of the intervention completed) and the type of intervention content accessed by participants were based on self-report, rather than objective usage data obtained directly from the UVC program. This was due to personnel and resource constraints that prevented the research team from being able to extract and examine this data. Thus, results and conclusions pertaining to adherence and engagement in this trial may be prone to social desirability bias and recall inaccuracies. We were also unable to ascertain levels of adherence and engagement among participants who did not return a post-intervention questionnaire. Missingness at post-intervention and follow-up was significantly higher in the treatment group compared to the control group, which may have led to biased or incorrect estimates of treatment effect, despite the use of statistical methods that are robust to data that are not missing at random. As mentioned previously, lack of consistency across participants in the intervention content they accessed impeded our ability to draw conclusions about specific or active components of the intervention, particularly when combined with an inability to examine objective data about which parts of the intervention were access by which participants. In order to complete the study effectively within the constraints of the academic calendar, a relatively short follow-up period was used, which prevented our ability to examine the utility of the intervention over a longer time period. Finally, the trial was conducted with a sample from one university, and thus, results may not be generalisable to university students more broadly.

Future directions

The internal validity of the trial was compromised by a focus on maximising external validity (e.g., testing the intervention under conditions that closely mirror how it is intended to be used ‘in the real world’). An important priority for future research on the UVC would be to establish the key components of the intervention, which components impact on which outcomes, and how intervention components interact with each other (Murray et al., 2016). For a complex intervention such as the UVC, trial designs that allow evaluation of the importance of particular intervention components would be a logical next step. A national, multi-site trial of the UVC would enable a large enough sample to be obtained to achieve this, and would also enable the intervention to be tested in a more diverse sample with a potentially more robust control group. Alternatively, a factorial experimental approach may be an efficient way to determine the most active components of the intervention (Collins et al., 2014). Given the wide scope and open design of the intervention, the UVC may function more effectively for students as a gateway, portal, or decision making tool to help them navigate to the most appropriate help, which could be tested in future trials involving different primary outcome measures. Finally, stepped care models have been proposed as a possible solution to the problem of high demand for face-to-face counselling services on university campuses (Cornish et al., 2017). Online programs have shown promise as low-intensity components in stepped care, and the utility of the UVC could be tested as part of a stepped care model, particularly in the Australian university context.

Conclusion

This was the first effectiveness trial of the Uni Virtual Clinic, a comprehensive online intervention for mental health problems and related issues in university students. The trial demonstrated some utility in reducing social anxiety and improving academic self-efficacy among students, and was rated well in terms of satisfaction and usability. Mental health problems developed and untreated at university have lifelong impacts in the workplace, on relationships, and on health in later adulthood. Interventions such as the UVC have the potential to re-imagine the way mental health services are delivered in universities, which may reduce the prevalence of mental disorders in this high-risk population.

Declaration of Competing Interest

The Uni Virtual Clinic was developed by LMF, AG, and NK. The authors derive no financial benefit from the program. The authors declare no other competing interests.