FitMindKit: Randomised controlled trial of an automatically tailored online program for mood, anxiety, substance use and suicidality.

PURPOSE: Online mental health programs can be effective in reducing symptoms of depression, anxiety disorders, substance use and suicidal ideation. However, most existing e-mental health programs focus on a single domain of mental health, neglecting comorbidity. Furthermore, few programs are tailored to the symptom patterns of the individual user. FitMindKit was designed to overcome the gaps of existing e-mental health programs, providing tailored, transdiagnostic therapeutic content to address a range of comorbid mental health symptoms. A trial was conducted to test the program's efficacy.
METHODS: Australian adults with elevated symptoms of depression, anxiety, suicidal ideation and/or substance use were recruited through social media, with n = 194 randomised into a fully-automated trial of a 10-day brief intervention. Participants were randomly allocated to receive FitMindKit tailored to their symptoms, an untailored generic version of FitMindKit, or an attention control.
RESULTS: Mixed model repeated measures ANOVA indicated that participants in both FitMindKit and the attention control had significant reductions in symptom composite scores. Effects were not significantly greater in the FitMindKit program relative to control, either at post-test or 3-month follow-up. No effects were detected for specific decreases in depression, generalized anxiety, social anxiety, panic, suicidal ideation or alcohol/substance use. There were no significant differences between the tailored and static versions in effectiveness or adherence. Participants in the tailored and static conditions were more satisfied than in the control condition, with some evidence favouring the tailored condition. High attrition reduced power to find effects.
CONCLUSIONS: FitMindKit provides a model for addressing comorbid mental health symptoms in an online program, using automated tailoring to symptom patterns. Modifications to the program are recommended, along with the need for larger trials to test the effects of tailoring on mental health outcomes.

Introduction

Mental and substance use disorder account for 6.6% of total disease burden and 19.0% of disability burden globally (Institute for Health Metrics and Evaluation, 2017). Unlike most other health conditions, this burden is disproportionately observed among young people. Despite the high prevalence and burden of common mental disorders, only one in three people with a mental health problem seeks help from a health professional (Burgess et al., 2009). There is a pressing need for better prevention, identification and early treatment of mental health problems in the community. Internet interventions have been shown to be effective, efficient and cost-effective in preventing and treating many common mental disorders (Andrews et al., 2010, Griffiths et al., 2010, Batterham et al., 2015b). Online programs may also be effective in preventing the development of mental disorders, with up to a 30% reduction in the risk of developing depression (Spek et al., 2007, Cuijpers et al., 2005, Calear and Christensen, 2010). However, the implementation of internet interventions into traditional treatment services is often challenging and may not improve efficiencies (Batterham et al., 2015b, Kenter et al., 2015).

In addition, there is high comorbidity between mental health and substance use problems that is not fully addressed by existing online programs. Up to 45% of adults with a common mental disorder meet criteria for more than one disorder (Kessler et al., 2005, Slade et al., 2009). Online programs aimed at treating comorbidity have been shown to be effective, even outperforming traditional services (Kay-Lambkin et al., 2009). In addition, programs that target depression and anxiety more broadly using a transdiagnostic approach have been shown to significantly reduce symptoms (Titov et al., 2011). The potential of online programs to address comorbidity has not been fully realised. Although there are transdiagnostic programs available (e.g., Titov et al., 2011) and emerging programs to address comorbid mental health and substance use problems (e.g., Kay-Lambkin et al., 2011, 2009), there are no integrated programs that simultaneously target depression, anxiety disorders, substance use and suicidal ideation. Suicidality has been shown to be resistant to depression interventions (Christensen et al., 2013) although interventions focused on suicidal ideation have been shown to be effective (van Spijker et al., 2014a). The development of integrated, tailored interventions that better address comorbidity is an important next step.

Online programs are not typically targeted to individual characteristics, needs and preferences. Tailored interventions may be a way to address comorbidity. There is preliminary evidence that tailoring is an effective way to facilitate behaviour change (Lustria et al., 2013), that tailoring to individual symptom profiles by clinicians can more effectively treat depression or anxiety (Carlbring et al., 2011, Johansson et al., 2012) and that programs that account for individual preferences tend to be more acceptable (Andersson et al., 2011, Batterham and Calear, 2017). However, there is a pressing need to develop greater understanding of the factors that lead to positive outcomes in internet therapy across a range of mental health problems, to enable the development of empirical guidelines for tailoring (Lustria et al., 2013). Furthermore, there has been little exploration of automated tailoring in the context of self-help online programs. Automated tailoring may use data provided by the participant on their symptom patterns to identify specific targeted therapeutic strategies that may be most relevant to their symptoms, based on theory, research findings or clinical judgement. No existing study has developed a transdiagnostic intervention for the treatment of depression, anxiety, substance use and suicidal ideation. Tailoring is one approach to handling the complexity of such a multi-faceted intervention, to ensure that intervention materials match the needs of the user.

The current study reports on the development and randomised controlled trial (RCT) of a low-intensity internet-based therapy program, called FitMindKit, that uses automated tailoring to address comorbid depression, anxiety disorders, substance use and/or suicidal ideation. The intervention was developed by adapting and abridging therapeutic techniques from existing evidence-based online programs, on the basis that brief, targeted interventions may achieve better engagement (Christensen et al., 2009). Evidence suggests that the adaptation of the face-to-face therapeutic model (multiple 50 min consultations) may not be necessary for effectively reducing symptoms of mental health problems online (Christensen et al., 2006). Rather, a new model of intervention focused on the core components of psychosocial therapies that encourage overlearning of key concepts (Scott, 1992) may be sufficient for effective outcomes and promote greater adherence. Briefer intervention components also facilitate tailoring to individual needs by decreasing the amount of content required to present a comprehensive, multifaceted program. The intervention selected and combined evidence-based therapeutic elements (e.g., behavioural activation, cognitive reframing, problem solving, exposure, motivational interviewing, mindfulness, relaxation) based on individual symptom profiles. The program was aimed at adolescents and young adults, although with relevance to the broader adult population.

The three-arm randomised controlled trial reported in this paper aimed to test: (1) whether the FitMindKit program (tailored or untailored) was effective in reducing overall symptom burden in an online population-based sample with elevated symptoms of one or more common mental disorders, relative to an attention control program, and (2) whether tailoring of the program to individual symptom profiles increased intervention effectiveness, adherence or satisfaction.

Method

FitMindKit intervention – static and tailored

The FitMindKit intervention was developed by using a narrative approach to introduce the core strategies of relevant behavioural therapies. A total of 18 modules were developed: ten core transdiagnostic modules, along with two domain-specific modules each for symptoms of distress/mood disorders (major depressive disorder, MDD, and generalized anxiety disorder, GAD), fear disorders (social anxiety disorder, SAD, and panic disorder, PD), substance disorders (substance use disorder, SUD, and alcohol use disorder, AUD) and suicidal ideation. Specifically, the ten core modules were: psychoeducation, sources of help for mental health problems, introduction to cognitive reframing, problem solving strategies, introduction to mindfulness, managing relationships/conflict resolution, exercise and diet, sleep hygiene, introduction to values and committed action, and the role of stigma. The two mood/distress modules introduced behavioural activation (for MDD) and worry time (for GAD); the two fear modules introduced exposure (for PD) and interpreting social cues (for SAD); the two substance use modules provided a normative comparison and introduced motivational interviewing/identifying triggers; and the two suicidal ideation modules introduced distress tolerance/self-soothing and emotion regulation strategies.

Each module was scripted as being presented by one of five characters, of whom four described having personal experience of one or more mental health problems (the fifth acted as an “expert” narrator). The approach of having material presented by individuals with personal experience of a mental health problem was intended to make the material more relatable and to provide examples of modeling specific strategies or behaviours. The modules were then developed into brief (2–6 min) videos. The videos were developed using professionally-designed cartoon characters in various poses and engaging backgrounds and animated with Microsoft PowerPoint. Appropriate graphics illustrated key concepts and the videos were overlaid with character voice-overs. The video within each module was accompanied by a relevant “homework” activity, comprising a worksheet that was designed to facilitate practice of the strategy presented.

The RCT compared a static version of FitMindKit (comprising the ten core modules) to a tailored version (comprising a mixture of core and specific modules based on symptom profiles) and an attention control condition. The scheme for tailoring is provided in Fig. 1, Fig. 2. Eleven versions of FitMindKit are defined in the Fig. 1 Venn diagram, on the basis of comorbid mental health problems, suicidal ideation and/or substance use. The modules delivered to participants who received each version of the intervention are shown in Fig. 2. In this RCT, all eligible participants met criteria (detailed in the measures Section 2.5 below) for at least one of the targeted domains. Participants meeting criteria for a single domain received the two specific modules for that domain, sandwiched within a selection of an additional eight non-specific modules (Fig. 2). Participants meeting criteria for any two domains received the four specific modules from those two domains, sandwiched within a selection of an additional six non-specific modules (Fig. 2). Where criteria were met for 3–4 domains, a hierarchy was used to select the most pressing areas of need, detailed in Fig. 1. The static version of the intervention is labelled as “version 1” in Fig. 1, Fig. 2.

Fig. 1

Schematic of the selection of tailoring pathways used to select modules of FitMindKit in the tailored condition.

Notes: SI: suicidal ideation; AS: alcohol/substance.

Fig. 2

Brief description of the 18 modules of FitMindKit, with 11 versions of the program used in tailoring (based on Fig. 1).

Notes: alc/sub: alcohol/substance use

Attention control condition

Participants in the attention control condition received the online HealthWatch program, which has no specific mental health or sleep-related content and is not associated with therapeutic reductions in depression (Glozier et al., 2013). Modules were fully text-based and contained information about bone health, sun exposure, food hygiene, use of vitamins and supplements, kidney health, microbes, household burns, respiratory viruses, heart health, and allergens.

Participants

Participants were recruited from the social networking site Facebook between January and August 2016, with data collection completed in December 2016. A Facebook page was set up to describe the study and paid Facebook advertisements were used to invite participation. A sample advertisement read, “Register for a mental health study: Complete a 20-min survey and a 10-day online program”, along with the logo for FitMindKit featuring five cartoon characters (Fig. 3). Initially, 18–25 year-old participants in Australia were targeted, as this was the target age range for the intervention [n = 77 of the 194 participants were recruited in this initial phase, with the remaining 117 (including 21 18–25-year-olds) recruited in the second phase]. Due to both challenges with recruitment and an interest in a broad age range, recruitment was later widened to include adults of any age in Australia. Eligibility criteria for the trial included elevated, but not clinically severe, scores on the symptom scales described below at the screening survey, with no suicide attempt reported in the past year or current self-reported high distress.

Fig. 3

Logo for the FitMindKit intervention, including the five characters featured in the videos.

The target sample size was 570, based on detecting a moderate effect size of Cohen's d = 0.4 at post-test with 90% power, assuming 30% attrition from the trial. It was assumed that 40% of all respondents to the screening survey would meet criteria for the study and 75% of these would consent to participate in the trial based on previous studies (Batterham, 2014, Christensen et al., 2016). In addition, based on similar recruitment methodology (Christensen et al., 2016), it was assumed that 90% of individuals who clicked the screening survey link would not proceed to complete the screening survey. Consequently, we anticipated that the study would require 19,000 individuals to click on the screening survey link to reach the recruitment target. The target sample size was not reached because the cost of advertising was higher than in previous studies (Christensen et al., 2016, Batterham, 2014, Thornton et al., 2016) – advertisements attracted fewer clicks, with approximately half as many clicks achieved within the allocated budget. In addition, there was greater non-completion of initial assessments than anticipated. The study also had higher attrition than anticipated in comparison to previous online trials (Christensen et al., 2016).

Procedure

Participants completed a 5 min screening survey to assess eligibility, followed by a 15 min baseline survey. They were then randomised to receive the static intervention, tailored intervention or attention control (described above) with approximately equal allocations across the three conditions using simple randomisation. The randomisation protocol was computer-based and devised by a researcher independent of the trial, such that the trial researchers were blinded to allocations.

On each of the first 10 days during the intervention period, participants were sent an email containing a daily link to a new module, along with links to modules that they had previously received to encourage practice of learned strategies. Participants were given a total of 14 days to complete and practice the 10 modules. On the day following the completion of the final module, participants were invited by email to complete a 15 min post-test survey. They received additional reminders to complete the survey if they had not done so after one and two weeks. An identical email reminder schedule was used for a 15 min 3-month follow-up survey (due 3 months after the final module). The post-test and 3-month follow-up surveys contained identical scales to the baseline assessment, although several scales testing potential moderators of effectiveness (not reported in this paper) appeared in the baseline only, while items regarding satisfaction and engagement with the intervention appeared only in the post-test survey. The CONSORT diagram of participant flow through the study is presented in Fig. 4. The trial is registered at the Australian New Zealand Clinical Trials Registry (ACTRN12615001211550) and ethical approval was obtained from the Australian National University Human Research Ethics Committee (protocol #2015/114).

Fig. 4

CONSORT diagram of participant flow through the trial.

Measures

The primary effectiveness outcome was a composite of multiple symptom measures, an indicator of overall severity of mental health symptoms. This measure was created using a standardised score (M = 0, SD = 1) of the sum of standardised scores for each of the seven measures of specific mental health problems, allowing trends across the multiple domains to be examined. The secondary effectiveness outcomes were based on each of the seven individual symptom scales. Tertiary outcomes were adherence to and satisfaction with the intervention. Specific measures are described below.

MDD symptoms were assessed using the Patient Health Questionnaire-9 (PHQ-9; Spitzer et al., 1999), with scores ranging from 0 to 27. GAD symptoms were assessed using the Generalized Anxiety Disorder-7 (GAD-7; Spitzer et al., 2006), range 0–21. PD and SAD symptoms were assessed using the Panic Disorder Screener (PADIS; range 0–13) and Social Phobia Screener (SOPHS; range 0–20) respectively (Batterham et al., 2015a, Batterham et al., 2016b). Symptoms of AUD and SUD were measured with the Alcohol Use Disorders Identification Test (AUDIT; range 0–40) and Drug Use Disorders Identification Test (DUDIT; 0–44) respectively (Babor et al., 2001, Berman et al., 2005). Finally, the Suicidal Ideation Attributes Scale (SIDAS; range 0–50; van Spijker et al., 2014b) was used to assess severity of suicidal thinking. Each of these brief self-report scales has been demonstrated to have robust psychometric properties for the construct of interest and accurately reflects presence of the mental health problem in the general population. The score ranges for inclusion in the trial were as follows (with participants needing to meet lower bound criterion on one or more scales): PHQ-9 score of 10–19; GAD-7 score of 6–15; PADIS score of 1–6; SOPHS score of 5–12; AUDIT score of 5–15; DUDIT score of 1–6; and/or SIDAS score of 1–20. Cut-points were chosen based on both existing criteria for moderate symptoms, but also population data, where available, corresponding to the 70th–90th percentile range of symptoms in general population samples (Anstey et al., 2011, Batterham et al., 2016a). Respondents with a score above the moderate clinical cutoff on any of these scales at screening were excluded from the trial, as were respondents with scores below the minimum on all seven measures. Those who were excluded due to high scores were provided with online and telephone help-seeking resources. All measures were repeated at post-test and follow-up assessments.

Adherence to the intervention was compared on the basis of number of modules completed (automatically collected within the research portal). Satisfaction was assessed using a scale of seven items devised by the authors, related to how much the participant: enjoyed the program, found it helpful, understood the content, found it interesting, would use it in the future, would recommend it to others, and learnt new skills from the program. Each satisfaction item was rated on a 0–10 scale from 0 “completely disagree” to 10 “completely agree”. Items were compared both separately and combined (sum score) to identify differences between conditions.

Analysis

Sample characteristics were provided to describe the participants and compare intervention groups. Primary (composite of symptom scales) and secondary (individual symptom scales) outcomes were assessed using mixed model repeated measures ANOVA (MMRM; Verbeke and Molenberghs, 2000) to account for missing data and to include all available data for participants analysed in the trial. This approach provides an intention-to-treat analysis to compare the intervention groups, yielding unbiased estimates of intervention effects. An unstructured variance-covariance matrix was assumed and degrees of freedom (df) were estimated with Satterthwaite's correction. In MMRM, the critical test of the effectiveness of the intervention is the two-way interaction between time and condition.

To test efficacy of the FitMindKit intervention, analyses were re-estimated excluding participants who did not complete any modules of the interventions, and among participants who completed the majority of the intervention. In addition, to test whether the intervention was more effective in specific age or gender groups, moderation was tested using three-way interaction effects between time, condition and moderator (18–24 vs. 25 +; male vs female). Differences in adherence (module completion) and satisfaction were assessed using one-way ANOVA across the three conditions and t-tests for pairwise comparisons. Analyses were conducted using SPSS v23.0 (IBM Corp, Chicago IL).

Results

Sample characteristics

Table 1 displays the sample characteristics across intervention conditions. There were no differences between conditions at baseline except that participants in the Static group had significantly higher alcohol use than in the other conditions. Participants were predominantly female, and young adults dominated the sample by design. Other demographic factors were largely reflective of the general population. Participants had slightly elevated symptoms across all mental health measures, although generally in the healthy-mild range. Based on the identified cut-points for each scale, only 11% of the sample had depression symptoms in the moderate range, 25% had moderate GAD symptoms, 22% had moderate symptoms of panic, 21% had moderate social anxiety scores, 14% had indication of harmful alcohol use, 9% reported any drug use, while 64% reported suicidal thinking. As anticipated, there was high comorbidity, with most participants (n = 152, 78%) meeting criteria in more than one domain, and 41% (n = 80) meeting criteria for four or more domains. There were no baseline differences between the 75 participants who completed the post-test assessment and the 119 who did not, except that PADIS scores were significantly higher among non-completers (t = 2.3, df = 191, p = 0.021).

Table 1

Sample characteristics within conditions.

Characteristic	Tailored FitMindKit(N = 66)		Static FitMindKit(N = 62)		Active control(N = 66)		χ2	p
	n	%	n	%	n	%
Gender							3.4	0.49
Male	7	(11%)	10	(16%)	7	(11%)
Female	58	(88%)	50	(81%)	59	(89%)
Other	1	(2%)	2	(3%)	0	(0%)
Age group							8.1	0.23
18–25	36	(55%)	29	(47%)	33	(50%)
26–45	12	(18%)	11	(18%)	9	(14%)
46–65	10	(15%)	20	(32%)	18	(27%)
> 65	8	(12%)	2	(3%)	6	(9%)
Education							4.1	0.67
High school or less	26	(39%)	20	(32%)	24	(36%)
Certificate/diploma	13	(20%)	10	(16%)	11	(17%)
Bachelor's degree	19	(29%)	16	(26%)	17	(26%)
Postgraduate degree	8	(12%)	16	(26%)	13	(20%)
Employment							2.3	0.89
Full-time	11	(17%)	16	(26%)	16	(24%)
Part-time	28	(42%)	25	(40%)	26	(39%)
Unemployed	9	(14%)	6	(10%)	6	(9%)
Not in the work force	18	(27%)	15	(24%)	17	(26%)
Location							1.6	0.82
Metropolitan	33	(50%)	32	(52%)	35	(53%)
Regional	25	(38%)	26	(42%)	25	(38%)
Rural	8	(12%)	4	(6%)	5	(8%)
Language spoken at home							1.0	0.61
English	60	(91%)	59	(95%)	62	(94%)
Other	6	(9%)	3	(5%)	3	(5%)
Symptom measures	M	(SD)	M	(SD)	M	(SD)	F	p
Depression (PHQ-9)	8.33	(4.16)	7.89	(4.75)	9.13	(4.51)	1.2	0.29
Generalized anxiety (GAD-7)	6.38	(4.23)	6.16	(3.98)	7.22	(3.84)	1.2	0.29
Panic (PADIS)	1.59	(2.14)	1.45	(2.01)	1.72	(1.88)	0.3	0.75
Social anxiety (SOPHS)	4.24	(3.20)	4.71	(3.83)	4.34	(3.66)	0.3	0.74
Alcohol use (AUDIT)	3.02	(3.48)	5.31	(3.85)	3.18	(3.03)	8.6	< 0.001
Drug use (DUDIT)	0.29	(0.87)	0.42	(1.18)	0.17	(0.82)	1.1	0.35
Suicidal ideation (SIDAS)	4.03	(4.91)	4.27	(4.66)	3.89	(4.86)	0.1	0.90

Note: bold values indicate p < 0.05

Effectiveness on the basis of the primary composite outcome

Participants in all conditions had a significant decrease in overall burden of mental health symptoms over time (F = 4.3, df = 2, 61.2, p = 0.017). However, no significant condition × time interaction was observed (F = 0.6, df = 2, 61.2, p = 0.640), indicating that there was no difference in symptom reduction between conditions. Fig. 5 shows mean (SE) composite scores at each time point for each condition, estimated from the MMRM. Effect sizes at post-test and 3-month follow-up are displayed in Table 2 for both the primary composite outcome and the specific symptom measures. Effect sizes were mixed and inconsistent. Participants in all groups tended to show small-moderate improvements over time. Particular exceptions were small increases in depression scores in the tailored condition at 3 months, small increases in social anxiety in the tailored condition at post-test, a large increase in drug use at 3 months for the static condition, a small increase in drug use at post-test in the control condition, and a small increase in suicidal thoughts at 3 months in the control condition. However, the calculation of effect sizes were based on completer data, and deteriorations were not significant in MMRM analyses.

Fig. 5

Estimated marginal means (SE) from mixed model repeated measures ANOVA for composite mental health/substance/suicidal ideation scores over time in the three conditions.

Table 2

Raw within-group effect sizes at post-test and 3 months.

	Tailored		Static		Control
	Post-test	3 month	Post-test	3 month	Post-test	3 month
Composite	0.18	− 0.02	0.56	0.16	0.24	0.47
Depression (PHQ-9)	− 0.10	− 0.25	0.31	0.21	0.06	0.38
Generalized anxiety (GAD-7)	0.47	0.23	0.48	0.67	0.35	0.51
Panic (PADIS)	0.14	0.07	0.42	0.35	0.33	0.38
Social anxiety (SOPHS)	− 0.15	− 0.07	0.36	0.45	0.21	0.24
Alcohol use (AUDIT)	0.09	0.03	0.20	0.13	0.04	0.33
Drug use (DUDIT)	− 0.05	− 0.22	0.19	− 1.39	− 0.17	− 0.02
Suicidal ideation (SIDAS)	0.24	0.04	0.18	0.14	− 0.08	− 0.27

Effectiveness on the basis of specific symptom measures

Similar to the composite measure, significant decreases were observed in some of the specific symptom measures over time, including GAD-7 scores (F = 10.6, df = 2, 68.2, p < 0.001), SOPHS scores (F = 4.4, df = 2, 67.1, p = 0.016) and AUDIT scores (F = 4.4, df = 2, 60.7, p = 0.017). No significant decrease was observed in PHQ-9, PADIS, DUDIT or SIDAS scores. Moreover, no significant condition × time interactions were observed.

Intervention adherence

Adherence to the intervention was low, with 43 (22%) participants not completing any modules of the intervention (16/24% in the tailored condition; 10/16% in the static condition; 17/26% in the control condition) and only 66 (34%) completing the majority of modules (≥ 5 modules: 33%, 31% and 38% respectively in the three conditions). Overall, participants completed a mean of 3.9 (SD = 3.8) of the ten modules. Among participants who began the intervention (n = 151), on average 5.0 (SD = 3.6) modules were completed. Although module completion was higher in the tailored condition (M = 4.0, SD = 3.7) than the static condition (M = 3.6, SD = 3.4), this difference was not significant (t = 0.713, df = 126, p = 0.48; Cohen's d = 0.12).

Due to low adherence, we also wanted to test whether the intervention showed a significant effect among individuals who had completed any modules, and among individuals who had completed the majority of the intervention. As with the primary analyses, there were no significant condition × time effects observed for the composite outcome, nor for the specific symptom outcomes.

Testing moderation by age or gender

No significant moderation effects were found by age group or gender, indicating that no differential effects existed within age or gender groups.

Comparison of satisfaction

Satisfaction with the intervention is shown in Fig. 6. Overall satisfaction ratings (sum score) at post-test were highest in the tailored condition (M = 7.0, SD = 1.9), followed by the static condition (M = 6.4, SD = 1.9) and the control condition (M = 5.7, SD = 2.2). Although the difference between all conditions was not significant (F = 2.7, df = 1, 50, p = 0.07), a priori analyses of group differences found a significant difference between the tailored and control groups (t = 2.3, df = 50, p = 0.029) in overall satisfaction. There were also significant effects on specific forms of satisfaction as shown in Fig. 6: tailored/static participants found the intervention more helpful (F = 3.7, df = 1, 50, p = 0.030), were more likely to recommend it to others (F = 6.9, df = 1, 50, p = 0.002), and reported that it taught them more skills (F = 6.3, df = 1, 50, p = 0.003) compared to control participants.

Fig. 6

Satisfaction ratings by condition (error bars indicate SD).

Discussion

The current trial found no evidence for the effectiveness of the FitMindKit program in reducing overall symptoms of mental health and substance use problems, nor evidence for reductions in specific symptom domains. There were only small and inconsistent improvements in most mental health symptom domains over time in all three conditions. Further exploration of interventions that address comorbid conditions may be worthwhile, particularly given the high rates of comorbid mental health problems seen in the present sample and in national population surveys (Kessler et al., 2005, Slade et al., 2009). This lack of between-group effects may reflect the control condition being an active comparator, volatility in the estimates due to small sample sizes, insufficient engagement with the intervention or insufficient intensity of the intervention. The sample also had limited scope for improvement in many of the symptom domains – for example, only 11% of the sample had moderate depression symptoms, 14% had indication of harmful alcohol use, and 9% reported any drug use, leading to potential floor effects. Consequently, subsidiary outcomes such as increased coping and self-efficacy may be useful to measure in future trials with relatively healthy samples. Possible explanations for the findings are discussed further below.

Adherence to the intervention appeared to slightly favour the tailored intervention condition, although differences were not significant. More importantly, participants in the tailored condition had significantly greater overall satisfaction with their program than control participants. Participants in the active intervention (tailored/static) also reported their program was significantly more helpful, were significantly more likely to recommend it, and reported learning significantly more skills than participants in the control condition. Participants in all conditions found the content easy to understand. As positive differences in satisfaction between intervention and active control conditions are not always observed (Kaltenthaler et al., 2008, Griffiths et al., 2017), it appears likely that participants found the format and content of the intervention to be engaging. The present study found clear evidence that participants had greater satisfaction with the FitMindKit program than an attention-control program. Findings also suggest that with a larger sample, there remains the possibility that benefits may be found in terms of symptom reductions and adherence related to the tailored program.

There were two major challenges of trialling the FitMindKit program in a fully-automated online setting. Firstly, when participants are not personally engaged with a project and have no interaction with a researcher, it is difficult to recruit and maintain engagement (Todkill and Powell, 2013, Christensen et al., 2016), even in brief trials such as this one. While we had hoped for greater engagement based on previous trials with online recruitment (Christensen et al., 2016), the absence of personal interaction may have been a barrier to ongoing engagement for many participants. There is a need for further research on how best to engage with participants in research that is conducted in an entirely online setting without personal contact. The trial did not have a sufficient sample to adequately test the effectiveness of the program. Secondly, the FitMindKit program addresses a broad range of mental health problems. As a result, it is challenging to recruit a sufficiently large sample of individuals who have elevated symptoms across a range of mental health conditions (seven domains in the current study). Therefore, it is difficult to capture change across all of the mental health domains targeted by the intervention. A less ambitious transdiagnostic intervention approach, potentially focusing only on internalising symptoms (e.g., 3–4 domains), might be warranted. This challenge was further compounded by the low adherence to the intervention and high attrition from assessments, considerably reducing the study's power to find effects. Acceptance facilitation (Ebert et al., 2015) and similar approaches could be further explored for increasing adherence in unguided interventions.

Furthermore, the intervention was designed to be low-intensity to enable targeting of multiple comorbid mental health and substance use problems. It is possible that the brevity of the intervention reduced its effectiveness. In addition, the rapid pace of module presentation (daily delivery) may have become a barrier to completing the intervention for some participants. Although our previous research has supported community preferences for rapid delivery of brief modules (Batterham and Calear, 2017), it may be the case that more time is needed for participants to fully engage with material, to consolidate their understanding of therapeutic concepts and to not become overwhelmed if they fall behind schedule. Full completion of the program as intended would require at least 2–3 h, including watching ten videos (~ 1 h) and completing activities (> 1 h). Such a “dosage” of internet therapy has previously been shown to have an effect on depression symptoms (Christensen et al., 2006). Although the videos appeared to be engaging and participants were largely satisfied with the content and delivery, greater interactivity may be needed to encourage users to engage with the therapeutic strategies presented. The activities presented in conjunction with the videos were static (text and images); greater interactivity and ability for participants to monitor their progress might be necessary to promote sufficient practice of the therapeutic material, with a greater emphasis on overlearning (Scott, 1992). Techniques to improve interactivity may include the use of persuasive systems design (Oinas-Kukkonen and Harjumaa, 2008) and/or gamification (Deterding et al., 2011) that actively emphasise, among other principles, behavioural feedback.

There were a number of other limitations of the trial that should be noted. All outcomes were based on self-reported symptoms. Some of the scales were not normally distributed; however, the lack of significant effects observed would be unlikely to change if non-parametric models were used. Participants were selected on the basis of elevated symptoms, an indicated prevention approach. It is possible that this program would be more appropriate in a universal prevention context, such as in a school setting (Calear and Christensen, 2010), or in a clinical context as an adjunct to therapy. While the present trial was underpowered to find differential effects by age or gender, it is possible that the design of the program (including the character artwork, character voices and examples provided) might be more appealing to a younger audience. The trial was opened to adults of all ages due to challenges in meeting recruitment targets. Young people may have been less inclined to engage with an online trial than older adults, as they may be time-poor and unable to commit to the completion of assessments in particular. The absence of direct human contact in the trial may have been an additional barrier to the completion of assessments and the intervention. In addition, alternative formats for delivery, such as email or an app, may also be more appropriate for such a low-intensity program.

Further examination of the effects of specific modules and their order of presentation may also be beneficial, as it remains unclear which specific strategies are most beneficial to individuals with a particular pattern of symptoms. Such research might take into account both needs (on the basis of symptom patterns) and individual preferences for addressing the specific problems that they perceive to be most salient. Ultimately, a dynamic tailored intervention that responds to the changing needs and preferences of the individual may further facilitate engagement with complex internet interventions. Assessment of clinical states using a structured clinical interview may also better determine whether such low-intensity interventions may have impact in a prevention or treatment context.

Conclusions

The FitMindKit program was designed as a tailored internet intervention targeting a broad range of comorbid mental health and substance use problems. This trial did not find evidence that the intervention was effective. Participants were more satisfied with the program than an active control condition, suggesting the content and design of FitMindKit were acceptable. A more comprehensive or more targeted version of the program with greater interactivity might have scope for reducing comorbid mental health problems in the community. Identifying appropriate content, duration, delivery methods and target populations is important to progressing the development of transdiagnostic interventions.

Funding

The development of FitMindKit was supported by a grant from AFFIRM, The Australian Foundation for Mental Health Research, and the John James Memorial Foundation. The development of the RCT and recruitment of young people into the RCT was supported by a grant from Australian Rotary Health. PJB and ALC are supported by NHMRC Fellowships 1083311 and 1122544 respectively.

Conflict of interest

FitMindKit was created by PJB with input from ALC and LF and additional review by Dr. Bridianne O'Dea and Dr. Fiona Shand. The authors derive no financial benefit from the program.

Role of the funding sources

The funding sources had no involvement in study design; in the collection, analysis and interpretation of data; in the writing of the report; or in the decision to submit the article for publication.