Adherence to a web-based pre-treatment for phobias in outpatient clinics.

BACKGROUND: Non-adherence in Internet interventions is a persistent and multifaceted issue and potentially limits the applicability and effectiveness of these interventions. Factors that influence non-adherence are poorly understood, especially in outpatient samples with more complex symptoms.
OBJECTIVE: The current study is a secondary analysis of data from a randomised controlled trial that examined the cost-effectiveness of offering an Internet-based exposure treatment to phobic outpatients on a wait-list to receive face-to-face psychotherapy.
METHODS: We collected baseline demographic and clinical information, and adherence data of the Internet-based intervention and conducted regression analyses to predict non-adherence to the intervention.
RESULTS: The adherence to the intervention was low, with only 13.3% of 105 patients completing all five lessons of the intervention. The median number of exercises completed (out of a possible 8) was 3. In a multi-predictor model, a higher baseline score of anxiety (OR = 0.94, 95% CI 0.90-0.99) was a risk factor for low adherence. Higher age (OR = 1.05, 95% CI 1.00-1.09) was a protective factor against non-adherence. Participants who adhered to the intervention were more likely to complete post-test assessments, further biasing results. However, overall participant attrition was high. The results are based on a small subset of participants and should be interpreted with caution.
CONCLUSIONS: Poor baseline clinical status and age are factors to consider when deciding whether to offer an Internet-based intervention to outpatients. Low adherence among those patients might be related to intrinsic motivation and might even be lower in outpatient settings where participants expect to receive face-to-face treatment. It might be worthwhile to develop a concise instrument to assess intrinsic motivation and treatment expectations for using Internet-based interventions, and for the therapist to review the range of possible (Internet-based) intervention options to suit personal preferences and expectations. TRIAL REGISTRATION: Netherlands Trial Register, NTR2233.

Introduction

The use of the Internet to deliver mental health interventions has been increasing almost exponentially in the past decade. The flexibility and easy implementation of Internet-based mental health interventions makes it possible to deliver therapies to a large number of people in their own environment, at a convenient moment. Furthermore, these interventions have the potential to reach populations hitherto unreachable, due factors such as distance to healthcare providers or stigma. Internet interventions in the form of Internet-based cognitive behavioural therapy (iCBT) have been shown to be as effective as face-to-face (FtF) psychotherapy for a number of common mental disorders (Andersson et al., 2014). Additionally, preliminary evidence suggests that these interventions may be cost-effective when compared to alternative treatments or wait-list control groups (Donker et al., 2015).

One of the main challenges of offering Internet-based interventions is premature discontinuation of the intervention (Christensen and Mackinnon, 2006, Eysenbach, 2005). Participants who discontinue prematurely are non-adherent to the intervention, as they do not experience the content of the intervention as it was intended. Non-adherence estimates in Internet interventions range from 2% to 83%, with an average of approximately 31% (Melville et al., 2010). It has been found that adherence to an intervention tends to be higher when the intervention has some of guidance by a professional (Christensen et al., 2009, Richards and Richardson, 2012). The non-adherence to guided Internet-based interventions is comparable to discontinuation of FtF psychotherapy (van Ballegooijen et al., 2014), as a recent large meta-analysis found that approximately 26.2% of patients discontinued FtF psychotherapy prematurely (Fernandez et al., 2015). Although the non-adherence to guided Internet-based interventions seems comparable to FtF treatments, the effects of these psychological interventions might be increased when the problem of non-adherence can be reduced. This makes it important to examine factors that influence non-adherence to psychological treatments.

Previously found factors associated with treatment non-adherence in FtF psychotherapy were a lower educational level (Keijsers et al., 2001), higher baseline depression scores, or being in specialised (outpatient) rather than in primary care, although the latter may be indicative of worse clinical status. Research on the influence of gender on adherence has produced equivocal results, with some indicating higher adherence in men (Issakidis and Andrews, 2004), some in women (Herbert et al., 2005, McEvoy, 2007), and others reporting no differences (Erwin et al., 2002, Hofmann and Suvak, 2006, Santana and Fontenelle, 2011).

With regard to Internet interventions, several categories of adherence predictors can be identified. Broadly, these are divided into socio-demographic variables, psychological problems (e.g., duration and severity of problems, comorbid anxiety and depression) and treatment-related variables such as e.g., treatment credibility, expectations and presentation and motivation to participate (Melville et al., 2010). In Internet-based interventions, sociodemographic variables that were associated with non-adherence in unguided Internet-based therapy included, as in FtF treatment, a low educational level. Furthermore, male gender, and younger age were also associated with non-adherence (Karyotaki et al., 2015). Psychological problems associated with lower online adherence are also similar to FtF psychotherapy, including higher pre-treatment symptoms and comorbidity (Christensen et al., 2009). Treatment-related variables include expectations of whether the intervention will work (Boettcher et al., 2013), feeling accountability to the therapist (Mohr et al., 2011) or a sense of duty to researchers carrying out the intervention and being able to identify with the programme (Donkin and Glozier, 2012). Additionally, persuasive design features – which explicitly motivate a user to keep using the intervention – also play a part. For example, more frequent intended usage, more frequent updates and more extensive use of dialogue support (e.g., an online coach), significantly predicted better adherence (Kelders et al., 2012).

However, one shortcoming of previous research on predictors of adherence is that it was mostly based on either non-clinical samples, self-selected samples who were offered no other treatment other than an Internet-based treatment (i.e., studies with wait-list control conditions), or samples from unguided Internet-based interventions (e.g., Karyotaki et al., 2015). Currently, we are not aware of any research on predictive variables of adherence to an Internet-based intervention in patients that are actively awaiting outpatient psychological treatment. The upcoming face-to-face psychological treatment may influence the attitudes of patients towards Internet-based treatment, or may influence attitudes or behaviours. For example, outpatients might have different clinical profiles (e.g., worse symptom severity) which influence adherence. Additionally, knowing that one is guaranteed to receive face-to-face psychotherapy could lower the threshold to discontinue the Internet-based intervention. Information on predictors of adherence can help therapists and healthcare workers to identify different patient profiles, e.g. those likely to complete an Internet-based treatment relatively independently, patients for whom an Internet-based intervention should be more closely monitored e.g. in form of extra guidance, and perhaps patients best directly referred to FtF treatment.

In short, treatment dropout and its causes remain a research priority. To add to currently available information, the current study uses data from a previously conducted RCT (Kok et al., 2012, Kok et al., 2014), which investigated the potential cost-effectiveness of offering an Internet-based guided pre-treatment to outpatients during wait-list for FtF psychotherapy. In short, the rationale was that offering an Internet-based pre-treatment might reduce the number of FtF sessions by relegating routine tasks to guided self-help.

In the current study, we investigate predictors of adherence in this clinical sample of phobic outpatients. We will examine prognostic variables related to intervention adherence in an outpatient sample of phobic patients. As Internet-based interventions increasingly find their way into clinical practice, it is necessary to identify predictive variables for adherence to these interventions in clinical samples.

Method

Design

This study is a secondary analysis from a randomised controlled trial examining the cost-effectiveness of guided Internet-based treatment for phobic patients waiting for face-to-face treatment (Kok et al., 2012, Kok et al., 2014). In this trial, 212 outpatients from eight Dutch outpatient clinics were randomised to either 1) a 5-week wait-list control group (n = 107) or 2) a 5-week Internet-based guided intervention (N = 105) based on exposure homework exercises. After wait-list control group or the Internet-based intervention, all participants were scheduled to receive face-to-face-psychotherapy; all but 4 participants (2 in either condition) did continue to face-to-face psychotherapy.

For this secondary analysis on intervention adherence, data from the 105 participants receiving the Internet-based intervention were used.

The study from which these data originated has been approved by the Medical Ethical Committee of the VU University Medical Centre (registration number 2010/77).

Participants and recruitment

Phobic patients were referred to the research team by outpatient clinic staff. If eligible for inclusion, the mood and anxiety sections of a diagnostic interview (CIDI; World Health Organization, 1990) were administered by telephone, informed consent was sought and baseline measurements were recorded. We included computer-literate, Dutch-speaking patients, 18 years or older, with a DSM-IV-TR diagnosis of any phobia (social phobia, agoraphobia or specific phobia). Stable psychotropic medication use was allowed. Patients with psychotic disorders or at elevated risk for suicide were excluded and received care as usual at the outpatient clinic.

Intervention

Intervention patients were offered a five-week Internet-based intervention, based on exposure therapy. This intervention comprised 8 exercises, which were to be completed in 5 weeks; the average time spent waiting for face-to-face psychotherapy. In the first week, patients were offered psychoeducational information on phobias and reviewed the treatment rationale for exposure therapy. Additionally, patients had to assess motivation for treatment and were asked to affirm they were ready to start confronting their fear by self-exposure. In the second week, patients categorised their fears and avoidance behaviours, constructed a fear hierarchy (ranking fear-inducing situation from ‘no fear’ to ‘complete terror’) and started planning to complete this fear hierarchy in the coming weeks by planning gradual exposure homework exercises. During week 3, 4 and 5 patients were asked to complete their planned exposure exercises. This means that patients needed to expose themselves to situations or objects that elicited fear. They had made a fear hierarchy and were supposed to expose themselves to fearful situations in a gradual way starting with the least fearful situation and increasing levels of intensity each week. After each lesson, patients reported back to their coaches. These trained coaches were Master's level clinical psychology students who were instructed to provide procedural (i.e., not therapeutic) guidance (approx. 15 min per week), which served to guide patients through the self-exposure exercises, to explain things more clearly if necessary and to motivate them to carry on. After these 5 weeks, all patients were to receive FtF psychotherapy. An overview of exercises per week can be found in Table 1. The intervention is described in more detail elsewhere (Kok et al., 2012, Kok et al., 2014). The intervention was tunnelled, i.e., participants had to complete a week's exercise before they could progress to the next week. However, they had unlimited access to previous weeks.

Table 1

Intervention planning and adherence percentages.

Week	Exercise	Content of the exercise	Percentage adherence
1	Exercise 1	Select fear inducing situations	20
	Exercise 2	Assess motivation for treatment
2	Exercise 3	Assess avoidance behaviour	20
	Exercise 4	Construct fear hierarchy
	Exercise 5	Construct exercise plan
3	Exercise 6	Complete exposure exercises	20
4	Exercise 7	Complete exposure exercises	20
5	Exercise 8	Complete exposure exercises, relapse prevention, prepare FtF therapy	20
Total	8 exercises	Intended use	100

Definition of adherence

The Internet intervention consisted of five lessons. Therefore, the completion of one week, including all exercises for that particular week, contributed 20% to the total adherence. During the first two weeks, patients were asked to perform more than one exercise. The 20% of adherence for those weeks is broken down by the number of exercises (week 1: 10% for completing each exercise; week 2: 6.7% for completing each exercise). Patients were considered to be adherent to the intervention when they had completed 50% of more of the intervention. The rationale for this cut-off point is that at 50%, participant had crossed from planning exposure exercises to actually performing in-vivo exposure, which we had anticipated to be the largest obstacle for participants. The remainder of the intervention after this is repetition of exposure exercises that do not necessarily need guidance.

Assessments

In the primary study from which data for the current analyses were used, anxiety, depression and quality of life were assessed using Internet-based questionnaires at pre-randomisation and at post-intervention (5 weeks). A short description of these instruments is provided below. Apart from these instruments, we recorded demographic variables (age, sex, use of psychotropic medication such as anxiolytics or antidepressants; also whether the participant was highly educated, this being defined as having finished a degree comparable to at least bachelor's level). Before randomisation, the prospective participants were screened using the CIDI (World Health Organization, 1990), but using only the sections screening for anxiety and mood disorders to minimise participant burden. As such, other comorbidities were not assessed as this would have results in a large burden for the participant in terms of assessment time.

Primary Outcome Measure

The Fear Questionnaire (FQ) (Marks and Mathews, 1979) was the primary outcome measure, which measures phobic fear severity and avoidance. Its psychometric validity has been established for the Dutch version (Van Zuuren, 1988).

Secondary Outcome Measures

Secondary outcome measures were the Beck Anxiety Inventory (BAI) (Beck and Steer, 1993), a 21-item self-report questionnaire measuring physiological anxiety symptoms and the Center for Epidemiological Studies-Depression Scale (CES-D) (Radloff, 1977). The BAI has been validated for patients with different anxiety disorders (de Beurs et al., 1997, Leyfer et al., 2006). Similarly, the Dutch version of the CES-D shows good psychometric validity (Donker et al., 2010). Internal consistency in this sample was excellent for the BAI (Cronbach's α = 0.92) and the CES-D (Cronbach's α = 0.92). Quality of Life was assessed using the EQ-5D (EQ-5D), which has previously been assessed for validity (Brooks, 1996, van Agt et al., 1994). These scores result in utility scores which range from death (0) to perfect health (1).

Analyses

Logistic regression models using adherence as a dependent variable were constructed using independent baseline demographic and clinical predictor variables. First, based on previous findings into the factors influencing non-adherence, predictor variables age and baseline measurements of the FQ, BAI, CES-D and EQ-5D were entered separately as univariate predictors with adherence status (adherent/not adherent) as a dependent variable. Secondly, variables significantly associated (p < 0.20) in the univariate analyses were entered in a backward conditional logistic multi-predictor regression model with adherence status as dependent variable. Reported confidence intervals (CI) are 95% and analyses were performed with SPSS 21 for Windows (SPSS, Chicago IL, USA).

Missing data

Out of 105 participants, adherence status for 4 participants (3.8%) could not be retrieved. Therefore, all analyses using adherence as a dependent variable were performed on the remaining 101 participants. Furthermore, 46 out of 101 participants (45.5%) did not complete the post-test assessment, non-completion of which was used to calculate the association between adherence and non-completion of post-test questionnaires.

Results

Adherence

Of 101 participants from the intervention group available for analyses; 20 participants (19.8%) never started using the intervention (pre-treatment dropout) and were coded as being 0% adherent; 50 participants (49.5%) used 1–50% of the intervention and 31 (30.7%) used at least 50% of the intervention and were considered adherent.

Table 2 reports demographic and clinical variables for the full sample and for the three usage groups. On average, participants were 36 years old; 57.4% were female and 57.4% were highly educated. Alongside one or more phobias, almost half of participants (44.6%) were diagnosed as suffering from a DSM-IV diagnosis of major depressive disorder (MDD) at baseline, and 78.1% of all participants was diagnosed with 2 or more disorders. Almost half of the sample (48.6%) was suffering from 3 or more disorders. The descriptive statistics suggest that those of older age, less severe baseline clinical scores and higher quality of life were more likely to be in a more adherent group. Moreover, the 20 participants that did not start using the intervention appear to have the least favourable baseline scores of all three groups.

Table 2

Descriptive statistics of baseline demographic and clinical scores in relation to adherence.

		% of adherence1
	All (N = 101)	Did not start (n = 20)	1 to 50% (N = 50)	≥ 50% (N = 31)
Demographic variables
Age	35.9 (11.9)	32.4 (9.1)	34.1 (11.5)	40.3 (11.9)
Female sex	58 (57.4%)	10 (50.0%)	27 (54.0%)	19 (61.3%)
Higher education	58 (57.4%)	11 (55.0%)	31 (62.0%)	20 (64.5%)
Baseline diagnoses
Dysthymia present	8 (7.5%)	3 (15.0%)	5 (10.0%)	0 (0%)
MDD present	45 (44.6%)	3 (15.0%)	24 (48.0%)	9 (29.0%)
SAD present	52 (49.5%)	5 (25.0%)	30 (60.0%)	15 (48.4%)
GAD present	36 (34.3%)	7 (35.0%)	17 (34.0%)	10 (32.3%)
Panic disorder present	64 (61.0%)	13 (65.0%)	34 (68.0%)	14 (45.2%)
Agoraphobia present	19 (18.1%)	5 (25.0%)	8 (16.0%)	6 (19.4%)
Specific phobia present	61 (58.1%)	12 (60.0%)	28 (56.0%)	18 (58.1%)
2 or more specific phobias	24 (22.9%)	4 (20.0%)	12 (24.0%)	7 (22.6%)
2 or more diagnoses	82 (78.1%)	16 (80.0%)	41 (82.0%)	21 (67.7%)
3 or more diagnoses	51 (48.6%)	8 (40.0%)	29 (58.0%)	12 (38.7%)
Comorbid anxiety/depression	46 (43.5%)	10 (50.0%)	25 (50.0%)	9 (29.0%)
Using psychotropic medication	14 (13.3%)	0 (0%)	5 (7.1%)	8 (25.8%)
Baseline clinical scores
FQ score (M/SD)	42.0 (23.4)	50.4 (21.9)	45.8 (23.7)	32.8 (21.7)
BAI score (M/SD)	45.0 (13.8)	50.8 (15.2)	48.2 (13.4)	36.9 (11.0)
CES-D score (M/SD)	25.0 (8.6)	27.7 (9.5)	27.1 (8.7)	20.2 (6.3)
EQ-5D score (M/SD)	0.60 (0.28)	0.51 (0.27)	0.53 (0.28)	0.74 (0.20)

Adherence is defined as % of intended usage.

Table 3 shows the flow of participants through the interventions. As can be seen, each week shows a marked drop in participants. The relative largest drops in adherence can be seen in the weeks in which actual in-vivo exposure was to be exercised; weeks 3–5.

Table 3

Frequencies and percentages of participants who completed the exercises.

Week	Exercise	Content of the exercise	Completed by (N/%)	Relative attrition (N/%)
0	–	Start of intervention	101 (100%)	–
1	Exercise 1	Select fear inducing situations	79 (78.2%)	22 (21.8%)
	Exercise 2	Assess motivation for treatment	77 (76.2%)	2 (2.5%)
2	Exercise 3	Assess avoidance behaviour	59 (58.4%)	18 (23.4%)
	Exercise 4	Construct fear hierarchy	49 (48.57%)	10 (16.9%)
	Exercise 5	Construct exercise plan	46 (45.5%)	3 (6.1%)
3	Exercise 6	Complete exposure exercises	33 (32.7%)	13 (28.3%)
4	Exercise 7	Complete exposure exercises	18 (17.8%)	15 (45.5%)
5	Exercise 8	Complete exposure exercises, relapse prevention, prepare FtF therapy	14 (13.9%)	4 (22.2%)

Association of baseline demographic, clinical variables and adherence

Univariate analyses show that younger age and not using psychotropic medication were significantly associated with higher adherence. Furthermore, higher baseline severity of phobic fears, higher scores of general anxiety, depression, and quality of life were all significantly associated with non-adherence in univariate analyses. Thus, patients with more severe symptoms at baseline, and patients with lower quality of life were less likely to adhere to the intervention (Table 4).

Table 4

Univariate association between baseline demographic and clinical scores and intervention adherence (logistic regression analysis).

	At least 50% adherent (N = 31)
	OR (95% CI)	p
Higher age	1.05 (1.01–1.08)	0.02
Female sex	0.75 (0.32–1.78)	0.51
Higher education	0.57 (0.24–1.36)	0.20
Using psychotropic medication	4.52 (1.34–15.23)	0.02
Baseline clinical scores
FQ	0.97 (0.95–1.00)	0.01
BAI	0.93 (0.89–0.97)	0.00
CES-D	0.89 (0.83–0.95)	0.00
EQ-5D	30.34 (3.73–245.42)	0.00

Next, we entered the variables of interest into a backward multi-predictor analysis (Table 5). The results show that higher baseline anxiety, lower quality of life and psychotropic medication use are significant predictors of a worse adherence status. Most predictors that were significant as univariate predictors were no longer significant in this model; age (p = 0.156), sex (p = 0.157), having a higher education (p = 0.438), baseline FQ score (p = 0.213) and baseline depression score (p = 0.288).

Table 5

Multi-predictor association between baseline demographic and clinical scores and dichotomous adherence status.

	At least 50% adherent (N = 31)
	OR (95% CI)	p
Age	1.05 (1.004–1.093)	0.03
Baseline BAI score	0.944 (0.901–0.990)	0.02

Note: R2Nagelkerke = 0.333. Model X2 = 26.96 (p = 0.000).

Association between adherence and post-test assessment completion

Finally, we explored if adherence was associated with completing post-test assessments (dropout attrition). Of those who were non-adherent to the intervention (< 50% adherent, N = 70), only 27 (38.5%) completed the post-test assessment, of those who were adherent to the intervention (N = 31), 26 (86.7%) completed the post-test assessment. Analyses show that in this sample, when adherence was defined at completing ≥ 50% of the intervention, adherence to the intervention was indeed associated with a higher probability of completing the post-test assessment (χ2 = 17.68, p = 0.000).

Discussion

General discussion

Identifying factors for non-adherence to Internet-based interventions in routine clinical practice is a key issue to improve acceptability of those interventions to therapists, researchers and patients. Moreover, improving adherence may serve to improve clinical effectiveness, dissemination and adoption and cost-effectiveness of Internet-based interventions. In the current study, we attempted to identify prognostic variables of adherence to a guided Internet-based intervention for outpatient phobic patients.

The clinical status was unfavourable for the sample, with almost 50% of the sample suffering from three or more comorbid diagnoses at baseline. The average adherence to the intervention was low, with only 13.3% of 101 patients completing all 5 weeks of the intervention. The median number of exercises completed (out of a possible 8) was 3. In a multi-predictor model, adherence was associated with baseline characteristics of more severe anxiety symptoms at baseline and higher age. Contrary to findings on unguided Internet-based interventions (Karyotaki et al., 2015), demographic variables such as sex and educational level were not significantly associated with adherence in this sample. Additionally, participants who did not adhere to the intervention were less likely to complete the post-test assessment.

The adherence to guided Internet interventions among patients recruited from the general population varies between 50% and 80% (Christensen et al., 2009, Richards and Richardson, 2012). However, research data on adherence to Internet-based psychological interventions from patients actively seeking psychological treatment in outpatient care is scarce. The current study, being one of the first in this outpatient setting, shows low adherence rates. This is corroborated by two recent studies in comparable outpatient settings (Kolovos et al., 2016, Mathiasen et al., 2016). The results suggest that patients with better clinical status at baseline are more likely to keep using the intervention, which is in line with previous research in non-clinical populations and unguided Internet interventions (Karyotaki et al., 2015).

Apart from severity of the primary disorder, it has been shown that comorbidity has profound implications for the severity, chronicity and impairment of patients (Klein Hofmeijer-Sevink et al., 2012) and is associated with a worse clinical prognosis (Penninx et al., 2011). Depression is a common comorbidity for patients with a current anxiety disorder (Lamers et al., 2011) and depressed patients are more likely to be non-adherent to both psychological (Ledley et al., 2005) and pharmacological treatments (DiMatteo et al., 2000). In Internet interventions, it is also associated with lower adherence and worse treatment outcomes (Boettcher et al., 2013, Ebert et al., 2013, Karyotaki et al., 2015). In the current sample, patients suffered from either anxiety-anxiety comorbidity, anxiety-depression comorbidity, or both (Kok et al., 2014), with almost 50% of patients suffering from three or more diagnoses at baseline. Additionally, most patients had baseline CES-D depression scores well over the commonly used cut-off scores of 16 and 20 (Lewinsohn et al., 1997). The high co-morbidity rates with other anxiety disorders as well as with depression may be an explanation of the relatively low adherence rates in this trial. This association has also been demonstrated in other trials (Johnston et al., 2013, Kolovos et al., 2016, Mathiasen et al., 2016).

With worse clinical status, one would perhaps expect less clinical improvement. However, a recent individual patient data meta-analysis demonstrated that both patients with severe and mild symptoms benefitted equally from guided Internet based interventions (Bower et al., 2013). As this meta-analysis was performed on depressed patients, the results may not be fully generalizable to our sample; although depression scores were also relatively high in the current trial (Kok et al., 2014) and comorbid depression was present in almost half of all patients. However, in the current sample too few patients were non-comorbid to perform meaningful analyses on subgroups. Despite this, based on the literature, comorbid depression may be an important factor to consider when offering Internet-based interventions to anxious outpatients, especially since comorbidity can be high in these patients.

Previously, no association between adherence to the intervention and improvement in terms of clinical scores was found in an unguided web-based intervention for depression (Donkin et al., 2013). A similar result was recently demonstrated in a small RCT on a guided Internet intervention for patients with social phobia or panic disorder (Mathiasen et al., 2016) and confirmed in a recent, broader meta-analysis on web-based interventions for a variety of conditions (Wildeboer et al., 2016). Given that the definition of adherence is open to interpretation, the drop-out at post-test was relatively high and the short-term effect sizes small (Kok et al., 2014), analyses on whether adherence was associated with therapeutic benefit would have been underpowered and inconclusive.

One explanation for the previously found lack of association between adherence and improvement is, as mentioned above, that some participants may have felt that they had derived sufficient benefit and no longer needed the intervention (Hilvert-Bruce et al., 2012). However, we lacked the short-term, repeated measurements to investigate this effect in a way that is psychometrically valid and not too burdensome for the patients. It should be noted that all participants in the current sample were scheduled to receive – and nearly all did receive – FtF treatment. Had they derived sufficient benefit, they would have opted out of FtF treatment. Indeed, it may be argued that participants experienced a low threshold to dropping out as they were assured of receiving FtF treatment regardless of their progress in the Internet-based intervention. These findings question the assumption of a dose-response relationship between exposure to the intervention and clinical benefit (Donkin et al., 2013).

Another reason for a lack of association might be caused by inconsistencies in defining and measuring the concept of (non)adherence (Kelders et al., 2012). In some studies, it is defined as e.g., whether a web page was accessed or content was displayed on a user's screen. This approach, however, implicitly assumes that the presented content was also read, understood and acted upon in the case of homework exercises. Unlike pharmacological interventions that require taking a pill, psychological interventions that require metacognitive techniques or behavioural techniques need practice and elaboration in real life before they can have an effect. As such, one possible explanation for the weak association between adherence and effect is that adherence is an inherently poor metric for assessing whether the intervention content was internalised as intended. Another possible explanation is that adherence and clinical effect have a reciprocal relationship, where participants need to experience improvement or success rapidly in the beginning of the intervention to ‘kick-start’ as an incentive to create adherence. In exposure therapy, although this is partly addressed by setting attainable exposure goals in the fear hierarchy, in the current intervention the first two of five weeks are theory-based and offer little in the way of such successful experiences. In the current study, however, non-adherence throughout the weeks with in-vivo exposure was steady.

Limitations

A number of limitations reduce the generalisability of the findings of this study. Firstly, we defined adherence as having performed at least 50% of the intervention. This might be considered a low threshold as some reviews have advocated higher thresholds (e.g., 75% or 80% (Karyotaki et al., 2015, van Ballegooijen et al., 2014). However, not only would this have resulted in too few participants in the adherent group to perform subgroup analyses, FtF treatment was scheduled to start after five weeks and finishing all of the intervention in this comparatively short time would have posed a rather large burden on the participants. Secondly, participants in this trial were scheduled to receive FtF treatment regardless of whether they participated in the Internet intervention, as the primary goal of this trial was to assess whether FtF psychotherapy sessions could be reduced (Kok et al., 2012, Kok et al., 2014). This limits the generalisability of the results as different motivational factors may play a role in samples that receive only an Internet intervention – for these samples, an Internet intervention is the only treatment they will receive and motivation to persist with the intervention may be greater. Knowing that one will receive face-to-face treatment regardless of whether one completes on Internet-based intervention may lower the threshold of discontinuing such an intervention.

The current analyses into the factors of non-adherence were grounded on previous literature and were explorative in nature. Therefore, we may not have measured important other determinants of non-adherence. Although our assumptions included demographic factors and symptomatology, these are probably not the sole causative factors of non-adherence. Alternative factors we did not assess are e.g., that some participants may have felt that they had derived sufficient benefit and decided to discontinue the intervention (Hilvert-Bruce et al., 2012). Other factors not explicitly assessed were low treatment motivation (Donkin and Glozier, 2012), side effects (e.g., worsening of symptoms), treatment expectations (Alfonsson et al., 2016) and practical barriers as possible reasons to be non-adherent to FtF and Internet-based treatments for anxiety (Taylor et al., 2012). Additionally, a worse clinical status such as in the currently described clinical population may aggravate or mediate generic factors, and further research should investigate whether a combination of these factors actually predict worse adherence in clinical populations.

Lastly, we were not able to analyse any influence of adherence status on short-term treatment outcome. Given the large drop-out and the small treatment effects at post-test (Kok et al., 2014) we would not be able to reliably detect any effect of adherence on outcomes, should they be present.

Clinical implications

Many clinical trials have now established the effectiveness of guided Internet-based interventions. As a result, a number of countries in the world now adopt those interventions for reasons of patient preference (therapy can be performed where and whenever the patient chooses) and initial results suggest possible cost-effectiveness of these interventions (Donker et al., 2015). However, most effectiveness studies have been performed among patients recruited in the general population.

The current trial was performed among patients recruited in routine outpatient mental health care settings. Our findings indicate that the sustained uptake among these patients is limited. This might indicate that guided Internet interventions are less suitable in outpatient settings. However, since about a third of the patients did use the intervention, we feel that it is too soon to disregard guided Internet interventions in routine care. Although our results suggest that clinicians might have to be more cautious with more severely ill patients, ultimately we think that better integration in mental health care will produce better effects.

Some recommendations can be given based on the current study and previous research. When offering an Internet-based intervention to patients, baseline severity and age are important factors to consider. Given that baseline anxiety, as measured through the BAI, but not more specific fear (as measured through the FQ) was a significant predictor of adherence, there is a possibility that non-adherence may be influenced more by general anxiety than specific anxiety. It is quite possible that the challenge of having to perform in-vivo exposure was not the decisive reason for patients to discontinue this intervention. However, little is currently known about how and why anxiety may influence non-adherence to interventions.

Although it is often assumed that younger patients would have a natural affinity for Internet-based programmes, the current study shows that it is older patients that are more likely to actually keep using the programme long enough to probably be beneficial. Again, it is unclear why older patients may persist in using an intervention, and this finding may be confounded with variables not measure in the current study, such as cultural factors, duration of illness or individual attitudes to psychotherapy.

Other factors to consider when offering these interventions to waitlisted patients, are that patients should ideally be guided by the therapist who performed the intake interview and carried out the treatment (rather than being coached by researchers outside of routine care) to improve a sense of continuity in treatment. This would likely increase the accountability to the therapist (Mohr et al., 2011), Alternatively, guided Internet interventions might be blended with face-to-face treatments, alternating sessions between FtF sessions and Internet-based sessions, which might increase motivation. Although number of such ‘blended’ interventions are currently being trialled (e.g., Kooistra et al., 2016, Romijn et al., 2015), clients and clinicians at present seem to prefer different ratios of online versus online content (van der Vaart et al., 2014). Furthermore, assessment of patient expectations (Greenberg et al., 2006), preferences (Beattie et al., 2009, Boettcher et al., 2013) and intrinsic motivation (Mohr et al., 2011) should be assessed carefully. Finally, treatment credibility can be discussed with the patient (El Alaoui et al., 2015).

Future research

A number of issues put forward in this discussion suggest areas of future research. Firstly, there is the myriad ways to define and measure adherence limit the cross-study generalisability of results (Kelders et al., 2012). Even if there were a perfect measure of adherence, it would be difficult to interpret these data as there is very probably no linear relationship between adherence and outcome. Notwithstanding, consensus or guidelines on the standardisation of adherence measurement would greatly improve further research. Additionally, reporting of adherence metrics should also be standardised, preferably following the proposed eHealth extension of the CONSORT reporting guidelines (Eysenbach, 2011). Analogous to this, there is currently little agreement on how to reliably assess patient expectancy of the effect of psychological interventions (Constantino et al., 2012). Moreover, to our knowledge, none of the proposed measures have been adapted and validated to be used for Internet-based intervention. Future research should aim to fill this gap.

Additionally, more data on the acceptability, feasibility and effectiveness of guided Internet interventions in outpatient mental health care settings is needed. Preferably, future research should incorporate at least the demographic and clinical predictors of adherence as suggested earlier, including treatment expectancy if a suitable instrument can be adapted or developed. In the future, greater access to full datasets from studies will allow us to perform individual patient data analyses (e.g., Karyotaki et al., 2015) or network meta-analyses (Leucht et al., 2016) that allow for greater statistical power to pinpoint predictors of non-adherence. Moreover, these large datasets may help to model the proposed type of relationship between adherence and treatment outcome (linear, quadratic, etc.).

Finally, the association between non-adherence to the intervention and not completing post-test measurements again highlights the issues with high drop-out, as previously noted (Cunningham, 2014).

Conclusions

In short, outpatient clinics may do well to consider baseline severity, age, but also intrinsic motivation, expectations when assessing outpatients for eligibility to use an Internet intervention. In all, we would like to stress that our results need not necessarily mean that it is unfeasible or ineffectual to offer the current intervention to patients in outpatient settings. After all, initial uptake of the intervention was good, and about a third of the patients adhered. In a shared-decision model, patient and therapist should take into account patient preferences and possible personal and circumstantial barriers to (continue) using Internet-based intervention.