Motivation and Treatment Credibility Predicts Dropout, Treatment Adherence, and Clinical Outcomes in an Internet-Based Cognitive Behavioral Relaxation Program: A Randomized Controlled Trial.

BACKGROUND: In previous research, variables such as age, education, treatment credibility, and therapeutic alliance have shown to affect patients' treatment adherence and outcome in Internet-based psychotherapy. A more detailed understanding of how such variables are associated with different measures of adherence and clinical outcomes may help in designing more effective online therapy.
OBJECTIVE: The aims of this study were to investigate demographical, psychological, and treatment-specific variables that could predict dropout, treatment adherence, and treatment outcomes in a study of online relaxation for mild to moderate stress symptoms.
METHODS: Participant dropout and attrition as well as data from self-report instruments completed before, during, and after the online relaxation program were analyzed. Multiple linear and logistical regression analyses were conducted to predict early dropout, overall attrition, online treatment progress, number of registered relaxation exercises, posttreatment symptom levels, and reliable improvement.
RESULTS: Dropout was significantly predicted by treatment credibility, whereas overall attrition was associated with reporting a focus on immediate consequences and experiencing a low level of intrinsic motivation for the treatment. Treatment progress was predicted by education level and treatment credibility, whereas number of registered relaxation exercises was associated with experiencing intrinsic motivation for the treatment. Posttreatment stress symptoms were positively predicted by feeling external pressure to participate in the treatment and negatively predicted by treatment credibility. Reporting reliable symptom improvement after treatment was predicted by treatment credibility and therapeutic bond.
CONCLUSIONS: This study confirmed that treatment credibility and a good working alliance are factors associated with successful Internet-based psychotherapy. Further, the study showed that measuring adherence in different ways provides somewhat different results, which underscore the importance of carefully defining treatment adherence in psychotherapy research. Lastly, the results suggest that finding the treatment interesting and engaging may help patients carry through with the intervention and complete prescribed assignments, a result that may help guide the design of future interventions. TRIAL REGISTRATION: Clinicaltrials.gov NCT02535598; http://clinicaltrials.gov/ct2/show/NCT02535598 (Archived by WebCite at http://www.webcitation.org/6fl38ms7y).

RCT Entities:   Population Interventions Outcomes

Introduction

It is well established that therapist-guided Internet-based cognitive behavior therapy (ICBT) and other behavioral interventions can be effective in improving psychological symptoms and well-being [1]. However, not all patients are helped and one reason for this could be that treatment adherence may be somewhat lower in Internet-based psychotherapy compared to traditional face-to-face therapy [2]. This difference may be important because adherence is associated with treatment outcome in both face-to-face and Internet-based therapy [3]. Little is known about factors that affect treatment adherence to Internet-based interventions, but it is clear that therapist online support improves adherence substantially [4]. With the growth of Internet-based interventions, there has been an increased interest in trying to find variables that affect treatment adherence and outcomes [5-7]. However, the association between treatment adherence and outcome is complex and in previous studies it has been shown that increased adherence does not automatically lead to better treatment outcomes [8,9]. It is also unknown to what degree treatment adherence can be affected by therapist behaviors because background variables such as education and personality traits affect patients’ ability to carry through and benefit from ICBT [10]. In order to develop and evaluate effective Internet-based interventions, it is important to further investigate factors that are associated with adherence and outcome [11,12].

Adherence to Internet-based treatments can be operationalized in different ways, including working with the intervention by reading texts and watching video clips on a webpage or adhering to the behavioral prescriptions in everyday life (eg, by completing homework assignments) [6,13]. In other words, one may separate the administration of the intervention (ie, providing the treatment content) and the proposed behavioral mechanisms (ie, everyday behavior change) that more directly lead to symptom change [14-16]. In Internet interventions, therapist support is hypothesized to help participants both to work with the online material and to complete behavioral assignments. The effect of human support can be explained by operant principles and by models such as self-determination theory (SDT), which is a framework that describes motivation and behavior change in congruence with operant conditioning principles [17]. According to SDT, behavior is governed by external motivation (behavior governed by external factors or consequences) and four forms of internal motivation: introjected motivation (behavior governed by reduced negative affect), identified and integrated motivation (behavior governed by goals or values), and intrinsic motivation (behavior that is in itself rewarding).

Self-determination theory has been applied to health behaviors and it seems that different forms of motivation may have different effects on treatment adherence [18]. For example, behaviors that are governed by intrinsic, identified, or integrated motivation are more likely to occur and to be enduring over time compared to behaviors that are governed by external motivation. Because some behavioral prescriptions in psychotherapy, such as exposure exercises, are not intrinsically rewarding, patients are helped by therapists to instead employ identified or integrated motivation to facilitate behavior change [19]. That therapists’ follow-up on assignments may also be an example where introjected or external motivation may be beneficiary for treatment adherence.

Previous studies have shown that adherence to an intervention, as well as treatment outcome, may be influenced by demographical variables, such as gender, age, and education [20-22]. Another background factor that may be important in this context is the ability to focus on either future or present consequences [23-25]. According to Zimbardo’s model of behavior and postponed reward, there are three types of time perspectives: future, hedonistic, and fatalistic. A future-oriented time perspective is goal-driven and has been associated with health behaviors, whereas the opposite is true for people with the hedonistic time perspective who are highly influenced by immediate consequences and stimulation [26]. Lastly, a person with a fatalistic time perspective may show a pattern of health-destructive behaviors [27]. Thus, time perspective is closely associated with executive functioning and the ability to postpone reinforcement and to act based on future goals rather than present needs [28-30]. Therefore, it may be an important construct that can help explain some of the variance in participants’ adherence to psychological interventions.

Finally, patients’ belief in a treatment may have a very strong effect on the outcome as seen in studies on placebo effects [31]. However, the credibility of a treatment may also explain why a patient engages in treatment and completes assignments that are not intrinsically rewarding. Thus, belief may be associated with internal motivation and can have an indirect but real effect on outcome in the case of psychotherapy [22,32].

This study aimed at investigating variables that may predict three different types of outcome variables in ICBT: (1) dropout and attrition from treatment, (2) treatment adherence, and (3) clinical outcomes. More specifically, the aim was to assess the predictive value of different background variables as well as the variables time perspective, treatment credibility, motivation, and therapeutic bond on early dropout, attrition, treatment progress, adherence to behavioral prescriptions, posttreatment symptoms, and reliable improvement in a previously conducted study of a brief online stress management treatment. A secondary aim was to investigate whether treatment adherence could predict clinical outcomes.

Methods

Design, Procedure, and Participants

Data for this study were retrieved from a previously conducted study on Internet-based relaxation training for people with mild to moderate stress and anxiety symptoms [9]. Participants were recruited from the general population primarily by online advertisement. Inclusion criterion was self-reported stress symptoms, whereas the exclusion criteria were elevated symptom levels of anxiety or depression or other severe psychological or somatic problems that warranted immediate care, younger than 18 years of age, insufficient mastery of the Swedish language, or lacking daily access to computer, Internet, and cell phone. In the study, a total of 162 included participants were presented with an online intervention and were randomized to either normal or enhanced treatment presentation and either normal or enhanced therapist support in a full factorial design [9]. In the normal treatment condition, the treatment was presented as black-and-white text files, whereas in the enhanced presentation condition, the treatment was presented in full color and also in video format. In the normal support condition, participants received support from trained therapists at least once a week, whereas in the enhanced support condition, participants received daily support from therapists based on motivational interviewing. The intervention consisted of a brief 4-week program of applied relaxation similar to what has been used and empirically tested in previous studies [33]. Each week of the program included webpage material, assigned relaxation exercises, and contact with a therapist via the webpage. All participants were asked to answer a battery of self-report instruments before, during, and after the intervention; of the 162 participants, 157 had complete data from the pre- and midtreatment assessments, whereas 96 participants had complete data also from the posttreatment assessment.

Outcome Variables

Early dropout (yes/no) was assessed by counting the number of participants who discontinued the study before completing the first week of the treatment program; attrition (yes/no) was assessed by counting the total number of participants who discontinued the study before the posttreatment assessment. Treatment adherence was divided into the two variables treatment progress and registered exercises, both on continuous scales. Treatment progress was assessed by measuring how much of the Web-based treatment material (eg, texts, examples, assignments) each participant accessed before dropping out or completing the treatment. Because the treatment consisted of 25 such items, this variable ranged from zero (not accessed the treatment) to 25 (accessed the whole treatment). Registered exercises were measured by the mean number of prescribed exercises of applied relaxation that the participant had registered on the webpage each week. The weekly number of registered exercises ranged from zero (not completed any exercises) to 14 (completed all prescribed exercises). As a measure of stress symptoms and treatment outcome, the Perceived Stress Scale (PSS) [34] was chosen. The short version of the PSS used in this study contains 14 items scored on a scale between zero and 4, which provides a total score between zero and 56 with a higher score indicating more symptoms of stress. In the current study, the PSS had an internal reliability of α=.72. In previous studies, the PSS has showed adequate psychometric properties [35]. To assess reliable improvement (yes/no), the Reliable Change Index for PSS was calculated (described subsequently).

Predictor Variables

The predictor variables consisted of the background variables age, gender, level of education (primary/secondary/university), occupation (student/unemployed/employed/sick leave/retired), and computer expertise (low/intermediate/high). In order to facilitate interpretation, education and occupation were transformed into dichotomous variables (nonuniversity vs university; employed/student/retired vs unemployed/sick leave). Four psychological predictor variables were collected by self-report instruments at baseline: time perspective, treatment credibility, and internal and external motivation. Intrinsic motivation and therapeutic bond were measured at midtreatment and stress symptoms were measured both at baseline as a predictor variable and at postmeasurement as an outcome variable. Because a previous analysis [9] had shown that enhanced support was associated with higher adherence to the online treatment program, treatment condition (normal vs enhanced therapeutic support) was also included as a predictor variable in the analyses. No other independent variable was significantly associated with any outcome variables.

Time perspective was measured with the Zimbardo Time Perspective Inventory Short Form (ZTPI) [36,37]. The version of the ZTPI used in this study has three subscales—future, hedonistic, and fatalistic—and comprise 22 items scored on a scale from 1 to 5 [38]. The ZTPI has been evaluated for research in health psychology and has shown adequate psychometric properties [39]. The ZTPI subscales had internal reliabilities of α=.69-.73 in this study.

Treatment credibility was measured with the Treatment Credibility Scale (TCS), which is often used in studies of Internet interventions and is an adaptation from Borkovec and Nau [40]. The TCS consists of five items scored on a scale from 1 to 10 with a higher score indicating more trust in the current treatment. The TCS had an internal reliability of α=.83 in this study.

Internal (ie, identified and integrated motivation) and external motivation were measured with the Treatment Self-Regulation Questionnaire (TSRQ) [41]. The TSRQ consists of two subscales—internal motivation (IM) and external motivation (EM)—each measured with six items that were adapted to suit the Internet intervention used in this study. Each subscale provides a score between 6 and 42 with a higher score corresponding to a higher degree of motivation. The TSRQ has been used in studies on motivation and health behaviors and has shown adequate psychometric properties [42]. In the present study, the TSRQ-IM and TSRQ-EM had internal reliabilities of α=.70 and α=.68, respectively.

Intrinsic motivation was measured with the Intrinsic Motivation Inventory (IMI) [43]. The IMI aims at measuring how pleasant, interesting, and meaningful a task is perceived and has nine items scored on a scale between 1 and 7 which provides a total score of 9 to 63 with a higher score indicating a more positive experience of the task. Due to mixed findings concerning the factor structure of the IMI, only the total score was used [44]. The IMI had α=.71 in this study.

Therapist bond was measured with the Working Alliance Questionnaire Short Form (WAI) [45]. The WAI has been widely used in psychotherapy research and has shown adequate psychometric properties [46]. The short form WAI consists of three subscales—Goal, Task, and Bond—each with a score between 4 and 28 where a higher score equals a higher degree of therapeutic alliance. The WAI total scale had an internal reliability of α=.73 in this study.

Analysis

Before analysis, data were screened for outliers and normality, linearity, and homoscedasticity were evaluated by scrutinizing the residual scatterplots between predicted variables and errors of prediction and found adequate. Because subscales were entered into the analyses, multicollinearity was assessed by analyzing the variance inflation factor for each predictor variable and found to be nonproblematic.

Reliable improvement was computed by dividing the difference between the pretreatment and posttreatment scores by the standard error of the difference between the two scores. If the Reliable Change Index was greater than 1.96, a change of that magnitude would not be expected due to the unreliability of the measure [47]. Using this procedure, the reliable improvement criterion for PSS was a change score of 10 or more in this study.

Modeled after deGraaf et al [48], bivariate regression analysis was first used to identify candidate (P<.10) predictor variables for each outcome variable. All identified predictor variables were included in subsequent multiple regression analyses using a backward deletion process for each outcome variable. Logistic regression was used for the dichotomous outcome variables dropout and attrition and linear regression was used for continuous outcome variables. Cox-Snell R 2 and Nagelkerke R 2 were used as a measure of overall model fit in the logistic regression analyses and R 2 was used in the linear regression analyses. Because some of the variables had distributions that deviated somewhat from normality, the final regression models were confirmed using robust regression analyses with bootstrap and bias correction. The sample size of 157 was deemed adequate for regression analysis of a maximum of eight predictor variables for each outcome variable except stress symptoms and reliable improvement for which the sample size of 96 was deemed adequate for six predictor variables. Single missing values (n<1%) were imputed using expectation-maximization estimates. A P value of .05 was considered the threshold for statistical significance if not stated otherwise; exact P values were reported for the final analyses.

Results

Of the 157 participants in this study, 115 (73.2%) were women and the mean age was 34.5 (SD 13.1) years. The background variables education, occupation, and computer expertise are shown in Table 1. A total of 39 of 157 (24.8%) participants dropped out before completing the first week of treatment; the total attrition was 61 (38.9%) participants at the postmeasurement. The mean treatment progress among all participants was 14.9 (SD 9.7) items out of 25 (60%) and the mean number of weekly exercises was 8.5 (SD 4.0) out of the prescribed 14 (61%).

Table 1

Background predictor variables (N=157).

Factor		n (%)
Education
	Primary	8 (5.1)
	Secondary	57 (36.3)
	University	92 (58.6)
Occupation
	Studying	43 (27.4)
	Unemployed	9 (5.7)
	Employed	81 (51.6)
	Sick leave	16 (10.2)
	Retired	8 (5.1)
Computer expertise
	Low	54 (34.4)
	Intermediate	51 (32.5)
	High	52 (33.1)

There were no significant differences for any of the predictor variables between the treatment groups of the original study. Of the background variables, only education and occupation were significant predictors for any outcome variable in the initial bivariate regression analyses. Of the self-reported psychological variables, the TSRQ-IM showed markedly higher standard deviation compared to other variables and it was the only variable that failed to significantly predict any outcome variable, so it was removed from further analyses. The results of these bivariate analyses for each outcome variable can be found in Multimedia Appendices 1 and 2.

Dropout and Attrition

The multivariate logistic regression analyses showed that early dropout could be significantly negatively predicted by the TCS (B=–0.14, χ2 1=10.5, P=.001), whereas total attrition to postmeasurement was predicted by baseline stress symptoms (B=0.08, χ2 1=3.2, P=.05), the ZTPI Hedonistic subscale (B=0.32, χ2 1=10.3, P=.001), and the IMI (B=–0.06, χ2 1=5.7, P=.02) (see Table 2). Early dropout was associated with a low belief in the treatment model, whereas dropout during the course of treatment was associated with having elevated stress symptoms, being more focused on the immediate consequences of behaviors, and finding the treatment uninteresting or unengaging.

Table 2

Significant predictor variables for early dropout and attrition after backward deletion (N=157).

Treatment dropouta		Cox-Snell R 2	Nagelkerke R 2	B (SE)	χ2 1	P	OR (95% CI)
Early dropout		.18	.39
	TCS			–0.14 (0.04)	10.5	.001	0.87 (0.80-0.95)
Attrition		.19	.28
	Baseline stress symptoms			0.08 (0.05)	3.2	.05	1.08 (1.00-1.18)
	ZTPI Hedonistic			0.32 (0.10)	10.3	.001	1.37 (1.13-1.66)
	IMI			–0.06 (0.03)	5.7	.02	0.94 (0.90-0.99)

aIMI: Intrinsic Motivation Inventory; TCS: Treatment Credibility Scale; ZTPI: Zimbardo Time Perspective Inventory.

Treatment Adherence

After controlling for level of support, treatment progress was positively predicted by level of education (beta=.24, t 153=2.33, P=.02) and by the TCS (beta=.35, t 153=3.36, P=.001), whereas registered exercises was significantly predicted only by the IMI (beta=.29, t 155=2.43, P=.02) (see Table 3). All other predictor variables were nonsignificant in the multiple regression analyses for treatment adherence. Accessing the online treatment material was associated with a priori belief in the treatment model, whereas complying with the prescribed homework assignments was associated with experiencing interest and engagement in the treatment.

Table 3

Significant predictor variables for treatment adherence after backward deletion (N=157).

Treatment adherencea		R 2	B (SE)	β	t (df)	P
Treatment progress		.36
	Enhanced support		2.84 (0.88)	.33	3.23 (153)	.002
	University education		1.95 (0.84)	.24	2.33 (153)	.02
	TCS		0.15 (0.04)	.35	3.36 (153)	.001
Registered exercises		.13
	IMI		0.10 (0.04)	.29	2.43 (155)	.02

aIMI: Intrinsic Motivation Inventory; TCS: Treatment Credibility Scale.

Treatment Outcome

Posttreatment stress symptoms were significantly and positively predicted by the baseline stress symptoms (beta=.47, t 91=4.43, P<.001) and by the TSRQ-EM (beta=.25, t 91=2.40, P=.02) while negatively predicted by the TCS (beta=–.28, t 91=3.53, P=.001) (see Table 4). Reliable improvement was positively predicted by baseline stress symptoms (B=0.11, χ2 1=4.5, P=.03), by the TCS (B=0.09, χ2 1=3.3, P=.05) and by the WAI (B=0.14, χ2 1=3.9, P=.049) (see Table 5). Reporting external pressure to complete the treatment was associated with worse treatment outcome, whereas a good therapeutic bond was associated with a substantial positive treatment effect. Treatment credibility predicted both overall symptom levels and substantial improvement.

Table 4

Significant predictor variables for post treatment stress symptoms after stepwise deletion (n=96).

Predictor variablea	R 2	B (SE)	β	t 91	P
Postmeasurement stress symptoms	.42
Baseline stress symptoms		0.53 (0.12)	.47	4.43	<.001
TSRQ-EM		0.48 (0.20)	.25	2.40	.02
TCS		–0.28 (0.08)	–.35	3.53	.001

aTCS: Treatment Credibility Scale; TSRQ-EM: Treatment Self-Regulation Questionnaire Extrinsic Motivation.

Table 5

Significant predictor variables for reliable improvement after stepwise deletion (n=96).

Predictor variablea	Cox-Snell R 2	Nagelkerke R 2	B (SE)	χ2 1	P	OR (95% CI)
Reliably improved	.32	.47
Baseline stress symptoms			0.11 (0.05)	4.5	.03	1.12 (1.01-1.25)
TCS			0.09 (0.05)	3.3	.05	1.10 (1.00-1.20)
WAI total			0.14 (0.07)	3.9	.049	1.15 (1.01-1.32)

aTCS: Treatment Credibility Scale; WAI: Working Alliance Inventory.

Associations Between Treatment Adherence and Outcomes

In bivariate regression analyses and after controlling for baseline PSS score, posttreatment stress symptoms was significantly negatively predicted by both treatment progress (beta=–.31, t 92=3.29, P=.001) and registered exercises (beta=–.21, t 92=1.98, P=.05). The same pattern was seen for reliable improvement, which was significantly predicted by both treatment progress (B=0.26, χ2 1=8.0, P=.005; OR 1.29, 95% CI 1.08-1.55) and registered exercises (B=0.20, χ2 1=7.5, P=.006; OR 1.23, 95% CI 1.06-1.42). Whether treatment effect was mediated through treatment adherence could not be further investigated in this study because the power calculations had not accounted for this type of analyses.

Discussion

Predicting Dropout, Attrition, and Adherence

The analyses showed that different ways of operationalizing (ie, measuring) dropout, treatment adherence, and treatment outcomes was related to somewhat different predictor variables, something that may be important to consider in psychotherapy process research. First, there was a difference in that early treatment dropout could be predicted by treatment credibility, whereas further attrition was predicted by a personality pattern of focusing of immediate consequences and by finding the treatment unengaging, two variables that are probably connected. People whose behavior is generally governed by immediate consequences and reward may find an online behavioral treatment largely consisting of abstract instructions and texts unsatisfying and boring. This pattern suggests that some people with low expectations of the treatment start the program, but very soon realize it does not suit them and quit. Whether these participants should receive more motivational support from therapists to stay in the online treatment or efforts should be made to guide them to other forms of treatment needs further investigation. Participants who dropped out later in the treatment reported a lower tendency to focus on future goals and also experienced the treatment as unrewarding. These results are in line with previous results showing that being able to focus on the future and to postpone rewards is associated with higher levels of education and health behaviors [23]. Although the treatment may be designed to be more interesting and engaging (eg, by employing media content or using features of gamification [49]), it may be difficult to counter the fact that most health behaviors rely on an ability to postpone rewards and a focus on future consequences. However, strategies such as repeated reminding of treatment goals and using everyday prompts may be alternatives for further investigation [50].

Treatment adherence showed a somewhat similar pattern to dropout with online treatment progress predicted by education and treatment credibility. Thus, working with the online material was associated with a probable familiarity of working with texts and thinking in abstract terms. Treatment credibility may correspond to a familiarity and interest in using the Internet for learning about health behaviors and perhaps previous positive experiences of online courses. Treatment credibility may also represent participants’ belief in the treatment model and specifically in using relaxation to target stress symptoms. Finally, the association between treatment credibility and treatment progress may represent the different reasons why people undergo Internet-based psychotherapy; for some it is a preferred choice, whereas for others it may be the only available option. Completion of homework assignments, similar to staying in the treatment program, was predicted by finding the treatment engaging. That the IMI was significantly associated with adherence in this study may be partly explained by the nature of relaxation exercises that may actually be pleasant in contrast to exposure exercises, for example. It is worth noting that the instrument used to measure intrinsic motivation, the IMI, is designed to measure different forms of intrinsic reward and comprise items concerning experiences of interest, enjoyment, and meaningfulness [41]. It may capture participants’ experience that the treatment suits them and their needs rather than the treatment being “fun.” Whether results from this study of relaxation exercises can be generalized to other types of assignments used in CBT needs further study. In short, the results from this study suggest that a high level of education and belief in the treatment model may be beneficiary for overcoming the effort needed to work with the online program, but participants who find the treatment suits them work with the prescribed assignments to a higher degree. However, few of the variables in this study could significantly predict adherence to the assignments, so there are probably other unknown variables that could help explain this behavior.

Predicting Treatment Outcomes

Among participants who did not dropout and remained in the study, treatment outcome, as measured by the posttreatment stress symptoms, was significantly positively predicted by baseline stress symptoms, but also by the TSRQ-EM measuring external motivation. External motivation corresponds to feeling pressured by others, mostly in a negative way, to complete tasks. In previous studies, external motivation has been associated with difficulties in sustaining behavior and a reliance on accountability [19]. That external motivation was associated with worse outcome in this study implies that people who complete the intervention because of feeling obliged to do so benefit less from the treatment. In a recent study of negative effects in online psychotherapy, some participants reported feeling pressured by their online therapist or the demands of the treatment schedule [51], but in this study, external motivation was measured at baseline and corresponded to the reasons why participants took part in the study. Why some participants reported external pressure is unknown and difficult to speculate about because this study was based on self-referral recruitment. It is important to note the different nature of psychotherapy compared to the workplace environment, where most of the research on motivation has been conducted. In studies of psychotherapy, results may rely more on participants being engaged and involved than in most workplace research. For example, unlike work tasks, psychotherapy aims at making participants generalize behaviors from the therapy situation to their everyday life, something that may be more probable if the patient experiences the treatment as meaningful. Also, the Internet format may deemphasize external motivation because of the perceived anonymity of both patients and participants [52,53]. Previous studies have shown that therapists’ prompts and reminders can have a relatively minor impact on treatment adherence among participants who may lack internal motivation [54]. Posttreatment stress symptoms were further negatively predicted by treatment credibility, an effect also seen in previous studies [21,22]. Belief in the treatment format and model seems to be a consistent predictor for positive outcome and may be used as a measure for identifying people for whom the online format and minimal therapist contact may be beneficiary. It is important to note that suggesting or persuading people to try Internet-based psychotherapy or a treatment model they do not believe in may be counterproductive, both given the effect seen for external motivation and the effect of treatment credibility on treatment outcome. However, given the positive results of many online behavioral interventions, it may be beneficiary to better inform skeptical participants about the treatment format (eg, by pretreatment vignettes or examples). After controlling for baseline stress symptoms, reliable improvement after the treatment program was predicted by treatment credibility, but also by therapeutic bond, an effect that has been seen in other studies as well [55]. Interestingly, this effect in this study was seen only among participants who improved substantially, further providing support for the notion that working alliance is as an important variable in online psychotherapy as in face-to-face therapy [56].

Identified Difficulties and Study Limitations

In this study, many of the proposed predictor variables could not significantly predict the outcome variables in the multivariate analyses. This suggests that several of the predictor variables covaried to a large degree; therefore, finding the best predictor variables is difficult. The multivariate analyses with backward deletion resulted in models with acceptable levels of model fit except for registered exercises (R 2=.13); so far, there is limited understanding of the processes that may be involved in following prescribed homework in ICBT. That personality traits associated with conscientiousness and delay discounting could predict carrying through with the treatment seemed reasonable, but it was somewhat surprising that working alliance could not. A strong working alliance is often associated with treatment adherence, but in this study this association was overshadowed by the impact of education and treatment credibility. The weak association between working alliance and treatment adherence was unexpected given that therapeutic alliance has shown to be strongly associated with adherence and outcome in several other studies of ICBT [57]. Finding the intervention engaging, interesting, and meaningful were crucial factors for participants in successful ICBT as seen in previous studies [58]. Although intrinsic motivation was a predictor for treatment adherence, it was not associated with treatment outcome; this was surprising given the association between treatment adherence and outcome. This study was not designed to investigate mediation effect; therefore; it was unfortunately underpowered to further analyze these processes.

This study has a number of limitations. First, several of the predictor and outcome variables were difficult to measure accurately. For example, the measurement of registered exercises was constructed to assess the treatment’s effective mechanism, but may have failed to fully capture changes in participants’ everyday behavior. The intervention encouraged participants to conduct relaxation exercises and highly engaged participants may have done so without registering on the webpage. Further, several of the self-report instruments have not been used in psychotherapy research before and their psychometric properties in this context are unknown. For example, the internal reliability of some questionnaires seemed to be somewhat lower in this study compared to previously reported figures. Second, a more complex design with repeated measurements of adherence could have been conducted to show causal mediation, but this would also have demanded a much larger sample size. Third, the lack of a control group or a face-to-face treatment condition limits the generalizability of the results. Therefore, whether the conclusions from this study are valid for other treatment modalities are unknown. Finally, and most importantly, there was large dropout (39%) at postmeasurement and although investigating dropout from Internet-based interventions was one of the aims of this study, it also meant loss of follow-up data. This also means that the conclusions regarding predictors of clinical outcome are only valid for participants who stay in the study or treatment, which limits the generalizability of the results. A more thorough analysis of participants who dropped out may have resulted in a better understanding of this group.

Conclusions

In conclusion, this study confirms the importance of treatment credibility and working alliance in Internet-based psychotherapy and also suggests that experiencing intrinsic reward from participating in the treatment may be important. In contrast, external pressure to try online therapy may be counterproductive and lead to worse outcomes. Apart from identifying people who believe in the online treatment format and continue to explore the best methods for online therapist support, it may also be valuable to further investigate what makes participants engage in an intervention and what features makes the intervention interesting and meaningful [55]. However, given that quite rudimentary online treatment programs seem to be effective, it is possible that Internet-based interventions simply suit some participants better than others and that the focus should be on identifying these participants early [59]. Future studies may investigate both what makes treatments engaging while recognizing that different participants may have different requests and wishes for an intervention [60]. A good match between participant expectations and needs on the one hand and the intervention content and design on the other hand may be one of the reasons clinical assessment before treatment and tailoring treatment to better suit different patients may be beneficiary for treatment outcomes in Internet-based interventions [61]. Believing in the online treatment format and finding working with the online program rewarding may be two aspects of a process that could be investigated further to develop even more effective treatment programs.