Text based internet intervention of Binge Eating Disorder (BED): Words per message is associated with treatment adherence.

Background: Some evidence suggests that in internet-based cognitive behavioral therapy (iCBT) the likelihood of adherence is increased when patients write longer messages to the therapist in the program. This association has not previously been investigated in iCBT for Binge Eating Disorder (BED). Objective: In this study, we hypothesized that the number of words written by patients with mild to moderate BED was associated with increased likelihood of treatment completion in a text-based iCBT program. Material and methods: We compared 143 BED patients (92 completers and 51 non-completers) on the number of messages and words written to their therapist during the treatment.
Results: Completers wrote significantly more words per message (words/message) than non-completers. The results remained significant after controlling for gender, age, educational level, marital status, children, source of income and intake measures of BED, BMI and depression symptoms (Wald = 14.48, p < .001). The odds ratio of completion increased by 1.5% for each additional word patients wrote per message (OR = 1.015). The model showed a 72.4% classification accuracy, and an optimal cut-off point of 68.99 words/message for differentiating completers and non-completers. The model accurately identified 80.9% of completers (sensitivity) and 54.9% of non-completers (specificity). Conclusions: The number of words/message patients write may have important implications for ascertaining likelihood of adherence and improving adherence rates. From a clinical perspective, therapists should encourage patients to use the option of writing messages to the therapist. Words/message may prove to be a transdiagnostic predictor of treatment adherence in text based iCBT.

Introduction

Binge Eating Disorder (BED) is an eating disorder characterized by recurring episodes of binge eating, where the person feels a lack of control over their eating and is markedly distressed by the symptoms (American Psychiatric Association [DSM ). BED is associated with at least three out of five symptoms of binge eating: (1) eating faster than normal; (2) eating until uncomfortably full; (3) eating when not feeling hungry; (4) eating alone; and (5) feeling disgusted, depressed or guilty afterwards. Unlike bulimia nervosa, BED is not associated with compensatory behaviors such as vomiting.

Internet-based interventions are efficient in treating mental health disorders, including eating disorders (Bauer and Moessner, 2013; Fairburn and Murphy, 2015; Melioli et al., 2016). Text based internet cognitive behavioral therapy (iCBT) has proven effective in treating BED (Jensen et al., 2020; Wyssen et al., 2021). Although iCBT is effective, some studies using text based internet interventions report high dropout rates (Beintner et al., 2020; Puls et al., 2020). Therefore, it is essential to identify predictors of completion to improve adherence rates. While some studies have examined the predictors of adherence and treatment effect in conventional treatment of eating disorders (Fassino et al., 2009; Vall and Wade, 2015), little is currently known about predictors of text based internet interventions.

Baseline measures (e.g., pre-test) show poor or mixed performance in identifying adherence and treatment effects in the iCBT literature (Bremer et al., 2018). Therefore, interest in process variables describing how patients respond to treatment during treatment have received increasing interest. For instance, early symptom improvement during treatment reliably predicts adherence and treatment outcome in iCBT among adults suffering from depression and anxiety (Lutz et al., 2017; Schibbye et al., 2014). Furthermore, early detection of reduced treatment response can help adjust treatment interventions, resulting in a better treatment outcome (Forsell et al., 2020; Forsell et al., 2019). For instance, Forsell et al. (2019) identified patients at risk of dropping out of an internet CBT program for insomnia. At-risk patients were randomly assigned to either a “step-up” condition with increased therapist contact or a “continue” condition with “treatment as usual”. At-risk patients in the step-up condition had treatment effects comparable to patients who were not at risk of dropping out of treatment, while at-risk patients in the “continue” condition showed significantly lower treatment effect.

Compared to conventional CBT, it is easy to assess objective measures of text based iCBT (Manwaring et al., 2008). In a recent randomized clinical trial using text based iCBT for BED, Puls et al. (2020) analyzed measures of both objective adherence (e.g., number of messages exchanged with therapists) and subjective adherence (self-report measures). The authors found that only objective adherence significantly predicted dropout.

Other objective measures such as the number of words and messages may also have the potential to predict treatment adherence and effect. Wallert et al. (2018) found that the number of written words in homework assignments predicted treatment adherence in a text based internet intervention of symptoms of depression and anxiety after myocardial infarction. Van der Zanden et al. (2014) found that the number of written words predicted treatment adherence in an internet-based intervention of depression, and that fewer discrepancy words (e.g., “should”) at baseline predicted a greater level of mastery after treatment. These studies suggest that patients who write more words in internet-based interventions have increased likelihood of treatment adherence. In sum, objective treatment measures may reliably predict treatment adherence, and may have transdiagnostic properties. However, currently little is known about the association between the number of words used by patients and treatment outcome in iCBT for BED.

The aim of the current study was to investigate the association between the number of words per message and adherence in a text based iCBT program for BED. We hypothesized that the number of words per message (words/message) would be positively associated with treatment-completion.

Material and methods

Design and ethics

Patients enrolled in a 12-session text based iCBT treatment program called “internet treatment of Binge Eating Disorder” (iBED), where they received weekly written support from a psychologist. The program is hosted on a digital treatment platform (Minddistrict). iBED consists of written psychoeducation, and therapeutic exercises on goal-setting, stable eating patterns, identification of binge eating triggers, problem-solving skills, new coping strategies and relapse prevention. The patients complete exercises and receive written therapist support on each exercise before progressing to the next session. Furthermore, the program has a chat function, where patients can write with their therapist. There is no word limit in the chat function and patients are encouraged to use this option to establish an alliance between patient and therapist. The chat function is asynchronous and the patient is informed that the therapist will respond to their message within seven days.

Patients can work on the sessions whenever it fits their schedule. Many patients emphasized this as a primary motivation for seeking online treatment, because it enabled them to fit the treatment program into their work or family schedule. Patients were encouraged to use the voluntary chat function in addition to completing the obligatory sessions. Patients often used the voluntary chat function to follow up on feedback from the obligatory sessions or bring up questions. Sometimes patients used the voluntary chat function to share experiences of success or difficulty.

Patients completed questionnaires prior to entering treatment (pre-test) before the first session (called “session 0”), and at the last session (“session 11”, post-test). We only have pre-test data from non-completers, since they did not complete all 12 sessions. All participants gave digital written informed consent to the study. All procedures were carried out in accordance with the ethical standards of the institutional and national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. The IRB committee at the Southern Region of Denmark approved the study (20212000-57).

Participants

A total of 143 participants (92 completers and 51 non-completers) were included in the program. Completers finished all 12 sessions of the program, while non-completers were defined as completing anywhere from 0 to 11 sessions. Participants were recruited through open enrollment at a website hosted by the Centre for Telepsychiatry at the Mental Health Services in the Region of Southern Denmark. Participants applied for treatment by completing questionnaires about eating disorder symptoms and were included if they met the DSM-5 diagnostic criteria for mild to moderate BED and had no or mild co-occurring mental health disorders. The online program was part of a larger project group, where other psychiatric units treated patients with severe to extreme BED and moderate to severe comorbidity using face-to-face therapy. Cut-offs of severity of BED were assessed using the Binge Eating Disorder Questionnaire (BED-Q). The screening and treatment did not include face-to-face, video, telephone or chat contact, but was solely based on the participants' written answers.

Materials

Binge-Eating Disorder Questionnaire (BED-Q) (Jensen et al., 2020; Lichtenstein et al., 2021) is a 9-item questionnaire. Items 1–7 make up the sum score (0–35). These items match the DSM diagnostic criteria: (1) eating a large amount of food within a short time (2 h); (2) losing control over one's eating; (3) eating faster than normally; (4) eating until uncomfortably full; (5) eating without being hungry; (6) eating alone; and (7) experiencing negative feelings after overeating.

Items 1–7 is rated as follows: 0 = no (none/week); 1 = <1/week; 2 = 1–3/week; 3 = 4–7/week; 4 = 8–13/week; 5 = >13/week). The sum score is interpreted as: 0 = no symptoms; 1–9 = subclinical symptoms of BED; 10–14 = mild BED; 15–21 = moderate BED; 22–28 = severe BED; 29–35 = extremely severe BED. The present sample included patients with mild to moderate BED symptoms, corresponding to a score between 10 and 21. Item 8 controls for compensatory behaviors such as self-induced vomiting, while item 9 assesses whether binges are experienced as distressing.

The Major Depression Inventory (MDI) (Bech et al., 2001; Cuijpers et al., 2007) was used to screen for depressive symptoms. The MDI has 10 items that are rated on a 6-point Likert scale from 0 to 5. The total score ranges from 0 to 50, with higher scores reflecting more symptoms of depression. Recommended cut-off points are 21 for mild depression, 26 for moderate depression, and 31 for severe depression. We computed a dichotomized depression variable to estimate the prevalence of “risk of depression”: nonsymptomatic/few symptoms (total score < 26) and symptomatic/severe symptoms (total score ≥ 26).

Words were defined as the total number of words patients wrote in the chat function with their therapist. This did not include obligatory exercises or therapist responses. Number of messages were defined as the total number of messages patients sent to their therapist. Number of messages did not include the obligatory exercises or messages sent from therapists. Messages sent from therapists were excluded due to confounders such as reminders (therapists reminding patients to complete sessions) and quotes (therapists inserting quotes from patients' previous answers to address a point or a question). To account for the possible confound of completers writing more simply because they stayed longer in the program, we calculated the number of words per message as: words/(messages +1). We added a constant of 1 in the denominator to avoid a possible division of zero among patients, who did not submit any messages.

Statistics

Data were analyzed using Statistical Package for the Social Sciences (SPSS) version 28. We used independent sample t-tests to test for group differences between completers and non-completers on age and intake scores of BED-Q, BMI and MDI. We used Chi-squares to test for group differences on gender, educational level, source of income, children and marital status. Three outliers among completers were qualitatively identified, who wrote more than 400 words/message. The outliers were removed.

We used binary logistic regression analysis to investigate the relationship between treatment completion and the number of words per message (words/message) among BED patients. We used group (completers vs. non-completers) as the dependent variable and words/message as the covariate, while controlling for gender, age, educational level, marital status, children, source of income and intake measures of BED, BMI and MDI as possible confounding variables.

We used a receiver operating characteristic (ROC) curve analysis to determine the classification accuracy, while a Kolmogorov-Smirnov analysis was used to estimate an optimal cut-off for differentiating completers and non-completers, and to calculate sensitivity and specificity using the cut-off.

Finally, we used Spearman correlations to measure correlations between words/message and changes in BED symptoms following treatment. Only completers were included in this analysis, as we did not have post-treatment measures from non-completers.

Results

Completers and non-completers did not differ significantly at intake on gender distribution, age, marital status, whether they had children, symptom severity of Binge Eating Disorder (BED-Q) or BMI, see Table 1. However, they did differ significantly on educational level (X2 = 3.97, p < .05) and depression scores (t = 2.09, p < .05). Completers and non-completers also differed significantly on the total number of words (t = 4.32, p < .001), messages (t = 6.20, p < .001) and words/message (t = 5.09, p < .001). Completers finished all 12 sessions, while non-completers finished an average of 2.73 sessions (SD = 1.73, range = 2–9). Patients spent on average 136.38 days (SD = 74.99) in the program, or approximately 20 weeks.

Table 1

Demographics and health characteristics of completers and non-completers.

	Completers(n = 89)		Non-completers(n = 51)		Χ2/t Sig. level
	Mean/N	St.D./%	Mean/N	St.D./%
Gendera
Male	10	11,2%	7	14,0%
Female	79	88,8%	43	86,0%
Age, mean (SD)	40,71	12,34	37,80	10,96
Marital statusa
Relationship, married	56	62,9%	32	64,0%
Single, divorced	33	37,1%	18	36,0%
Has childrena	67	64,0%	29	58,0%
Educationa					<0,05
Lower education	32	36,0%	26	52,0%
Higher education	57	64,0%	24	48,0%
Primary income
Job/Salary	58	65,2%	31	60,8%
Other	31	34,8%	20	39,2%
Binge Eating Disorder – Q (BED-Q)a	16,80	3,21	17,26	2,82
Body Mass Index (BMI)a	36,57	9,23	39,21	10,93
Major Depression Inventory (MDI)a	21,73	8,01	24,90	9,09	<0,05
Total Words	1721,45	1419,57	628,47	777,91	<0,001
Total Messages	12,08	7,70	5,47	5,30	<0,001
Words/message	128,43	77,52	72,01	59,91	<0,001

1 missing from non-completers.

The binary logistic regression analysis showed that completers wrote significantly more words/message than non-completers when controlling for gender, age, educational level, marital status, children, source of income and intake measures of BED, BMI and MDI (Wald = 14.48, p < .001). The odds ratio showed that for each additional word participants wrote per message, they increased their odds of completing by 1.5% (OR = 1.015), see Table 2. It can also be seen from Table 2 that patients with larger BMIs were less likely to complete treatment (Wald = 4.10, p < .05) when adjusting for number of words/message and potential confounders. Each additional BMI point reduced the odds of completing by 4.3%.

Table 2

Association between completion status and words per message.

Variables	B	df	Sig.	Exp(B)
Words/messages	0,015	1	<0,001	1,015
BMI, intake	−0,043	1	0,043	0,957
BED-Q, intake	−0,051	1	0,466	0,950
MDI, intake	−0,022	1	0,405	0,978
Age	0,023	1	0,213	1,024
Gender	0,093	1	0,878	1,097
Education	0,155	1	0,726	1,168
Income type	0,356	1	0,432	1,428
Civil Status	0,138	1	0,784	1,148
Children	0,278	1	0,594	1,320
Constant	0,803	1	0,609	2,231

The ROC analysis showed that words/message had a 72.4% CI [63.2, 81.4] classification accuracy. The Kolmogorov-Smirnov metric showed that the optimal cut-off for differentiating completers and non-completers was 68.99 words/message, which accurately identified 80.9% (sensitivity) of completers and 54.9% (specificity) of non-completers.

Finally, Spearmans correlations showed no significant correlation between words/message and changes in BED-Q or MDI scores among completers. On the individual items on BED-Q and MDI only changes in BED-Q sub-scale item 3 (eating faster: r = 0.30, p = .005) and item 6 (eating alone: r = 0.27, p = .01) were significantly correlated with words/message among completers.

Discussion

In this study we found that: (1) completers wrote significantly more words/message than non-completers; and (2) that these differences were not confounded by other variables. This is consistent with previous findings that an increased number of words written to the clinician is associated with treatment adherence (Van der Zanden et al., 2014; Wallert et al., 2018). Our results support the growing body of literature suggesting that iCBT is well suited for using objective measures of therapeutic interaction to predict treatment adherence.

Words/message had a 72.4% classification accuracy, suggesting that it accurately identified almost three out of four completers and non-completers. Studies of other diagnostic groups also find that the number of words is associated with treatment adherence (Van der Zanden et al., 2014; Wallert et al., 2018). The number of words, messages and words/message patients write may therefore be a good predictor of treatment adherence. Further, they may be particularly suitable for transdiagnostic prediction of treatment adherence, since the number of words and messages are easily generalizable and objective, compared with subjective measures or diagnosis specific content (e.g., what patients write, or whether they mention specific topics like “anxiety” during anxiety treatment). Future studies should investigate the role of the number of words or keystrokes in online CBT in other patient populations.

Although words/message in our study had a high sensitivity rate identifying four out of five completers (80.9%), the specificity was only 54.9%. This means that 45.1% of non-completers were false positives, who wrote more than the Kolmogorov-Smirnov metric optimal cut-off of 68.99 words/message. People may drop out of treatment for many different reasons other than a lack of writing in the program such as a sudden job loss, divorce, a family member falling ill, having to pass a final examination, pregnancy, moving and so on. Anecdotal observations suggest that such obstacles often prohibit program completion even though patients are active in the program. While words/message may be a good sensitivity measure of adherence, other measures such as life stressors might be better specificity measures for identifying people at risk for dropping out.

Another important point is that while words/message was a good predictor of treatment adherence, it was not a good predictor of treatment effect. In other words, while our data suggest that writing more is associated with completing the program, writing more is not associated with better treatment effect. Our previous study (Jensen et al., 2020) shows that the iBED program generally has a good treatment effect among completers measured as improvement in BED symptoms (BED-Q) and depressive symptoms (MDI). Hence, while words/message might reflect an engagement in treatment, it may not reflect a change in symptom level. For instance, a patient might experience a worsening in symptoms, prompting them to write to their therapist to seek help. Conversely, another patient might experience symptom improvements, and want to share their success with the therapist. Both instances might be associated with a higher number of words/message and thereby increased likelihood of completing treatment, but for very different reasons.

Several factors could explain why completers wrote more words/message. One possibility is that completers wrote longer messages because they developed an alliance with the therapist; another is that they engaged more with the program. While such hypotheses are plausible, we cannot firmly conclude on such questions in the present study, since we did not measure patient alliance with the therapist or patient engagement in the program.

This study has several limitations. First, we only investigated the number of words written by patients. We did not investigate, e.g., the length of words or the content of the messages. However, further studies may reveal differences in outcomes related to, e.g., the use of longer or infrequent words or sentence structure (e.g., active vs passive language). For instance, the study by Van der Zanden et al. (2014) found that fewer discrepancy words (e.g., “should”) at baseline predicted a greater level of mastery after treatment. While such studies are important for understanding the process variables of text based iCBT, they may be difficult to reproduce transdiagnostically, unless researchers have a shared understanding for measuring, e.g., discrepancy and mastery. Further analyses of the content of messages are needed to determine the role of content in relation to adherence.

Second, the Kolmogorov-Smirnov metric optimal cut-off at 68.99 words/message needs to be replicated both within BED populations and across diagnoses, treatment programs and languages. While our data support the growing literature suggesting that writing more is associated with treatment adherence, we currently know very little about how much and how often therapists and patients should interact to increase the likelihood of treatment completion.

Third, in this study we only looked at voluntary conversations between patients and therapists. While patients were obligated to complete the treatment sessions, they were free to decide whether to engage in conversations with their therapist. Thus, we do not know how patients' responses to obligatory sessions would affect our current results if included. Indeed, the rationale for this study originated from clinical observations that patients who engaged in voluntary conversations with their therapist tended to complete the program at higher rates. Our data support this observation, and therefore, the findings may have real life implications for therapists. For instance, therapists may be able to better ascertain the likelihood of treatment completion and help increase adherence rates by encouraging patients to write more, e.g., by prompting questions or elaborations.

Conclusion

Our findings suggest that BED patients who write more words per message with their therapist are more likely to complete text based iCBT for BED. Words per message may have several important implications for therapists such as ascertaining the likelihood of patient treatment completion and help increase adherence rates by encouraging patients to write more. Words per message may also prove to be a transdiagnostic predictor of treatment adherence in text based iCBT.

Data availability statement

The data supporting the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy and/or ethical restrictions.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.