Feasibility of a Software agent providing a brief Intervention for Self-help to Uplift psychological wellbeing ("SISU"). A single-group pretest-posttest trial investigating the potential of SISU to act as therapeutic agent.

BACKGROUND: Software agents are computer-programs that conduct conversations with a human. The present study evaluates the feasibility of the software agent "SISU" aiming to uplift psychological wellbeing.
Methods: Within a one-group pretest-posttest trial, N = 30 German-speaking participants were recruited. Assessments took place before (t1), during (t2) and after (t3) the intervention. The ability of SISU to guide participants through the intervention, acceptability, and negative effects were investigated. Data analyses are based on intention-to-treat principles. Linear mixed models will be used to investigate short-term changes over time in mood, depression, anxiety. INTERVENTION: The intervention consists of two sessions. Each session comprises writing tasks on autobiographical negative life events and an Acceptance- and Commitment Therapy-based exercise respectively. Participants interact with the software agent on two consecutive days for about 30 min each.
RESULTS: All participants completed all sessions within two days. User experience was positive, with all subscales of the user experience questionnaire (UEQ) M > 0.8. Participants experienced their writings as highly self-relevant and personal. However, 57% of the participants reported at least one negative effect attributed to the intervention. Results on linear mixed models indicate an increase in anxiety over time (β = 1.33, p = .001). Qualitative User Feedback revealed that the best thing about SISU was its innovativeness (13%) and anonymity (13%). As worst thing about SISU participants indicated that the conversational style of SISU often felt unnatural (73%).
CONCLUSION: SISU successfully guided participants through the two-day intervention. Moreover, SISU has the potential to enter the inner world of participants. However, intervention contents have the potential to evoke negative effects in individuals. Expectable short-term symptom deterioration due to writing about negative autobiographical life events could not be prevented by acceptance and commitment therapy-based exercises. Hence, results suggest a revision of intervention contents as well as of the conversational style of SISU. The good adherence rate indicates the useful and acceptable format of SISU as a mental health chatbot. Overall, little is known about the effectiveness of software agents in the context of psychological wellbeing. Results of the present trial underline that the innovative technology bears the potential of SISU to act as therapeutic agent but should not be used with its current intervention content. TRIAL-REGISTRATION: The Trial is registered at the WHO International Clinical Trials Registry Platform via the German Clinical Studies Register (DRKS): DRKS00014933 (date of registration: 20.06.2018). Link: https://www.drks.de/drks_web/navigate.do?navigationId=trial.HTML&TRIAL_ID=DRKS00014933.

Introduction

Digitalization evokes societal changes in various fields (World Economic Forum, 2018). Fully automated and intelligent machines within the areas of work, economy and science become more and more integrated into people's daily lives. Innovative technologies are entering the context of mental health and could be part of a “next generation” of corresponding health care services.

Worldwide, mental disorders are a public health concern. However, the majority of individuals living with mental disorders do not receive any health care supply (Alonso et al., 2004; Kessler et al., 2005; Thornicroft, 2007). In Europe, only about 25% of people with mental disorders receive professional treatment (Wittchen and Jacobi, 2005). Either societal aspects keep them from receiving health care supply or people choose not to pursue supply offers due to various reasons (Andrade et al., 2014). Discussed are stigmatization (Barney et al., 2006), restrictions of time and location (Paganini et al., 2016), negative attitudes towards pharmacological and psychotherapeutic treatments (Baumeister, 2012), negative experiences with professionals (Rickwood et al., 2007) or too little conscientiousness for mental health (Zobel and Meyer, 2018).

Psychological Internet- and mobile-based interventions (IMIs) are frequently evaluated as medium to overcome some of the aforementioned barriers, with the potential of delivering psychological interventions on a large scale and a widespread roll-out (Carlbring et al., 2018; Ebert et al., 2018; Sander et al., 2016). IMIs are highly standardized, manualized computer-programs, which can be provided on an online-platform (Barak et al., 2009). They can help to reduce treatment barriers and expand supply offers (Carlbring et al., 2018; Ebert et al., 2018; Baumeister et al., 2018). Guided IMIs can provide low-threshold, flexible interventions that can be as effective as traditional face-to-face psychotherapy, e.g. in reducing depression and anxiety (Carlbring et al., 2018; Andersson et al., 2014). However, the scope of IMIs seems to be limited by a low uptake and intervention adherence of individuals (Baumel and Yom-Tov, 2018; Baumel et al., 2017). A high level of implemented persuasiveness could help to overcome these limitations (Baumeister et al., 2019).

A fully automated version of IMIs is software agents (Bendig et al., 2019). Software agents can automatically hold a text-based conversation in real-time via a chat-interface (e.g., smartphone application) (Abdul-Kader and Woods, 2015). Via software agents, components of persuasiveness (Baumeister et al., 2019) like personalising featured contents or adopting a social role to the software agent could easily be integrated (Araujo, 2018). Therefore, delivering IMIs by software agents could compensate for some of the disadvantages of conventional computer program-based IMIs (Baumeister et al., 2014). Thus, software agents gain importance especially as natural interaction port between people and innovative technology-driven or technology–enabled forms of psychological treatment (Bendig et al., 2019).

Software agents could deliver low-threshold, flexible and cost-effective interventions aiming at promoting psychological wellbeing in a large number of individuals. They could be used in the future as add on in the context of traditional face-to-face therapy, e.g. by delivering brief interventions which can be processed by clients independently from time- and location. Another application could be the use of software agents to deliver stand-alone interventions. Software agents in the context of clinical psychology and psychotherapy are on the rise. This is predictable from a growing body of research in that field (Brandtzaeg and Følstad, 2017; Dale, 2016) and increasing online offers of healthcare providers (e.g. health-apps with chat-support). Initially they will be primarily driven by technologies such as algorithmic decision trees and full-text search engines and not by advanced artificial intelligence (Yuan, 2018) - at least in the near future. Thus, within the context of psychotherapy, technological progress will initially mainly allow to implement brief, definable interventions which are able to converse with people on the basis of predefined, manualized scripts (Becker, 2018; Dowling and Rickwood, 2013).

Such a brief intervention could include a therapeutic writing paradigm. Writing with the aim of improving health has a long history (Pennebaker and Beall, 1986; McKinney, 1976). In the current literature, the labelling of corresponding writing approaches varies. Terminology includes e.g. expressive (Pennebaker and Beall, 1986), narrative or therapeutic writing (Bolton et al., 2004). Regardless of terminology, the writing approach in this study will refer to the written expression of thoughts and feelings in the context of upsetting or traumatic life events (Baum and Rude, 2013; Pennebaker, 1997; Spera et al., 1994). Meanwhile, writing approaches have been frequently investigated in the context of physical and mental wellbeing (Krpan et al., 2013; Baikie et al., 2012; Meston et al., 2013; Kállay, 2015). Previous research reports consistently short-term increases in emotional distress, negative mood and in physical symptoms immediately after writing prior to longer term physical and mental health benefits (Baikie and Wilhelm, 2005; McGuire et al., n.d.; Wetherell et al., 2005; Frisina et al., n.d.; Frattaroli, 2006; Bell-Pringle et al., 2004).

As instructions to write can be highly standardized and directive, writing interventions are easily translatable into an online offer (Baikie et al., 2012; Stockton et al., 2014; Goss and Anthony, 2004). Additionally, the paradigm could easily be implemented as a low-threshold, minimal time- and cost-effective intervention to uplift psychological and physical wellbeing (Goss and Anthony, 2004; Smyth and Helm, 2003).

However, software agents delivering brief interventions are few in numbers (Bendig et al., 2018). Furthermore, they have not yet been evaluated in the context of the implementation of a therapeutic writing intervention. Nevertheless, using the paradigm of therapeutic writing may clarify if participants are able to disclose themselves to a software agent. As the majority of psychological paradigms require at least some sort of identifying, labelling and disclosing emotional life experiences (Smyth and Helm, 2003; Smyth et al., 2008), this might be a key requirement of software agents delivering an psychological intervention. We developed a software agent gender neutrally called SISU (Software agent providing an Intervention for Self-help to Uplift psychological well-being and finnish word [ˈsisu] for inner strength). This study investigates the potential of SISU to act as therapeutic agent. To answer this research question, we investigated 1) whether the application is suitable to deliver the paradigm of therapeutic writing as well as 2) the acceptability and 3) potential negative effects of SISU. Research questions were as follows:

Has SISU the potential to deliver the paradigm of therapeutic writing?

How is the acceptability of SISU?

Are there negative effects of SISU?

Materials and methods

Recruitment

Recruitment started on 21.06.2018 and the study was closed on 07.08.2018. We designed this trial with a planned enrolment of 30 individuals. Within the period of 6 weeks, N = 30 eligible participants provided informed consent. We recruited following an open recruitment strategy with offline and online recruitment strategies. Posters and Flyers were distributed at several universities, information on the study was provided in lectures and via e-mail to students who needed credits for their courses. Additionally, the study was advertised on facebook. As an incentive, participants could win six gift cards with a value of 30 € each. Participants were eligible to take part in this study if they were 18 years of age or older, were German native speakers, had a smartphone and provided informed consent.

Intervention

SISU is a script-based software agent and was developed at the Ulm University. It provides a two-day therapeutic writing intervention on autobiographical negative life events. The instructions delivered by the software agent as well as psychoeducation on experiential acceptance follow the principles of therapeutic writing enriched with Acceptance and Commitment Therapy-based principles (Hayes et al., 2011) as well as narrative psychology. The software agent supports participants in labelling and structuring information into a coherent narrative (Smyth et al., 2001). After each writing task, SISU delivers an ACT-metaphor (floating leaves on a moving stream, Hayes and Smith, 2005), supporting participants with distancing themselves from negative thoughts and feelings (Wengenroth, 2008). The latter is being implemented to account for the well-established finding, that therapeutic writing is often followed by an immediate increase of negative mood (Baikie et al., 2012; McGuire et al., n.d.; Wetherell et al., 2005; Frattaroli, 2006; Nyssen et al., 2016). Psychological benefits in terms of e.g. increasing wellbeing are usually followed in the longer run (Baikie and Wilhelm, 2005), thus not expectable shortly after writing. SISU imitates a human conversational style. Participants are guided by SISU to write each day at the same time for 10–20 min about a self-selected autobiographical negative life event (S1). On day 1 there is psychoeducation prior to the instructions for the writing task which is followed by the narratives of the participants. Participants are instructed to write about a meaningful negative life event on day 1 and on the same or another negative autobiographical life event on day 2. After each writing, SISU facilitates coping with the narrated life event in the present moment, gives brief psychoeducation on negative thoughts and feelings and provides possibilities for dealing with them. Afterwards, SISU applies strategies for cognitive defusion to help participants to distance themselves from the life event. Furthermore, SISU is able to automatically send prompts to participants (e.g. reminders via push-up messages to do their writing task on day 2). Additionally, SISU reacts to external triggers e.g. from the survey software (e.g. participant completed survey, SISU switches to the next state and e.g. thanks the user for participating). More details on intervention contents can be derived from Table A1, a schematic representation of the interaction with SISU can be seen in Fig. A1.

Table A1

Content and techniques of the intervention delivered by SISU.

Component	Content	Focused ACT technique
1 Introduction	Introduction, Psychoeducation on psychological wellbeing, therapeutic writing, ACT exercise	Psychoeducation
2 Writing tasks	Instructions for writing about a negative autobiographical life event
3 Thoughts and feelings	Definition of negative thoughts and feelings and dealing with them	Contact with the present moment
4 Exercises	Leaves on a moving stream	Cognitive defusion

Notes. ACT = Acceptance and Commitment Therapy.

Fig. A1

Simplified, schematic representation of the interaction with SISU.

Technology and privacy

SISU is founded on a state machine-based conversation engine custom software system for computer-mediated communication with instant messaging applications that has been developed at Ulm University. It represents a modular middleware that can interface with existing instant messaging endpoints and act on any ongoing conversations and scripted events. This includes the interception and analysis of incoming messages, execution of internal handling processes, triggering of predefined rules or actions, and the dispatch of outgoing messages. Internally, SISU is backed by an execution engine with pre-defined, rule-based conversation scripts (i.e., conversation flow graph, associated rule sets and corresponding word lists). Here, SISU can also take contextual and session-related information for individualized conversation paths into account. In addition, SISU supports chat-external triggers (e.g., pre-scheduled timers) to resume conversations. SISU's only conversational input channel is arbitrary text sent by the user, which is then parsed, matched, and mapped into the ongoing conversational state machine. The state machines contain branches and alternative routes for individual conversational paths. Hence, the only means of interaction between the participants and SISU are the ongoing chat conversations.

The SISU application interfaces with the online messaging Wire Services SDK (https://wire.com/de/), a text-based instant messaging application with end-to-end encryption for its messages. SISU is embodied by a dedicated Wire user account that is under full control of the middleware system. Thus, SISU appears like any other conversation partner with the messenger of its users. Each new conversation partner spawns a separate session instance in the system so that SISU can conduct multiple conversations with different chat partners simultaneously.

Due to end-to-end encryption of any communication and the restricted and secured storage of conversation logs, SISU ensures the privacy for its users. In particular, conversations are not accessible to any third parties, including the providers of Wire.

Ethics and approval

All subjects gave their informed consent for inclusion before they participated in the study. The study was conducted in accordance with the Declaration of Helsinki. This trial was approved by the Ethics Committee of the Ulm University (No. 158/18, 04.06.2018) and registered in the German Clinical Trail Registry (DRKS DRKS00014933).

Implementation

Eligible participants who provided informed consent were asked to fill in the first online survey (t1). After this, participants were given instructions for downloading Wire and to connect with SISU. From that point, all processes were fully automated. Participants were invited by SISU to the second online-survey (t2) after completing the first session and they received the third (t3) online-survey after completing second session on day two. Outcomes, measurements and points of assessment are shown in Table A2. On each day, participants received instructions to write about an emotionally negative autobiographical life event. Subsequently participants processed through an ACT-metaphor exercise which was provided via audio. The exercise was followed by the second survey (t2). On the next day (24 h later), SISU reminded participants to continue the writing task. Participants could follow the reminder or reschedule the participation to a later time on that day. However, participants were again instructed to write about a personal negative life event and subsequently received the ACT-metaphor exercise via audio. After that, participants completed the third survey (t3) and were discharged from the study. If problems or uncertainties occurred during the study, participants were able to communicate with the study team via E-mail. Additionally, we assessed participants' probability of living with post-traumatic stress disorder symptoms PTSD-7 (Siegrist and Maercker, 2010) to detect potentially traumatised persons seeking support (PTSD-7-score ≥ 4). Persons which indicated elevated symptoms were provided automatic online feedback and a pamphlet with aids offered by the German health system. The flowchart of study procedures can be seen in Fig. 1.

Table A2

Constructs, measurement instruments and points of assessment.

Construct	Instrument	Point of assessment
		T1	T2	T3
Feasibility
Demographics	Self-report	X
Adherence	Intervention and study attrition			X
Attitudes and expectations	APOI	X		X
User experience	UEQ			X
Negative effects	INEP			X
Postwriting questionnaire	Manipulation check			X
User feedback (best/worst thing about SISU, improvement proposals)	Open questions			X
Trauma	PTSD-7	X		X
Mood	PANAS	X	X	X
Anxiety	GAD-7	X		X
Depression	PHQ-8	X		X
Emotion regulation	ERQ	X		X
Psychological Flexibility	FAH-II	X		X

Notes. t1 = Baseline; t2 = Intermediate; t3 = post measurement; PHQ-8 = Patient Health Questionnaire-8; GAD-7 = Generalized anxiety disorder 7-item scale; APOI = Attitudes towards psychological online interventions questionnaire; INEP = Inventory for the assessment of negative effects of psychotherapy.

Fig. 1

Studyflow.

Measurements of feasibility

Data were collected at baseline (t1) and after each interaction with SISU (t2, t3). We collected demographic data, feasibility data and user feedback on SISU in order to inform modifications to the prototype. Outcomes, measurements, and points of assessment are shown in Table A2.

Functionality: potential of SISU to deliver the paradigm of therapeutic writing

The post-writing questionnaire (Pennebaker and Beall, 1986; Herbert et al., 2019) was administered after each writing session for a manipulation-check. It measures whether participants are able to express their emotions with regards to a personally meaningful life event which is crucial to the paradigm. As literature suggests that the length of the narratives might be a further indicator of the personal significance of the events (Thompson, 2013), we assessed the amount of written words. To assess whether the ACT-based contents have the potential to keep immediate negative mood small after writing, we measure the mood on the positive and negative affect schedule (PANAS; Kroenke et al., 1996) immediately after writing.

PANAS - Positive Affect Negative Affect Schedule

Participants filled in the German state version of the PANAS (Kroenke et al., 1996) after each writing. Answers indicate current effect of participants on two nearly orthogonal dimensions, positive and negative affect, with 10 adjectives describing each. Participants rate the intensity of the different moods on a 5-point-Likert scale (1 = “very slightly” or “not at all” to 5 = “extremely”). A mean score can be calculated for every dimension. Higher scores indicate greater positive or negative affect. The German version shows good internal consistencies between α = 0.84 and α = 0.86 (Kroenke et al., 1996).

Postwriting-Questionnaire

To assess if participants adhered to the instructions, participants were asked questions after each writing. Answers were rated on a 5-point-likert-scale (1=”not at all”, 3 = “few”, 5 = “very much/extremely”). Participants indicated the degree to which they wrote in emotional and personal terms, the degree to which privacy was important to them to enable self-disclosure and the degree to which it was difficult to express one's feelings during writing. The questionnaire was adapted from the original version of Pennebaker and Beall (1986).

Acceptability

To collect data on acceptability, we assessed general attitudes and expectations of individuals towards chatbots delivering a psychological intervention (APOI; Schröder et al., 2015, t1, t3). Furthermore, intervention dropout as well as study attrition were investigated. To assess whether study assessment procedures are acceptable, we included likely outcomes as well as potential moderators/mediators of a future randomized-controlled trial (PANAS, PHQ-9, GAD-7, PTSD-7, ERQ, FAH-II). Furthermore, we investigated user experiences after the two-day intervention at t3 (UEQ; Laugwitz et al., 2008). These information on acceptability were complemented with qualitative user feedback on SISU collected at t3 on the best and the worst aspects of SISU as well as on improvement suggestions. The feedback was collected by means of three open questions (“What did you like best/least about your experience with SISU?”, “How can SISU be further improved?”) and evaluated following Fitzpatrick et al. (2017) using thematic maps.

APOI - Attitudes Psychological Online Interventions

The Attitudes towards Psychological Online Interventions Questionnaire (APOI) measures the position of a person regarding psychological Internet Interventions (Schröder et al., 2015). There are four subscales: Scepticism and perception of risks, confidence in effectiveness, threat due to technology and anonymity benefits. Participants rate 16 Items on a 5-point-Likert scale (1 = “I do not agree at all” to 5 = “I agree completely”) and a sum score (range: 15–75 points) is generated. Internal consistency of the APOI total score is acceptable with α = 0.77 (Schröder et al., 2015).

UEQ - User Experience Questionnaire

The User Experience Questionnaire (UEQ) measures the experience of interacting with the chatbot (Laugwitz et al., 2008). The UEQ consists of 26 items grouped into six scales (“Attractiveness”, “Perspicuity”, “Efficiency”, “Dependability”, “Stimulation” and “Novelty”). Attractiveness refers to the look of the product, the degree to which it is perceived as enjoyable, friendly and pleasant (Schrepp, 2017). Efficiency refers to the degree to which the user interface looks nice and tasks can be performed fast and in a pragmatic way. Perspicuity refers to the degree to which the product is easy to understand, clear and simple. Dependability refers to the degree to which the interaction with the product is predictable and secure. Stimulation refers to the degree to which the product is perceived as interesting and exciting. Novelty refers to the degree to which the product is innovative, inventive and creatively designed (Schrepp, 2017). Each item of the UEQ consists of a pair of terms with opposite meanings and can be rated on a 7-point Likert scale (−3 = “fully agree with negative term” to 3 = “fully agree with positive term”). The standard interpretation of scale means is that values between −0.8 and 0.8 represent a neutral evaluation of the corresponding scale whilst values >0.8 represent a positive evaluation and values <0.8 represent a negative evaluation (Schrepp, 2017). Internal consistencies range between the scales from α = 0.65 to α = 0.89 (Laugwitz et al., 2008).

Open questions

For feedback on SISU, three open questions following Fitzpatrick et al. (2017) about the interaction with SISU were provided (t2). The answers are individually evaluated and thematically summarized (Fig. A2, Fig. A3).

Fig. A2

Thematic map on the best aspects of SISU.

Fig. A3

Thematic map on the worst aspects of SISU.

ERQ – Emotion Regulation Questionnaire

The Emotion Regulation Questionnaire (ERQ; 72) is a 10-item questionnaire measuring positive and negative feelings as well as their regulation. Items refer to two different emotion regulation strategies: Reappraisal and suppression. Participants rate the items on a scale from 1 (“strongly disagree”) to 7 (“strongly agree”). Means show the preference for each strategy indicating higher preference at higher mean scores. Internal consistencies are acceptable to good and differ from α = 0.75 to α = 0.82 (Gross and John, 2003).

FAH-II – Questionnaire for acceptance and action

The Questionnaire for acceptance and action (FAH-II; Hoyer and Gloster, 2013) consists of 7 items. On a 7-point-Likert scale that ranges from 0 = “never true” to 6 = “always true”, participants rate processes of experiential avoidance and psychological inflexibility with higher scores indicating greater levels of psychological inflexibility. The questionnaire shows good to excellent psychometric properties in a German sample (α = 0.84 to 0.97) (Hoyer and Gloster, 2013).

Negative effects

We measured negative effects of SISU using the inventory (INEP; Ladwig et al., 2014) to assess potential risks. Additionally, we assessed if SISU has the risk of worsening depressive (PHQ-9; Kroenke et al., 2001), anxiety (GAD-7; Spitzer et al., 2006) or symptoms of posttraumatic stress (PTSD-7; Siegrist and Maercker, 2010).

PHQ-9 - Patient Health Questionnaire

The Patient Health Questionnaire (PHQ-9; Kroenke et al., 2001) records depressive symptoms regarding the 2 weeks prior to the measurement. Nine items are rated on a 4-point-Likert scale (0 = “not at all“to 3 = “almost every day“). There is an additional item, which asks if functionality is limited in daily life. The items are computed into a sum score. A meta-analysis concluded that it is a good screening tool for the detection of depression (Manea et al., 2012). Cronbach's α is high (α = 0.89; Kroenke et al., 2001).

GAD-7 - Generalized Anxiety Disorder Scale

The Generalized Anxiety Disorder 7-item Scale (GAD-7) measures anxiety symptoms regarding the 2 weeks prior to the measurement (Spitzer et al., 2006). There are seven Items to rate on a 4-point-Likert scale (0 = “never“to 3 = “almost every day“). A sum score is computed. Sum scores of fifteen points or higher indicate severe anxiety symptoms. Internal consistency is very good with Cronbach's α = 0.92 (Spitzer et al., 2006).

PTSD-7 post-traumatic stress

The PTSD-7 is a 7-item screening tool to examine symptoms of post-traumatic stress disorder (Siegrist and Maercker, 2010). A 4-point-Likert scale measures the frequency of PTSD-Symptoms (“not at all” to “5 times a week”) during the last month. Symptoms exist if they occur at least 2 to 4 times a week. Sum scores are computed with a total of 0 to 7 symptoms and a cut-off with at least 4 symptoms marking a possible PTSD. The German version of the screening instrument has an excellent internal consistency of α = 0.90 (Siegrist and Maercker, 2010).

INEP – inventory to assess negative effects of psychotherapy

Self-reported negative effects of SISU are recorded with the 21-item inventory for the assessment of negative effects of psychotherapy (INEP; 63). Items 1 to 6 are rated on a bipolar 7-point scale ranging from 1 to 3 (=positive change), 0 (=no change) and −1 to −3 (=negative change). The remaining Items (7 to 21) are rated on a unipolar 4-point-scale (0 = “no agreement” to 3 = “total agreement”). Negative effects are assessed across six life areas: friends and family, intrapersonal changes, partnership, stigmatization, workplace and therapeutic misconduct. For each of the 21 items, participants rate whether the negative effects occur because of the intervention or due to other current life circumstances. The absolute number of negative effects per patient allows a statement about the side-effect load (Ladwig et al., 2014; Abeling et al., 2018; Nestoriuc and Rief, 2012; Grüneberger et al., 2017). For this purpose, all of the items answered in the negative range on the bipolar scales (−1 to −3) and all positively answered items of the unipolar scales (1 to 3) which were attributed to the intervention by the participants, were summed up. Excluded from this procedure are the three items of the misconduct scale (Item 18, 19, 21), since misconduct attributed to the chatbot is not a negative effect that can be attributed to a chatbot-based psychological intervention per se. The 21-item inventory has a good internal consistency of α = 0.86 (Ladwig et al., 2014).

Sample size and data analysis

Viechtbauer et al. (2015) stated that at least 29 participants are sufficient to detect problems with a probability of 0.10 and a 95% confidence interval. Therefore, screening at least 29 participants will ensure a high level of confidence (i.e. at least 95%) for the chosen minimum problem probability to investigate feasibility. Statistical analyses were conducted using SPSS 26 software. Data for evaluation was collected at baseline (t1) and after each interaction with SISU (t2, t3). For the characterisation of the sample and the assessment of negative effects (INEP), absolute and relative frequencies are calculated. For metric variables descriptive statistics with mean values (M), median (Md), range and the standard deviation (SD) were calculated. To evaluate if SISU has the potential to affect depression, anxiety and symptoms of posttraumatic stress as well as to investigate whether there are immediate intervention-related emotional responses, linear mixed models will be applied. Linear mixed models take into account that answers are nested within persons and the resulting dependency between variables. In this study, the repeated measurements (level 1) are nested within person (level 2) (Nezlek et al., 2006; Luke, 2004). All analyses are based on an intention-to-treat (ITT) principle.

Results

Baseline participant characteristics

Recruitment ended in August 2018 with N = 61 individuals expressing their interest to participate within one month. N = 30 individuals consented. Mean age of participants was M = 23.17 (SD = 3.85), 80% identified as female, all participants had fluent German language skills. 73% have had no prior psychotherapeutic experience, 20% had prior psychotherapy (>3 months ago), and two participants (6.7%) received psychotherapeutic treatment while participating in this study. Baseline participant characteristics are presented in Table 1. At baseline, the mean depression score (PHQ-9) was M = 5.00 (SD = 3.54) indicating that participants were on average not to mildly depressed. The mean anxiety score (GAD-7) was M = 3.1 (SD = 2.69) on average, indicating no to minimal levels of anxiety symptoms. Participants preferred on average the emotion regulation strategy (ERQ) of reappraisal (M = 4.96, SD = 0.97) over suppression (M = 3.14, SD = 0.97) (t(29) = 5.13, p < .001) and were on average psychological flexible (M = 17.97, SD = 7.36) (FAH-II; Hoyer and Gloster, 2013) (Table 2).

Table 3

Sociodemographic sample characteristics.

Demographics	Total
Age (M, SD)	23.17 (3.85)
Gender (%)	80% female
Education (%)	High school or less 76.6
	Diploma 3.3
	University 20.0
Employment status	Full-time 23.3
	Part-time 43.3
	Student 26.7
	Unemployed 6.7
Prior psychotherapy experiences (%)	26.7
Marital status (%)	Single 50
	Solid partnership 43.3
	Married 3.3
	Divorced 3.3

Table 4

Clinical characteristics over time.

Measurement	T0 (M, SD)	T1 (M, SD)	T2 (M, SD)
Depression, PHQ-9	5.00 (3.54)		5.00 (3.75)
Anxiety, GAD-7	3.1 (2.69)		4.43 (3.52)
Mood, PANAS
Positive affect	2.97 (0.65)	2.54 (0.82)	2.51 (0.78)
Negative affect	1.40 (0.37)	1.47 (0.45)	1.60 (0.58)
Emotion-regulation, ERQ
Reappraisal	4.96 (0.97)		4.95 (1.17)
Suppression	3.14 (1.40)		3.25 (1.57)
Psychological flexibility, FAH-II	17.97 (7.36)		18.00 (7.99)
Posttraumatic stress, PTSD-7	9.03 (3.71)		9.1 (3.05)

Functionality: potential of SISU to deliver the paradigm of therapeutic writing

Participants indicated on the 5-point-likert scale of the post-writing questionnaire, that they had expressed their emotions in their writings (M = 4.03, SD = 0.41, t2 and M = 4.03, SD = 0.56, t3). In addition, participants experienced their writings as highly self-relevant and personal (M = 4.53, SD = 0.63, t2 and M = 4.50, SD = 0.57, t3). Participants reported that privacy mattered to them for enabling self-disclosure (M = 4.13, SD = 0.90, t2 and M = 4.07, SD = 1.14, t3). Participants reported no difficulties with writing about an emotional negative life event (M = 2.63, SD = 1.16, t2 and M = 2.47, SD = 0.97, t3). Participants wrote on average M = 357.13 (SD = 158.9) words in the first and M = 271.33 (SD = 112.61) words in the second writing session. Writing-related changes in positive and negative affect after each session were assessed by linear-mixed models. Immediate writing-related emotional responses comprised an increase of negative affect over time (β = 0.09, p = .04) and a decrease of positive affect over time (β = 0.35, p = .002).

Acceptability

All participants (N = 30) completed all measurements as well as all sessions of the brief intervention within two consecutive days.

Attitudes and expectations towards psychological chatbots were comparable to those towards psychological internet interventions in general M = 49.83 (SD = 7.88) at t1, M = 48.17 (SD = 11.48) at t3 (APOI; Schröder et al., 2015).

User experience (UEQ; Laugwitz et al., 2008) was rated as good on each of the six subscales (Perspicuity, Dependability, Stimulation, Novelty, Attractiveness, Efficiency), all M > 0.8 (Fig. 2). In comparison to general benchmarks provided by Schrepp (2017), Perspicuity (M = 0.97, SD = 0.36), Dependability (M = 1.11, SD = 0.35), Stimulation (M = 0.94, SD = 0.37), Attractiveness (M = 0.96, SD = 0.22) and Efficiency (M = 0.89, SD = 0.52) are in the average range of interactive products. Originality (M = 1.43, SD = 0.34), the degree to which the product is experienced as innovative and creatively designed exceeds the average benchmark thus achieving the attribute “good” in comparison to 452 product evaluations with a total of 20,190 participants in all evaluations (M = 1.14). Fig. 2 depicts and summarizes the scale means.

Fig. 2

Scale means of user experience.

The evaluation of the qualitative feedback following Fitzpatrick et al. (2017) revealed two core topics respectively which were summarized as “Hardware” and “Software” of the Chatbot. Hardware-related feedback comprised technical execution, privacy aspects and the possibility of using SISU on any digital terminal device. As best hard skills, participants explicitly indicated innovativeness (13%), anonymity (13%) as well as the independence of the application from time and space (16%). “Software” comprised features concerning the programme of SISU. As best soft skills, participants indicated the therapeutic writing tasks (16%), the ACT-metaphor (20%) and the formulations/phrases of SISU (27%).

As worst Hardware-related aspects, participants mentioned that they had technical complications with SISU (e.g. no immediate answers from SISU) (33%) or insecurities with regards to confidentiality (“I'm not sure about what happens with my narratives now”) (6%). As worst software-related aspects, participants mentioned that the conversational style of SISU felt unnatural (73%). Fig. A2, Fig. A3 give an overview of the feedbacks ordered by thematic maps of the best/worst aspects of SISU. Improvement suggestions comprised more individualized feedback (e.g. more personal answers, addressing described issues) (37%), less repetitions (e.g. more language variation) (20%), more conversational contents after the writing tasks (e.g. chatting about a problem) (13%).

Negative effects

56.67% (n = 17) of the participants reported at least one negative effect attributed to the intervention (t3) (Table A3). On average, participants reported 1.88 negative effects (SD = 1.22) (Total value adjusted by the items for misconduct, item number = 18) (Table 3).

Table A3

Negative effects: frequencies part I.

Life areas	Positive changes since intervention completion (n, %)	Attributed to intervention(n)	Attributed to other life circumstances since intervention completion(n)	Negative changes since intervention completion(n, %)	Attributed to intervention(n)	Attributed to other life circumstances since intervention completion(n)	No changes(n, %)	Attributed to intervention(n, %)	Attributed to other life circumstance since intervention completion(n)
1) Individual well-being	17 (56.6)	11	5	2 (6.7)	–	2	11 (36.7)	1	10
2) Trust others	5 (16.7)	5	–	2 (6.7)	–	2	23 (76.7)	3	20
3) Suffering from past events	13 (43.3)	11	2	2 (6.7)	2	–	15 (50.0)	1	14
4) Partnership	5 (16.3)	3	2	2 (6.7)	–	2	23 (76.7)	4	19
5) Family	3 (10.0)	2	1	2 (6.7)	–	2	25 (83.3)	4	21
6) Friends	5 (16.3)	5	–	2 (6.7)	–	2	23 (76.7)	1	22

Notes. 1) Since completion of the Interaction with the Chatbot I feel… 2) To trust others has been on my mind since Interaction with the Chatbot compared to the time before the interaction with the chatbot… 3) I suffer from past events since completion of the interaction with the Chatbot compared to the time before interaction with the Chatbot… 4) Since completion of the interaction with the chatbot I experience in my partnership in the comparison with the time before the interaction… 5) My relationship with my family after the Completion of the interaction with the chatbot in comparison with the time before the interaction with the Chatbot… 6) My relationship with my friends after the Completion of the interaction with the chatbot in comparison with the time before the interaction with the Chatbot.

With regards intra- or interpersonal changes (Items 1 to 6), the frequencies of positive changes after the intervention (t3) ranged between 10.0 and 56.3%, with most of the positive changes attributed to the interaction with SISU rather than to other life circumstances (Table A3). Across Items 1 to 6 no negative effects attributed to the interaction with SISU occurred, except for Item 3, “I suffer more from past events since the interaction with SISU”. Two participants (6.7%) indicated to suffer more from past events since the interaction with SISU and attributed this to intervention contents. The frequencies across the individual areas of life are shown in the Table A3, Table A4.

Table A4

Negative effects: frequencies part II.

Areas and items	0 = not at all(n, %)	1 = a little bit(n, %)	2 = partly(n, %)	3 = entirely true(n, %)	Rated 1 = a little bit to 3 = entirely true and traced this back to the intervention, (n)	Rated 1 = a little bit to 3 = entirely true and traced this back other life circumstance, (n)
Stigmatisation
7) Fear of colleagues/friends finding out about the therapy or telling them about it	18 (60.0)	7 (23.3)	3 (10.0)	2 (6.7)	10	2
Financial and legal disadvantages
8) Problems with insurance (e.g. life insurance) or fear that problems could arise.	28 (96.7)	1 (3.3)	1 (3.3)	–	2	0
9) More financial worries	29 (96.7)	1 (3.3)	–	–	0	1
Therapeutic alliance
10) I feel dependent on the Chatbot	26 (86.7)	3 (10.0)	1 (3.3)	–	3	0
16) I felt that statements of the chatbot were hurting	29 (96.7)	1 (3.3)	–	–	1	0
17) I had the feeling that the chatbot was making fun of me	26 (86.7)	3 (10.0)	1 (3.3)	–	4	0
11) It's harder for me to make important decisions alone	26(86.7)	2 (6.7)	1 (3.3)	1 (3.3)	0	4
20) The chatbot made me do things which I didn't really want to	26 (86.7)	2 (6.7)	2 (6.7)	–	4	0
Partnership, family, friends
12) My partner is/was jealous of my relationship with the Chatbot. (omit if no partnership)	20 (66.7)	1 (3.3)	1 (3.3)	–	1	1
Symptoms
13) …I've had longer periods where I was feeling sick…	22 (73.3)	5 (16.7)	3 (10.0)	–	6	2
14) I've changed for the worse as a person	28 (93.3)	2 (6.7)	–	–	1	1
15) I had suicidal thoughts/intentions for the first time	29 (96.7)	1 (3.3)	–	–	1	0
Misconduct
18) During the interaction there were unpleasant, direct attacks by the chatbot	29 (96.7)	1 (3.3)	–	–	1	0
19) I felt physically assaulted by the chatbot	29 (96.7)	1 (3.3)	–	–	0	1
21) It has come to my attention that the chatbot broke confidentiality	29 (96.7)	–	1 (3.3)	–	1	0

Notes. 12) n=8 participants indicated to have no partnership, 17) n=1participant indicated, that the chatbot answered basically the same after each writing session, n=1 participant indicated that the chatbot did not answer the question of how it knew that the participant was burdened, n=1 participant wrote that instructions of how to write were exaggerated, 20) N=1 participant indicated that he felt uncomfortable with writing the event to the chatbot as the negative event could possibly be traced back to him/her N=1 participant indicated that he/she felt uncomfortable to be encourage to write more although he/she did not want to write more, 21) N=1 participant felt like writing a story to a chatbot could possibly break confidentiality.

Changes in anxiety, depression and posttraumatic stress were assessed by linear-mixed models. Results show an increase in anxiety over time (β = 1.33, p = .001). Symptoms of depression and posttraumatic stress stayed unchanged (p > .05) (Table 2). One serious adverse event occurred, concerning the suicidal tendency of a participant. One person indicated suicidal thoughts/intentions for the first time (Table A4). An overview on feasibility variables can be found in Table 5.

Table 5

Feasibility-related outcomes.

Measurement	T0 M (SD)	T1 M (SD)	T2 M (SD)
Attitudes and expectations, APOI	49.83 (7.88)	–	48.17 (11.41)
Scepticism and perception of risks	9.80 (2.31)	–	10.27 (2.99)
Confidence in effectiveness	14.97 (2.04)	–	14.17 (3.65)
Threat due to technology	9.50 (2.62)	–	9.60 (3.23)
Anonymity benefits	12.17 (3.30)	–	11.87 (3.86)
User experience	–	–	1.04 (0.18)
Attractiveness	–	–	0.96 (0.22)
Efficiency	–	–	0.89 (0.52)
Perspicuity	–	–	0.97 (0.36)
Stimulation	–	–	0.94 (0.37)
Novelty	–	–	1.43 (0.34)
Dependability	–	–	1.11 (0.35)
Negative effects, INEP	–	–	1.88 (1.22)
Postwriting questionnaire
How much did you reveal your emotions in your writing?		4.03 (0.41)	4.03 (0.56)
How personal was the text you wrote?		4.53 (0.63)	4.50 (0.57)
How important is it for you that you wrote this text anonymously?		4.13 (0.90)	4.07 (1.14)
How difficult was it for you to write something about this question?		2.63 (1.16)	2.47 (0.97)

Discussion

To the best of our knowledge, this is the first trial of a fully automated text-based conversational agent designed to deliver a brief self-help therapeutic writing intervention. The goal of our study was to investigate the feasibility of SISU for the implementation in a full-scale trial.

The study demonstrates, that the SISU-application can fully automated guide participants through a two-day intervention, including delivering study assessments. However, immediate emotional responses comprised an increase of negative (β = 0.09, p = .04) and a decrease of positive affect over time (β = 0.35, p = .002). Thus, ACT-based exercises could not prevent participants from immediate mood changes for the worse. These results are in line with frequent findings from therapeutic writing interventions (Baikie and Wilhelm, 2005; McGuire et al., n.d.; Wetherell et al., 2005; Frisina et al., n.d.; Frattaroli, 2006; Bell-Pringle et al., 2004). From the current trial we cannot say, if the short-term symptom deterioration is associated with medium to long-term effectiveness regarding psychological wellbeing. Future research needs to establish the effectiveness of SISU in long-term follow-ups and establish further safety measures for short-term symptom deterioration if the writing contents should be retained.

Results on the paradigm of therapeutic writing show that it is possible to enter the inner world of participants with SISU and thus without human involvement. Participants expressed their emotions with regard to self-relevant negative life events. To this, feeling safe/private was relevant for enabling disclosure. As the majority of psychological paradigms require at least some sort of affect management, identifying, labelling and disclosing emotions (Smyth and Helm, 2003; Smyth et al., 2008) might be a key requirement for the effectiveness of a software agent delivering a psychological intervention (Ritterband et al., 2009). The average length of participants' narratives was comparable to those in other studies online and offline (Baum and Rude, 2013; McGuire et al., n.d.; Stockton et al., 2014; Smyth et al., 2001).

High adherence in terms of no study and no intervention attrition as well as positive experiences of and with SISU suggest that the software agent provides a useful and acceptable format for participants. User experience (UEQ; Laugwitz et al., 2008) on each of the six subscales (Perspicuity, Dependability, Stimulation, Novelty, Attractiveness, Efficiency) was positive (all M > 0.8). Ratings on any subscale achieved at least an average level when compared to general benchmarks derived from 452 product evaluations and 20,190 participants (Schrepp, 2017). In comparison to these evaluations, SISU was experienced as outstanding innovative and creatively designed (M = 1.43, SD = 0.34). Thus the user experience of SISU could likely compete with interactive products which were evaluated in various contexts (web-based applications, business software) with the UEQ (Schrepp, 2017). This is important, as it is a current challenge in the e-mental-health sector to reduce attrition and improve uptake rates in internet interventions, which can likely be achieved by means of an attractive, persuasive intervention design (Baumeister et al., 2019).

Qualitative responses on acceptability complemented quantitative findings with regards to necessary changes to hardware- and software-related aspects of SISU. SISU should be further developed with regard to the experienced naturalness of the conversational style. SISU is driven by a predefined conversation flow graph as well as associated rule sets and corresponding word lists. On several branches, participants are asked to enter predefined answers (e.g. “End of my narrative”) to trigger the next condition in the flow graph. This might feel unnatural in the conversational process. More individualized feedback, a heightened number of possible responses in the flow graph and associated rule sets as well as the avoidance of repetitions could help to improve the naturalness of the conversational style.

Regarding potential risks, 56.67% of the participants reported at least one negative effect attributed to the intervention (t3). One participant indicated to have suicidal thoughts or ideation for the first time. This emphasizes the absolute importance of corresponding systems for suicide risk detection. These ethical considerations have to inform the process of system design, construction, and use of chatbot-based interventions for mental health issues. One possible access could be via data-driven mechanisms to detect participants at suicide risk. Suicide risk detection systems can e.g. be based on combinations of questionnaires and text inputs of users (Tielman et al., 2019). People at risk could then be referred back to human contact persons (Tielman et al., 2019). Furthermore, participants reported on average M = 1.88 negative effects (SD = 1.22). Most frequently they reported negative changes with regards to the degree to which they suffer from past events since the interaction with SISU (6.7%). On the one hand, this result fits literature that shows, that psychological interventions, online and offline, can initially evoke negative effects and symptom deteriorations (Ladwig et al., 2014; Rozental et al., 2016). On the other hand, results could be explained by the fact that participants indicated on average to live with mild symptoms of depression and anxiety. Prior studies indicate, that therapeutic writing could be less effective in such vulnerable populations (Frisina et al., n.d.), which could again be associated with increased anxiety and depression after.

Negative effects could be directly related to the fact that a large part of the intervention focusses on personally meaningful negative life experiences. Bell-Pringle et al. (2004) state, that focusing on personally meaningful, unprocessed negative events may flood individuals with anxiety (Bell-Pringle et al., 2004). Encouraging participants in that manner to externalize their deepest thoughts and feelings with regards to a certain negative life event could cause harm (Connolly Baker and Mazza, 2004). The lack of collateral therapeutic support which e.g. provides external ideas on problem solving towards the narrated events might leave individuals behind with their current patterns (Lestideau and Lavallee, 2007). It can be derived from literature, that therapeutic writing can be helpful for people and some researcher highlight the potential of the paradigm: Small benefits are consistently reported and the paradigm can be easily provided to a large number of individuals online at low-cost (Kállay, 2015; Frattaroli, 2006; Nyssen et al., 2016). However, innovation isn't all fun and games and there might be other approaches with comparable benefits and lower risks for the individual.

A growing body of literature emerges which suggests the integration of positive psychological interventions to promote wellbeing (Koydemir et al., 2020). Currently, there is rising evidence for the effectiveness of therapeutic writing interventions oriented towards more positive approaches (e.g. writing about positive experiences (Burton and King, 2004) or gratitude (Wong et al., 2018)). A more positive writing approach could be chosen as intervention delivered by SISU as it may come along with less negative effects than the pathological writing focus chosen in the present study.

There are limitations of this study which weaken the generalizability of the findings and motivate future investigations. First, this study was a feasibility trial not designed for the detection of significant effects in psychological outcome measures. The intervention needs to be investigated within the context of a robust large scale study design to reliably detect eventual effects between intervention and control groups.

Second, high adherence in this study might not be over interpreted, as the intervention comprised only two sessions in two consecutive days. Additionally, future studies should comprise more sessions, as more sessions have been found to have a larger effect on overall wellbeing (Frattaroli, 2006). Recent studies reported equally good user adherence rates when investigating chatbot-based interventions (Ly et al., 2017).

Conclusions

Overall, the study showed the potential of SISU to act as therapeutic agent delivering a brief psychological intervention, but not necessarily an intervention that delivers the paradigm of therapeutic writing. Although therapeutic writing can certainly be a powerful tool to improve psychological health (Baum and Rude, 2013; Baikie and Wilhelm, 2005; Craft et al., 2013), the paradigm of has been challenged several times with regards to its effectiveness and potential negative effects (Nyssen et al., 2016; Mogk et al., 2006). In the present study, the use of the paradigm mainly underlines that SISU has the potential to deliver an intervention that is associated with the need for emotion expression. Thus, SISU has the potential to enter the inner world of participants. This could be used to adapt the content of SISU towards a more positive approach. As writing about positive aspects and life events bears the potential to have similar positive effects on physical and psychological wellbeing, SISU could deliver an intervention which is more built upon the importance of positive emotions in the promotion and maintenance of wellbeing (Folkman and Moskowitz, 2000). The trial testing this hypothesis based on a further developed version of SISU alongside the results of this study is registered at the WHO International Clinical Trials Registry Platform via the German Clinical Studies Trial Register (DRKS): DRKS00016799.

Author contributions

Conceptualization and idea, E.B.; methodology, E.B., N.B.; software, B.E., D.M., E.B.; validation, H.B.; formal analysis, E.B.; investigation, E.B.; resources, H.B.; data curation, E.B.; writing – original draft preparation, E.B., writing – review and edition, all Co-Authors; supervision, H.B.; visualization, E.B.; project administration, E.B. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Declaration of competing interest

E.B., B.E., D.M. and N.B. declare no conflict of interest. HB received consultancy fees, reimbursement of congress attendance and travel costs as well as payments for lectures from Psychotherapy and Psychiatry Associations as well as Psychotherapy Training Institutes in the context of E-Mental-Health topics. He has been the beneficiary of study support (third-party funding) from several public funding organizations.