Implementing guided ICBT for chronic pain and fatigue: A qualitative evaluation among therapists and managers.

INTRODUCTION: Internet-based cognitive behavioural therapy (ICBT) for chronic pain and chronic fatigue syndrome (CFS) has a high potential to increase the number of patients who can receive an evidence based treatment aimed to reduce symptoms and/or disability and to lower burden on (mental) health care. However, implementing a new behaviour-change intervention, and especially an online intervention, has shown to be a challenge. This study aimed to identify factors influencing the implementation process of ICBT for chronic pain and CFS in mental health care.
METHODS: A qualitative study using semi-structured interviews with therapists and managers from twelve mental health care clinics was conducted. Questions and analysis were guided by the Consolidated Framework for Implementation Research (CFIR), covering five domains: (1) the implemented intervention, (2) individual characteristics of the users, (3) the inner setting of implementation, (4) the outer setting, and (5) the implementation process.
RESULTS: In all five domains important facilitators and barriers were found. Key themes were: (1) the quality of the content, its perceived effectiveness and usability, (2) the attitude, self-efficacy and ability to learn new skills among therapists, and motivation to start online treatment among patients, (3) internal communication within a team, existing workload, and top-down support from the management, (4) availability of reimbursement options and marketing strategies, and (5) involvement of all key stakeholders, steering towards independence of the implementation sites during the process and adequate training of therapists.
CONCLUSIONS: This study provides insight in the challenge of implementing ICBT for chronic pain and CFS in daily clinical practice. Several lessons can be learned from the interviews with therapists and managers which can also be more broadly applied to (ICBT) implementation projects in general. Development of practical tools to support the implementation process would be a valuable next step to overcome certain challenges at forehand and to properly prepare for those expected to come along.

Background

In recent two decades, much research has been done on providing traditional mental health treatment in a more flexible, accessible and affordable manner. Currently there is thorough empirical support that cognitive behavioural therapy via the internet (called ICBT) can be (cost)effective for a multitude of psychological disorders and chronic somatic conditions (Carlbring et al., 2018; Hedman et al., 2012; van Beugen et al., 2014). An ICBT trajectory generally has approximately the same length and content as traditional face-to-face CBT treatment (Andersson et al., 2014). ICBT often contains psycho education and interventions aimed at altering unhelpful beliefs and behaviour. It can be unguided (purely self-help), but it is often used in a guided self-help format, in which a patient works largely independently on a set of (tailored) online modules. Guidance is often done by a therapist via encrypted e-mail messages. When needed, phone calls or face-to-face meetings can sometimes be scheduled to provide extra support, motivation or explanation.

For patients suffering from chronic somatic conditions, such as chronic pain and chronic fatigue syndrome (CFS), evidence based treatments exist, that are increasingly provided in mental health care. However, patients do often not receive them. ICBT may be particularly appropriate to overcome barriers for access to treatment. It has been shown to improve both psychological and physical functioning, quality of life and coping behaviour and can reduce symptoms (van Beugen et al., 2014; McCombie et al., 2015; Cuijpers et al., 2008; Janse et al., 2018). Moreover, it can overcome help-seeking barriers related to physical limitations and stigma of attending mental health care (Webb et al., 2017). Furthermore, ICBT may increase treatment capacity, and thereby access, as it tends to take less therapist time.

Although the scientific support for the benefits of ICBT is mounting, implementation in routine healthcare practice falls behind (Vis et al., 2018; Paldam Folker et al., 2018). Many theories and models have been proposed over time, to implement new (online) interventions in daily clinical practice in a structured way (Nilsen, 2015). A comprehensive framework that aims to combine constructs found in a broad array of these theories is the Consolidated Framework for Implementation Research (CFIR) (Damschroder et al., 2009). This framework takes the perspective of individuals in organizations and suggests that barriers and facilitators for the implementation of health services can be identified in five domains: 1) the intervention, 2) the individuals involved in implementation (e.g., healthcare providers and patients), 3) the inner setting (e.g. the organization or units within the organization), 4) the outer setting (e.g. the organization or larger healthcare system), and 5) the implementation process. With regard to the implementation of internet interventions, previous literature reviews have indeed described facilitators and barriers in all these domains (Granja et al., 2018; Mair et al., 2012; Ross et al., 2016). The framework is developed for interventions delivered by caregivers in health services. It could therefore help identify factors influencing implementation of ICBT for chronic somatic conditions in mental health care.

To our knowledge, the barriers and facilitators of implementation of ICBT for chronic somatic conditions in mental health care have not yet been described. In this paper, the evaluation of the implementation of two specific ICBT programs, one for chronic pain and one for chronic fatigue syndrome, is described. These programs, called “Master your Pain” and “Master your Fatigue” respectively, were implemented in a wide range of mental health care clinics in the Netherlands. Both programs were translated from evidence-based regular CBT protocols for these specific conditions. The interventions were found to be effective regarding outcome measures on psychological functioning and fatigue (Janse et al., 2018; Worm-Smeitink et al., 2019a; Worm-Smeitink et al., 2019b; Van Koulil et al., 2010; Ferwerda et al., 2017). The aim of this study was to distill facilitators and barriers experienced by mental health care practitioners and managers, which contribute to the success of implementation of ICBT programs. More insight into these factors may help the integration of ICBT for these populations in clinical practice, by tailoring implementation strategies to the most relevant factors, and probably also may enhance the implementation in mental health care in general.

Methods

The implementation of both ICBT programs was part of a National implementation project to improve the care for patients with medically unexplained somatic symptoms, called Master your Symptoms. Both Leiden University and Radboud UMC were responsible for implementing their own developed and evidence-based ICBT program but during half-yearly meetings with all project partners the strategy and progress was discussed. The implementation process included: finding agreement with managers with respect to the use of the treatment, treatment capacity; instruction of therapists; therapist training and supervision; monthly contact with therapists of each centre to discuss progress and possible threats, such as reorganizations or changes in procedures of routine clinical care that may hamper the project; and PR actions to notify possible referrers of the treatment options.

Interventions

The ICBT programs made use of the same digital platform and are build up in a highly comparable manner. Both programs start with a face-to-face intake, to assess whether the ICBT is indicated. For CFS this was when patients met CDC criteria (revised, 2003 criteria) for CFS or idiopathic chronic fatigue (Worm-Smeitink et al., 2019c), both stating that patients should have severe and ongoing fatigue that leads to impairment in daily functioning and is not explained by a medical or psychological condition. Co-morbidity was allowed when this could not explain the presence of severe fatigue. ICBT was offered to all these patients. For chronic pain ICBT was indicated when the patient had severe pain. It was advised to not use it when comorbidity was present, but this was no strict exclusion criterion. Therapists decided whether ICBT was suitable. The intake was also used to explain the treatment and the online program and (for chronic pain) to set personal treatment goals. Thereafter, patients continued to work via the online platform, on which they could access the six treatment modules that together form the complete online CBT. Examples of modules for chronic pain are: ‘Your goals’, ‘Your mood’ and ‘Your social environment’ (see Appendix A, Appendix B for the content of the program and a screenshot of the interface). Examples for CFS are: ‘Getting started and goal setting’, ‘Regulate sleep-wake cycle’, ‘Gradually increasing my activity’ (see Appendix C for a screenshot of the interface), the full ICBT is described in more detail elsewhere (Janse et al., 2015). The content of both programs included psycho-education, assignments and diary registrations. Master your Fatigue also included video's with patient examples and Master your Pain included a relaxation training program with audio exercises. Patients are guided by the same therapist they had their intake session with. They received therapist feedback weekly or fortnightly in a secured e-mail box in the program. Therapists were trained in a 1.5 day training. The training for chronic pain focused on the delivery of ICBT (Terpstra et al., 2018), as well as the CFS training, while the latter also included using ICBT in stepped care. This was because the ICBT was implemented as a first step of stepped care, in which patients who were still severely fatigued or impaired after ICBT would be offered additional face-to-face CBT (Worm-Smeitink et al., 2019c).

Study design

A qualitative study using semi-structured interviews with therapists and managers from twelve mental health care clinics was conducted between November 2017 and January 2018. All clinics were participating in an implementation project which was being finished at that time. All had been using “Master your Pain” and/or “Master your Fatigue”, during 2 to 4 years. The Consolidated criteria for Reporting Qualitative Research (COREQ) checklist was used as guideline to write this paper (Tong et al., 2007).

Study populations

A purposeful sample of therapists and team managers was planned. From each participating mental health care clinic one or two therapists and a manager (if applicable) were invited via e-mail to participate in a single-time interview in order to evaluate the implementation project. The invited managers were team managers who were involved in the planning and coordinating phase of the implementation. Four clinics were smaller independent practices, in which the manager and therapist role were fulfilled by the same person. Four other clinics were all part of the same larger institute, with a shared manager. In total, interviews were conducted with 14 therapists and four managers. Two invitations were not responded to and one invitation was declined due to time constraints. The mean age of the interviewees was 41.9 (SD = 9.0) for the therapists and 51.8 (11.2) for the managers. Four therapists and two managers were male and 10 therapists and two managers female. The mean number of years of experience with treatment of pain/fatigue was 9.1 (SD = 6.9), with a range from 1 to 20 years. The sample included five health care psychologists, two clinical psychologists, one psychotherapist (these are all protected titles in the Netherlands, based on post-master licensed tracks) and six MSc Psychologists.

Procedure and measures

Interviews were performed by RV and MW, both researchers and project leaders. The interviews took place either face-to-face, at the health care clinic or at the university, or via telephone and lasted approximately 30 min (median: 32 min; range: 21 min–65 min). The interview guide covered the five domains of the CFIR (Damschroder et al., 2009). The interviewee was first invited to reflect about his or her experience with the implementation in general, after which further questions were prompted to ensure that facilitators and barriers regarding all domains of the CFIR were covered. The interviews were audiotaped and transcribed verbatim.

Data analysis

Analysis of the interviews took place in three steps. First, two authors (RV and YB) independently categorized fragments within the interviews and placed them within one of the five CFIR domains. Discrepancies in choice of domain were discussed by the two researchers until consensus was reached. As CFIR is based on research in medical settings and the current study focused on the mental health care setting, the second step in the analysis used an inductive approach to bottom up identify themes within the five main domains. This way, specific factors related to ICBT use in mental health care practice could be distilled. One of the authors (YB) created a coding scheme in which the themes that had come up in a subset of three interviews were divided in the five levels of the CFIR model. This coding scheme was checked by a second author (RV) before it was used to code all other interviews. During the second phase of coding, themes could be combined and any new themes that emerged could be added, resulting in the definitive subdivision in themes. Finally, all themes were divided into either facilitators or barriers.

Results

Facilitators and barriers as mentioned in each of the five CFIR domains are summarized below. Each domain is also represented in a table.

The intervention (Table 1)

An essential facilitating factor as mentioned by the respondents was the perceived quality of the content. Overall the ICBTs were perceived as a proper substitute for their regular face-to-face therapies, covering the same content, providing useful assignments and having a clear structure. Respondents reported to perceive the ICBTs as being effective, seeing improvements among patients who adhered to the programs. Furthermore, the user-friendliness of the ICBTs was mentioned, both regarding the usability of the platform and the level of the content. Many respondents experienced the ICBTs overall to be easy and intuitive to use, both for themselves and for their patients and they valued the option to tailor the modules and assignments to each specific patient. Also, respondents were positive regarding the empowerment of patients. The ICBTs encourage patients to largely work on their treatment independently. This was reported to provide flexibility for the patients and to support them to work on their problems in their own environment and involve significant others in their treatment process. A last facilitator which was mentioned a few times, specifically for the chronic pain module, regarded the possibility to combine the ICBTs with other protocols, for example to use it as an additional support tool, combined with a face-to-face treatment protocol. At the same time, this characteristic was also mentioned as a barrier for implementation. Most therapists reported to feel the need for face-to-face contact with their clients alongside the online programs, as the online intervention did not always suffice in their experience. Therapists reported that patients also indicated this need. This also related to the level of difficulty of the programs. The techniques that patients need to familiarize themselves with can be quite challenging, especially for patients to master mainly on their own. The programs are also rather text-based and communication occurs via text as well, which asks quite a high basic level of reading and writing skills from patients. This resulted in some themes to be mentioned as both facilitators and barriers, namely that the ICBTs were not easy to use and not effective for everyone. For some therapists the large number of treatment components was challenging to keep an overview of what was done by each patient, what needed feedback and what messages still needed an answer. This asked for a new set of treatment skills, which demanded a large time investment during the first times they used the treatment.

Individual characteristics of therapists and patients (Table 2)

Regarding individual characteristics that influenced implementation, respondents mentioned both determinants among therapists and among their patients. The attitude of the therapist is key, which is often expressed in a feeling of confidence and trust in the ICBT, and also confidence in therapist's own skills and working with a strict protocol. Also, the ability to use the ICBTs in a flexible manner was frequently mentioned. Skills to tailor the ICBT to the needs of each individual patient are a prerequisite in order to use the program beneficially. For example, therapists who mentioned they still saw their patients face-to-face from time to time, or who skipped certain assignments if they did not seem appropriate, valued the ICBTs a lot more. This also relates to the self-efficacy that therapists report. Feeling in control of the program and the treatment process was essential. Clear and positive communication about the program towards patients was perceived as very beneficial, also increasing the motivation of patients to work with the program. A large barrier for implementation was that working with the ICBTs asks for quite a large set of new skills, such as writing feedback, keeping patients motivated, and another way of time-management. Gaining these new skills asks for an investment in both time and energy, which was not always available. Communicating via e-mail was also a new skill which was mentioned as barrier several times by therapists involved in the ICBT for chronic pain. They found it for example difficult to formulate their feedback in a correct way via e-mail. For example, they feared that by keeping the e-mails readably short, they would come across as too strict or uninviting. This barrier was not mentioned by the therapists involved in ICBT for CFS. Their therapist training invested more time in training these skills and was partly given by a language expert.

Barriers that related to patients often involved the presence of comorbidity, according to the respondents. Patients with chronic pain or fatigue regularly experience other psychological problems, such as PTSD, depression, or personality disorders. Respondents indicated to expect that ICBT would not be enough to help these patients effectively and would therefore not start an online therapy. Furthermore, the attitude regarding online therapy among patients could be a barrier. Some patients did not want to start with ICBT at all, because they lacked trust, felt hesitance to take responsibility and/or had no interest in computers. Other patients did start, but had problems staying engaged. It was also mentioned that the ICBTs seemed to be particularly useful for a specific subgroup within the patient population, according to some respondents. This subgroup was described as being younger, of the male gender, intelligent, and with an existing interest in computers. Finally, patients often struggle with a low level of energy and concentration, which was described as a “low load capacity”. This made it difficult for some to read the texts in the programs or to even sit behind a computer.

Inner setting - treatment teams and broader organization (Table 3)

Within the treatment teams, internal communication was reported as the most essential facilitator for effective implementation. This communication (e.g. during team meetings) usually lead to internal referrals to ICBT and made sure that the programs were on top of mind of the whole team. Furthermore, within the organization, people on all levels including the managers and the board of directors need to support and encourage the use of ICBT. In teams where this was the case, managers acknowledged the value of ICBT and encouraged their usage. The most frequently named barrier was a lack of support from team members and/or from the management. Often a part of the team was not sufficiently informed about the ICBTs, was not involved in the project and/or was skeptical about the use of online interventions. A too heavy general workload was a second major barrier. Therapists explained that there usually was no extra time available to get acquainted with ICBT programs, and it was not seen as a project that needed investment (e.g. going to the training, set up intervision meetings). Lastly, internal developments were mentioned as a barrier, such as re-organizations (in which sometimes also trained therapists were laid off) or competitive treatment programs taking place during the ICBT implementation process.

Outer setting – health care context (Table 4)

Essential as a facilitator in the outer context of the mental health care clinics is the availability of a reimbursement of the therapy by insurance companies. In the Netherlands this is quite well arranged and online treatment is part of standard compensated health care. Still, many respondents presumed the ICBT programs to be a better fit with primary care, closer to the GP where patients' complaints are less complex, or in clinics where psychological support and physiotherapy is combined. According to them, this would probably provide a better fit between the programs and the patient populations intended to use it. Additionally, marketing for ICBT outside the clinic, towards patients and referrers was also seen as important, since referrers were often not properly aware of the treatment options offered by the clinics. On a more general level, some therapists experienced that online care, or eHealth, was set as an end-goal in their clinic and in Dutch health care in general, creating a wrong incentive to use the ICBTs. They felt as if the use of ICBT was forced upon them by their clinic and by insurance companies, by asking to treat a certain percentage of their patients via ICBT, without attention for feasibility, fit or effectiveness.

Implementation process (Table 5)

Within the broader implementation process, the external support from the National project team was valued as positive and helping. This support included answering questions, supervision during treatment and regular contact via e-mail and phone calls for updates. The quality of the training was named as a second facilitating factor. Respondents valued the information and exercises during the training day and felt it was a good start to take up ICBT as a new form of treatment. Nevertheless, lack of involvement of all stakeholders was mentioned as a barrier. Respondents mentioned that not all essential managers and other team members were involved in the implementation process. Also, some respondents would have appraised more regular reminders, or even extra booster trainings, to keep the ICBT on top of their minds and to further develop their online treatment skills throughout the project. On the other hand, dependence on support from the project team was also mentioned as a barrier, since the support team was always available for questions, a treatment team or individual therapist was not challenged to find their own way around (relatively) simple problems or questions. Lastly, a bad timing of the training was mentioned. Some therapists experienced a large gap between the moment of training and their first ICBT patient.

Discussion

This qualitative evaluation study provides an overview of facilitators and barriers when implementing ICBT for chronic pain and chronic fatigue syndrome in daily mental health care practice, according to therapists and managers. The interviews showed that a broad range of issues on a broad range of levels interact in implementing a new way of working, resulting in a genuine complex process. It seems that part of these issues could probably have been avoided in the preparation and planning phase of the project, which could be improved or overcome in future projects. Some other issues, however, might be unavoidable and are perhaps inherent to implementation of a new intervention. They cannot simply be overcome, but they need to be dealt with along the way, which takes time, effort and a long breath. All in all, from this study it becomes clear that implementing ICBT for somatic symptoms in mental health care is feasible, but nevertheless is a process that needs to evolve and grow over time. Looking at the lessons learned from the whole project and this evaluation study in particular, some core necessities come forward that will probably hold for many implementation processes in general.

First of all, it is key to create overall support from all involved stakeholders. That is, before the start of an implementation project, but also during the course of the project to ensure both initial success and maintenance. Our results indicate that support and enthusiasm to start implementation of a new project needs to be present on all levels of an organization (among the board, team managers, and individuals within a team). It seems that, the more an implementation process feels as a shared goal, the larger the chances are for success. Herein it is valuable to make use of so-called implementation champions, which are the motivated early adopters within a team, but for a sustainable implementation the other members need to be at least informed and positive about the new way of working. This can be difficult in the face of all sorts of practical issues such as time, money and other priorities. That is why a second key to success, which came forward in our data, seems to be a thorough involvement of a few people who are willing to invest and who have the power of persuasion to motivate the larger group. This asks for an extensive preparation, talking about goals and perceived facilitators and barriers from an early point on, before taking actual steps in starting the use of a new intervention. In this phase it also needs to become clear whether there is a right match between what the new intervention can offer and what an organization or team is expecting. Without a proper match or without proper management of expectations disappointments can appear right at the start of a project, abolishing all momentum in one go. Related to that, our study showed that implementation of a new intervention or new way of working is a process, which asks constant effort from many people and which will cause ups and downs which need to be evaluated and which need to be acted on along the way. Lastly, we found that a clinic or institute needs to feel ownership in the implementation process and needs to be able to operate largely independently. If an implementation process is seen as a project, for which an external party is responsible or accountable, it will not last.

Concerning the use of ICBT specifically there are some additional conclusions to be drawn. The key matter that comes forward in our results seems to be that the use of ICBT asks for a certain attitude and a new set of skills among therapists. Especially since these type of interventions are quite demanding for the client or patient who starts working with it, sufficient knowledge and self-efficacy of the therapist with respect to the delivery of ICBT is indispensable. This may be especially relevant for implementation of treatment for CFS or chronic pain, as most therapists in mental health care will have less experience with treating somatic symptoms than mental disorders and feel perhaps less confident in delivering ICBT for somatic symptoms. Proper training could help increase confidence of therapists in ICBT (Terpstra et al., 2018). Furthermore, the therapist needs to be confident that providing therapy via ICBT is indicated for their client, also considering their skills and motivation. A triage system or screening tool (Wentzel et al., 2016) or assessment instruments (Van der Vaart and Drossaert, 2017) could provide relevant support to come to a decision. What also came forward is that it is important that the ICBT is seen as a means to provide therapy, and not as a goal to use eHealth, to lower cost or to increase production time. Therapists really feel that the benefits and content of ICBT need to be leading in the choice to start an ICBT trajectory.

Some strengths and weaknesses of this study may be identified. In this evaluation, a large number of different clinics participated, which implemented ICBT for chronic pain and CFS over a longer period of time. Also, both therapists and managers were interviewed, using an interview scheme based on a theoretical model (Damschroder et al., 2009). This combination provides a thorough insight into the process of implementation of these specific interventions. The total sample size in this study is modest, but data saturation was reached, in which no new information came to the front in three consecutive interviews. An aspect to take into account is that social desirability might have played a role in the interviews, causing the interviewees to be less critical, since the interviews were held by the project managers who had been the contact person throughout the project.

One other aspect to consider in the interpretation of the results, is that the findings are not linked to objective results of the implementation. For example, we did not test whether ICBT was less effective for patients with co-morbidity. This is a perception of (a group of) respondents. To put the statements in perspective, we attempted to give some insight into the proportion of respondents that stated a case (some, many, most). However, as we did not use closed questions, it is not possible to give exact numbers of respondents agreeing with an issue.

Research on implementation of ICBT is still in its infancy and strategies to optimize the process of implementation need to be further developed and shared in the implementation science community. For this particular project and purpose the CFIR very much helped to create a thorough overview of important variables and also in adding structure to all the information available. Despite the existence of this, and many other valuable theories, models and frameworks (Nilsen, 2015), the development of practical tools on how to involve all stakeholders and key people and how to create momentum and continuity within an organization would help to systematically approach implementation.

Conclusions

This study provides insight in the facilitating and hindering of implementing ICBT for chronic pain and chronic fatigue syndrome in mental health care, using all domains of the CFIR. Several lessons can be learned from the interviews with therapists and managers which can also be more broadly applied to (ICBT) implementation projects in general. Development of practical tools to support the implementation process would be a valuable next step to overcome certain challenges at forehand and to properly prepare for those expected to come along.

Abbreviations

Internet-based cognitive behavioural therapy

Consolidated Framework for Implementation Research

Ethics approval and consent to participate

The study has been granted an exemption from requiring ethics approval by the Medical Research Ethics Committee (MREC) of Leiden University Medical Center (LUMC). I confirm all personal identifiers have been removed so the persons described are not identifiable and cannot be identified through the details of the story. All participants provided written consent to participate in the study.

Consent for publication

Not applicable.

Availability of data and material

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Funding

This work was supported by Innovatiefonds Zorgverzekeraars (Innovation Fund Health Insurances; grant number: 2619). Innovatiefonds Zorgverzekeraars was not involved in the study design, data collection, data analysis, data interpretation, nor in publication decisions.

Authors' contributions

RV and MWS created the interview scheme, recruited participants and conducted the interviews. RV and YB analyzed the data. RV, MWS and YB were major contributors in writing the manuscript. MW, HK and AE revised the manuscript critically for important intellectual content. All authors read and approved the final manuscript.

Declaration of competing interest

None of the authors have competing interests to declare.

Table 1

Characteristics of the intervention influencing implementation.

Theme	Example quote
Facilitators
Content quality	The assignment in which patients need to distribute energy points to their activities was a very valuable exercise. It was a real eye-opener for patients (R3)
Perceived effectiveness	I experienced several times that patients with quite complex problems were very satisfied at the end of the treatment (R7)
User-friendliness/usability	Many patients experienced the program to be clear. Patients could find the information by themselves. (R12)
Independence of patients	What patients liked about the program was that they could choose their own pace and own direction, and that I as their therapist was still available through the message box. (R11)
Combination with other protocols	The program can also be done very well in combination with a group. Patients can work with the program at home and then come to group sessions. Therapists can give feedback once a week via the computer, and the other time during the group session. (R7)
Barriers
Need for a combination with face-to-face consults	I personally would prefer to make it a blended program, since I missed the personal contact with my clients (R3)
Level of difficulty	With the ‘think-feel-do assignment’ I sometimes see patients struggle to distinguish between these three. That's quite difficult for many people. (R6)
User-friendliness/usability	I think the layout and interfaces of the programs don't work so well. For me it is not intuitive where to find what. (R13)
Time investment	The main problem with chronic pain patients is that they want to much. They are often overburdened. I regularly heard back that it was too much for them to keep track of everything. (R7)
Insufficient effect for subgroups of patients	I have treated eight patients with GOP until the end, others quitted before the end. I think that for maybe two or three of these eight patients the GOP program was sufficient. The rest really needed more therapy after finishing GOP. (R6)
Keeping an overview	Patients fill in three different diaries and they read pieces of text. I got an e-mail for every diary entry, and these e-mails were not coupled together, so I had to click in each e-mail to see which assignment it was about. Eventually I began to know that during the first module I would receive 5 e-mails after each other, and then a minimum of 3 e-mails per day. So, in five days I had 15 e-mails about diary entrees, adding 5 to 15 e-mails for read texts. That adds up to around 30 e-mails. (R8)

Table 2

Characteristics of the therapists (T) and patients (P) influencing implementation.

Theme	Example quote
Facilitators
Attitude (T)	I really see it as a standard treatment option that we offer. I think it can be just as effective and useful as face-to-face therapy. (R7)
Ability to be flexible (T)	When I noticed patients were struggling, I would call them and discuss which parts of the program were do-able for them and which were not. (R8)
Self-efficacy (T)	You have to sit down and make time to give feedback, but I do know immediately what feedback I want to give. (R7)
Communication about GOP (T)	During the face-to-face intake I try to clarify that they have a choice. Many patients then say that an online program would not work for them. But when I take the time to explain the possibilities and content of the program, it sometimes changes their mind. (R6)
Motivation (P)	Patients who came to us for the ICBT always had found the program themselves, via our website. They came to us and told us ‘this is what I want to do’. (R1)
Barriers
New skills required (T)	Online therapy is a bit difficult for me. I'm doing it, I have to, but I've never really learned about the do's and don'ts. (R12)
Time management (T)	I think that this program takes me relatively more time, because I'm often thinking about how to plan and how to respond. I keep checking the program to see if my patients have done their assignments. (R6)
Communication via e-mail (T)	I stayed very positive and reinforcing. Maybe I had to be a bit more strict, or not. That's something I find difficult with online therapy. (R4)
Comorbidity (P)	What I found difficult, is that many people do not only suffer from fatigue, but also have other psychological problems. Some more than others. That can make it difficult to stick to the process and make progress, to work with the ICBT exactly as intended. (R10)
Negative attitude/non adherence (P)	Patients feel more obliged when I see them face-to-face. So, the fact that they know they have to see me again increases their responsibility to perform their assignments. I notice that online therapy results in less motivation. (R8)
Suitability for a specific subgroup (P)	When I look at the patients who worked with ICBT and accepted it, I see these people are younger, and relatively more often men. They are a bit whizzkids. (R3)
Low load capacity (P)	Some people don't like the idea of reading many texts, or read any text at all. Often because of low energy, or concentration problems. (R4)

Table 3

Characteristics of the inner setting influencing implementation.

Theme	Example quote
Facilitators
Internal communication	I sometimes received emails from colleagues: “would you like to see this patient? Would he/she be someone for ICBT?” (R8)
Support from management and board of directors	The organisation is supporting us. They acknowledge the somatic issues, that it is a specialization that needs attention. So the organisation is facilitating. (R10)
Promotion of ICBT in treatment team	I named the program very frequently during meetings, during treatment indication meetings or other team meet-ups (R8)
Involvement of the right people	I know that in [city] maybe not the right people were sent to the training. There are a couple of informal leaders in that team, and those were not at the training. The difficulty is in choosing the right representative from a treatment team. (R1)
Barriers
Lack of support from the team and management	You're working in a team, but the rest of the team does not know about the program. So I cannot go to anyone with questions. (R2)
General workload of therapists	I find the workload too heavy, so I had difficulty finding time to keep up with the program. I also need to reach my other targets. So I feel like such an extra task feels unwelcome, something I lack the time and space for. (R12)
Internal developments and organization	The misfortune of this project is that it started at an unfortunate moment. It started in the period that we were transitioning and new teams were formed (R1)

Table 4

Characteristics of the outer setting influencing implementation.

Theme	Example quote
Facilitators
Insurance reimbursement	The medical expenses were never an issue. ICBT was compensated by insurance companies, just like other treatments. I have not experienced any obstacles with that. (R10)
Marketing for ICBT towards patients and referrers	We're in newsletters, I have contacted doctors, we have put it on our website and on LinkedIn. (R9)
Barriers
Better fit with primary care	I can imagine that the program would fit at the GP. Because people are often more complex when they come to our clinic. (R4)
eHealth as end-goal	Every year we have to show the insurance companies what percentage of our treatments are digital. So you have to use ICBT. (R3)

Table 5

Characteristics of the implementation process influencing implementation.

Theme's	Example quote
Facilitators
Support from research project	Something I found pleasant was that you were very approachable. That you took initiative yourself to find space in my busy schedule, that you e-mailed and called. This way it stayed in my mind. (R13)
Quality of the training	In the training we received the tip to reserve time for the patient, as it was a real appointment. I found it very important that you offered those trainings. I have learned when to react and when not, use short messages, keep it positive. (R1)
Barriers
Lack of involvement from stakeholders	Taking a look at how my colleagues were thinking about the program, I think it might have helped to get more people involved. If there was more clarity for everyone, the team would be more supporting. (R2)
Lack of reminders and extra training	I only think it would be smart to offer more booster trainings. Or provide frequent tips via e-mail of newsletters, to keep us focused, such as “this is how you send an e-mail on topic A, or this is how you can handle topic B.” […] (R4)
Dependence on support	With technical problems I found it difficult at the beginning that it could take some time before I got an answer. They need the feeling that there is support a la minute. (R10)
Training at bad timing	It took some time before we could start after the training, and that made me forget a lot. So the information that we received during the training was sufficient, we understood. But it took some time before we had our first client, and then I had forgotten about the procedures, it took me time to remember those. (R3)