Implementing internet-delivered cognitive behavior therapy for common mental health disorders: A comparative case study of implementation challenges perceived by therapists and managers in five European internet services.

OBJECTIVE: Internet-delivered cognitive behavior therapy programs have been developed and evaluated in randomized controlled trials during the past two decades to alleviate the rising demand for effective treatment of common mental health disorders such as anxiety and depression. While most of the research on internet-based cognitive behavior therapy (iCBT) has focused on efficacy and effectiveness only little attention has been devoted to the implementation of iCBT. The aim of this study was to identify the main implementation challenges perceived by therapists and managers involved in the practical operation of iCBT services in routine care settings in five European countries.
METHOD: The study was designed as a multiple comparative case study to explore differences and similarities between five different iCBT services in Sweden, Norway, Denmark, The Netherlands and Scotland. Field visits were carried out to each of the five services including interviews with the management of the service (n = 9), focus group interviews with key staff (n = 15) and demonstration of online programs. The data material was processed through thematic, comparative analysis.
RESULTS: The analysis generated four transversal themes: 1) integration in the mental health care system; 2) recruitment of patients; 3) working practice of therapists; and 4) long-term sustainability of service. The main results concerned the need to address the informal integration in the health care systems related to the perceived skepticism towards iCBT from GPs and face-to-face therapists, the role of referral models and communication strategies for the stable recruitment of patients, the need for knowledge, standards and material for the training of therapists in the provision of online feedback, the need to improve the possibilities to tailor programs to individual patients, and the need for considerate long-term sustainability planning of the transitions from local projects to permanent regional or national services.
CONCLUSION: The present study gives an overview of the main implementation challenges regarding the practical operation of iCBT services perceived by the therapists and managers of the iCBT services. Future studies into specific details of each challenge will be important to strengthen the evidence base of iCBT and to improve uptake and implementation of iCBT in routine care.

Introduction

Meeting the demands for treatment of common mental health disorders such as depression and anxiety is a major public mental health challenge. Nearly 40% of the population is estimated to be in need of treatment at some time during their life for anxiety and depression (Wittchen et al., 2011) and mental disorders are considered the leading contributor to the burden of disease in both high-income and low-to-middle-income countries (Whiteford et al., 2015). Despite the existence of effective treatments such as psychotherapy (i.e. cognitive behavior therapy (CBT) or Interpersonal Psychotherapy (IPT)), and psychopharmacology, a substantial proportion of people suffering from mental disorders go untreated (Demyttenaere et al., 2004; Thornicroft et al., 2017; Alonso et al., 2007). Structural barriers such as waiting lists, out-of-pocket payments, and physical difficulties in accessing services as well as individual and social obstacles such as perceived stigma, negative attitude to psychological and drug treatment and inadequate health literacy play an important role in choosing to seek diagnosis and access treatment (Andrade et al., 2014).

Internet-based psychological treatment for common mental health disorders may be one way to minimise barriers to treatment delivery and increase access to evidence-based treatment (Andersson, 2016). Potential benefits of internet-based psychological treatment include their ease of access (accessible 24/7 from different locations), cost-efficiency (due to less clinician face-to-face time) and ability to reach a wide range of users. Most internet-based psychotherapy is Internet-delivered cognitive behavior therapy (iCBT). This treatment is typically delivered as an educational software package with interactive components such as symptom questionnaires, multimedia content, and exercises in which users practice skills that can help them cope with their illness. iCBT is delivered over a set period either as pure self-help program or alongside therapist support. In this way, the treatments become automated and geographically independent, which positively impact patient access and therapist capacity.

Several randomized controlled trials and meta-analyses have demonstrated that iCBT can be effective for various mental and somatic disorders with outcomes that are comparable to face-to-face interventions when delivered with some form of professional guidance (Andersson et al., 2014; Andersson and Hedman, 2013; Arnberg et al., 2014; El Alaoui et al., 2015; Hedman et al., 2013; Hedman et al., 2014; Karyotaki et al., 2017; Olthuis et al., 2016). However, the route from promising results based on efficacy studies to routine clinical practice is less travelled with treatment effect at times being miniscule or absent (e.g. Mathiasen et al., 2016). According to Andersson and Hedman (2013), it is possible to transfer iCBT into routine clinical practice with moderate to large effect sizes; however, which service model should be used for iCBT and how to disseminate iCBT is an open question, since few implementation studies of internet-delivered treatments have been performed in routine care settings (Hadjistavropoulos et al., 2014; Ebert and Baumeister, 2017; Hadjistavropoulos et al., 2017; Titov et al., 2017; Nordgreen et al., 2018).

Despite the promising results of iCBT, it has been, and still is, an enormous challenge to transcend the gap from research projects to implementation into routine care. In Europe, the Joint Action on Mental Health and Wellbeing recognises the need for concerted efforts to mainstream e-mental health interventions such as digital treatment programs into routine care (European and Commission, 2016). Known barriers to mainstreaming e-mental health into routine practice include legislative, regulatory, safety and political restrictions, incompatible reimbursement systems, negative attitudes towards internet-based treatment among providers and patients, lack of awareness of internet-based treatment options among patients and providers, high drop-out numbers from treatment, limited availability of adequately trained professionals and limited evidence for cost-effectiveness (Batterham et al., 2015). Research concerning iCBT has so far primarily focused on efficacy and effectiveness trials, with little attention to the process of implementation (Drozd et al., 2016; Vis et al., 2015). Although some authors have identified key components to consider for successful implementation (Andersson and Hedman, 2013; Andersson et al., 2008) very little research has been done on implementation challenges and facilitators relating to the practical operation of iCBT services (Hadjistavropoulos et al., 2017; Nordgreen et al., 2018).

For a diffusion of an innovation to occur, the determinants of implementation, i.e. factors impeding or facilitating the diffusion process, must be known and addressed (Rogers, 2003). As an example, Fleuren et al. (2015) described how determinants of implementation of clinical guidelines in health care should be described on three levels: i) the developers, ii) the professionals/coordinators in the organizations, and iii) the end users (Fleuren et al., 2015). Applying the same logic to determinants of implementation of iCBT in healthcare systems, the efficacy and effectiveness studies are conducted at the first level and studies of the patients on the third level. Not much is known about determinants belonging to the second level.

The aim of the present study was to identify the main implementation barriers perceived by the key actors involved in the practical operation of iCBT services in routine care settings in five European countries with established experience in the field placing it on the second level of the above. The study was designed to inform decisions on dissemination and implementation of internet-based therapy services in routine care settings. Due to the challenges facing mental health care services, it is of great importance that the implementation of iCBT services is informed and qualified by the experiences already accumulated in other European countries. However, no systematic overview exists of European experiences in offering internet-based therapy services as part of routine clinical settings, including target groups, practice in relation to daily operation and organization, as well as barriers and facilitators relating to the implementation and sustainability of internet-based therapy services. We use the term sustainability to denote the continued use of an innovation in practice. This knowledge is crucial to existing services as well as for the development of new internet-based services. The aim of the present study was to describe five European services as examples of iCBT service designs and to identify the main implementation challenges perceived by therapists and managers involved in the online services.

Methods

The study was designed as a multiple comparative case study to explore differences and similarities between cases (Yin, 2014; Crowe et al., 2011; Stake, 2006). To complement earlier implementation studies and to provide an operational focus for the explorative investigation of each of the five iCBT services the study was mainly focused on implementation challenges relating to the practical operation of iCBT. The data collection in each case study combined the following data sources: desk research on each iCBT service, a short online survey covering background information about the iCBT service, a semi-structured interview with the management of the service, and a focus group interview with key staff involved in organizing or performing the clinical activities, as well as relevant researchers attached to each of the five services (Table 2).

Table 2

Case study design.

Aim of study and key study questions	Method	Data sources	Data analysis
To identify the main implementation challenges perceived by the key actors involved in the practical operation of iCBT services in routine care settings in five European countries with established experience in the field.1. How is the practical organization of each iCBT service in terms of relation to the health care system, financing, treatment setting, operation of service and patient pathways? 2. What are the experiences of implementation barriers to the practical operation of iCBT services?	Desk researchShort online surveyQualitative interviewsFocus groups interviews	Scientific literatureiCBT services websitesManager of serviceClinical staffAdministrative staffResearchers	Combined inductive and deductive method focused on thematic categories emerging during data analysis aligned with the aim of the study and the study questions

Field visits were carried out to each of the five services including interviews with the management of the service (n = 9), focus group interviews with key staff (n = 15) and demonstration of online programs.

Case study selection

The study comprised five European supported iCBT services which operate within a routine care context. Each participating iCBT service was conceptualized as a ‘case’, i.e. as a specific unit of analysis in relation to the practical operation of iCBT services. The five iCBT services were selected because of their experience with iCBT as part of the general range of services (either in special iCBT services or services offering blended treatment regimens or iCBT as one option among other kinds of treatment), as well as because of their variation in relation to patient population and uptake area. The selected cases were all embedded within European public sector operated healthcare systems comparable in terms of client need, service provision and funding requirements. Internationally, there are other examples of iCBT services operating in routine care such as THIS WAY UP Clinic (Newby et al., 2016) and the MindSpot Clinic (Titov et al., 2017, Titov et al., 2015) in Australia sponsored by the Australian Government and the Online Therapy Unit (Hadjistavropoulos et al., 2016, Hadjistavropoulos et al., 2017) in Canada. However, for the purpose of this study, European services were included in order to understand the local context for European provision of iCBT. Additionally, European iCBT services were included due to practical considerations such as available time and resources to conduct the study as well as ease of access due to the existence of previous collaboration among the included iCBT services. European research services solely funded through short-term or pilot projects rather than through daily operation were not included in the study. The following five iCBT services were part of the case study (Table 1):

Table 1

Overview of the five iCBT services.

Service and year of introduction into routine care	Diagnoses treated	Support offered and sector affiliation	Duration of intervention	Referral source	Type of diagnostic interview	Funding base	Number of clinical staff
Internetpsykiatrin, Stockholm, Sweden2007	Depression, social anxiety, panic disorder and irritable bowel syndrome	Guided iCBT, secondary care	12 weeks	Self-referral	Face-to-face	Permanent funding, reimbursement and co-payment as in regular care	20 (full time and part-time staff)
eMeistring, Bergen, Norway2015	Anxiety, depression	Guided iCBT, secondary care	14 weeks	GP and self-referral	Face-to-face	Permanent funding, reimbursement and co-payment as in regular care	15 (full time and part-time staff)
Internetpsykiatrien, Odense, Denmark2015	Depression, anxiety	Guided iCBT, primary care	10–12 weeks	Self-referral	Video-based	Temporary funding, no reimbursement and free of charge in contrast to regular care	5 (full time and part-time staff)
eHealth@Mind, Amsterdam, the Netherlands2016	Depression, generalized anxiety, social anxiety, panic disorder, obsessive compulsive disorder	Blended care, secondary care	16–20 weeks	GP and mental health provider referral	Face-to-face	Temporary funding and basic insurance	6 (full time and part-time staff)
Beating the Blues, NHS Tayside, Scotland2007	Depression and anxiety	Unguided iCBT with administrative support, primary and secondary care	10 weeks	GP referral	Face-to-face at referring GP	Temporary funding, no reimbursement and free of charge as in regular care	N/A

Internetpsykiatrin is part of a university psychiatric hospital providing specialist mental health care for adults at Karolinska University Hospital in the southwestern part of Stockholm, Sweden. Internetpsykiatrin is integrated within a regular psychiatric outpatient clinic where patients receive face-to-face treatment. The clinic was established in 2007 preceding rigorous iCBT research since the late 1990s. Swedish mental healthcare is tax-based and public funded. Swedish mental healthcare is mainly government-funded and decentralized, although private health care exists. The health care system in Sweden is financed primarily through taxes levied by county councils and municipalities.

eMeistring is an internet clinic operating at Bjørgvin District Psychiatric Centre, Haukeland University Hospital in Bergen, Norway. eMeistring started out as a research project in 2012 and became part of routine mental health care by January 2015. Today, eMeistring is a permanent part of Bjørgvin District Psychiatric Centre, offering iCBT to patients from western Norway. The clinic is involved in several research programs in collaboration with university departments. Mental healthcare in Norway is financed by national and municipal taxation in combination with user payments.

Internetpsykiatrien is an internet clinic which is part of Centre for Telepsychiatry, a research and development center within the Mental Health Services in the Region of Southern Denmark. Internetpsykiatrien has been operating in routine care since March 2015, with temporary funding running to the end of 2019. It followed a three-year development project running from 2013 to 2015 initiated by a government action plan for the dissemination of telemedicine. Internetpsykiatrien is part of one research project in collaboration with two university departments. Danish mental healthcare is public funded and financed through regional and local taxes.

eHealth@Mind is a newly established e-mental health clinic operating within GGZ inGeest, a large academic mental health organization serving the greater Amsterdam area, the Netherlands. eHealth@Mind opened in March 2016 and is temporarily funded for the starting period. Subsequently the service will be integrated in the regular care financial system. eHealth@Mind continues iCBT work carried out at GGZ inGeest and the VU University since 2012. The clinic is research based and works closely with several university departments. The financing of mental health care in the Netherlands is based on social health insurance, with a small contribution from general taxation and out-of-pocket payments.

The iCBT service Beating the Blues is offered at six different NHS Health Boards in Scotland. The service was introduced in NHS Tayside in 2007 following the service introduction to NHS Forth Valley in 2005. In contrast to the preceding four clinics, Beating the Blues is an automated service without therapist support. The service is managed by an administrative coordinator within the Psychological Therapies Service of NHS Tayside. The program is used as a stand-alone service in primary care and in conjunction with face-to-face psychological treatment services in secondary care. NHS Tayside participates in research project on iCBT. The National Health Service in Scotland is publicly funded from general taxation and National Insurance contributions.

Data collection

The data collection was carried out from June 2016 to October 2016 by an interdisciplinary project team of five researchers with backgrounds in public mental health, clinical psychology and social anthropology (article authors). The data collection in each case study combined the following data sources: desk research on each iCBT service, a short online survey covering background information about the iCBT service, a semi-structured interview with the management of the service, and a focus group interview with key staff involved in organizing or performing the clinical activities, as well as relevant researchers attached to each of the five services (Table 2).

Each case study was initiated with desk research focusing on the online description of the iCBT service as well as relevant research literature covering the efficiency, effectiveness and implementation of the respective iCBT services. Field visits were then carried out to each of the five services. The field visits lasted two days and included interviews with the management of the service, focus group interviews with key staff and demonstration of online programs. Three researchers from the project team participated in each case visit. The interviews followed an interview guide based on desk research focused on the key study questions and scientific literature on core components for effective implementation of interventions (Fixsen et al., 2009, Fixsen et al., 2005). Interview questions for therapists and managers were followed by specific probes and questions were asked in an open and nondirective manner, allowing participants to speak freely about their experiences. At the field visits the researchers recorded detailed notes and observations.

The case studies were carried out in two phases. First, two case studies were carried out in Denmark (June 2016) and in the Netherlands (August 2016). This was followed by a midterm analysis workshop focusing on data collection experiences, the result of which was minor adjustments to the interview guides. Subsequently, the additional three case studies were carried out in Norway (September 2016), Sweden (October 2016), and Scotland (October 2016). In total, 9 managers and 15 key staff members participated in the study. The key staff members were involved in organizing or performing the clinical activities. Out of 15 key staff 3 were administrative staff while 12 were therapists working as clinicians in the iCBT services.

Data analysis

We used qualitative content analysis to analyse the data in which coding categories are derived directly from the text data (Hsieh and Shannon, 2005). Qualitative data from the desk research, the semi-structured interviews, the focus group interviews and the field notes were imported and analysed in the qualitative data analysis software program NVivo. Triangulation (multiple informants and data sources) was used to increase the validity of the results. Initially, all interviews were transcribed according to a detailed transcription guide. Data was analysed through systematic inductive and deductive categorization and text coding (Crabtree and Miller, 1999; Ritchie et al., 2013). The data analysis proceeded in several steps. First, three of the researchers inductively identified 16 preliminary meaning units based on a thorough reading of the field and interview notes. These were discussed at a workshop with all researchers and assessed according to the aim of the study and the key study questions. This lead to a template of 10 thematic codes which were subsequently used to categorize the textual material in NVivo.

The data analysis was further organized in two tracks. In the first track, the qualitative data was analysed with a focus on the details of each specific case in terms of the practical organization of each iCBT service (key study question 1, Table 2). The analysis of each case study was subsequently returned to the managers of each iCBT service for respondent validation. This resulted in a few, mainly factual adjustments. In the second track, a comparative analysis of the data was carried out with a focus on the experiences of implementation barriers to the practical operation of iCBT services (key study question 2, Table 2). In the comparative analysis, the thematic codes were analysed across cases to identify broader transversal themes and to systematize the thematic codes accordingly (Baxter and Jack, 2008). This led to the identification of four transversal themes and the adjustment of the ten initial thematic codes into 12 more specific subthemes. All the researchers have been involved in the data analysis of both tracks. Detailed data of each specific case study, including case-specific barriers and facilitators to implementation, has been published separately (Folker et al., 2017). This article reports the results of the comparative analysis.

Results

The comparative analysis identified four transversal themes relating to the aim of the study, i.e. to identify the main implementation challenges across cases: (i) integration in the mental health care system; (ii) recruitment of patients; (iii) working practice of therapists; and (iv) long-term sustainability of service. Table 3 details the transversal themes and the typical subthemes of each. Typical subcategories are those in which the core ideas within the theme represented 4–5 cases. Each transversal theme and attached subthemes are described below. In quotes, each iCBT service is represented with a number (1–5) the identity of which is known only to the researchers.

Table 3

Transversal themes and subthemes.

Themes	Subthemes
Integration in mental health care system	Formal integration
	Informal integration
Recruitment of patients	Unstable intake
	Model of referral
	External communication
	Match between patients and programs
Working practice of therapists	Professional background
	Required iCBT competences
	Training and supervision
	Recruitment and maintenance
Long-term sustainability of service	Transition from project to permanent service
	Transition from local service to national implementation

Integration in the mental health care system

The five iCBT services included in the study were all part of publicly funded healthcare systems. The services in Stockholm, Odense, Bergen and Tayside were part of healthcare systems funded largely through national and/or regional taxation. In Bergen and Stockholm, the secondary financial resources were out-of-pocket payments. In Amsterdam, social health insurance was the main financial source, along with a small contribution from taxation and user payments. Despite different health care systems, different legislation and different financial systems, the five iCBT services shared the same context in terms of rising numbers of people with symptoms of anxiety and depression, waiting lists for specialized psychiatric and psychological treatment, a rising demand for alternative or flexible treatment modalities and the need to develop effective services that are easy to expand and to disseminate.

The iCBT services in the study were all embedded within larger specialized secondary mental health care organizations. In Stockholm, Bergen, Amsterdam and Tayside, the iCBT service was embedded within a regular outpatient service where patients received treatment face-to-face. In the case of Stockholm, Bergen, Amsterdam and Tayside, the majority of therapists delivering the internet-based treatment also worked clinically in a regular outpatient setting. In Odense, the iCBT service was embedded within a mental health research and development center. Here therapists worked either full- or part-time in the iCBT service or with other development tasks at the center; thus, they did not work in an outpatient service of the host organization.

It was a main finding across interviewees that the integration of iCBT service within the hosting mental health care organizations was judged to be necessary to complete the transition from project to permanent (national) service. Integration was also perceived to be crucial to ensure a steady referral of patients from GPs and other mental health professionals and to recruit and retain qualified health professionals to work in the iCBT services. Finally, integration was found to be vital to establish a sustainable funding base for the service through reimbursement, co-payment or health insurance coverage. All services appeared to be rather well-integrated within routine mental health care systems – especially regarding integration in the existing IT-infrastructure. Several services reported that they shared an electronic journal register system with the health care system where they could communicate with GPs. In some cases, it was also possible for the GPs to refer patients to the iCBT service electronically.

A second subtheme within this theme concerned the recognition of iCBT from other health care professionals. All iCBT services reported that they had to deal with prejudices and negative attitudes regarding iCBT treatment of the referring health professionals and fellow CBT therapists. The iCBT therapists reported that other therapists worried about the quality of the therapeutic relationship and the overall therapeutic quality of iCBT.

“The challenge is often with the other therapists' attitude to internet therapy, because it is new, and they are a bit skeptical: ‘Does this work?’ ‘Is it too superficial?’”

Negative attitudes of GPs were also experienced as an obstacle mainly because of reservations on the part of GPs about referring patients to iCBT.

“A lot of GPs have an attitude like ‘Well I'm not really too keen on it, I don't really believe in it’, but they have not actually looked at it, so it's like saying I don't like this kind of stuff but I have never tasted it.”

Although the services were rather advanced in terms of formal or organizational integration in the mental health care system they all seemed to struggle with issues relating to the more informal or attitudinal integration in the health care system.

Recruitment of patients

Most of the iCBT services experienced an unstable intake of patients, i.e. there was no continuous supply of patients and patient flow was reported to vary in unsystematic and rather unpredictable ways. The managers generally had an aspiration to increase the intake of patients and they all believed in the existence of more patients suitable for iCBT. Three thematic categories were found to be important for the recruitment of patients. The first subtheme concerned the model of referral. The investigated iCBT services operated with either self-referral (n = 2), referral from other clinical services including GPs (n = 2), and a mixture of self-referral and referral (n = 1). Self-referral was clearly considered to be the most viable and promising referral model across interviewees. Two main advantages were envisaged; that patients who actively chose iCBT would be more motivated than referred patients, and that self-referral would ease access and thus reduce waiting time in the system.

“We want to get more patients. I think that is the main reason. And also, because if we have to get referral through GPs or others, it often takes very much time. Generally, it takes a month before a patient gets into the treatment, so self-referral is a way to get them in faster.”

Stigma was also stressed as an access barrier to treatment that would be reduced if patients could enter treatment online without having to contact their GP or to show up in a mental health care setting.

“We hope to recruit patients who say, ‘I don't want to go to a mental hospital and I don't want to sit here. […] So, we are looking for those people who need help but resist going to the mental health hospital. […] It is something with stigma – to lower the stigma.”

No disadvantages were associated with self-referral. However, the role of GPs as gatekeepers on the intake of patients was mentioned as a potential advantage if the services would begin to receive more patients than they could handle.

The second subtheme in this theme included considerations over external communication and marketing of the iCBT services as a vehicle for recruitment. All iCBT services were aware of the need to promote their services by various dissemination channels; they all had a dedicated website, and most of the iCBT services used social media, leaflets and educational videos to attract patients. However, only two of the services had an explicit, systematic communication strategy; one strategy was directly targeting patients through social media such as Facebook, while the other was targeting GPs. Both strategies were tailored to the adopted model of referral designed to address key obstacles to referring. For instance, the communication strategy targeting GPs was deliberately designed from a GP's perspective, i.e. it was easy to approach from the GP's electronic system and the content addressed potential negative referrer attitudes towards iCBT through peer-recommendations, newsletter-updates and oral presentations at conferences. The remaining three services employed a more ad hoc and mixed strategy experimenting with several target groups at the same time. This approach was associated with the services being relatively new and still experimenting with different ways of running their operations.

Generally, the managers were aware that systematic communication is key to increase patient intake and that external communication is an essential aspect of implementation that need to be addressed as an integral part of running an iCBT service.

“Our biggest issue is that we focus so much on the technology aspects that we completely miss the fact that implementation makes it work. […] …The most important thing is not technology, it's marketing, communication and staff engagement.”

The third subtheme that was found to influence the intake of patients concerned the match between patients and programs. Several managers explained that the services spend a lot of time and resources on the admission process because patients are referred who do not match the service. This seemed to be a common problem for services relying on self-referral and services based on referrals from clinical decision-makers.

“Today we offer everyone who has done a self-referral an assessment by a physician. To be honest, I don't think that it's reasonable for us to keep on doing this year after year. […] We have become some sort of unit for diagnosing people and just referring them elsewhere, and we don't have this kind of resources. So, I think that we have to think about how, in quite a near future, we make sure that we get the right patients – the patients to whom we can actually offer treatment programs.”

A related aspect concerned the ability of the programs to properly match the patients. It was mentioned by therapists across services that they found it difficult to change programs in a flexible way during treatment and thus difficult to tailor the treatment to patients' individual needs, if for instance their problems or symptoms changed during treatment.

“The main problem when you can't be flexible enough is when the patient is not - their problems are not related to the program. Like if I get a patient face-to-face, I can start off, ‘This seems like social anxiety’. And then suddenly, ‘No it's post-traumatic stress.’ And then I can change, and I can be flexible. You can't do that in online therapy. […] We are using very primitive technology. And it is very inflexible for different reasons […] We ought to increase the types of programs, improve the programs and the user experience.”

According to the therapists the current nature of the programs posed not only a barrier to the proper treatment of some patients but also a barrier to the intake of patients that do not clearly match the programs for instance because of comorbidity or changing symptomatology. It was envisaged that improved flexibility and functionality of programs would increase the actual intake of patients.

Working practice of therapists

Most of the iCBT services experienced a strong relation between adequate implementation and the working practice of therapists. Four subthemes were found to be related to the working practice of therapists. The first concerned the professional background of therapists. Most of the therapists in the services were psychologists or psychiatrists. The interviewees generally emphasized that knowledge of the patient group in terms of symptoms and disorders was necessary for the therapists to identify lack of progress in the treatment or prevent the treatment from steering in a less effective direction. However, a few interviewees pointed out that people with other professional backgrounds such as psychiatric nurses or specialized social workers might be able to conduct iCBT if they received proper training and supervision.

A second subtheme was the required iCBT competences of therapists. All interviewees stressed that good writing skills were a necessary requirement for therapists. This was mainly seen to be a matter of being very clear and concise in communicating with the patient. According to most interviewees good writing skills were crucial but also difficult to acquire. For most of the therapists, providing written feed-back was not something they had been trained to do as part of their education or postgraduate clinical training. Rather, they have had to develop their style and competences in a trial and error fashion.

“When I wrote my first feedback, there were a lot of assumptions in there that I shouldn't really make. When you are face-to-face, you can check in or use a question mark in your sentence with the intonation of your voice. It doesn't work like that in the text. […] Writing good, concise, and clear feedback, I find difficult.”

Skills in written communication were also perceived to be a matter of being sensitive to the kind of feed-back and style of presentation to which the patient responds including an ability to ‘read between the lines’ for the communication to move forward and to limit the need for further clarifying questions that may slow down the speed of communication and the progress of therapy.

A third subtheme within this theme was the training and supervision of therapists. This was generally considered very important by interviewees. It was a general experience that most of the programs could be relatively easily taught, e.g. in a two-hour training-session, and that they did not require specific technological skills on the part of therapist. Both managers and therapists underscored that online communication was very different than face-to-face communication, and that it should be part of the training to practice written feedback. However, it appeared that no standards had yet been formulated for the provision of online, written feedback.

“In the training, we try to teach people essential parts of written feedback and how you use the feedback […] We have organized the supervision so that it is possible to talk about that kind of problems. What's the best way to do it? What do you think of it? How can we do it better? But we still have to develop lots of knowledge on the scale that therapists need for this way of providing therapy. […] We are only scratching the surface of that topic in our training. We don't go into depth.”

Most services had recurring supervision sessions, where therapists could come and discuss issues relating to online therapy. Use of this supervision feedback option varied considerably among therapists. It was common for therapists to supervise or ‘coach’ each other, and coaching was generally considered to be the most valuable form of supervision. The possibility to consult a colleague during treatment was highlighted as an especially valuable form of support.

“It is very important that we sit together and guide each other, the contact between the therapists is very important. So, that is one thing we hope to do more – to work a little bit closer together.”

A fourth subtheme within this theme concerned the recruitment and retention of therapists. Several managers reported that it was difficult to recruit therapists. Some interviewees suggested that difficulties in recruiting therapists could be tied to assumptions among therapists that online therapy somehow constrains professional freedom and autonomy, thus making it less attractive from a professional point of view.

“We're a bit up against some myths, I think, about at least a hesitation among psychologists about going into this field. And I don't quite know what it stems from. In any case, it's not that they are standing in line, and this puzzles me, you know. But they aren't.”

Retention of therapists was also a critical issue for managers across services. It was experienced as difficult to retain therapists and to provide working conditions sufficiently attractive for qualified therapists to remain at the services for a longer period.

“We are losing good therapists, so this is something that we have discussed more and more during recent years, that we need to find solutions to get people to stay here.”

A related issue was the appropriate amount of iCBT work in a day's work load. Most managers were concerned about finding the proper ratio between iCBT and other tasks including face-to-face therapy. Based on their practical experience some managers suggested that iCBT should not markedly exceed 50% of the workload. Therapists across services pointed out that iCBT can be boring and strenuous if not combined with other kinds of clinical work, such as face-to-face therapy.

“I don't think you can do full time internet therapy. It's too boring. You need the human contact. It is too boring just to sit with the computer all day. […] If things were great in the internet clinic and there were a lot of variation, maybe it would feel differently. I am not sure if that kind of clinic exists anywhere, where there is a large menu of programs and modules and everything is top notch.”

The structured and somewhat inflexible format of online therapy was repeatedly mentioned as a challenge for therapists. The challenge seemed mainly to be connected to the lack of variation and the very structured approach of CBT rather than the online medium as such. Generally, the managers were very aware of the issue of retention and the relative importance of factors such as case load, variation between online therapy and face-to-face therapy, sociability and colleagues, previous experience and the flexibility of online programs in terms of possibilities to tailor programs to individual patients.

Long-term sustainability of iCBT services

All the five iCBT services had successfully managed the shift from project to service operating in routine care. Two related transitions were found to be important for achieving long-term sustainability of service. The first transition concerned the movement from project to permanent iCBT service.

The service managers across the five iCBT services described the difficulties of managing this transition. In addition to the planning decisions about development of programs, stable recruitment of patients, providing attractive working conditions for therapists, and achieving sustainability in terms of operation effectiveness and cost-effectiveness of services the task of mobilizing executive management support for continued operation and working towards peer recognition of service was a recurrent theme across the five iCBT services.

“The task of the project was to create the program and to test it with a certain number of patients. It is an entirely different thing to run a clinic with a long-term view. […] It is a mental shift in the direction of, O.K., we're going to be here for many years, what should the service look like? How do we get a stable patient intake? How do we get hold of the target group that is suited to this form of treatment, and to whom we can make a relevant offer? We need to have some work procedures that are good for the patients and that are good to work with for the psychologists. We need to have a patient flow that makes it sustainable, but which does not produce an excessive overload in the long run either […].”

A related difficulty going from project to routine service was to strike a proper balance between making the service fully operational in terms of patient intake, recruitment of therapists and proper work procedures as well as retaining the spirit of innovation, research and development. All iCBT services in the study were initiated by dedicated researchers and entrepreneurs who had pushed the initial experimental research trials forward and established the iCBT service based on extensive research. Keeping a strong link with research, innovation and development was considered necessary given the fact that there were still many areas of service provision which are not yet evidence-based and that evidence-based practice in most cases has been the main argument for funding and establishing the service. Thus, all the five services were actively pursuing iCBT research trials in parallel with providing iCBT to patients as part of routine care, however, organized differently. Internetpsykiatrin in Stockholm and eHealth@Mind in Amsterdam were research-based to the extent that therapists worked part-time in research and clinical iCBT work. Therapists working at eMeistring in Bergen and Internetpsykiatrien in Odense did not time-share between providing iCBT and doing iCBT research. Instead, staff was hired to do either the one or the other activity. Clinical iCBT work and iCBT research were organized as separate work units.

The second transition relevant for achieving long-term sustainability of service concerned the movement from providing a local and geographically restricted iCBT service to a regional or national provision of iCBT service. Most of the iCBT services in the study were moving towards national implementation at the time of data collection. The degree of national support for wider regional or national implementation varied between the included cases. In Sweden and Scotland, the move towards national implementation was supported by dedicated government funding and implementation support for iCBT service provision. In Sweden, government initiatives included the development of national it-platform for iCBT services and the maintenance of a national quality register for internet-based treatments for common mental health disorders. This contrasts with the situation in Denmark and Norway where national support had mainly taken the form of strategic policy action plans without dedicated funding for national implementation. Consequently, the speed of implementation on a national scale was different, albeit moving in that direction across the five cases.

The scenario of expanding uptake area spurred interest in reorganizing services, such as introducing video-based assessment and procuring new and larger iCBT platforms capable of accommodating larger number of patients. It also introduced organizational difficulties such as negotiating reimbursement systems and registering patient data in electronic health care records across health care regions.

At a policy level, the shift towards iCBT service provision in routine care and the push towards national implementation had been driven by national and regional health policy priorities of preventing common mental health disorders and improving access to psychological therapy. At an institutional level, the service managers across the five services faced the task of leveraging new sources of funding for a permanent service as well as mobilizing the support of executive management of the hosting organization and convincing relevant decision making bodies about the feasibility of providing internet-based therapy alongside conventional care delivery.

Navigating the health policy push towards wider dissemination, mobilizing hospital management support for continued existence as well as ensuring that the service is operating efficiently in terms of patient flows, uptake area, efficient therapist time and convincing treatment outcomes was ongoing and tremendous task for service managers across the five iCBT services.

Discussion

iCBT is evolving rapidly. The potential of digital mental health care and iCBT appears great, though studies of the actual implementation of services remain few. Appraisal of how iCBT operates in real life settings is central to further development of the field. The present case study clearly indicates that iCBT has taken hold in the routine mental health care systems of the respective countries. Though the transition from temporary projects to stable and permanent services is not complete, important steps have been taken in terms of the integration of iCBT into routine care systems.

Facing skepticism

Challenges may remain regarding skepticism towards iCBT that seems to prevail among GPs and face-to-face therapists. In our study, the perceived skepticism of CBT colleagues was mainly linked to worries about the quality of the therapeutic relationship and the overall quality of iCBT aligning with previous studies of CBT therapists and referring-providers' perceptions and attitudes towards iCBT (Topooco et al., 2017; Waller and Gilbody, 2009; Musiat et al., 2014; Hadjistavropoulos et al., 2017). In our study, GPs were also perceived to have negative attitudes towards iCBT though in a more unspecific way. Particularly, reservations about referring patients to the services were perceived to be part of the explanation why the iCBT services grappled with unstable patient intake. Only limited research is available on referrers attitudes towards iCBT. More research on this topic is necessary to properly assess and address the potential barrier regarding referrers attitudes towards iCBT. One Norwegian study among eleven selected GP's has found positive attitudes towards applying iCBT (Wilhelmsen et al., 2014). However, despite their positive attitudes the usage of iCBT was low even after iCBT training of the GPs.

Regarding patients' attitudes it has been found that patients worry about the clinical effectiveness of iCBT (Topooco et al., 2017) which has been shown to correlate with poorer treatment outcome (El Alaoui et al., 2016). In a British study of attitudes towards iCBT in the general population it was found that web-based treatments were expected to be convenient but to have lower appeal and credibility than face-to-face therapy (Musiat et al., 2014). In a study of the use of the iCBT program FearFighter for anxiety on waiting list patients in a Danish secondary routine care clinic no treatment effect was found (Mathiasen et al., 2016) even though the program had been shown to be effective in earlier studies (Marks et al., 2004). As the authors speculate, low treatment expectancy may partly have hampered the treatment effect which was also indicated from interviews conducted with the participants (Mathiasen et al., 2016). Taken together, the present study confirms existing literature on clinicians' worries about therapeutic relationship and patients' worries about treatment effect, emphasizing a need to carefully investigate and address such expectations when implementing iCBT services.

Stable recruitment

A second main implementation challenge in our study was related to the recruitment of patients. Most of the iCBT services experienced an unstable intake of patients, and all wanted to increase their patient intake. Three factors were perceived to be important for recruitment of patients: to increase possibilities for self-referral; to develop explicit systematic communication strategies targeted to the adopted model of referral; and to improve the match between patients and programs e.g. by adapting programs to individual patient needs.

While improved possibilities for self-referral was identified in the study as a way to increase patient numbers this finding should be investigated further based on quantitative data. E.g. results from the MindSpot Clinic in Australia suggest that only 25% of self-referring patients take up free iCBT (Titov et al., 2017). Hence, it may turn out that the demand for iCBT is not as high as assumed by the iCBT community.

While GP referral may be judged to be a barrier for the uptake of iCBT, one study on the effect of referral source on patient characteristics and treatment outcome showed similar characteristics of patients at baseline but superior effect of treatment by the group of patients referred by GPs compared to self-referrals and referrals by mental health professionals (Mataix-Cols et al., 2006). Contrary to the result of our study regarding the potential of self-referral for the recruitment of patients the study by Mataix-Cols et al. (2006) indicates that the potential of GP referral could be explored further.

Also, it has been observed that patients in the clinic, i.e. outside research trials, often do not present one clear psychiatric diagnosis but rather suffer from multiple co-morbid disorders (Andersson et al., 2008). This may support the perception of the interviewees in study that improved possibilities to tailor programs to individual needs and problems rather than specific diseases may serve to increase the intake of patients. The perception of low adaptability of iCBT programs in terms of individual patient needs that often involved multiple conditions shows consistency with a recent implementation study of iCBT in community mental health clinics in Canada (Hadjistavropoulos et al., 2017). Few studies have specifically targeted tailored treatment in iCBT, but it has been indicated that programs which can be adapted to individual patients' conditions may be superior to standardized iCBT programs for higher severity levels of mental health disorders (Carlbring et al., 2011; Andersson et al., 2011; Johansson et al., 2012; Andersson, 2016). The present study highlights the need for issues of patient recruitment and treatment adaptation to be further elucidated particularly with regards to implementation efforts.

Therapists working practice

A third line of implementation challenge in our study was related to the working practice of therapists. Several factors were found to be important in the implementation of iCBT: to develop knowledge, standards and training material for the training of therapists in the provision of online, written feedback, to provide opportunities for continuous peer-supervision, and to improve possibilities to tailor programs to individual patients and for therapists to change between iCBT and face-to-face therapy. Emphasis on the working practices of iCBT therapists as important factors in the implementation of iCBT is also stressed in the literature (Wilhelmsen et al., 2014; Bengtsson et al., 2015; Mol et al., 2016).

It was a main finding of our study that good writing skills are considered necessary to provide effective iCBT. This was found to be a matter of being clear and concise in communicating with the patient, to develop a high degree of sensitivity about what kind of feedback and style of presentation individual patients responds to, and to ‘read between the lines’ for the communication to move forward and to limit the need for further clarifying questions that could slow down the progress of therapy. Our results thus provide supplementary detail to one of the lessons learned from a series of Swedish trials on internet-delivered CBT, where the need for therapists ‘to be good communicators and have sufficient writing skills’ was stressed by the authors (Andersson et al., 2008).

Another lesson from the Swedish trials was the importance of proper training and supervision of therapists and the need for more implementation research on the required form and content of iCBT competence development (Andersson et al., 2008). In a recent scoping review of 164 studies on internet interventions for depression in regular care settings, it was found that the training of therapists was typically brief and consisted of lectures, videos and written material (Drozd et al., 2016). It was also found, that supervision or other forms of therapist support were often not formalized, and that the content appeared to be heterogeneous (Drozd et al., 2016). The findings of the present study confirm these findings. Most services had recurrent supervision sessions where therapists could come and discuss issues relating to online therapy. However, there seemed to be much variation in the degree to which the supervision option was used and therapists generally found the possibility to consult with a colleague during treatment to be the most valuable form of support.

It was also clear from the present study that issues relating to the recruitment and retention of therapists could be investigated further, e.g. the relative importance of case-load, variation between iCBT and face-to-face therapy, possibility for sociability and colleagues, and the flexibility of online programs in relation to individual patients' needs and problems. There seems to be a genuine risk that therapists will find iCBT monotonous and insufficiently challenging after a while. As a solution, it has been suggested that therapists work with both face-to-face and online therapy or work with blended care (Andersson, 2016). Our study supports the need to explore this possibility further. Future studies and service development of iCBT should be just as concerned about issues relating to staff recruitment and retention than previous studies have been on issues relating to the selection of study participants.

Long-term sustainability

The final main implementation challenge of our study concerned the achievement of long-term sustainability relating to the transitional process, whereby the iCBT services were evolving from local, project-like conditions to permanent organizations at a wider regional or national scale. The case study has shown that transitioning from temporary projects to more permanent services is a continuous effort which requires dedicated commitment from therapists and managers alike.

This finding is confirmed by the core implementation component framework where Fixsen et al. describe six functional stages of implementation: exploration, installation, initial implementation, full implementation, innovation and sustainability (Fixsen et al., 2009). According to Fixsen et al. (2009) and Fixsen et al. (2005) implementation stages are not linear or separate. The stages are dynamic within organizations and change as individual and organizational circumstances change. In the scientific literature on implementation most of the commonly cited models of implementation invoke sustainability as a key component. However, as pointed out by Greenhalgh et al. (2004), Fixsen et al. (2005) and Ross et al. (2016) sustainability or maintenance is rarely singled out as a separate implementation challenge and there have been little empirical work examining factors that either facilitate or limit sustainability of health innovations such as iCBT services.

The present study has identified several, potentially interlinked, barriers to successful implementation: skepticism among colleagues, technical limitations and inadequacy to tailor programs to complex needs among individual patients. However, the study has also revealed possible solutions to these barriers. Thus, the study suggests that the innovative scope of iCBT in at least four different dimensions has not yet been fully realized. First, for most iCBT services the geographical uptake area of their services is under discussion. Typically, the debate is about organization and whether iCBT should remain a regional or become a countrywide service. Secondly, the patient target group has not yet been settled. All iCBT services presently offer services to patient groups with a specific set of diagnoses but several institutions consider enhancing their service to include additional diagnoses (bipolar disorder, OCD, stress and various kinds of chronic physical and co-morbid conditions) transdiagnostic treatment options and new kinds of patients (e.g. young people, older people and prisoners). Thirdly, the service models are not yet fully developed. For instance, more iCBT services still experiment with finding the right balance between some level of face-to-face interaction, using the phone as a means of communication and written feedback in addition to blended treatment formats and developing ways to embed iCBT services with the rest of the healthcare system. Finally, most of the iCBT services are in search of more modular IT-platforms that would allow institutions to provide iCBT to patients with co-morbid diagnoses i.e. patients who suffer from several diagnoses rather than one diagnosis.

Future research could aim to answer these questions through systematic experiments with service models and implementation strategies (Powell et al., 2014; Ross et al., 2016). Particularly, development and investigation of implementation methods and strategies specifically aimed at iCBT would be beneficial. Various frameworks and conceptual models of implementation exist within the field of implementation science e.g. the Consolidated Framework for Implementation Science (Damschroder et al., 2009; Hadjistavropoulos et al., 2017), Theory of Diffusion (Rogers, 2003), the RE-AIM model (Bennett and Glasgow, 2009), and Normalisation Process Theory (Murray et al., 2010). Some have already been applied to the field of e-mental health (Hadjistavropoulos et al., 2017), but further studies are needed, which take advantage of these theoretical frameworks and study methods to further illuminate the mechanisms involved in these processes when implementing iCBT. Of particular interest would be studies which were also instrumental or directional about implementation of iCBT as part of their outcome such as the European ImpleMentAll study (http://www.implementall.eu/).

Strengths and limitations

It is a strength of our study that the included iCBT services have been studied in detail in their real-life context. Through observations, detailed interviews and focus group interviews we have been able to provide data on key actors' perceptions of the main implementation challenges in relation to the practical operation of iCBT services that was not previously available. Also, it is a strength of the study that it has involved a purposeful selection of cases operating in routine care in several European countries with different public healthcare systems yet comparable in terms of client need, service provision and funding requirements. Additionally, the data collection has focused mainly on the research questions, while allowing for unexpected issues to emerge. The researchers have taken care to collect sufficient data to allow for an in-depth analysis of individual cases and to provide a comprehensive analysis across cases (Crowe et al., 2011). Finally, it is a strength of the study that respondent validation and triangulation (multiple informants and data sources) have been used to increase the validity of the results.

A limitation of the study stems from its exploratory nature which has not included the collection of empirical data to cross validate the findings. It is also a limitation of the study is that we have mainly focused on implementation challenges. This choice was made to provide a uniform frame of reference for a concise reporting of the explorative data of the case studies but we acknowledge that studies on implementation facilitators could have provided additional important information. It is also a limitation of the study that we have only included European iCBT services. Given the limitations of time and resources, we have not had the opportunity to include iCBT services outside of Europe, nor first-hand perspectives of patients, referring health care providers or the perspectives of political decision makers. Patient data would have provided valuable insight into the interaction between therapists and patients from the patient's perspective, including their experience of the referral process and course of treatment. Interview data from referring health care providers such as GPs would have validated our findings on perceived skepticism towards iCBT. Data from political decision makers would have provided insight into implementation challenges at another level such as the perception of the role of iCBT in the health care system, and the assessment of the economic and transformative potential of iCBT in the development of future mental health care.

Conclusion

The present study adds important information on the main implementation challenges regarding the practical operation of iCBT services in routine care settings in five European countries. The study points to the need to address several issues to improve implementation of iCBT. One major issue is to address the informal integration in the health care systems related to the perceived skepticism towards iCBT from GPs and face-to-face therapists. Another important issue relates to the stable recruitment of patients. This seems to be a matter of investigating the role of referral models, to develop effective communication strategies, and to improve the match between patients and programs and to attract patients most likely to benefit from iCBT. A third major implementation issue relates to the working conditions of therapists. This appears to be a question of further development of knowledge, standards and training material on the provision of online, written feedback. It also seems to be a matter of opportunities for continuous peer-supervision, to improve the possibilities to tailor programs to individual patients and for therapists to change between iCBT and face-to-face therapy. Finally, a major implementation issue concerns the long-term sustainability of iCBT services relating to the two transitions from projects to permanent services and from local to regional or national services. Both transitions are laborious tasks involving the considerate mobilization of institutional and policy level actors.

It is, however, important to highlight the limitations of the results given that the data of the study stems from only five iCBT services. As the field develops it will provide better opportunities to conduct implementation studies which include a larger number of iCBT services. In future studies, it will also be highly relevant to include the perspectives of patients and political decision makers. While the reporting of the present study has focused on the provision of an overview of the main implementation challenges from the point of view of the key professional actors involved in iCBT this has been at the expense of depth in analyzing the details of the specific subthemes of the main transversal implementation themes. Future studies into the details of each theme and subcategories will be important for the further development of the field and the effort to consolidate the evidence base and to improve uptake and implementation of iCBT in routine care.