Clinical and Cost-Effectiveness of PSYCHOnlineTHERAPY: Study Protocol of a Multicenter Blended Outpatient Psychotherapy Cluster Randomized Controlled Trial for Patients With Depressive and Anxiety Disorders.

Introduction: Internet- and mobile-based interventions (IMIs) and their integration into routine psychotherapy (i.e., blended therapy) can offer a means of complementing psychotherapy in a flexible and resource optimized way. Objective: The present study will evaluate the non-inferiority, cost-effectiveness, and safety of two versions of integrated blended psychotherapy for depression and anxiety compared to standard cognitive behavioral therapy (CBT).
Methods: A three-armed multicenter cluster-randomized controlled non-inferiority trial will be conducted comparing two implementations of blended psychotherapy (PSYCHOnlineTHERAPYfix/flex) compared to CBT. Seventy-five outpatient psychotherapists with a CBT-license will be randomized in a 1:1:1 ratio. Each of them is asked to include 12 patients on average with depressive or anxiety disorders resulting in a total sample size of N = 900. All patients receive up to a maximum of 16 psychotherapy sessions, either as routine CBT or alternating with Online self-help sessions (fix: 8/8; flex: 0-16). Assessments will be conducted at patient study inclusion (pre-treatment) and 6, 12, 18, and 24 weeks and 12 months post-inclusion. The primary outcome is depression and anxiety severity at 18 weeks post-inclusion (post-treatment) using the Patient Health Questionnaire Anxiety and Depression Scale. Secondary outcomes are depression and anxiety remission, treatment response, health-related quality of life, patient satisfaction, working alliance, psychotherapy adherence, and patient safety. Additionally, several potential moderators and mediators including patient characteristics and attitudes toward the interventions will be examined, complemented by ecological day-to-day digital behavior variables via passive smartphone sensing as part of an integrated smart-sensing sub-study. Data-analysis will be performed on an intention-to-treat basis with additional per-protocol analyses. In addition, cost-effectiveness and cost-utility analyses will be conducted from a societal and a public health care perspective. Additionally, qualitative interviews on acceptance, feasibility, and optimization potential will be conducted and analyzed. Discussion: PSYCHOnlineTHERAPY will provide evidence on blended psychotherapy in one of the largest ever conducted psychotherapy trials. If shown to be non-inferior and cost-effective, PSYCHOnlineTHERAPY has the potential to innovate psychotherapy in the near future by extending the ways of conducting psychotherapy. The rigorous health care services approach will facilitate a timely implementation of blended psychotherapy into standard care. Trial Registration: The trial is registered in the German Clinical Trials Register (DRKS00023973; date of registration: December 28th 2020).

RCT Entities:   Population Interventions Outcomes

Introduction

The effectiveness of psychotherapy in the treatment of mental disorders has been well-documented (1, 2). However, even in countries with a well-developed health care system, treatment rates are low despite the given demand. In Germany, about 28–63% of people with varying mental disorders in need of treatment remain untreated (3). Psychotherapy as one of the first-line treatments for depressive and anxiety disorders (4–6) is provided for only 10–15% of those who receive treatment (7). One reason for this low utilization rate is the shortage of health insurance covered psychotherapies, documented by waiting times of 3–12 months (3). Other reasons might result from conflicting life tasks and challenges to realize time-consuming psychotherapeutic on-site sessions.

Internet- and mobile-based interventions (IMIs) can provide a means of making evidence-based psychotherapeutic interventions available in a timely manner, thereby contributing to reducing the shortage of care (8, 9). The aim is not only to provide information on possible causes, symptoms, and courses of mental disorders, but also to provide parts or the entire psychotherapeutic process digitally (8). The research on IMIs has so far almost exclusively focused on stand-alone IMIs, i.e., online interventions that are used as an alternative to on-site treatment (8, 9). Numerous clinical trials summarized in several meta-analyses have now shown the effectiveness of IMIs in the treatment of mental disorders (8), particularly well-studied in depressive disorders and anxiety disorders (10, 11). Thereby, guided IMIs for mental and somatic disorders are seemingly as effective as the respective on-site treatments (12). However, the evidence refers to participants who are willing to be treated via IMIs. Studies on the acceptance of IMIs show that this only applies to a small part of the target population regarding both patients (13–16) and therapists (17, 18). In addition, the exclusive remote psychotherapeutic treatment of mental disorders, as is the case with stand-alone IMIs, is largely restricted in Germany by the applicable professional regulations.

Blended psychotherapy, i.e., the combination of online intervention elements with standard psychotherapeutic care, is a rather new field of research (19–22). Recent surveys and qualitative studies amongst psychotherapists indicate that blended therapy approaches would be acceptable considering perceived advantages over conventional psychotherapy, including e.g., bridging distances, flexibility, patient empowerment, and therapist support by standardized materials (17, 18, 23, 24). Acceptance rates are seemingly higher amongst psychotherapists with a background in cognitive behavioral therapy (CBT) compared to other therapeutic backgrounds (17, 18).

From a conceptual point of view, blended therapy approaches can be subdivided into (1) sequential and (2) integrated blended therapy concepts focusing on (a) maximizing the effectiveness of psychotherapy or (b) maximizing the efficiency of psychotherapy (19). Examples for sequential blended therapy concepts are IMIs provided prior to on-site psychotherapy, e.g., during waiting-time (25), or IMIs following on-site psychotherapy, e.g., as inpatient aftercare and relapse prevention (26–28).

Regarding integrated blended therapy, Berger et al. (29) examined whether an Internet-based self-help intervention, when used adjunctive to standard psychotherapy, has an additional effect compared to standard psychotherapy for depression. In this randomized controlled trial, integrated blended therapy was superior over standard psychotherapy (d = 0.51; n = 98). Similarly, Zwerenz et al. (30) documented an incremental effectiveness (d = 0.44; n = 229) of an Internet-based self-help program in addition to psychodynamic inpatient psychotherapy for depression compared to inpatient psychotherapy only.

IMIs could also be used to optimize the efficiency of psychotherapy. It has been well-established that therapeutic guidance is an active component of IMIs, however, with a yet to be examined ceiling effect from which onward more therapist time does likely not translate into clinically significant higher therapeutic benefits for the average patient (31, 32). Hence, one possible way of implementing blended therapy is to iteratively provide standard psychotherapy combined with Internet- and mobile based self-help modules, with the assumption of non-inferiority. As such, blended therapy could represent a means of providing psychotherapy to more patients in need, against the background of restricted resources as present in most health care systems around the world.

PSYCHOnlineTHERAPY aims to examine the potential of integrated blended therapy by comparing standard CBT-focused outpatient psychotherapy (CBTstandard) with two implementation variants of integrated blended therapy, (1) PSYCHOnlineTHERAPYfix as a standardized blended therapy concept combining equal numbers of standard therapy sessions and online intervention modules and (2) PSYCHOnlineTHERAPYflex, providing therapists with the means of combining standard therapy with online interventions modules as perceived fitting for the respective therapy process at hand. In more detail, the project aims to examine:

the non-inferiority of PSYCHOnlineTHERAPYfix/flex in comparison to CBTstandard.

the cost-effectiveness of PSYCHOnlineTHERAPYfix/flex in comparison to CBTstandard.

the safety of PSYCHOnlineTHERAPYfix/flex in comparison to CBTstandard.

qualitative and quantitative details of the implementation variants with regard to acceptance, feasibility, barriers, and facilitators to identify optimization potential.

moderators and mediators of the therapy success as well as potential risks and side effects.

Methods

Study Design

A three-armed multicenter large-scale pragmatic, cluster-randomized controlled trial (cRCT) will be conducted comparing the clinical and cost-effectiveness of two implementations of blended therapy (PSYCHOnlineTHERAPYfix/flex) compared to standard CBT (CBTstandard). Quantitative trial outcomes will be complemented by qualitative interview data on the acceptance, feasibility, and optimization potential of PSYCHOnlineTHERAPY in order to gain in-depth insights in participants' experiences. A smart-sensing sub-study, examining ecological day-to-day digital behavior variables via passive smartphone sensing, aims to provide psychotherapy process insights (33).

This clinical trial has been approved by the ethics committee of the German Psychological Society (DGPs no. BaumeisterHarald2020-07-29VADM) and will be reported in accordance with the Consolidated Standards of Reporting Trials (CONSORT) Statement 2010 and the extensions for reporting pragmatic trials, non-inferiority trials, cluster randomized trials, multi-arm parallel group trials, and trials on psychological interventions (34–39). Qualitative data analyses will be reported following the Consolidated Criteria for Reporting Qualitative Studies (COREQ) checklist (40). Cost-effectiveness analyses will be reported according to Consolidated Health Economic Evaluation Reporting Standards statement [CHEERS; (41)] and the guidelines from the International Society for Pharmacoeconomics and Outcomes Research [ISPOR; (42)]. This trial protocol was created according to SPIRIT guidelines (43). The study has been registered in the German clinical trial register under DRKS00023973.

Participants and Procedure

Cluster Definition

The trial will be conducted in psychotherapy outpatient practices in South-West Germany (Baden-Wuerttemberg) that take part in the PNP (Psychotherapy, Neurology, Psychosomatic, and Psychiatry) selective health care services contract of the health insurance companies AOK Baden-Wuerttemberg and Bosch BKK, managed by MEDIVERBUND AG according to §73c SGB V. This contract defines (amongst others) specific psychotherapy services for patients as outlined below. Clusters are defined by psychotherapy outpatient practices that are run by licensed psychotherapists who are PNP contract partners (i.e., authorized to bill according to the PNP contract; https://www.mediverbund-ag.de/file/4922). Psychotherapists who are not PNP contract partners themselves may participate in the study, if they are employed in a practice owned by a PNP contract partner. More than one therapist per practice is allowed to participate in the study.

Psychotherapists are eligible for inclusion in case of a given informed consent and if they (a) are actively working as a psychological psychotherapist, a medical specialist for psychiatry and psychotherapy or psychosomatic medicine and psychotherapy, another physician working as psychotherapist, or a children and adolescent psychotherapist, (b) are (employed in a practice owned by) a PNP contract partner, (c) hold a CBT license, (d) are available during recruitment and assessment period (self-report regarding no already known time-offs), and (e) are capable of including 12 patients into the study during the recruitment period (18 months; self-report). Enrolment and opening of clusters will take place over the entire recruitment period in order to reach the recruitment target of 900 patients. The recruitment target is a median of 12 patients per participating psychotherapist in order to achieve recruitment of 900 patients.

Inclusion and Exclusion Criteria of Patients

Psychotherapy outpatients of enrolled psychotherapists are eligible for inclusion in case of a given informed consent and if they (a) are ≥ 18 years, (b) have a depressive disorder or anxiety disorder diagnosis eligible to be treated under the PNP contract (medical record of an ICD-10 F32/33.1-0.3; F34.1; F40.00/0.01; F40.1; F40.2; F41.0-0.3), (c) have a health insurance contract with AOK Baden-Wuerttemberg or Bosch BKK as part of the PNP contract according to §73c SGB V, (d) complete the baseline assessment (online assessment and telephone-based standardized clinical interview), (e) have Internet-access and an Internet-capable device (self-report), (f) have sufficient knowledge of the German language (therapist rating), (g) have no ICD-10-F2 diagnosis (therapist rating) as IMIs are not well-examined for this patient group yet, and (h) show no clinical reasons for exclusion (psychotherapist rating). Exclusion criteria are kept at a minimum in this effectiveness trial embedded in standard psychotherapeutic outpatient care. Suicidal tendencies are not defined as exclusion criteria and will be therapeutically handled by the treating psychotherapist according to established standards for crises interventions in standard psychotherapeutic care. In case of acute suicidal tendencies psychotherapists might judge patients as not clinically suitable for blended therapy (criterion h). Respective therapist decisions will be recorded weekly.

Recruitment

Initial recruitment of psychotherapists started in July 2020 and is expected to be finished in July 2021, supported by MEDIVERBUND AG as well as the collaborating professional associations MEDI Baden-Wuerttemberg e.V., Freie Liste der Psychotherapeuten and Deutsche Psychotherapeutenvereinigung (DPtV). First patient in is expected to be included in January 2021. Psychotherapists taking part in the PNP contract were contacted via e-mail and invited to one of four online information events. Furthermore, psychotherapists can express their study interest at www.psychonlinetherapie.de. After informed consent has been given and inclusion and exclusion criteria confirmed, eligible psychotherapists complete baseline assessment. Psychotherapy outpatient practices are consecutively randomized to one of the three trial arms. Psychotherapists are subsequently invited to a 1-day training, which will be tailored to the respective trial arm (PSYCHOnlineTHERAPYfix/flex; CBTstandard). MEDIVERBUND AG assigns a practice structural feature for billing project-specific blended therapy services in accordance with the PNP contract.

Once psychotherapists are allocated to one of the three trial arms and trained for intervention and study protocol adherence, patient recruitment starts. This is expected to take place from January 2021 until June 2022. The psychotherapists assess if patients are potentially eligible for the trial. If this is the case, patients are informed by their psychotherapists about the possibility to take part in PSYCHOnlineTHERAPY. Interested patients will give their informed consent and schedule an appointment for the telephone-based clinical interview online on a tablet provided for study purposes only in the psychotherapy practice. Patients are then invited for the online baseline assessment and the baseline clinical interview is conducted. Once the baseline assessments have been completed, patients enter the trial and are treated for their condition according to trial arm allocation of their psychotherapist following the intervention rational as described below. Follow up assessments will take place as outlined in the flow chart (Figure 1).

Figure 1

Flow chart.

Randomization, Allocation, and Masking

Psychotherapist outpatient practices will be consecutively randomly allocated to (a) PSYCHOnlineTHERAPYfix, (b) PSYCHOnlineTHERAPYflex, or (c) CBTstandard and informed about group membership via e-mail. Randomization will take place at a psychotherapist practices level. That means, therapists in joint practices are randomized jointly into one of the three trial arms to avoid trial arm contamination. Randomization will be conducted by an independent researcher who is blind regarding the study conditions. Whereas, blinding of psychotherapists is not possible, data collectors are blinded regarding treatment condition. Treatment condition is only known by the study personnel administering allocated treatments to psychotherapy practices. Randomization will be stratified by the number of therapists per practice (single therapist vs. more than one therapist). Two randomization lists will be generated by using the web-based programme Sealed Envelope (www.sealedenvelope.com). Randomization will happen on an individual level and an allocation ratio of 1:1:1 will be performed. In case of dropouts and if it becomes apparent that more than 75 therapists are needed to reach the sample size of N = 900 patients, additional therapists will be randomized.

Intervention

PSYCHOnlineTHERAPY provides the mean of combining standard psychotherapeutic care as described below (see control condition) with Internet- and mobile-based intervention modules. Based on psychotherapy practice study arm allocation patients receive either PSYCHOnlineTHERAPYfix/flex or CBTstandard.

All patients with a diagnosed depressive or anxiety disorder as defined in the inclusion criteria can take part in a psychotherapy reimbursed based on the PNP contract. In adult psychotherapy, acute care of up to 10 h is reimbursed (as of October 20th 2020) with 120 €. Initial treatment of a maximum of 20 h is reimbursed at 115 €. Further treatment is possible with up to 30 sessions (105 €). The first max. 16 psychotherapy sessions are defined as trial intervention of PSYCHOnlineTHERAPY. Thereby, the trial definition of 16 sessions follows both trial feasibility considerations as well as psychotherapy dose-response findings, with 4–24 sessions being reported as optimal dose in routine treatment settings (44). Patients of all three trial arms will receive standard PNP-based psychotherapy following the first 16 sessions in case of still existing need of psychotherapy as defined by the therapist. As only psychotherapists with a CBT background are eligible for this trial, PNP-based psychotherapy will be CBT-based.

Control Group—CBTstandard

Patients enrolled in PSYCHOnlineTHERAPY and allocated to the control group will receive standard psychotherapy as described before, following the obligatory diagnostic process in the first few sessions (Table 1). Thereby, therapy follows standard care without a predefined treatment protocol. Details of the psychotherapy provided will be assessed via therapy documentation sheets in order to provide a post-hoc description of standard psychotherapy as provided in standard care.

Table 1

PSYCHOnlineTHERAPYfix/flex vs. CBTstandard.

PSYCHOnlineTHERAPYfix	PSYCHOnlineTHERAPYflex	CBTstandard
on-site diagnostics, indication for psychotherapy and evaluation of suitability for PSYCHOnlineTHERAPY as well as informed consent
max. 16 sessions in a fixed order alternating an online-intervention module followed by a standard psychotherapy session	max. 16 online or standard sessions, amount and order of online and standard sessions as defined by their therapists	max. 16 standard psychotherapy sessions
end of PSYCHOnlineTHERAPY (standard PNP-based psychotherapy following the first 16 sessions in case of still existing need of psychotherapy as defined by the therapist)

Intervention Groups – PSYCHOnlineTHERAPY

Patients allocated to the intervention group will receive PNP-based psychotherapy as aforementioned, combined with Internet- and mobile-based modules (Table 1).

PSYCHOnlineTHERAPY was developed by the Department of Clinical Psychology and Psychotherapy, Ulm University. The content has been specifically tailored to the needs of psychotherapy outpatients as well as their psychotherapists based on experience from a multitude of prior Internet- and mobile-based intervention development processes and clinical trials [e.g., (29, 45–47)]. Patient and psychotherapist feedback on former versions of the intervention modules has been used to further optimize the modules. The overall web-design was revised based on persuasiveness principles (48). Various interactive design elements such as videos, audio files, pictures, and exercises are included in order to optimize user experience and facilitate intervention adherence.

Intervention content consists of seven Internet- and mobile-based modules for depressive disorders, seven for each of the included anxiety disorders, 22 transdiagnostic modules as well as one introductory and one closing module (Table 2) with an estimated proceeding duration of 45–60 min each. They are based on CBT-principles including psychoeducation, many exercises and homework assignments to promote transfer into everyday life. To illustrate therapeutic principles and exercises within the intervention, fictional patients are introduced in the beginning of the intervention and used to illustrate processes, challenges, and possible solutions throughout the intervention modules.

Table 2

Intervention content.

Modules	Content
Introduction	Platform features, presentation of example patients
Depression modules
Psychoeducation	Introduction, symptoms, Lewinsohn's depression model
Lifeline and therapy goals	Drawing lifeline, risk factors, resources, therapy goal setting
Activities	Depression spiral, activity planning
Depression pitfalls	Behavioral patterns, self-observation, problem solving skills
ABC model	Presentation of the model, individual formulation of components
Cognitive restructuring	Beneficial and impeding thoughts, consequences and connection with emotions, restructuring methods
Emotions	Introduction to and cultural rules of emotions, components of emotion
Anxiety modules
Psychoeducation: development of anxiety*	Introduction, symptoms, anxiety levels, diathesis-stress model
Psychoeducation: maintenance of anxiety*	Vicious circle, individual formulation of initiating and maintaining factors
Dealing with anxiety reactions	Safety seeking and avoidance behavior, perception control, exercises
Anxiety process*	Situation exploration, fear hierarchy, worst possible consequences
Motivation	Cost-benefit relation, neglected activities, goal setting
Confrontation*	Confrontation therapy, protocol, exposition exercises in sensu and in vivo
Pleasant thoughts	Reflection of thoughts, stress reducing thoughts, exercises
Closing module	Resources, goal setting, emergency plan, motivation
Transdiagnostic modules
Mindfulness	Introduction, effects of mindfulness, exercises
Physical activity	Importance, recommendations, assessment, goal setting, everyday activity
Sleep	Healthy sleep, sleep disorders, stimulus control exercises
Social competence training I	Social situations and competence, confident behavior, asserting rights
Social competence training II	Social situations and competence, confident behavior, recognizing emotions, managing relationships
Grief	Grief reaction, secondary losses, rumination, detrimental thoughts
Pain	Understanding pain, activity despite pain, rumination, relaxation
Relaxation	Introduction, relaxation and meditation exercises
Relationship and sexuality	Communication, needs, relaxation, massages
Self-esteem and self-image	Importance of self-esteem and self-image, elevating self-esteem, values
Self-compassion	Importance of self-care, learning self-compassion
Loneliness	Loneliness vs. being alone, dealing with loneliness, building relationships, exercises
Gratitude	Importance of gratitude, learning gratitude
Perfectionism	Illusion of perfection, origin and consequences, tolerate and accept imperfection
Procrastination	Understanding procrastination, connection with Internet, importance of self-regulation, exercises to overcome procrastination
Substance use	Problematic substance use and reflection, distraction, alternative behavior
Stress	Meaning of work, connection between work-related stress and mental illness, problem- and emotion-focused stress management
Social media	Problematic usage, SORCK-Model, alternative activities, strategies of self-regulation
Somatoform symptoms	Understanding health and illness, connection between stress and physical complaints, ABC model, physical activity
Acceptance	Short-term problem-solving strategies, primary vs. secondary suffering, learning acceptance
Values and goals	Definition of individual values, goal setting, beneficial key assumptions
Stigma	Diagnostic label, self- and public-stigma, sharing diagnosis

Individual modules for agoraphobia, generalized anxiety disorder, panic disorder, social phobia, and specific phobia.

The intervention is available to participants on eSano, an open source e-health platform developed by Ulm University for providing a technological infrastructure to create and deliver a multitude of IMIs. The platform is divided into three sub-platforms. Intervention content is designed and created in the web-based Content Management System. The intervention is then made available to participants in the cross-platform patient application (web-based, Android, iOS). During the intervention therapeutical guidance can be provided using the web-based e-coach platform. Communication and data transfer between all sub-platforms are end-to-end encrypted with TLS. All eSano systems are located in an isolated network environment, whose interfaces to adjacent networks are regulated by firewalls with appropriate rules. These rules are defined to allow a necessary minimum of communication. The platforms are developed oriented on the requirements of the German Medical Devices Act and the Medical Device Regulation (MDR). Thus, the software development and validation process takes into account the IEC 62304 (safety class B), the GAMP5 (category 4), the General Principles of Software Validation of the FDA as well as the Pharmaceutical Inspection Cooperation Scheme (PIC/S) 11-3.

PSYCHOnlineTHERAPY modules can be used by therapists and patients in varying forms, operationalized in the present study in two versions:

PSYCHOnlineTHERAPYfix: Patients receive alternating online-intervention modules and standard psychotherapy sessions with a fixed ratio of max. eight online and eight standard sessions. Thereby, therapists are free to choose amongst the available intervention modules and in their decision of module order.

PSYCHOnlineTHERAPYflex: Patients receive a flexible number of up to 16 online or standard sessions as defined by their therapist. Thereby, therapists are free to choose amongst the available intervention modules and in their decision of module order as well as frequency.

Both conditions, PSYCHOnlineTHERAPYfix and −flex do not comprise therapeutic guidance in an Internet-based self-help intervention sense of way (31). Therapists are requested to check patients online-session activities prior to the next online or standard session. This process is reimbursed at 20 € within the PNP contract per therapists' check of patients' online-session activities. It is possible to provide a written feedback within the eSano platform, however, the PNP billing code is not designed for this therapeutic intervention guidance, but rather a quality assurance check of estimated 5–15 min time per patient and online session.

Sample Size and Power Calculation

The sample size calculation is based on a random intercept model comparing the primary outcome (PHQ-ADS at T3) between treatment conditions while accounting for the nested structure of the data. Although this model is simpler than the target statistical analyses for the primary outcome presented below, it allows to avoid speculative assumptions about numerous unknown model parameters. The focal hypothesis is that both intervention groups PSYCHOnlineTHERAPYfix/flex are not inferior to CBTstandard (= non-inferiority trial). Non-inferiority is assumed if the confidence interval (CI) of the standardized mean comparissson between PSYCHOnlineTHERAPYfix/flex is completely above SMD = −0.24, which is considered as a lower threshold of clinical significance (49). We assume one-sided tests with α = 0.025 (Bonferroni-adjusted) and 1-ß = 0.8, an intra-cluster correlation coefficient (ICC) of 0.01 with a median cluster size of 12 eligible patients as feasible number to be recruited within the recruitment period. Based on these assumptions, the present trial aims at a sample size of 25 psychotherapy outpatient practice clusters and n = 300 patients per study arm with an allocation ratio of 1:1:1 (PSYCHOnlineTHERAPYfix, PSYCHOnlineTHERAPYflex, CBTstandard; for formulas see (50).

Assessments

All assessments will be conducted online (patient and therapist self-reports) or telephone-based (standardized clinical interview SCID-5, HAM-A, QIDS-C). For an overview of instruments at baseline (T0), inter-session assessments at six (T1) and 12 weeks (T2) follow-up, as well as 18 weeks (T3; assumed as post-treatment), 24 weeks (T4) and 12 months post-inclusion follow-up (T5) see Tables 3, 4. PSYCHOnlineTHERAPY might be continued with further follow-up assessments (2–5-year follow-up assessments) in case of patients' informed consent and given follow-up assessment resources.

Table 3

Overview of the assessments (patients).

Variable	Instrument	Time of measurement
		T0	T1	T2	T3	T4	T5
Online-questionnaires (self-rated)
Depression and anxiety	PHQ-ADS	X	X	X	X	X	X
Quality of Life	AQOL	X			X	X
Patient satisfaction	ZUF-8				X
Working alliance	WAI-SR		X		X
Cost-effectiveness	TIC-P	X			X	X
Sociodemographics	SR	X	X
Risk factors	SR	X
Medication	SR	X			X	X	X
Reasons for dropout	SR				X
Childhood trauma	CTS	X
Social support	F-SozU K-6	X			X
Negative effects	NEQ				X
Suicidal and self- injurious thoughts and behaviors	C-SSRS	X			X
Personality functioning	LPFS-BF 2.0	X
Self-care	SR	X	X	X	X
Self-management	SMST	X	X	X	X
Self-efficacy	MHSES	X	X	X	X
Therapeutic agency	TAI		X	X	X
Empowerment	SR				X
CBT-Skills	CBTSQ	X	X	X	X
Homework implementation	SR		X	X	X
Loneliness	UCLA	X			X
Expectations/Credibility	CEQ	X			X
Attitudes toward blended therapy	APOI	X			X
Individual therapy goals	FRAPT	X			X
Interviews (clinician-rated)
Comorbid disorders, severity, chronicity; remission	SCID-5	X			X
Depression response	QIDS	X			X
Anxiety response	HAM-A	X			X
Serious adverse events	Checklist				X

T0, Baseline; T1, 6 weeks; T2, 12 weeks; T3, 18 weeks post inclusion; T4, 24 weeks post inclusion; T5, 12 months post inclusion follow-up; PHQ-ADS, Patient Health Questionnaire Anxiety and Depression Scale; AQOL, Assessment of Quality of Life; ZUF-8, Questionnaire for Patient Satisfaction-8; WAI-SR, Working Alliance Inventory—Short Revised; TIC-P, Trimbos Institute and Institute of Medical Technology Questionnaire for Costs Associated with Psychiatric Illness; SR, Self-Report Assessment; CTS, Childhood Trauma Screener; F-SozU K-6, Perceived Social Support Questionnaire brief form; NEQ, Negative Effects Questionnaire; C-SSRS, Columbia-Suicide Severity Rating Scale; LPFS-BF 2.0, Level of Personality Functioning Scale Brief Form 2.0; SMST, Self-Management Self-Test; MHSES, Mental Health Self Efficacy Scale; TAI, Therapeutic Agency Inventory; CBTSQ, Cognitive-Behavioral Therapy Skills Questionnaire; UCLA, UCLA Three-Item Loneliness Scale; CEQ, Credibility-Expectancy-Questionnaire; APOI, Attitudes toward Psychological Online Interventions; FRAPT, Fragebogen zur Messung persönlicher Therapieziele [Questionnaire for measuring personal therapy goals]; SCID-5, Structured Clinical Interview for DSM-5; QIDS, Quick Inventory of depressive Symptomatology; HAM-A, Hamilton Anxiety Rating Scale.

Table 4

Overview of the assessments (therapists).

Variable	Instrument	Time of measurement
		T0	T1	T2	T3	T4	T5
Working alliance	WAI-SR-T				X
Sociodemographics	SR	X
Attitudes toward blended therapy	APOI	X			X*
Attitudes toward evidence-based practice	EBPAS-D36	X
Willingness to use digital health interventions	SR	X
Barriers/facilitators in the use of digital health interventions	SR	X
Acceptance of components of blended therapy	SR	X
Determinants of behavioral change	TDF				X*
Degree of normalization	NoMAD				X*

T0, Baseline; T1, 6 weeks; T2, 12 weeks; T3, 18 weeks post inclusion; T4, 24 weeks post inclusion; T5, 12 months post inclusion follow-up; WAI-SR-T, Working Alliance Inventory—Short Revised therapist version; SR, Self-Report Assessment; APOI, Attitudes toward Psychological Online Interventions; EBPAS-D36, Evidence-based Practice Attitude Scale-36; TDF, Theoretical Domain Framework Questionnaire; NoMAD, Normalization Measure Development Questionnaire;

Assessment at the end of the entire study.

Primary Outcome

The primary outcome is depression and anxiety severity at post-treatment 18 weeks post-inclusion (T3), assessed with the Patient Health Questionnaire Anxiety and Depression Scale [PHQ-ADS; 16 items, score range: 0–48; (51)]. Depression and anxiety symptoms at all other assessments will be considered as secondary outcomes. PHQ-ADS is the combined sum score of the questionnaires Generalized Anxiety Disorder Screener (GAD-7) and Patient Health Questionnaire (PHQ-9) as a composite measure of depression and anxiety, with good internal consistency [α = 0.88–0.92; (51)].

Secondary Outcomes

Depression and anxiety remission will be assessed with the Structured Clinical Interview [SCID-5; (52)] as a comprehensive, structured interview designed to be used by trained interviewers for the assessment of mental disorders according to the definitions and criteria of DSM-5. It enables a reliable, valid, and efficient assessment of depressive disorders (52). The SCID-5 will also be used in order to obtain additional information about comorbid disorders, severity of disorders, and chronicity.

Depression response will be assessed with the Quick Inventory of Depressive Symptomatology in its clinician-rated version [QIDS-C; 16 items, score range: 0–27; (53)]. It encompasses the criteria sleep, depressed mood, appetite/weight change, concentration/decision making, self-outlook, suicidal ideation, loss of interest or pleasure, energy/fatigability, and psychomotor changes. QIDS depressive symptom scores are used to determine depression response in accordance to the recommendations of Jacobson and Truax (54). Trained interviewers will conduct the clinician-rated version QIDS-C for which good psychometric properties and internal consistencies between α = 0.81 and α = 0.95 are reported (55).

Anxiety response will be assessed with the Hamilton Anxiety Rating Scale [HAM-A; 14 items, score range: 0–56; (56, 57)] which measures psychic and somatic symptoms of anxiety. Like the QIDS-C and the SCID-5, the HAM-A is clinician-rated and therefore will be conducted by a trained interviewer. It is characterized by a high inter-rater reliability and internal consistency (α = 0.85; 57). HAM-A anxiety symptom scores are used to determine anxiety response in accordance to the recommendations of Jacobson and Truax (54).

Health-related quality of life will be assessed with the self-report questionnaire Assessment of Quality of Life [AQoL-8D; 35 items; (58, 59)] including the unweighted responses subscales physical super-dimension (range: 10–51) and psychosocial/mental super-dimension (range: 25–125) and a total score (range: 35–176). The AQoL-8D is characterized by a high Cronbach's Alpha of 0.96 and good psychometric properties (58).

Patient satisfaction will be assessed with a German short version [ZUF-8; eight items, score range: 8–32; (60)] of the Client Satisfaction Questionnaire [CSQ; (61)]. Higher scores are indicative for higher satisfaction. Internal consistency of the ZUF-8 is reported with a Cronbach's Alpha of 0.90 (60). In addition, reasons for dissatisfaction with the intervention will be assessed with 9 self-developed items.

Working alliance will be assessed with the German version (62) of the Working Alliance Inventory [WAI-SR; 12 items, score range: 12–60; (63)]. It covers the three subscales (a) agreement on tasks (four items), (b) agreement on goals (four items), and (c) development of an affective bond (four items). For the German Version, internal consistencies between α = 0.81 and α = 0.91 were reported for the subscales and internal consistencies between α = 0.90 and α = 0.93 for the total score (62, 64). Participants will complete the WAI-SR at T1 and at T3. Therapists only at T3. This will allow for a comparison between patients' and therapists' view on working alliance.

Psychotherapy adherence will be assessed by means of the number of completed online- and standard sessions. Per-protocol (PP) adherence is operationalized by the percentage of participants that completed their psychotherapy as recommended by their therapist (therapist-rating). Reasons for dropout are assessed by six items at post-treatment (T3; patient-rating).

Covariates

As potential moderating variables, demographic (e.g., gender, age, education) and medical information (e.g., previous treatment, medication) will be recorded at baseline. Further, a variety of potential predictors will be included to assess moderators and mediators of psychotherapy effects.

The following information of therapists will be assessed with six items: age, gender, time since licensed as therapist, number of inhabitants at the location of the practice, and experience with digitally supported psychotherapy. Patient characteristics will be assessed with 15 self-report items including information on age, gender, body height, weight, education, employment, income, relationship status, children living in the household, ethnicity, migration, previous treatment, number of inhabitants of the place of residence, and distance to the practice of the therapist.

Further patient characteristics (risk factors) that potentially predict depression and anxiety symptoms will be assessed by means of 27 self-report items. The following factors will be assessed: smoking, drug use, alcohol consumption, diet quality, social status, minority, discrimination, self-perceived energy, family history of mental illness, adverse childhood experiences (parental death or divorce), accidents, physical and sexual abuse, and physical activity.

Existing medication at the beginning of the study will be assessed at baseline (three items). Potential initiation of new medication will be assessed at post-treatment (two items) and at follow-ups (three items).

Childhood trauma will be assessed with the Childhood Trauma Screener [CTS; five items, score range: 5–25; (65, 66)]. Its internal consistency is reported to be α = 0.76 (65).

Social support will be assessed with the brief form of the Perceived Social Support Questionnaire [F-SozU K-6; six items, score range: 6–30; (67)]. Higher scores indicate higher perceived social support. The measure is characterized by a high Cronbach's alpha of 0.90 (67).

Personality functioning will be assessed with the German version of the Level of Personality Functioning Scale—Brief Form 2.0 [LPFS-BF 2.0; 12 items, score range: 12–48; (68)]. Personality functioning is divided into the two subscales interpersonal-functioning (six items) and self-functioning (six items). The internal consistency of the total scale is reported to be α = 0.82 (69).

Self-management will be assessed with the Self-Management Self-Test (SMST; five items, score range: 0–20; 70). Higher scores are indicative for better self-management competence. The SMST has been shown to have good psychometric properties with a Cronbach's Alpha of 0.80 (70).

Self-efficacy will be assessed with the Mental Health Self Efficacy Scale [MHSES; six items, score range: 6–60; (71)]. The instrument shows a Cronbach's Alpha of 0.89 (71).

Self-care will be assessed by means of four items (score range: 0–34) that were newly developed for this study. They consist of statements about having and taking time, personal resources for oneself, and the engagement with therapy contents at home.

Therapeutic agency will be assessed with two selected subscales of the Therapeutic Agency Inventory [TAI; 10 items; (72)]. The dimensions therapy-related processing (five items, score range: 1–5) and therapist-oriented passivity (five items, score range: 1–5) will be used within our study. Higher scores indicate higher levels of therapeutic agency. Internal consistency of the subscales is reported with α = 0.79 (therapy-related processing) and α = 0.73 [therapist-oriented passivity; (72)].

Empowerment will be assessed with two open-ended questions. Patients will be asked whether and how the intervention contributed to a feeling of strength and confidence and how they experience this in their everyday life.

CBT-skills will be assessed with the Cognitive-Behavioral Skills Questionnaire [CBTSQ; 16 items; (73)]. The measure comprises the two subscales behavioral activation (seven items, score range: 1–5) and cognitive restructuring (nine items, score range: 1–5). Higher scores are indicative of greater use of CBT-skills. Internal consistency is reported to be α = 0.88 (cognitive restructuring) and α = 0.85 [behavioral activation; (73)].

Homework implementation will be assessed with three self-developed items regarding adherence to exercises between the sessions, levels of difficulty in homework completion, and reasons for non-adherence.

Loneliness will be assessed with the UCLA Three-Item Loneliness Scale [UCLA; three items, score range: 3–9; (74)]. Higher scores are indicative for greater loneliness. It is characterized by a Cronbach's Alpha of 0.72 (74).

Expectations and credibility regarding the intervention will be assessed with the Credibility/Expectancy Questionnaire [CEQ; six items; (75, 76)]. The CEQ consists of the two distinct factors credibility (three items, score range: 3–27) and expectancy (three items, score range: 3–27). In order to re-evaluate expectancies of patients, treatment credibility will be assessed again at post-treatment (T3). Cronbach's Alpha of the total scale is ranging between α = 0.84 and α = 0.85, between α = 0.79 and α = 0.90 for the expectancy factor, and between α = 0.81 and α = 0.86 for the credibility factor (76).

Attitudes toward blended therapy will be assessed with selected subscales of the Attitudes toward Psychological Online Interventions Questionnaire [APOI; four items patients/12 items therapists; (77)] which was adapted to blended therapy for this study. For patients the subscale confidence in effectiveness (four items, score range: 1–5) will be used. From the therapist version we will use the following three subscales: skepticism and perception of risk (four items, score range: 1–5), confidence in effectiveness (four items, score range: 1–5), and technologization threat (four items, score-range: 1–5). Depending on the specific subscale higher scores indicate a more negative or positive attitude toward blended therapy. Internal consistency of the total scale is reported to be α = 0.77 with the subscales ranging from α = 0.62–α = 0.72 (77).

Individual therapy goals will be assessed with 4 selected subscales of the “Fragebogen zur Messung persönlicher Therapieziele” [Questionnaire for measuring personal therapy goals] [FRAPT; 31 items; (78)]. The subscales cover the overall goal categories trust in yourself and others (14 items, score range: 0–3), active confrontation with oneself and the disease (nine items, score range: 0–3), coping with depression and anxiety (four items, score range: 0–3), and family and improvements in the family and socioeconomic conditions (four items, score range: 0–3). Internal consistencies of the subscales are ranging from α = 0.67–α = 0.92 (78). Individual therapy goals will be assessed at T0 and their achievement will be evaluated with an adapted version of the measure at T3.

Attitudes toward evidence-based practice will be assessed with selected subscales of the German version of the Evidence-based Practice Attitude Scale-36 [EBPAS-D36; 12 items; (79)]. We will use the scales openness (three items, score range: 0–4), divergence (three items, score range: 0–4), limitations (three items, score range: 0–4), and balance (three items, score range: 0–4). For the English version Cronbach's Alpha is reported to range between α = 0.60 and α = 0.90 for the used subscales (80).

Three newly developed scales will be employed to assess therapists' willingness to use digital health interventions (six items, score range: 0–4), their experiencing of barriers and facilitators in the use of digital health interventions (eight items, score range: 0–4), and their acceptance of components of blended therapy (nine items, score range: 0–4).

Determinants of behavioral change that influence the behavior in health care settings when new interventions are implemented will be assessed with a theoretical domains framework (TDF) questionnaire [32 items; (81)] including the following dimensions: knowledge (four items, score range: 1–7), skills (three items, score range: 1–7), social/professional role and identity (four items, score range: 1–7), beliefs about capabilities (three items, score range: 1–7), optimism (two items, score range: 1–7), beliefs about consequences (two items, score range: 1–7), intentions (four items, score range: 1–7/1–10), memory, attention, and decision processes (four items, score range: 1–7), environmental context and resources (two items, score range: 1–7), social influences (two items, score range: 1–7), and emotion (four items, score range: 1–7).

The Normalization Measure Development Questionnaire [NoMAD; 20 items, score range: 0–80; (82)] assesses the extent to which the newly implemented intervention PSYCHOnlineTHERAPY is a normal part of the daily working routine of therapists. The measure is characterized by a Cronbach's Alpha of 0.89 (83).

Documentation of the therapy process will be performed by participating therapists for each of their patients via tablet at the beginning and at the end of the therapy process and after each session regarding homework assignments, topic of the session, therapeutic techniques, serious adverse events, and orientation toward individual therapy goals.

Mobile Sensing

Mobile sensing data will be collected via the “AWARE” framework [https://awareframework.com; (84)]. In short, the AWARE app allows the collection of data on smartphone usage (e.g., usage time and frequency, GPS data, communication behavior) as well as ecological momentary assessments in form of questionnaires. For technical details on how the data is collected as well as an in-depth description of privacy and data security of the app and the server please see the concept paper of the AWARE framework (84).

After giving consent to participate in the main trial patients are informed about the optional mobile sensing sub-study and asked whether they would like to participate. Participants that provided their additional informed consent are instructed to install the mobile application AWARE on their personal smartphones. After installation participants can choose which of the following data points are collected over the 6 months:

Active Data

Anxiety and depression via the four-item version of the Patient Health Questionnaire [PHQ-4; (85)], drive, sleep quality; data on the use and acceptance of mobile sensing; quality of the application from the user's perspective via the German version of the Mobile Application Rating Scale [MARS-G; (86, 87)].

Passive Data

Duration and frequency of smartphone usage, calls, and SMS; number of words in SMS; usage duration and frequency of installed apps, keyboard input, GPS, type of movement, other movement information (acceleration, rotation, gravity), battery status, screen status, phone events, ambient light, ambient noise, and weather at the location.

Side Effects and Adverse Events

We include different ways of monitoring and assessing side effects and (serious) adverse events [(S)AEs] adapted from the National Institute for Health Research recommendations (88) and Horigian et al. (89) who give general principles to define (S)AEs.

We define AEs as adverse or unintended symptoms or conditions that are inconsistent in nature or severity with the present information about the effects of the intervention. SAEs include the following events: (1) emergency hospitalization due to mental illness, (2) breakdown of a close, important relationship, (3) intoxication with a psychotropic substance requiring medical care, (4) self-injury (intentional) requiring medical care, (5) suicide or suicide attempt, (6) acute psychosis.

(S)AEs may be reported in telephone interviews and during psychotherapy sessions. Psychotherapists and interviewers are required to report (S)AEs to the trial evaluation administration. In addition, the possible occurrence of SAEs is systematically queried at the end of the telephone interview at T3.

Additionally, possible negative effects of the psychotherapies are assessed by means of the 20-item version of the Negative Effects Questionnaire [NEQ; 20 items; (90, 91)] which measures the frequency (score range: 0–20) and impact (score range: 0–80) of several different negative effects during the treatment period. Internal Consistency of the NEQ is reported to be α = 0.95 (90). Moreover, depression and anxiety symptom deterioration are determined by means of the QIDS-C and HAM-A.

Within the online-questionnaires suicidal and self-injurious thoughts and behaviors will be assessed by means of a modified version of the Columbia-Suicide Severity Rating Scale (92) at baseline (T0) and post-treatment (T3). At all times of measurement suicidal tendencies are measured by PHQ-ADS Item 9.

Key Economic Outcomes

Health-Related Outcomes

In the cost-effectiveness analyses, the main outcome will be response according to PHQ-ADS. In the cost-utility analysis, quality-adjusted life years (QALYs) will be the health-related outcome based on the AQoL-8D. The AQoL-8D generates patient preference–based utilities on a scale of 0 (death) to 1 (perfect health), using the time-trade-off method (93). QALY gains will be estimated by calculating the area under the curve (AUC) of linearly interpolated AQoL-8D utilities between measurement points to cover the follow-up period.

Cost Measures

The health-economic evaluation will be based on claims data provided by the statutory health insurance companies AOK Baden-Wuerttemberg and Bosch BKK and the German version of the Dutch cost questionnaire Trimbos Institute and Institute of Medical Technology Questionnaire for Costs Associated with Psychiatric Illness [TiC-P; (94, 95)], which was specifically adapted to the population of psychotherapy outpatients in Germany. Claims data will contain basic information on the insured persons like sex, age, and profession as well as information on costs, diagnoses, and treatments for the following areas: in- and outpatient care, rehabilitation, prescribed medication, and therapeutic appliances and remedies, as well as sickness benefits and disability pension. Each online session will be charged with 20 €. In addition, Ulm University will provide information on costs for providing the digital treatment to psychotherapists. We will use the TiC-P for collecting data on patient and family costs (e.g., out-of-pocket expenses) and productivity costs due to presenteeism (i.e., reduced efficiency while at work). Lost workdays due to presenteeism will be computed by taking into account the number of work days for which the participant reported reduced functioning weighted by the reported corresponding inefficiency score for those days (96).

Qualitative Semi-Structured Interviews

Qualitative semi-structured interviews will be conducted with a sub-group of psychotherapists in a focus group setting and with patients in individual interviews. Trained interviewers will explore acceptance, usage behavior, barriers, and facilitators of PSYCHOnlineTHERAPYfix/flex. The interview-guides will be developed theory-based after a literature review and with the involvement of experts.

A qualitative method with a theory-based approach should gain insights into the perspectives and experiences of both participant groups. Hence, interviews with psychotherapists will investigate barriers to and facilitators for the implementation of blended psychotherapy for depression and/or anxiety. The interview guide for psychotherapists will take into account the TDF (97), that aims to identify domains that influence the implementation of interventions and professionals‘behavior change.

Interviews with patients will provide insights into the participants' experience (e.g., acceptance, feasibility of intervention usage) with the Internet-based interventions which are blended with face-to-face sessions. The interview guide will take into account the Unified Theory of Acceptance and Use of Technology (UTAUT) model (98).

The sample size and composition will be planned to consider the different intervention groups and gain sufficient theoretical data saturation. Both PSYCHOnlineTHERAPYfix and −flex will be represented. Based on sample size guidelines (99), ~20 participants per group (PSYCHOnlineTHERAPYfix/flex) in each study with psychotherapists and patients are estimated to be necessary. However, final sampling follows theoretical data saturation principles (100, 101).

Reimbursement

Trial participants will receive the following compensations for their study related efforts:

Trial psychotherapists will receive 1.000 € for the one-day training course as compensation for their non-realized incomes, 44.80 € for each successfully recruited study patient as compensation for the time necessary to conduct the informed consent process, and 120 € for every provided complete therapy process documentation sheet per patient after completion of the treatment within the study. Additionally, psychotherapists taking part in the qualitative interview will receive 100 € as compensation for their effort and not-realized income in this time.

Trial patients will receive 20 € for completed 18 weeks (T3) and 6 months (T4) follow-up. T5 onwards are not covered by the PSYCHOnlineTHERAPY grant. Hence, compensation can only be realized in case of additional funding. Additionally, patients taking part in the qualitative interview will receive 30 € as compensation for their effort.

Data Management, Quality Assurance, and Safety Measures

All online assessments will be completed via LimeSurvey (installed on an internal server of the University of Erlangen-Nuernberg) and data entered will be transmitted directly and in pseudonymized form to the data-handling center at the University of Erlangen-Nuernberg. For each individual data assessment, an individualized link including the study ID of the participant is used. Participants will receive their individual links by email. For the management of participants in the therapy setting, a tablet with an integrated app is used for the therapy documentation which does not save any data but forwards them to LimeSurvey. The evaluating center will monitor the quality and completeness of the data and the compliance of the measurements with the assessment schedule determined by the study protocol.

Procedure on (S)AEs

Information about potential (S)AEs are obtained within diagnostic interviews, online questionnaires, and during therapy sessions. Whenever a (S)AE is detected, the incidence will be documented by either the therapist or the study personnel. During the treatment period especially the therapist is responsible for dealing with occurring (S)AEs and is informed by the study personnel about (S)AEs identified through answers of patients within the interviews or questionnaires. Critical answers within interviews are identified and documented by the interviewers who are trained in recognizing (S)AEs and provided with a detailed instruction in dealing with suicidality during interviews. Critical answers within questionnaires as defined by a score >1 on PHQ-ADS Item 9 will be forwarded to the therapist. Therapists will have to document (S)AEs and inform the study administration within 48 h on weekdays (SAEs) or within 1 week (AEs). Events occurring during the follow-up period are documented and handled by the study personnel by automatically sending information on help and emergency numbers and offering a meeting appointment via telephone for patients who give critical answers within the interviews or questionnaires. Critical answers as defined as a score >0 on PHQ-ADS Item 9 and selected items of our modified version of the Columbia-Suicide Severity Rating Scale will lead to automatic messages to patients with information on help and emergency numbers at all times of measurement. Documentations of (S)AEs will be forwarded to an independent Data Safety Monitoring Board (DSMB) which will monitor the frequency and severity of the occurring (S)AEs. Every 6 months the DSMB will be informed about documented (S)AEs and the recruitment process and if necessary can give recommendations for discontinuation or modification of the study.

Statistical Analyses

Clinical Analyses

The primary outcome will be analyzed using a three-level random slope model with measurement points nested in patients and patients nested in therapists. We will include data on depression and anxiety severity assessed at four measurement points, including baseline (T0), inter-session assessments at six (T1) and 12 weeks (T2) follow-up, and post-treatment at 18 weeks post-inclusion (T3). Depression and anxiety severity will be predicted from log-transformed time (i.e., weeks since baseline), dummy-coded treatment groups, and their interactions. Random intercepts and slopes will be included to account for unexplained heterogeneity in baseline status and rate of change, both at the level of patients and therapists. The focal tests of non-inferiority will be based on the bootstrap CIs of the interactions between time and groups. All statistical analyses will be performed based on intention-to-treat (ITT) principle. Patterns of missing data will be examined and analyses will be corrected for missing data by applying multiple imputation. Additional multiple PP analyses will be conducted including only patients that provided data.

Economic Evaluation

The economic evaluation will be performed from a societal and a public health care perspective. Two multilevel models (MLMs) will be specified, one for costs and one for effects, which consider the hierarchical structure of the data. For effects, normal-based 95% CIs will be estimated. For costs, 95% CIs will be estimated using bias-corrected and accelerated (BCa) bootstrapping with 5,000 replications (102). MLMs will be combined with cluster bootstrapping, which is recommended for resampling clustered data (103). Across the three treatment groups, the adjusted mean costs and QALYs will be compared to assess if any of the treatments are less effective and more expensive than the other treatments. If so, incremental cost-effectiveness ratios (ICERs) will not be estimated in relation to that treatment (104). Otherwise, ICERs will be estimated by calculating the difference in costs between two treatment options divided by the difference in effectiveness of these two treatment options. The joint uncertainty surrounding costs and effects will be summarized using cost-effectiveness acceptability curves [CEACs; (105)]. CEACs show the probability of an intervention being cost effective in comparison with the alternatives for a range of different ceiling ratios.

Moderator Analyses

Predictors and moderators of treatment outcome will be analyzed on an exploratory basis with a priori defined potential moderators. We will conduct univariate exploratory analyses by entering the respective baseline variable as a three-way interaction term in the three-level random slope model. Moreover, we will also investigate interindividual differences in treatment effects by utilizing the EffectLiteR approach (106), that allows to include interactions between the treatment variable and a range of categorical and continuous (latent) covariates, because it uses a multigroup structural equation model for the estimation of parameters.

The analysis of the mobile sensing data will be divided into two parts. Firstly, correlation analyses will be conducted to investigate associations between digital features (e.g., smartphone usage time) and mental health outcomes. Correlation coefficient r will be used for the analysis, which ranges between 0 (no relationship) to 1 (perfect relationship; −1 perfect negative relationship). For all correlation analyses, the alpha-level will be 5%. P-values will be adjusted for multiple testing using the procedure proposed by Holm (107, 108). Full information maximum likelihood will be applied to deal with missing values in the correlation analysis (108, 109). Secondly, we will build different prediction models to predict mental health. Both “traditional” multilevel models relying on significance tests and machine learning will be used. Building and modifying machine learning models is a highly iterative process with many changing factors to find the optimal model (e.g., architecture, optimizer, included features). Hence, defining a-priori model is impossible. However, a range of approaches will be tested, this includes: Random Forest models, Support Vector Machine (SVM), XGBoost (XGB), K-Nearest Neighbor (KNN), and Logistic Regression [LR; (110–114)].

Qualitative Data Analysis

All qualitative data will be audiotaped and transcribed verbatim using the software MAXQDA. The analysis of qualitative data will be based on qualitative content analysis. An inductive-deductive approach will be applied linked to the theory-based interview guide. To establish reliability of results (indicated by intercoder agreement), two independent raters will code all transcripts on the basis of coding guide and rules. The development of this coding guide follows an iterative process with consensus finding. A follow-up survey with the interviewed samples enable the validation of emergent themes. A higher representativity of the results should enable the subsequent validation of the identified themes by a survey with all participants. Meaningful differences in the identified themes between the two groups PSYCHOnlineTHERAPYfix and −flex will be described through the report of group differences of at least 25%.

Discussion

PSYCHOnlineTHERAPY will be one of the largest ever-conducted psychotherapy trials with 900 patients from 75 psychotherapy standard outpatient care practices. At the same time PSYCHOnlineTHERAPY has the potential of innovating psychotherapy in the near future by extending the ways of conducting psychotherapy on-site, video-conference based, Internet-and mobile-based, or blended using potentially all of these possibilities as examined in PSYCHOnlineTHERAPY. The rigorous health care services approach of PSYCHOnlineTHERAPY, embedded into standard care with even new billing codes as part of the underlying PNP contract for the new psychotherapeutic service of the Internet- and mobile-based intervention modules will ensure the timely implementation of blended therapy into standard care, in case of PSYCHOnlineTHERAPY being effective and cost-effective. Moreover, the elaborated evaluation concept on moderators and mediators including a smart sensing sub-study, will provide deeper insights into the black box psychotherapy, i.e., the active components and the mechanisms of change (32, 115) as well as the differential indication question on which type of psychotherapy and specific technique works for whom at what time in the treatment course (9, 116–118).

Potential Problems and Solutions

In order to avoid selection bias, PSYCHOnlineTHERAPY follows strict trial role division with Ulm University as principle investigator, intervention content and IT-solution provider and interface to health care service providers and stakeholders. Evaluation of PSYCHOnlineTHERAPY takes place at University of Erlangen-Nuernberg. Thereby, the University of Erlangen-Nuernberg has established a work flow ensuring that randomization, administration of participants, and outcome assessments (interviews) will be conducted independent of each other. Particularly outcome assessors are kept blind toward trial arm allocation. At the beginning of each telephone interview, patients are asked to keep their trial arm allocation disclosed.

Trial arm contamination might occur in joint practices. Hence, psychotherapists in joint practices are randomized and allocated jointly in the same trial arm condition. Resulting clustering effects will be compensated statistically. Different to cRCTs in general practices patient-level trial arm contamination is rather unlikely, as patients do not switch psychotherapists in due course or during times of absence (e.g., vacation) of their psychotherapists. Patients will be asked whether they already took part in PSYCHOnlineTHERAPY before.

Considering that therapeutic alliance, which depends also on the characteristics of the psychotherapists, is one of the most important variables predicting the outcome of psychotherapy (119, 120), randomizing therapists runs the risk of clinically important baseline imbalance. Cluster size was optimized for the pre-specified non-inferiority margin, however, between trial arm baseline imbalance regarding therapist competency needs to be considered carefully when interpreting the final trial results.

Study and intervention protocol adherence will be improved by structured training courses, regular contacts with study psychotherapists as well as assessed via therapists' documentation of their psychotherapies.

Conclusion

The present cRCT PSYCHOnlineTHERAPY aims to examine the non-inferiority and cost-effectiveness of two blended therapy versions embedded in standard care. As one of the largest psychotherapy trials ever conducted, PSYCHOnlineTHERAPY has the potential to innovate the way of how psychotherapy is provided, thereby exploring its mechanisms of change at the same time.

Trial Status

Patient Recruitment start is scheduled for January 2021.

Dissemination

Trial results will be presented on national and international conferences and published in peer-reviewed journals. Access to the final trial dataset can be provided on request depending on to be specified data security and data exchange regulation agreements.

Ethics Statement

The studies involving human participants were reviewed and approved by Ethics Committee of the German Asssociation of Psychology. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

HB, DDE, MR, and RP, MEDIVERBUND AG, AOK Baden-Wuerttemberg, and Bosch BKK obtained funding for this study. HB, NB, A-CZ, LB, CH, PM, TS, MS, LSteu, YT, IT, and DE contributed to the study design. ARI, RK, TN, RP, MR, and LSten developed the platform eSano and contributed to the technical study design. CB contributed to the design of the health-economic evaluation. HB drafted the manuscript and is principle investigator of PSYCHOnlineTHERAPY. All authors contributed to the article and approved the submitted version.

Conflict of Interest

Authors at Ulm University were partly involved in the development of PSYCHOnlineTHERAPY. Therefore, evaluation of the trial will be independently conducted by the evaluator at University of Erlangen-Nuernberg. HB received consultancy fees, reimbursement of congress attendance, and travel costs as well as payments for lectures from Psychotherapy and Psychiatry Associations as well as Psychotherapy Training Institutes in the context of E-Mental-Health topics. He has been the beneficiary of study support (third-party funding) from several public funding organizations. DE possesses shares in the GET.On Institut GmbH (HelloBetter), which works to transfer research findings on IMIs into standard care. DE has received payments from several companies and health insurance providers for advice on the use of IMIs. He has received payments for lectures from Psychotherapy and Psychiatry Associations and has been the beneficiary of third-party funding from health insurance providers. IT has received fees and travel costs for lectures or workshops in the eHealth setting from congresses and psychotherapy training institutes. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.