Acceptance of E-mental health interventions and its determinants among psychotherapists-in-training during the first phase of COVID-19.

Background: Although E-mental health (EMH) interventions have been shown to be effective in the treatment of mental health problems and empirical knowledge regarding EMH acceptance for different occupations in health care is established, little is known regarding EMH and psychotherapists-in-training. This seems particularly relevant as psychotherapists-in-training will shape the future health care system since they are as being the next generation of psychotherapists. With social distancing measures in place, COVID-19 has led to an increased demand for EMH, which is broadening the way psychological treatments are delivered. Objective: The present study aims to assess the acceptance of EMH and its determinants among psychotherapists-in-training of different EMH modalities and to retrospectively compare current acceptance with pre-COVID-19 times.
Methods: Altogether, 29 training institutions in Switzerland and 232 training institutions in Germany were contacted, resulting in a sample of N = 216 psychotherapists-in-training (88.4 % female) who filled out the self-administered web-based questionnaire in summer 2020. The acceptance of EMH was assessed considering several different modalities (e.g., videoconference, guided self-help programs) as well as further possible predictors of EMH acceptance based on the Unified Theory of Acceptance and Use of Technology. Acceptance scores were categorized as low, moderate or high based on prior research and predicted using multiple regression.
Results: Acceptance of EMH was moderate (M = 3.40, SD = 1.11) and increased significantly (t(215) = 12.03, p < .01; d = 0.88) compared to pre-COVID-19 (M = 2.67, SD = 1.11); however, acceptance varied significantly between modalities (F(2.6, 561.7) = 62.93, p < .01, partial η2 = 0.23), with videoconferencing being the most accepted and unguided programs the least. Stepwise regression including three of 14 variables (R2 = 0.55, F (14, 201) = 17.68, p < .001) identified performance expectancy, social influence and concerns about the therapeutic alliance as significant determinants of EMH acceptance. Discussion: Acceptance by psychotherapists-in-training was moderate and in line with prior research and comparable with other clinicians' acceptance scores. Performance expectancy, social influence and concerns about the therapeutic alliance were predictive of EMH acceptance, indicating their significance in the implementation of EMH in health care.
Conclusion: These findings underline the importance of the aforementioned determinants of EMH acceptance and the need for further studies investigating EMH acceptance in order to derive adequate educational programs and to facilitate dissemination among psychotherapists-in-training.

Introduction

The mental health system faces the challenge of providing the best possible care through effective and efficient treatment for persons with mental disorders. However, earlier studies showed that the prevalence of mental disorders in the general population and the proportion of people with mental disorders receiving treatment diverge widely, resulting in a so-called treatment gap (Kohn et al., 2004; Kohn et al., 2018; Patel et al., 2010). This treatment gap is, for example, reflected in delays in initial treatment contact after the onset of mental disorders (Pratt and Brody, 2014; Wang et al., 2002) and in the selection of treatments being often not evidence-based. Possible reasons are a preference for self-help (Kessler et al., 2001), stigmatization (Collins et al., 2004) and limited availability of evidence-based treatment (Bower and Gilbody, 2005). E-mental health (EMH) could help to counteract the previously mentioned barriers and supplement routine care (Casey et al., 2013; Kazdin and Blase, 2011; Kazdin and Rabbitt, 2013; Vis et al., 2018). Even though the effectiveness of EMH interventions and their research methods are still being assessed, recent RCTs and meta-analyses are highlighting the effectiveness for certain mental disorders and specific interventions (Berryhill et al., 2019; Mayo-Wilson and Montgomery, 2013; Richards and Richardson, 2012; Riper et al., 2011; Simblett et al., 2017). The generalizability of results on the effectiveness of EMH interventions should be interpreted with caution; thus, EMH as an umbrella term encompasses virtual reality (e.g., Diemer and Zwanzger, 2019), smartphone apps for depression (e.g. Fitzpatrick et al., 2017) or therapy by videoconference (e.g. Steel et al., 2011). Overall, the development and evaluation of interventions to complement mental health care has increased significantly in recent years and will continue to increase in the coming years (Gaebel et al., 2020), as EMH was found to be cost-effective (Donker et al., 2015) and to have the potential to overcome several barriers to patient care (Casey et al., 2013; Mistry, 2012) as well as to improve outcomes and processes (Lingg and Lütschg, 2019).

Acceptance of EMH among providers in mental health care

The acceptance of EMH is an important factor that has a significant influence on its implementation in routine care (Vis et al., 2018). The Unified Theory of Acceptance and Use of Technology (UTAUT; Venkatesh et al., 2003) is used in many studies to explain the adaptation of various technologies and has proven useful in studies regarding the acceptance of EMH in a medical context (Hennemann et al., 2017; Philippi et al., 2021). The UTAUT postulates four different positive predictors of behavioural intention: performance expectancy (individuals believe that the use of a technology will be beneficial), effort expectancy (expected ease of use), social influence (expected attitude of significant others towards using the technology) and facilitating conditions (organizational or technical resources and preconditions for technology use). Evidence regarding different technologies in a medical context suggests that performance expectancy is the most important predictor of acceptance (Dünnebeil et al., 2012; Dwivedi et al., 2011; Hennemann et al., 2017; Li et al., 2013; Taiwo and Downe, 2013). In the meantime, however, there are also many extended UTAUT models (cf. Apolinário-Hagen et al., 2019; Hennemann et al., 2016, Hennemann et al., 2017) with additional predictors of acceptance (e.g., “Perceptions of Risk” cf. Apolinário-Hagen et al., 2019). Earlier studies showed that acceptance of EMH in general among a mixed sample of clinicians working in inpatient treatment (e.g., nurses, psychologists and physical therapists) can be rated as moderate (Hennemann et al., 2017) and that most psychotherapists perceive themselves as not trained well enough in EMH (Perle et al., 2013).

Another study showed that the acceptance of EMH among health psychotherapists seems to be significantly lower than that among patients (Schröder et al., 2017), and it is argued that the resulting low adoption of EMH by clinicians seems to be a systematic pattern in multiple European countries (Hennemann et al., 2017). A systematic review by Waller and Gilbody (2009) revealed that people with mental disorders and general practitioners hold more positive attitudes towards EMH than psychotherapists, but studies again show that psychotherapists have reported a somewhat positive attitude towards EMH during COVID-19 (Békés and Doorn, 2020). Studies regarding the acceptance of EMH among psychotherapists-in-training have not been conducted until now Considering the facts that psychotherapists-in-training will shape the future health care system and that they provide insight to the educational system of psychotherapy training, they seem to be an undervalued source of information. Psychotherapists in training are at the very beginning of their career, which can, in turn, be used to improve the health care system at its very roots in a long-term and effective way. Past research has shown that qualified psychotherapists are influenced by their professional training (Lucock et al., 2006), this underlines the importance of carefully developed training curriculums and the consideration of psychotherapists-in-training's current attitude towards EMH. Additionally, there are several differences between psychotherapists-in-training and fully qualified therapists. Evidently, qualified psychotherapists are older than their counterparts in training, presumably have less experience with EMH and current psychotherapists-in-training have been growing up with the internet (Prensky, 2001). Studies also found that psychotherapists-in-training value different aspects of their training compared to practicing therapists (Rocco et al., 2019). Accordingly, it remains unclear whether acceptance of EMH among psychotherapists-in-training is comparable with prior findings in qualified therapists.

Determinants of EMH acceptance

Existing research has found a positive correlation between knowledge about EMH and its acceptance (Donovan et al., 2015). Ebert et al. (2015) were able to show in a randomized controlled study that increasing knowledge about an internet intervention through a short information video led to significantly higher acceptance. Békés and Doorn (2020) showed that psychotherapists who had already experienced online therapy also had more positive attitudes towards it. This was also shown by Hennemann et al. (2017), who found significantly higher acceptance of EMH among people in the health sector with previous experience (M = 3.31, SD = 1.06 vs. M = 2.33, SD = 0.9). Lazuras and Dokou (2016) also found more positive evaluations of online counselling among test persons with experience in online counselling than among persons without experience. A further determinant of EMH acceptance might be the subjective assessment of estimation of evidence towards EMH interventions; however, research on this issue is currently missing. Additionally, there might be further specific concerns about EMH influencing acceptance and adoption (Apolinário-Hagen et al., 2019; Featherman and Pavlou, 2003) and, thus, constituting barriers to the uptake of EMH (Waller and Gilbody, 2009): Frequently mentioned concerns refer to data insecurity (Wells et al., 2007), impersonality (Bengtsson, 2014), irresponsibility (Wells et al., 2007), legal concerns (Chakrabarti, 2015; Li et al., 2013) and concerns about the therapeutic alliance (Berger, 2015; Hennemann et al., 2017). In this study we subsume knowledge, experience and estimation of evidence of EMH as facilitators, and we equally subsume data insecurity, impersonality, irresponsibility, legal concerns and concerns about the therapeutic alliance as barriers to EMH acceptance.

COVID-19

Due to the COVID-19 pandemic, with >167 million infected people worldwide (as of 01.06.2021; European Centre for Disease Prevention and Control, 2021), our daily lives are suddenly determined by social distancing measures and quarantine. An increase in stress, anxiety, loneliness, and depression, as well as harmful alcohol or drug use and self-harming behavior, was expected (World Health Organization, 2020), and evidence supports those wide-ranging negative psychological outcomes due to COVID-19 (Brooks et al., 2020). One way to counteract the effects of the pandemic is to increase access to psychosocial services through EMH (Van Daele et al., 2020; Wind et al., 2020). EMH allows mental health care to be provided at a distance by video conferencing and internet interventions. During COVID-19, as expected, patient contact through EMH increased, while at the same time, a decrease in the face-to-face setting was observed (Humer et al., 2020). Earlier studies showed that different EMH modalities (e.g., unguided programs and psychotherapy by telephone) are perceived and accepted differently (Klein and Cook, 2010; Wildauer and Apolinário-Hagen, 2018), thus showing websites being accepted the most and unguided programs accepted the least. In a recent study by Parisi et al. (2021) that involved a diverse sample of practitioners consisting of clinical and counselling psychologists with a variety of theoretical orientations and professional backgrounds, the provision of evidence-based interventions by videoconferencing was rated as moderately acceptable. In times of COVID-19 where an increased usage of different EMH modalities is observed, it seems of particular interest to investigate differences in acceptance between EMH modalities.

Aims of study

The present study aimed to investigate the overall level of acceptance among psychotherapists-in-training and to compare the acceptance of different EMH modalities. The influence of facilitators (knowledge, experience, estimation of evidence) and barriers (data insecurity, impersonality, irresponsibility, legal concerns, concerns about the therapeutic alliance) on EMH acceptance is tested separately. The UTAUT predictors (performance expectancy, effort expectancy, social influence, facilitating conditions) will be supplemented with the aforementioned facilitators and barriers. Improvement of prediction of EMH acceptance will also be tested. Lastly, the study compares current assessments of EMH acceptance to pre-COVID-19 times.

Material and methods

Study design & recruitment

A cross-sectional online survey for psychotherapists-in-training (psychologists and physicians) at accredited training institutes in Switzerland and Germany was conducted between 06/2020 and 07/2020. A checklist of the ethics committee of the University of Zurich indicated that the ethical safety of the study was guaranteed and that no further approval of the ethics committee was necessary. Twenty-nine institutions in Switzerland and 232 institutions in Germany were contacted and asked to inform their trainees about the study by forwarding the link to the questionnaire. Since only a few institutions gave feedback on forwarding the questionnaire, no statement can be made about the response rate on an institutional level. In total, the questionnaire was opened 692 times (repeated openings cannot be ruled out), resulting in a dropout rate of 68.7 %.

Measures

Determinants of EMH acceptance

Acceptance was operationalized according to the UTAUT model (Venkatesh et al., 2003) as the intention of using EMH interventions, which was adapted from previous studies (Apolinário-Hagen et al., 2019; Hennemann et al., 2016, Hennemann et al., 2017; Jewer, 2018). The acceptance, the facilitators and the perceived barriers of EMH were calculated as an average of five different modalities (psychotherapy by telephone, psychotherapy by videoconference, virtual reality, guided and unguided programs) to cover the range of different EMH applications. EMH modalities were introduced and defined briefly in the survey (see Appendix A, Table S2). Acceptance items were rated on a 5-point Likert scale ranging from 1 (totally disagree) to 5 (totally agree), with higher scores indicating greater acceptance. The EMH acceptance scale showed good internal consistency (α = 0.89). Investigated determinants were age, gender, facilitators (knowledge, experience and estimation of evidence), barriers (data insecurity, impersonality, irresponsibility, legal concerns and concerns about the therapeutic alliance) and UTAUT predictors (performance expectancy, effort expectancy, social influence, facilitating conditions). Knowledge was measured by self-rated prior knowledge about EMH. Experience was assessed with the quantitative amount of experience with EMH since March 2020. Estimation of evidence was assessed using therapists own-rating of scientific evidence. These facilitators were assessed at an individual level in contrast to the facilitating conditions from the UTAUT, which are at an organizational-systemic level. UTAUT predictors, barriers and knowledge were rated on a 5-point Likert scale ranging from 1 (totally disagree) to 5 (totally agree). Experience with EMH and estimation of evidence were rated on visual analogue scales (VAS) ranging from 0 (very little) to 101 (very high). UTAUT predictors were measured each with two items and were partly adapted from previous studies (Hennemann et al., 2017; Venkatesh et al., 2003). To avoid underestimating the true reliability of the two-item subscales, the internal consistency was not calculated (Eisinga and Pelzer, 2013). The acceptance of EMH pre-COVID-19 was assessed retrospectively (“Thinking back to autumn 2019, how strong was your intention to ever use EMH in your job?”) and contained only items concerning EMH in general and not all different modalities. Supplementary material 1 (Table S1) contains a full overview of the content and reference studies of all assessed constructs and the corresponding scales. The research model with all investigated predictors is depicted in Fig. 1.

Fig. 1

Research model showing the core UTAUT predictors and further investigated EMH acceptance predictors.

Statistical analysis

Only completed surveys were entered in the data analysis using SPSS version 26 (IBM Analytics), RStudio (Version 1.2.5042) and R (Version 4.0.0). Based on prior research (Hennemann et al., 2016, Hennemann et al., 2017), the mean score of EMH acceptance was categorized as low (1–2.34), moderate (2.35–3.67), or high (3.68–5). We extended this categorization to knowledge and to the level of concern regarding the barriers. Estimation of evidence and experience were categorized as low (0–33), moderate (34–67), or high (68–101). To assess the acceptance between different EMH modalities, repeated-measure ANOVAs and Bonferroni-corrected pairwise comparisons were calculated. The influence of the five barriers and facilitators on the acceptance of EMH was also determined separately through multiple regressions. Predictors of acceptance were selected to enter a hierarchical stepwise regression analysis. Block 1 contained sociodemographic variables (age and gender), block 2 contained the four core UTAUT determinants performance expectancy, effort expectancy, social influence, and facilitating conditions, and block 3 contained the predictors data insecurity, impersonality, irresponsibility, legal concerns, concerns about the therapeutic alliance, experience, knowledge and estimation of evidence. Differences in mean scores for acceptance, facilitators, barriers and pre-COVID-19 estimates were assessed using paired t-tests and were only calculated for EMH in general. As the online questionnaire was configured in such a way that one could only go to the next page when all questions had been answered, there were no missing values. The significance level in this study was alpha < 0.05. An a priori power analysis using G*Power (Faul et al., 2007) version 3.1 resulted in a required sample size of at least N = 150.

Results

Sociodemographic characteristics

The survey contained items regarding sex, age, current country of residence, prior education, therapeutic orientation and how far they have advanced in training. A total of 228 people completed the questionnaire in full, and detailed sample characteristics are described in Table 1. Twelve people were excluded because they had just started their training as psychotherapists (had only participated in the theoretical part of training and did not have any clinical experience thus far; cf. Supplementary Material Table 1), resulting in a sample size of 216. The time taken to complete the questionnaire ranged from 6.5 to 37.7 min (M = 19.1, SD = 5.9).

Table 1

Sample characteristics (N = 216).

Variables	Participants, n (%)
Gender
Female	191 (88.4)
Male	25 (11.6)
Age
20–24	5 (2.3)
25–29	90 (41.7)
30–34	61 (28.2)
35–39	28 (13.0)
40–44	19 (8.8)
45–49	7 (3.2)
50–54	1 (0.5)
55–59	5 (2.3)
Country
Switzerland	60 (27.8)
Germany	156 (72.2)
Prior education
Psychology	197 (91.2)
Medicine	6 (2.8)
other	13 (6)
Therapeutic orientation
Cognitive/cognitive-behavioural	145 (67.1)
Psychodynamic/psychoanalysis	35 (16.2)
Systemic	27 (12.5)
Humanistic	9 (4.2)
Others	22 (10.2)

General acceptance and differences between modalities

Initially, the overall acceptance of EMH was investigated (corresponding scales in Supplementary Material 1). The overall acceptance of EMH in our sample can be rated as moderate (M = 3.40, SD = 1.11). Descriptive statistics for all modalities are shown in Table 2. Acceptance varied significantly depending on the EMH modality F(2.6, 561.7) = 62.93, p < .01, partial η2 = 0.23. Psychotherapy by videoconference was accepted the most, whereas unguided programs were accepted the least. Acceptance of psychotherapy by videoconference can be categorized as high; for the other modalities, it can be categorized as moderate. Bonferroni-corrected pairwise comparisons were used to investigate whether acceptance between modalities was different from each other. It was found that acceptance towards psychotherapy by telephone was significantly (p < .01) different from all other modalities. The same is also true for the acceptance of psychotherapy by videoconferencing, which is significantly (p < .01) different from the other modalities. However, the acceptance of virtual reality is not significantly different compared to the acceptance of unguided programs (p = .46) and the acceptance of guided programs (p = .10). The acceptance between unguided programs and guided programs was not significant (p = .06).

Table 2

Descriptive statistics of the acceptance of different EMH modalities.

Modality	Descriptive statistics							Correlations
	N	M	SD	Min	Max	Skewness	Kurtosis	Gender	Age
Psychotherapy by telephone	216	3.36	1.21	1	5	−0.37	−0.98	−0.19⁎⁎	0.11
Psychotherapy by video conference	216	3.70	1.15	1	5	−0.66	−0.48	−0.13	0.10
Virtual reality	216	2.70	1.10	1	5	0.13	−0.87	−0.04	−0.11
Unguided programs	216	2.54	1.14	1	5	0.47	−0.79	−0.09	−0.14⁎
Guided programs	216	2.88	1.14	1	5	0.11	−0.90	−0.06	−0.12

Note. Min = minimum (not at all accepted), max = maximum (very much accepted). Spearman's correlation for sex (1 = male; 2 = female) and age.

p < .05.

p < .01.

Barriers were perceived as moderately concerning, ranging from the lowest being data insecurity (M = 3.02, SD = 0.93) to the highest being concerns about therapeutic alliance (M = 3.65, SD = 0.83). The distribution is left-skewed for all five barriers, indicating the perceived relevance of the aforementioned barriers. Age was positively correlated (p < .05) with the barrier data insecurity and concerns about therapeutic alliance, and no significant correlations with gender were observed.

Self-rated knowledge was also moderate (M = 3.53, SD = 0.69), experience with EMH was low (M = 10.36, SD = 10) and estimation of evidence was moderate (M = 46.90, SD = 19.26). There were no significant correlations between the facilitators and age or gender. Descriptive statistics for the investigated determinants of EMH acceptance can be found in Table 3.

Table 3

Descriptive statistics of the investigated barriers and facilitators to EMH acceptance.

	Descriptive statistics
	N	M	SD	Min	Max	Skewness	Kurtosis
Barriers
Data insecurity	216	3.02	1.21	1.00	5.00	−0.19	0.06
Impersonality	216	3.09	1.15	1.00	5.00	−0.09	0.40
Irresponsibility	216	3.55	1.10	1.00	5.00	−0.41	0.14
Legal concerns	216	3.41	1.14	1.00	5.00	−0.42	−0.29
Therapeutic alliance	216	3.65	1.14	1.20	5.00	−0.35	−0.25
Facilitators
Knowledge	216	3.53	0.69	1.33	4.87	−0.34	−0.21
Experience	216	10.36	10.00	1.00	61.40	1.30	2.52
Estimation of evidence	216	46.90	19.26	1.00	97.00	−0.15	0.33

Note. Min = minimum, max = maximum; Barriers and Knowledge on a scale from 1 to 5; experience and estimation of evidence on a VAS from 0 to 101.

Barriers and facilitators predicting EMH acceptance

In Model 1 acceptance was predicted by the five barriers. In total, 28 % of the variance can be explained by the five barriers, whereby acceptance could be significantly predicted, F(5, 210) = 16.48, p < .01. The barriers impersonality and concerns about therapeutic alliance were significant (p < .01) predictors of EMH acceptance. In Model 2 acceptance was predicted by the three facilitators. In total, 23 % of the variance can be explained by the three facilitators, which are statistically significant predictors of acceptance, F(3, 215) = 11.12, p < .01. Knowledge, experience and estimation of evidence were significant (p < .01) predictors of EMH acceptance. A summary of results for Model 1 and Model 2 can be found in Table 4.

Table 4

Multiple regression analysis using barriers and facilitators to predict EMH acceptance independently.

	Predictors	B	β	p value	R2	Adjusted R2
Model 1					0.28	0.27
	Data insecurity	0.06	0.68	.302
	Impersonality	−0.23	−0.21	.006
	Irresponsibility	0.09	0.09	.250
	Legal concerns	−0.07	−0.09	.199
	Therapeutic alliance	−0.41	−0.41	<.001
Model 2					0.24	0.23
	Knowledge	0.17	0.13	.034
	Experience	0.02	0.20	.001
	Estimation of evidence	0.02	0.34	<.001

Note. Model 1: barriers to predict EMH acceptance; Model 2: facilitators to predict EMH acceptance.

Extension of UTAUT

Stepwise regression including 14 variables (cf. Fig. 1) identified performance expectancy, social influence, and concerns about therapeutic alliances as significant predictors of acceptance in the extended UTAUT model (R2 = 0.55, F (14, 201) = 17.68, p < .001). Block 1 with the predictors age and gender was not significant (R2 = 0.02, F (2, 213) = 2.61, p = .08). The core UTAUT predictors (performance expectancy, effort expectancy, social influence, facilitating conditions) added in Block 2 could significantly predict EMH acceptance (R2 = 0.49, F (6, 209) = 33.99, p < .01). Block 3 showed a significantly better prediction of EMH acceptance than Block 2 (p < .01). The extended UTAUT Model and the associated predictors are shown in Table 5.

Table 5

Extended UTAUT model and its predictors.

Predictors	B	β	p value	R2	adjusted R2
				0.55	0.52
Age	0.00	0.00	.973
Gender	−0.12	−0.04	.372
Performance expectancy	0.45	0.39	<.001
Effort expectancy	0.12	0.09	.150
Social influence	0.25	0.18	<.001
Facilitating conditions	0.00	0.00	.957
Data insecurity	0.07	0.08	.146
Impersonality	−0.07	−0.06	.307
Irresponsibility	0.03	0.03	.615
Legal concerns	0.02	0.02	.700
Concerns about therapeutic alliance	−0.22	−0.21	.003
Experience	0.01	0.08	.089
Knowledge	−0.02	−0.02	.764
Estimation of evidence	0.00	0.07	.161

Pre-COVID-19 comparison

The acceptance of EMH was significantly lower (t = 6.53, p < .01) pre-COVID-19 (M = 2.67, SD = 1.11). The distribution of EMH acceptance changed from positively skewed (0.25) to negatively skewed (−0.40). Pre-COVID-19, 50 % of all answers were between 1.7 and 3.3, while during COVID-19, 50 % of all answers were between 2.7 and 3.7. Additionally, all five investigated barriers (data insecurity, impersonality, irresponsibility, legal concerns and concerns about the therapeutic alliance) were significantly higher (p < .01), and all three facilitators (knowledge, experience and estimation of evidence) were significantly lower (p < .01) pre-COVID-19. Comparisons between retrospectively assessed barriers and facilitators pre-COVID-19 and during COVID-19 are summarized in Table 6.

Table 6

Comparisons between pre-COVID-19 and during COVID-19.

Determinant	Pre-COVID-19 M (SD)	During COVID-19 M (SD)	Test statistics
Barriers
Data insecurity	3.65 (1.11)	3.30 (1.08)	t = −4.45, p < .01
Impersonality	3.77 (1.06)	3.11 (1.14)	t = −7.86, p < .01
Irresponsibility	3.83 (1.11)	3.57 (1.07)	t = −3.41, p < .01
Legal concerns	3.61 (1.20)	3.48 (1.18)	t = −1.61, p < .01
Therapeutic alliance	3.93 (1.12)	3.68 (1.12)	t = −3.06, p < .01
Facilitators
Knowledge	3.28 (1.14)	3.64 (0.86)	t = 5.58, p < .01
Experience	7.64 (13.66)	29.72 (29.80)	t = 10.79, p < .01
Estimation of evidence	43.94 (24.90)	53.56 (24.49)	t = 8.521, p < .01

Note. Barriers and knowledge on a scale from 1 to 5; Experience and estimation of evidence on a VAS from 0 to 101.

Discussion

This study assessed the acceptance of EMH and its determinants among psychotherapists-in-training. Acceptance ratings were moderate and seems to be comparable to a mixed sample of health professionals (Hennemann et al., 2017) and a recent study from Parisi et al. (2021) investigating acceptance of psychotherapy by videocall in a diverse sample of practitioners. Moderate to rather positive attitudes towards EMH were also observed in two other studies with psychotherapists (Békés and Doorn, 2020) and healthcare practitioners (Netter et al., 2022). The current assessments were compared with pre-COVID-19 times retrospectively and showed that the acceptance of EMH was significantly lower pre-COVID-19. One possible explanation could be that the acceptance has increased because psychotherapists-in-training were forced to engage with EMH and thus could develop experience and expand their knowledge of specific EMH tools. This assumption is supported by the study by finding from Békés and Doorn (2020), which showed that psychotherapists who already had experience with online psychotherapy also had more positive attitudes towards it but still did not allow for causal interpretation. Our findings support the fact that psychotherapists-in-training could acquire knowledge and experience during COVID-19 as knowledge and experience prior to COVID-19 received lower retrospective ratings. This accumulation of experience due to the restrictions caused by COVID-19 was also observed in several countries across Europe, where patient contact in a face-to-face settings decreased and the use of EMH increased (Humer et al., 2020). It is likely that the increased use of EMH, which was triggered by COVID-19, will be more than a temporary phenomenon and that once mental health care institutions have adopted and implemented EMH, there is little reason to stop this development in view of the many advantages EMH offers in broadening treatment options (Wind et al., 2020). Due to the moderate acceptance ratings and the influence of COVID-19, it seems likely that EMH will be increasingly and sustainably used in the future.

Although previous studies showed that more knowledge about EMH is associated with greater acceptance (Donovan et al., 2015; Ebert et al., 2015), knowledge was not found to be a significant predictor in the extended UTAUT model. Neither did the predictors experience and estimation of evidence reach a significance level in the extended UTAUT model, contrary to our exploratory assumption based on Model 2. The only significant predictors in the extended UTAUT model were performance expectancy, social influence and the barrier concerns about therapeutic alliances. These findings support previous evidence on performance expectancy as a key predictor of acceptance (Dünnebeil et al., 2012; Dwivedi et al., 2011; Hennemann et al., 2017; Li et al., 2013; Taiwo and Downe, 2013). Out of the four core UTAUT predictors, social influence was the other significant predictor of EMH acceptance, which is in line with the results of Hennemann et al. (2017). The added EMH predictors (data insecurity, impersonality, irresponsibility, legal concerns, concerns about therapeutic alliance, experience, knowledge, estimation of evidence) could only explain 6 % more variance than the core UTAUT predictors, which is very little compared to previous studies that extended the UTAUT model (Apolinário-Hagen et al., 2019; Hennemann et al., 2016). One possible explanation is the neglected moderation in our prediction model (Van Raaij and Schepers, 2008), which should be investigated further in future studies.

Additionally, our results reveal substantial variability in acceptance across different EMH modalities. Psychotherapy by video conference was found to be accepted the most, and unguided programs were found to be accepted the least. Earlier research also supports the differences in acceptance and likelihood of future use across a broad range of EMH applications (Klein and Cook, 2010; Wildauer and Apolinário-Hagen, 2018). Interestingly and in contrast to prior research, which focused on the efficacy of (Baumeister et al., 2014) and attitudes towards (Apolinário-Hagen et al., 2018) unguided and guided programs, no difference in acceptance was found between guided and unguided programs in this study. The findings of this study also indicate that calculating an EMH sum score might not be the right approach to investigate EMH acceptance as a larger construct and might, thus, be an explanation for many predictors not reaching significance. However, differentiated assessments of EMH modalities can help to identify specific not well-accepted modalities and the specific barriers, which hinder their implementation in routine clinical care. Interventions that can be derived from specific barriers and modalities were found to be effective in patients as in psychologists. Short educational videos or texts concerning data security or effectiveness can lead to enhanced acceptance and improved adoption in routine care (Donovan et al., 2015; Ebert et al., 2015).

For practical implications, it might be particularly interesting to focus on the significant predictors found in this study (social influence and concerns about therapeutic alliances). One possible implication could be that psychotherapists-in-training could be informed about how the therapeutic alliance manifests itself in EMH and could share experiences in supervision or intervision. At that point, it might also be noted that 79 % of clinical psychologists consider themselves insufficiently trained in the use of EMH (Perle et al., 2013), which is also reflected in the limited level of experience the psychotherapists-in-training in our sample had with EMH in general. However, 75 % would consider using it again if they had the appropriate training, showing the importance of further training in the field of EMH (Perle et al., 2013). Van Daele et al. (2020) also addresses the importance of EMH training for psychotherapists: psychotherapists-in-training should ensure that they are sufficiently trained and are familiar with the field of EMH and its developments. Implementing EMH in the curriculum of training institutions and universities to prepare psychotherapists-in-training as well as possible so that they have at least already had contact with EMH could be considered (De Witte et al., 2021). As the results of the pre-COVID-19 comparison suggest that experience could have an important role when it comes to EMH acceptance, continuing education could be helpful to provide guidance for psychotherapists-in-training, which is also highlighted by Kuso et al. (2021).

For future research, it would be of great interest to determine whether the differences in acceptability between EMH modalities are also found in other health care workers. Likewise, the perception of EMH might vary between psychotherapists working in an inpatient or outpatient setting. Additionally, it is conceivable that psychotherapists working with children and young people have different requirements and expectations of EMH than psychotherapists working with elderly people or adults. To our knowledge, no study has looked for differences concerning the acceptance of EMH across different settings in which psychotherapy is delivered, although it seems essential to understand the different stakeholders and their interests to adopt EMH successfully.

Limitations and strengths

To the best of our knowledge, this study is the first of its kind to investigate the acceptance of EMH among psychotherapists-in-training. In this context, all accredited German-speaking training institutes in both Switzerland and Germany were contacted. Furthermore, this study followed a well-established theory-based framework of acceptance. In addition to a general assessment of EMH acceptance, we also assessed the acceptance of different modalities, which allowed us to refine our global assessment of EMH acceptance. Thus, in addition to general statements about EMH, modality-specific statements can be made.

While this study helps deepen the understanding of determinants that influence the acceptance of EMH among psychotherapists-in-training, some limitations must be acknowledged. As this study was conducted cross-sectionally, the results do not refer to the development of EMH acceptance over time or to the influence of COVID-19 on EMH acceptance. Another limitation is the low response rate, especially among the German training institutions, which may be due to the high workload during COVID-19. However, the recruited sample exhibited comparable demographic characteristics to Nübling et al.'s (2020) representative sample with >2500 psychotherapists-in-training. Even though our sample has demographic features that are comparable to the general population of psychotherapists-in-training, calculations including the variable of gender need to be interpreted with caution as group sizes are vastly different. Lastly it should be added, that despite assessing EMH modality specifically, some barriers do not apply to certain modalities (e.g., concerns about the therapeutic alliance and unguided programs).

Conclusions

This study showed that EMH acceptance among psychotherapists-in-training is moderate. The results showed that retrospective EMH acceptance was rated lower pre-COVID, possibly due to acquiring knowledge and more intensive usage of EMH during the pandemic. For the extended UTAUT model, a wide range of possible predictors were investigated. The predictors performance expectancy, social influence and therapeutic alliance were significant predictors of acceptance. Since we know from previous findings that the acceptance of technology is influenceable, this study underscores the need to investigate the facilitators and barriers of EMH acceptance among psychotherapists-in-training. This is needed to derive adequate educational programs as well as to prepare and support psychotherapists-in-training in the appropriate use of EMH. Such research would also enable the broadening of treatment options available in mental health care.

Abbreviations

Unified theory of acceptance and use of technology

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.