Psychotherapists' acceptance of telepsychotherapy during the COVID-19 pandemic: A machine learning approach.

OBJECTIVE: This study aimed to develop predictive models of three aspects of psychotherapists' acceptance of telepsychotherapy (TPT) during the COVID-19 pandemic, attitudes towards TPT technology, concerns about using TPT technology and intention to use TPT technology in the future.
METHOD: Therapists (n = 795) responded to a survey about their TPT experiences during the pandemic, including quality of the therapeutic relationship, professional self-doubt, vicarious trauma and TPT acceptance. Regression decision tree machine learning analyses were used to build prediction models for each of three aspects of TPT acceptance in a training subset of the data and subsequently tested in the remaining subset of the total sample.
RESULTS: Attitudes towards TPT were most positive for therapists who reported a neutral or strong online working alliance with their patients, especially if they experienced little professional self-doubt and were younger than 40 years old. Therapists who were most concerned about TPT were those who reported higher levels of professional self-doubt, particularly if they also reported vicarious trauma experiences. Therapists who reported low working alliance with their patients were least likely to use TPT in the future. Performance metrics for the decision trees indicated that these three models held up well in an out-of-sample dataset.
CONCLUSIONS: Therapists' professional self-doubt and the quality of their working alliance with their online patients appear to be the most pertinent factors associated with therapists' acceptance of TPT technology during COVID-19 and should be addressed in future training and research.

Therapists with neutral or strong online working alliance had the most positive attitudes towards telepsychotherapy.

Therapists with higher levels of professional self‐doubt and especially with higher vicarious trauma were most concerned about using telepsychotherapy.

Therapists with low working alliance were least likely to use telepsychotherapy in the future.

INTRODUCTION

Providing psychotherapy remotely has multiple advantages, such as providing access to patients in remote areas and with reduced mobility, avoiding the stigma of visiting a mental health clinic and reduced travel time and associated costs (e.g. commute and childcare; Békés, Grondin, & Bouchard, 2020; Simpson, 2009). The COVID‐19 pandemic has created a unique context in which many therapists have abruptly transitioned to telepsychotherapy (TPT), without much preparation or training in advance. Therapists' experience of this sudden transition to a new therapy format might determine whether they will continue conducting therapy online, once pandemic restrictions are lifted. It is therefore important to understand which therapist, patient and treatment factors contribute to therapists' acceptance of the TPT format and their intention to continue providing therapy online. Various terms have been used to refer to remote therapy (Rochlen et al., 2004), and in this study, based on the most recent literature, we use the term ‘telepsychotherapy’ to refer to psychotherapy sessions provided synchronously online via videoconferencing (Markowitz et al., 2020; Poletti et al., 2020; Rosen et al., 2020; Van Daele et al., 2020).

Preliminary studies have shown that despite the involuntary transition to TPT during the pandemic, therapists have a somewhat positive view of TPT (Feijt et al., 2020, McBeath et al., 2020) and are inclined to use it in the future (Békés & Aafjes‐van Doorn, 2020). In earlier studies, several factors have been proposed that may impact therapists' views of TPT (for a review, see Connolly et al., 2020). One important factor is the perceived ability to develop a strong therapeutic relationship remotely (Roesler, 2017; Sucala et al., 2013). Pre‐pandemic studies showed that in fact, the therapeutic relationship is generally strong in TPT (see review by Norwood et al., 2018) and similar to in‐person therapies (e.g. Bouchard et al., 2020; Watts et al., 2020). This was true not only pre‐pandemic but during the pandemic as well (Aafjes‐van Doorn, Békés, & Prout, 2020; Aafjes‐van Doorn, Bekés, Prout, & Hoffman, 2020). Moreover, during the pandemic, many therapists also reported that they were as authentic and genuine in their online sessions as they were before in in‐person (Békés, Aafjes‐van Doorn, et al., 2020; Békés, Grondin, & Bouchard, 2020).

Before the start of the pandemic, few therapists had attended training in TPT, and most felt uncomfortable using technology to provide psychotherapy (e.g. Essig & Russell, 2017; Glueckauf et al., 2018). Therapists' sense of uncertainty regarding their ability to provide effective therapy online likely further increased during this sudden transition to TPT during the pandemic. Especially among therapists with less clinical and online experience at the start of the pandemic, and those struggling to build strong therapy relationships, professional self‐doubt has been found to be salient (Aafjes‐van Doorn, Békés, & Prout, 2020).

Moreover, another salient factor in therapists' attitudes towards TPT during the COVID‐19 pandemic is their experience of vicarious trauma. Due to the global health crisis and its negative mental health impact, therapists may have been more exposed to more stressful and traumatic material during their sessions than before the pandemic. Indeed, previous studies showed that on average, therapists experienced a moderate level of vicarious trauma during the pandemic and about 15% of therapists experienced high levels (Aafjes‐van Doorn, Bekés, Prout, & Hoffman, 2020).

These preliminary findings suggest that therapists' ability to create a strong therapeutic relationship online, their professional self‐doubt and experienced vicarious trauma likely impact therapists' acceptance of the TPT format. However, little is known about which of these factors are most strongly associated with TPT acceptance and how they interact with one another when relating to acceptance of TPT technology during the pandemic.

Although traditional regression models and machine learning models, such as decision trees, may both base their variable selection on pre‐existing theory, a decision tree model has certain statistical benefits. Traditional statistical approaches test each independent variable as a separate hypothesis, which can lead to erroneous conclusions because of multiple comparisons (inflated Type I errors), model misspecification and multicollinearity. In contrast, machine learning models are not constrained by model assumptions and are particularly helpful for finding patterns in complex datasets (Bi et al., 2019). A decision tree model, for example, is able to predict the mean level of the dependent variable based on exploring interactions between potential independent variables (Loh, 2008). In an attempt to improve the ability of identified models to generalize to new data that will be collected in the future, a machine learning model is usually developed in the training set and subsequently tested in a separate test set (Aafjes‐van Doorn et al., 2021). This process of cross‐validation avoids model overfit on the original data and mimics a replication of the model on an independent dataset, and it thus increases the confidence in the model's generalizability.

The aims of the current study were twofold: (1) to examine the extent of therapists' acceptance of TPT technology during the pandemic and their attitudes towards TPT technology, concerns about the TPT technology and intention to use TPT technology in the future and (2) to develop predictive models for each of these three aspects of therapists' acceptance of TPT technology, based on the therapists' characteristics, reported quality of the therapeutic relationship in TPT and their levels of professional self‐doubt and vicarious trauma during the pandemic by applying decision tree machine learning analyses.

METHODS

Procedures and participants

This cross‐sectional study was based on Qualtrics‐based online survey data about psychotherapists' experiences and attitudes towards TPN during the pandemic. Preliminary results on small subsamples based on less than 20% of the presented data and focused on different research questions have been published in previous reports (omitted for peer review). The present study reports the results on all survey data collected between 25 March and 29 June 2020. Psychotherapists were recruited via professional email listservs, social media and individual contacts across the United States, Canada, China and Europe. Licensed and trainee therapists, currently seeing patients online, were eligible to participate. Interested participants were directed to an online survey platform, which provided additional information about the study. After providing consent, participants were directed to the anonymous questionnaire. Several individual items and standardized scales were administered in a fixed order, taking approximately 15 min to complete. The study was approved by (the local—omitted for peer review) institutional review board.

In the present study, out of the 1911 participants who clicked on the survey link, we included participants who completed all the relevant measures (n = 795). Due to the forced response setting in the survey, there were no missing data. The participating therapists were mostly female (n = 603; 75.8%), and White (n = 562, 70.7%); their average age was 52.72 (SD = 16.36), and more than half of the sample (n = 399, 57.2%) had more than 17 years of clinical experience, with only 7.9% of therapists reporting 4 years of clinical experience or less. This sample is representative of the larger population of US psychologists which is 65% female and 84% White and has a mean age of 45.9 (American Psychological Association, 2018). Most participating therapists were located in North America (n = 650; 81.8%). Detailed demographic data about the study sample are presented in Table 1.

TABLE 1

Therapist demographics and descriptives (n = 795)

	n	%
Ethnicity
White European, European American	562	70.7
Asian or Asian Indian	70	8.8
Native Hawaiian or Pacific Islander	2	.3
Hispanic, Latinx, Spanish	30	3.8
Black or African American	6	.8
American Indian or Alaska Native	3	.4
Middle Eastern	9	1.1
Other	40	5.0
Country (declaration of national emergency/lockdown a )
USA (March 13 Trump declares national emergency)	650	81.8
	33	4.2
Canada (March 16)	45	5.7
China (January 29)	20	2.5
United Kingdom (March 25)	29	3.6
Other European countries (March 9–March 25)	13	1.0
	8	1.0
Other—Latin America (March 16–17)	10	1.3
Profession
Psychologist	362	45.5
Social worker	119	15.0
Counsellor	104	13.1
Medical doctor	115	14.5
Psychoanalyst	69	8.8
Marriage family therapist	17	2.2
Psychotherapist	27	3.4
Other	2	0.3
Licensure status
Licensed	718	90.3
Trainee	77	9.7
Clinical experience in years
0–4	55	7.9
5–8	102	13.0
9–12	82	11.8
13–16	59	8.5
17 or more	399	57.2
Work setting b
Private practice	653	83.0
Outpatient clinic	142	18.0
Hospital	71	8.9
Other	62	7.9
Theoretical orientation b
Psychodynamic	528	67.1
Psychoanalytic	406	51.6
Integrative	280	35.6
CBT	182	23.1
Humanistic	133	16.9
Systemic	99	12.6
Other	106	13.5
Previous experience of providing TPT
No, never	353	44.4
Yes, once or twice	75	9.4
Yes, but only after seeing them in‐person first	223	28.1
Yes several patients	143	18.0
Previous training in TPT
Yes	136	17.1
No	659	82.9

Abbreviation: TPT, telepsychotherapy.

Retrieved from https://www.ajmc.com/view/a‐timeline‐of‐covid19‐developments‐in‐2020; https://en.wikipedia.org/wiki/Timeline_of_the_COVID‐19_pandemic_in_Canada; https://www.dw.com/en/coronavirus‐what‐are‐the‐lockdown‐measures‐across‐europe/a‐5290517; https://en.wikipedia.org/wiki/Timeline_of_the_COVID‐19_pandemic_in_the_United_Kingdom_(January%E2%80%93June_2020)); https://www.nytimes.com/article/coronavirus‐timeline.html; https://en.wikipedia.org/wiki/COVID‐19_pandemic_in_mainland_China.

Multiple answers were possible per respondent.

Measures

The online survey included individual items that assessed for therapists' demographics and professional activities as well as several standardized scales that assessed for variables of interest, including acceptance of TPT, vicarious trauma, professional self‐doubt and scales about the therapeutic relationship.

Demographics and professional activities

Items included age, gender, race/ethnicity, highest degree, treatment orientation and setting, patient population, licensure, years of experience and number of patients. Additional items inquired about previous TPT experience and training.

Attitudes towards TPT

The novel Unified Theory of Acceptance and Use of Technology Therapist Version (UTAUT‐T; Békés et al., 2021) was used to assess attitudes towards acceptance and usage of TPT. The UTAUT framework (Venkatesh et al., 2003, 2012) offers a comprehensive model of acceptance and subsequent utilization of technological innovations. It has been adapted for various contexts, including occupational and physical therapy (Liu et al., 2015). The UTAUT theoretical framework was recently used in a systematic review of pre‐pandemic empirical studies as an overarching conceptualization of different aspects of acceptance of TPT technology to help frame their inclusion criteria and description of their findings (Connolly et al., 2020; see Venkatesh et al., 2016). In line with the focus of the present study, the phrasing of the items was adapted to reflect TPT. For example, the item ‘People who are important to me think that I should use the system’ in the original UTAUT measure was adapted to ‘People who are important to me think that I should use online therapy’, whereas other items needed more modifications, for example, the original ‘Using the system enables me to accomplish tasks more quickly’ was adapted to therapists as ‘Using online therapy saves me time and/or money’ (for more details on the development of UTAUT‐T, see Békés et al., 2021). The original and extended UTAUT models have been validated in a variety of settings, including academic research (Gruzd et al., 2012), higher education (Pynoo et al., 2011) and healthcare settings (Alapetite et al., 2009; Chang et al., 2007; Liu et al., 2015), for a review, see Venkatesh et al. (2016), whereas the novel UTAUT‐T has not been validated on therapist samples yet.

The UTAUT for psychotherapists includes three scales: Attitudes, Anxiety and Behavioural Intention. The 21‐item UTAUT‐T Attitudes subscale includes items related to performance expectancy, effort expectancy, social influence and facilitating conditions regarding using TPT. The six‐item UTAUT‐T Anxiety subscale includes items regarding feelings of apprehension or concerns about using technology (Compeau et al., 1999). The Anxiety subscale has been shown to have a direct negative effect on actual technology use and is thus seen as an important inhibitory variable (Cenfetelli & Schwarz, 2011). Finally, the two‐item UTAUT‐T Behavioural Intention subscale represents explicitly declared intent and plan to use technology in the future.

As with all UTAUT adaptations, items of the UTAUT‐T scales are scored on a Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree), with higher scores indicating more positive attitudes towards technology (Attitudes Scale), more concerns (Anxiety Scale) and more intention for future use of TPN (Behavioural Intention Scale). In the present study, the internal consistency of the UTAUT‐T subscales was α = .78 (Attitudes), α = .80 (Anxiety) and α = .94 (Behavioural Intention).

Working alliance

Therapeutic alliance was measured with the Working Alliance Inventory – Short Revised – Therapist (WAI‐SRT; Hatcher & Gillaspy, 2006) which includes four positively worded items for each of three subscales, based on Bordin's (1979) pantheoretical model: bond, agreement on tasks and agreement on goals. The WAI‐SRT uses a 5‐point Likert scale, ranging from seldom (1) to always (5), and has been validated by associations with other alliance measures and by prediction of therapy outcomes (Munder et al., 2010; Zilcha‐Mano, 2017).

A global working alliance rating of 4 (sometimes), the middle point of the scale, is interpreted as a neutral working alliance (Horvath & Greenberg, 1989). The WAI‐SRT has shown adequate reliability and validity (Hatcher & Gillaspy, 2006). The mean score of the WAI‐SRT was used, with higher scores indicating a stronger working alliance. Cronbach's α for the total scale in the current sample was .85.

Real relationship

The Real Relationship Inventory Therapist Form (RRI‐T; Gelso et al., 2005) assesses the genuine human relationship between patient and therapist from the therapist perspective. In Gelso's tripartite model (Gelso et al., 2018), the RR is conceptualized as an ongoing quality of the relationship distinguished from transference and the working alliance. The RR has been found to be highly correlated with the Working Alliance Inventory (WAI; Horvath & Greenberg, 1989), especially with the WAI's Bond subscale (Kelley et al., 2010); however, ratings of real relationship predict treatment progress and outcome above and beyond the variance explained by the WAI (Lo Coco et al., 2011; Marmarosh et al., 2009). The RRI has 24 items that use a 5‐point Likert scale from strongly disagree (1) to strongly agree (5), with higher overall scores reflecting a more genuine and authentic relationship. Reliability has shown to be high, with coefficient alphas ranging from .80 to .90 in various samples (e.g. Fuertes et al., 2013, 2019; Marmarosh et al., 2009). In this study, Cronbach's α was .73.

Professional self‐doubt

The Professional Self‐Doubt Scale (PSD; Nissen‐Lie et al., 2017) is a nine‐item scale derived from the larger Development of Psychotherapists Common Core Questionnaire (DPCCQ; Orlinsky et al., 1999). The PSD assesses the level of uncertainty a therapist has in their ability to help a patient. Items are rated on a 6‐point Likert scale from 0 (never) to 5 (very often), with higher total score indicating more self‐doubt. The PSD includes nine items about how confident and competent the therapist feels, and items are rated on a 6‐point Likert scale from 0 (never) to 5 (very often) and summed for a total score, with higher scores indicating more self‐doubt. In our study, Cronbach's α was .84.

Vicarious trauma

The Vicarious Trauma Survey (VTS; Vrklevski & Franklin, 2008) is a self‐report measure of distress associated with working with traumatized patients. The VTS includes eight items, from which the first two ask about vicarious trauma exposure, whereas the other six ask about distress due to the exposure. Because the first two items are screening questions and do not assess the subjective experience of vicarious traumatization (Aparicio et al., 2013), in the present study, we used the calculated sum of the six distress items. Items are rated on a 7‐point Likert scale from strongly disagree (1) to strongly agree (7), where higher scores indicate more VT. The VTS has strong psychometric properties (Aparicio et al., 2013; Benuto et al., 2018; Michalopoulos & Aparicio, 2012), with alpha .77 for the six‐item version (Aparicio et al., 2013). Cronbach's α for the total scale in the current sample was .81, and for the six‐item subjective VTS, it was .83.

Data analysis

Machine learning regression decision tree analyses

The primary aim of the analyses was to develop a prediction model of the therapists' acceptance of the TPT format and their attitudes, concerns and intention for future use of videoconferencing. The three dependent variables were UTAUT‐T Attitudes subscale, UTAUT‐T Concerns subscale and UTAUT‐T Behavioural Intention subscale. Based on the aforementioned clinical theory, we entered the following potential independent variables in the models: therapists' age and gender, as well as reported mean ratings of the real relationship, working alliance, professional self‐doubt and vicarious trauma scale. Based on general recommendation for machine learning models (Vrigazova, 2021) and common practice (e.g. Prout et al., 2020), the dataset was randomly split into two sets using PROC SURVEYSELECT of SAS: a training set (70%, n = 556) and a test set (30%, n = 239).

For this purpose, regression decision tree analyses were run with the R party package (Hothorn et al., 2006) using Monte Carlo simulation for multiple‐testing adjustment (Strasser & Weber, 1999). The workings of the decision tree algorithm can be described as follows. First, the null hypothesis of independence between each of the six independent (i.e. input) variables and the respective response dependent (i.e. response) variable was tested. If the independent variable had no association to the dependent variable, this variable was not included in the model. A random selection of independent variables was performed based on the strength of their association with the dependent variable; the first selected independent variable had the strongest association to the dependent variable. This association is estimated by a p‐value corresponding to a test for the partial null hypothesis of a single independent variable and the dependent variable. Subsequently, the algorithm implemented a binary split in the selected independent variable, based on the corresponding p‐value of less than .01. These steps were repeated recursively for each of the independent variables that were significantly associated with the dependent variable, until no remaining variable was associated with the dependent variable. As is common and recommended in prediction models (Loh, 2008), performance of decision trees was evaluated by the root mean square error (RMSE; Kenney & Keeping, 1962) and R (Glantz & Slinker, 2001) of the test set compared to these values in the training set. The R Foundation for Statistical Computing Version (3.6.1) was utilized for the tree analysis.

RESULTS

Therapists' acceptance of TPT technology during COVID‐19

Descriptives

For an overview of the descriptive statistics of all standardized measures, see Table 2. Overall, therapists reported somewhat positive attitudes towards TPT. On average, therapists were relatively concerned about providing therapy as well as undecided as to whether they would like to continue using TPT in the future (i.e. expressed as a neutral response on the UTUAT Behaviour Intention subscale), with large differences among therapists. The real relationship scores were similar or higher compared to scores found in previous studies in in‐person treatments before the pandemic (Bhatia & Gelso, 2018; Gelso et al., 2012). The real relationship has not been previously assessed with regard to online therapies. The working alliance scores on the WAI‐SRT in our sample were similar to therapist‐rated alliance scores on this same measure in previous studies in blended (combined in‐person and online) therapy that was conducted before the abrupt transition to online therapies due to the pandemic (Vernmark et al., 2019).

TABLE 2

Descriptive statistics for the standardized independent variables and each of the dependent variables in comparison to previously published scores on the same measures (n = 795)

Measure	M (SD)	Min‐max	Published comparison in in‐person therapies	Two samples t‐tests a
1. UTAUT‐T Attitude	3.29 (.81)	1.00–5.00	N/A	‐
2. UTAUT‐T Anxiety	2.67 (.81)	1.00–4.67	N/A	‐
3. UTAUT‐T Behavioural Intention	3.17 (1.18)	1.00–5.00	N/A	‐
4. RRI	3.80 (.47)	1.92–5.00	Therapists: M = 3.81; SD = 1.01 (Bhatia & Gelso, 2018)	t(1042) = .77, p =. 83
			Therapists: M = 2.94, SD = .12 (Gelso et al., 2012)	t(812) = 7.96, p < .001
5. WAI‐SRT	3.89 (.63)	1.00–5.00	Therapists in outpatient blended therapy M = 3.93; SD = .63 (Vernmark et al., 2019)	t(805) = 0.52, p = .83
6. PSD	2.42 (.84)	1.00–5.78	Trainees: M = 1.52, SD = .94 (Odyniec et al., 2019)	t(833) = 28.03, p < .001
			Therapists: M = 1.24, SD = .70 (Nissen‐Lie et al., 2013).	t(863) = 11.41, p < .001
7. VTS	3.69 (1.17)	1.00–7.00	Social workers: M = 2.88 SD = 0.99 (Aparicio et al., 2013)	t(950) = 8.12, p < .001

Abbreviations: PSD, Professional Self‐Doubt; RRI, Real Relationship Inventory; UTAUT‐T, Unified Theory of Acceptance and Use of Technology–Therapist Scale; VTS, Vicarious Trauma Scale; WAI‐SRT, Working Alliance Inventory – Short Revised – Therapist.

Comparison with previously published scores on the same scales.

On average, therapists experienced professional self‐doubt sometimes or frequently in TPN during the pandemic, which is higher than the level of self‐doubt experienced by therapists in a prior naturalistic study of PSD (Nissen‐Lie et al., 2013) but still on the lower end of the 5‐point Likert scale. The sample in our study reported significantly higher subjective VT compared to previously found pre‐pandemic in‐person sessions (Michalopoulos & Aparicio, 2012).

Preliminary analyses

The UTAUT Attitude Scale significantly correlated with all independent variables, except for age and gender. The UTAUT Anxiety Scale significantly correlated with all independent variables. The UTAUT Behavioural Intention Scale correlated with UTAUT Attitude and Anxiety Scales, working alliance and professional self‐doubt, but correlations with age, gender, real relationship and vicarious trauma were non‐significant. After applying a Bonferroni correction (i.e. p < .005), the correlation between UTAUT Anxiety and gender was no longer significant. All the other correlations remained significant (when applying p < .005; see Table 3).

TABLE 3

Pearson correlations for the eight variables (n = 795)

Measure	1	2	3	4	5	6	7	8	9
1. Age	‐‐
2. Gender	.19***	‐‐
3. UTAUT‐T Attitude	.05	−.01	‐‐
4. UTAUT‐T Anxiety	−.28***	−.08*	−.53***	‐‐
5. UTAUT‐T Behavioural Intention	.03	.04	.60***	−.36***	‐‐
6. Real relationship	.01	−.40**	.22***	−.20***	.07	‐‐
7. Working alliance inventory	.29**	−.04	.28***	−.18***	.10**	.55***	‐‐
8. Professional self‐doubt	−.35***	−.12***	−.23***	.53***	−.11***	−.24***	−.25***	‐‐
9. Vicarious trauma	−.25***	−.12***	−.13***	.33***	−.01	−.08*	−.09**	.47***	‐‐

Abbreviation: UTAUT‐T, Unified Theory of Acceptance and Use of Technology Therapist Version.

p < .05.

p < .01.

p < .001.

Prediction of therapists' acceptance of TPT technology during COVID

Predicting models were trained on the train set (n = 556) and were assessed for predictive performance on the test set (n = 239). Three separate decision tree models were developed to predict the three aspects of acceptance of TPT technology.

Predicting therapists' attitudes towards TPT

The regression tree model showed that higher alliance ratings (WAI > 3.9) was associated with more positive attitudes. Those with the higher alliance ratings (WAI > 3.9) with relatively low professional self‐doubt (PD < =2.1) and younger than 40 reported the most positive attitudes. Therapists who reported the lowest alliance rates (WAI = <3) also reported the most negative attitudes (Figure 1).

FIGURE 1

Regression tree for attitudes towards telepsychotherapy technology (UTAUT‐T Attitudes), n = 556

Predicting therapists' concerns about TPT

The regression tree model showed that therapists with the highest levels of professional self‐doubt (PSD > 2.89) reported the most concern on UTAUT Anxiety. Those with relatively low levels of professional self‐doubt (PSD < 1.67) and low levels of vicarious trauma (VTS < 1.17) reported the least concern about using the online platform (Figure 2).

FIGURE 2

Regression tree for concerns about using telepsychotherapy technology (UTAUT‐T Anxiety), n = 556

Predicting therapists' intention to use TPT in the future

The regression tree model showed that therapists who reported higher working alliance (WAI > 3) also reported more intention to use TPT in the future (Figure 3).

FIGURE 3

Regression tree for intention to use telepsychotherapy technology in the future (UTAUT‐T Behavioural Intention), n = 556

Predictive performance

The performance of the model of prediction for UTAUT‐T Attitudes was relatively good in that it had the second best R 2 in the models (0.15) and the best RMSE values (0.51), the test set's RMSE value was 0.48 and the R 2 was 0.46. Prediction of UTAUT‐T Anxiety had the highest R2 in the models (0.54) and the second best RMSE value (0.57), and the test set's predictive criteria values were RMSE = 0.63 and R 2 = 0.46. Prediction of UTAUT‐T Behavioural Intention had the lowest R 2 (0.04) in the models and the highest RMSE values (1.16), and the test set had poorer predictive criteria values that were the same as the train set correspondence values. Taken together, the predictive criteria results of the test set were the same as in the training set when predicting UTAUT‐T Behavioural Intention and UTAUT‐T Anxiety. The ability to predict UTAUT‐T Attitudes was marginally poorer in the test than in the training set. The relatively similar model performances in the training and test set indicate that the model prediction abilities remained when applied to new data. Table 4 summarized the predictive criteria values on the three outcomes. In addition, we tested a dummy model and found higher RMSE for all three subscales (Attitudes RMSE = .51, Anxiety RMSE = .85 and Behavioural Intention RMSE = 1.18) compared to the test set, which indicates that the reported model has better predictive power than the dummy model.

TABLE 4

Predictive criteria values for training and test sets

Dependent variable	Train n = 556		Test n = 239
	RMSE	R 2	RMSE	R 2
UTAUT‐T Attitudes	0.51	0.15	0.48	0.13
UTAUT‐T Anxiety	0.68	0.34	0.69	0.35
UTAUT‐T Behavioural Intention	1.16	0.04	1.16	0.03

Abbreviations: RMSE, root mean square error; UTAUT‐T, Unified Theory of Acceptance and Use of Technology Therapist Version.

DISCUSSION

In the present study, we examined different aspects of therapists' acceptance of TPT technology (i.e. therapists' attitudes, concerns and intention for future utilization of TPT) as well as the therapists' reported therapeutic relationship with their online patients, their levels of professional self‐doubt and vicarious trauma, as experienced in online sessions during the COVID‐19 pandemic. We explored how these TPT experiences (the therapist‐reported therapeutic relationship with their online patients, their levels of professional self‐doubt and vicarious trauma) might relate to the different aspects of therapists' acceptance of TPT technology. In order to develop and cross‐validate predictive models, we divided our collected survey responses from 795 therapists into a larger training‐set subsample and a smaller test‐set subsample. We then used a machine learning decision tree approach to build predictive models in the training set, before cross‐validating the performance of these models on the test set.

Our survey responses from 795 therapists indicated that therapists had neutral or moderately positive attitudes while also many concerns regarding TPT, and on average, they were undecided about whether or not they would use TPT in the future. Therapists experienced the therapeutic relationship with their patients in their online sessions positively: They reported similar levels of working alliance and higher levels of real relationship in their online sessions during the pandemic compared to in‐person levels in previous studies conducted pre‐pandemic (e.g. blended therapy, i.e. online modules combined with therapist sessions; Vernmark et al., 2019 and in‐person therapy; Bhatia & Gelso, 2018, respectively). Therapists experienced higher levels of professional self‐doubt and higher levels of vicarious trauma than was reported in previous studies on in‐person therapies (Aparicio et al., 2012).

When we tested three aspects of the acceptance of TPT technology (attitudes, concerns, future intention to use), the first model showed that therapist‐reported working alliance was most pertinent to be associated with attitudes towards TPT, in that sufficiently high alliance scores were associated with more positive attitudes, and most positive attitudes were reported by those who also experienced little professional doubt and were 40 years old or younger. Therapists who reported the weakest quality of the working alliance also reported the most negative attitudes towards TPT technology. Therapists' concerns about TPT technology was most pertinently associated with the therapist's professional self‐doubt, in that therapists with very little professional doubt (i.e. the most professionally confident therapists) had the least concerns about TPT, and concerns were the lowest for those who also reported low levels of vicarious trauma during the start of the pandemic. The prediction model for the intention to use TPT technology showed that the only pertinent predictor was working alliance. The majority of therapists who reported at least a medium level of working alliance were inclined to use TPT in the future, whereas a small subgroup of therapists who reported weak alliance also reported the least likelihood to use TPT in the future.

Unlike previous research on in‐person therapies pre‐pandemic that suggested that besides the working alliance, the quality of the real relationship is also important in predicting treatment efficacy (Bhatia & Gelso, 2018), the real relationship did not appear to play an important role in therapists' acceptance of TPT technology. In none of the three models, their reported level of the real relationship (RR) with their online patients was associated with any of the three dependent variables, once the independent variables that were more strongly associated with each of the dependent variables were included in the models, and therefore did not show in the respective decision trees. Similarly, gender of the therapist was not relevant in predicting any of the aspects of the therapists' acceptance of TPT technology.

Overall, these results indicate that therapists' professional self‐doubt and the working alliance that they perceive with their online patients are the most prominent factors in predicting their reported acceptance of TPT. Our study findings partly confirmed previous results that therapists are often reluctant to use TPT due to fears of being unable to build a strong therapy relationship with their patients online (Roesler, 2017). In line with previous studies, based on patient self‐reported alliance measures, that reported that the working alliance in online therapies may be just as strong as in in‐person therapies (Norwood et al., 2018), we found similar ratings of the online working alliance in our participating therapists.

Our results also highlight that concerns about the use of TPT is not pertinent to the therapists' ability to build a strong working alliance online, but rather to their professional self‐doubt and experience of vicarious traumatization in their online work. This might imply that therapists can benefit from their own professional support in order to reduce their professional anxiety during the pandemic and more generally. Moreover, although professional self‐doubt was linked to better treatment outcomes in a previous study of trainees (Nissen‐Lie et al., 2013), our study showed that the experience of more professional self‐doubt was negatively related to therapists' attitudes and positively related to concerns about TPT technology. Thus, professional self‐doubt is an important area to address in practitioners in order to promote the acceptance of TPT as a potential treatment format. Future research is needed to assess whether professional self‐doubt relates specifically to concerns about TPT or to providing therapy in general, including in‐person settings. Future research might also be warranted in examining how best to target and reduce therapists' sense of professional competence and confidence when applying new therapy technology.

Finally, vicarious trauma was only associated with concerns about TPT; it was unrelated to attitudes towards or to future intention to use TPT. And even within the model predicting concerns about TPT, it was not as pertinent as the therapists' experienced professional self‐doubt. Earlier research has shown that vicarious trauma has a negative impact on treatment effectiveness (Sexton, 1999), and this impact is especially strong when therapists and patients experience a disaster simultaneously (Boscarino et al., 2004; Culver et al., 2011). In our study, during the pandemic, providing treatment remotely to patients dealing with traumatic stress represents a serious challenge. Before the pandemic, therapists considered severe psychopathology and acute crisis situations as counter‐indicators of online treatment (Connolly et al., 2020; Simms et al., 2011); however, during the pandemic, there was no alternative to TPT. Our findings about heightened concerns regarding TPT in those with more subjective vicarious trauma experiences possibly reflect the therapists' general sense of anxiety during the pandemic or the impact of therapists' subjective experiences of stress and overwhelm on their ability to be open‐minded to new technology/ways of working. Also, due to the pandemic, therapists were probably more frequently exposed to highly stressful material in their sessions, and given that the stress of the pandemic impacts the therapists as well, they might have less emotional capacity to deal with their patients' trauma presentations. The heightened vicarious trauma experience in itself is an alarming fact, and its impact on concerns regarding TPT need to also be taken seriously when considering ways of using TPT with patients under high stress.

Our results show that despite concerns described in previous literature about relational aspects of TPT (Roesler, 2017; Sucala et al., 2013), the actual experience of working alliance and real relationship in online sessions was relatively positive. In fact, the quality of the real relationship was higher than in previous reports in in‐person therapies (Bhatia & Gelso, 2018; Gelso et al., 2012), even though due to the different study designs, these comparisons should be taken with some caution. Given that the quality of the therapeutic relationship is known to predict positive treatment outcomes, this finding provides some optimism regarding the potential impact of ongoing TPT. However, our study also indicated that therapists' relational challenges are a central factor in determining their attitudes and future intention to utilize TPT. Difficulties with building strong relationships with patients may not only decrease treatment efficacy but also hinder the utilization of TPT in the future. Future research on how therapists' can be helped to build a sufficient working alliance with patients, especially within an TPT format could be important to ensure therapists will feel comfortable enough offering the option of TPT to their patients.

The moderate levels of professional self‐doubt found in this study are understandable in light of the sudden transition to a TPT format and related practical and professional challenges, as well as the general stress of the pandemic on both therapists and their patients, that may have triggered doubts about professional skills and abilities. However, the experiences during the pandemic may have long‐term consequences on the continued use of the TPT format.

Clinical implications

Our findings might give an indication of which therapists might need more support if required to deliver therapy online. There is a need for training and support on (1) reducing and tolerating professional self‐doubt related to TPT technology; (2) further developing skills to strengthen the relationship in TPT sessions; and (3) identifying therapists who are experiencing vicarious trauma and providing additional support and care.

Providing therapy remotely has its own unique challenges, including the lack of certain aspects of non‐verbal communication, which may hinder understanding of the patients' emotional state or the therapist's ability to sufficiently transmit their own emotions and empathy (Wagnild et al., 2006). Training courses could provide tools to compensate for these aspects of TPT and help therapists to improve their online communication skills to enhance the therapeutic relationship online, and maybe most importantly, their sense of professional competence and confidence. Professional training and peer support groups are also valuable resources for ameliorating the subjective sense of vicarious traumatization while working during a global pandemic that impacts both patient and therapist.

In many countries, therapy is delivered as part of government funded services, so decisions about mode of delivery are likely to be out of clinicians' hands. However, the increased clinical and research interest in TPT might mean that TPT will be more readily accepted by patients, therapists and payers. Given that more positive attitudes towards a treatment process tend to relate to better treatment outcomes, this bodes well for patients' well‐being. Although previously clinicians were required to provide in‐person therapy to ensure governmental funding, in the near future, clinicians might be able to make a clinical decision to determine which therapeutic format might be most beneficial to a patient at a given time. Of course, the use of in‐person therapy or TPT does not have to be a binary clinical decision. Therapists and patients might consider integrating TPT sessions within an in‐person treatment or do the full treatment via TPT technology (Van Daele et al., 2021). Patients may receive a combination of in‐person and remote sessions based on changing needs over the course of therapy (Yellowlees & Nafiz, 2010).

Limitations

This study had several limitations. First, even though in our study we built the prediction models based on the well‐supported assumption that attitudes towards TPT may be predicted by various factors (Connolly et al., 2020; Venkatesh et al., 2016), including the relationship, professional doubt and vicarious trauma variables, a reverse causation may also be argued for. For example, it is possible that preconceived attitudes towards TPT may prevent therapists from trying to build strong therapeutic relationships in their online sessions.

Second, even though the comparison between our sample and samples reported in previously published studies may be informative, these comparisons need to be interpreted with caution due to various confounders (e.g. different therapist and patient and treatment characteristics in these studies).

Third, we used a convenience sample of therapists willing to report about their experiences during the pandemic in an online survey. Although our sample was relatively diverse, most respondents were female (as is the case in many survey studies; Moore & Tarnai, 2002). At the same time, this is representative of the larger population of psychologists (American Psychological Association, 2015). Therapists who volunteered to participate might also have been especially interested in TPT, which possibly biased the results.

Fourth, a well‐known limitation, common to all survey research, is that all variables were self‐reported responses, which means that the relationship between these variables might have been spuriously inflated. Future research could benefit from using complementary methods to gain information about therapists' acceptance of TPT technology, for example, by having observer codings of behaviour of therapists or by interviewing supervisors and patients to gain their perspective on the therapist's attitudes.

Finally, although the present study reports on a rigorous cross‐validation method for producing results that are likely to be generalizable to the broader population, this must be viewed as internal validation, rather than external validation, as the test set is still a subsample of the original dataset, rather than an entirely new or separate sample (Aafjes‐van Doorn et al., 2021). Thus, it is possible that the identified independent variables may not generalize to new samples of other therapists and that an additional out‐of‐sample external validation would further strengthen the generalizability of our findings (Sammut & Webb, 2017).

Conclusion

This large survey study of therapists' acceptance of TPT technology during the first weeks of COVID‐19 indicates that therapists had relatively positive attitudes towards TPT technology and some concerns about using TPT in the future. The most consistent factor influencing therapists' acceptance levels was their professional self‐doubt as well as their perceived quality of the online working alliance. This implies that professional support and training might be helpful in increasing therapists' sense of professional and relational competence and confidence online, and therefore generating a more positive attitude, fewer concerns and a higher likelihood of continued use of TPT technology in the future, when it is no longer a health requirement.

CONFLICT OF INTEREST

The authors have no conflicts of interest to disclose.