The Good, The Bad, The Uncertain: Diverse Provider Experiences with Telemental Health During COVID-19.

This study assessed mental health provider attitudes and perceptions of telemental health (TMH) prior to and during the COVID-19 Pandemic. The study expands on earlier work by providing a more detailed qualitative analysis of provider perceptions of TMH, including its efficacy, advantages, and limitations. The current study is part of a larger mixed methods project utilizing a repeated cross-sectional design. An online survey was administered to a sample of 1,448 mental health providers. Of the survey participants, 934 offered narrative responses to open-ended questions and were included in the present study. Qualitative data was analyzed using a coding team and the Consensual Qualitative Research paradigm. Providers described both positive and negative feelings about using TMH during the COVID-19 Pandemic. Several advantages were identified, with providers clearly appreciating the role of TMH in allowing them to work continuously and safely during the public health emergency. An array of negative views and concerns were also expressed, including that TMH may not be optimal or effective in certain settings or situations. A portion of respondents also indicated a preference for face-to-face care and illuminated ways they found TMH lacking or limited.

Introduction

Mental health care systems and protocols were dramatically impacted on a global scale as a result of the COVID-19 pandemic [1, 2]. To balance risk and continuity of care, many systems and providers responded by rapidly shifting to remote delivery of mental health services via telemental health (TMH). Prior to the pandemic, TMH was increasingly used to address provider shortages, treatment gaps, and access issues, albeit was still relatively underutilized as compared to traditional face-to-face care [3]. Following the W.H.O. March 2020 pandemic declaration, TMH use rose exponentially as state and federal agencies issued executive orders to reduce barriers to implementation and use and expand TMH coverage [4-6]. For instance, within the U.S. Department of Veterans Affairs (VA), the United States’ largest health care system, by April 22, 2020, TMH video encounters had risen by 556%, and 77.5% of providers were first time TMH users [7].

As a result of the pandemic, the urgent and rapid adoption of TMH created wide variability across TMH utilization and protocols [1]. Research evaluating the clinical efficacy and feasibility of specific TMH modalities used in clinical care suggests that synchronous video conferencing is preferable to telephone [8-10] and is comparable to face-to-face treatment delivery across a wide range of populations, settings, and diagnoses [11-13]. However, a recent review of evaluation approaches for TMH programs found that there is inconsistency in assessment methods and no clear gold standard for TMH program measurement [14]. There is also a dearth of literature examining mental health care providers’ perceptions about their use of TMH, and qualitative analysis of important metrics including provider satisfaction [14]. Thus, qualitatively examining how TMH is perceived by providers is critical to evaluating its barriers and informing policy and long-term usefulness and effectiveness.

There has been an overall increase in qualitative and mixed methods research examining the pandemic related TMH shift for healthcare providers across specialties [15, 16]; however, a relatively small subset of this body of research focuses specifically on mental health providers’ experiences. Findings suggest overall positive experiences, with cited benefits for the patients and health care systems, and satisfaction with maintaining continuity of care and increased scheduling flexibility [17]. However, challenges and barriers persist regarding technology problems and outcome monitoring, lack of TMH trainings, and concerns around using TMH for new visits and for populations for which current practice of TMH may be ill-suited or building therapeutic alliance would be more challenging, including children [18], vulnerable groups such as refugees [19] and rural or homeless populations [5, 20, 21].

A recent systematic review found that mental health providers generally have positive feelings about TMH while also noting several disadvantages. While satisfaction with TMH is high overall, satisfaction tends to be higher for patients than providers [22]. Provider attitudes and underlying factors shaping these attitudes are important to understand, as they have been shown to influence the adoption and implementation of clinical practices, such as evidence-based treatments [23, 24]. Prior research has also identified provider expectations of perceived usefulness as a key predictor of TMH uptake [25]. While provider attitudes about TMH generally improve over time and with use [8], there may still be an overall preference for face-to-face care, with the latter typically rated higher in terms of provider satisfaction [26, 27].

Even though there is now general consensus that TMH is here to stay [28, 29] there remains scant qualitative research on provider perceptions of, attitudes towards, and beliefs about TMH, including the presence of potential biases that may limit their willingness to utilize this modality in their own practice. These important factors may also differ when considering the initial period of pandemic-related TMH use compared with long-term utilization.

The COVID-19 Pandemic presented a unique clinical situation where most, if not all, mental health providers were faced with utilizing TMH regardless of previous familiarity with the modality, beliefs about its clinical utility, or desire to incorporate it into their work. In this way, the pandemic presented the mental health workforce with one of the biggest paradigm shifts in the practice landscape most have experienced in their careers. Assessing and understanding diverse experiences with TMH as a modality of care is important in terms of continued adoption and expansion. Provider data will be crucially important in terms of offering bidirectional feedback to the field and helping to shape the future of healthcare delivery. Existing research has generally found positive outcomes for TMH, but there are notable ways we know that this research is limited or otherwise biased [22] and it is necessary for the field to obtain a fuller picture across all types of providers and clinical services. The present study expands upon previous work using quantiative methods that demonstrated that the COVID-19 Pandemic resulted in significant shifts in provider perceptions and attitudes about TMH as a clinical care modality [8]. This study takes a qualitiative approach to explore and understand the complex and nuanced experiences and viewpoints of mental health providers; it synthesizes and summarizes existing provider attitudes and concerns using real-world examples. We hypothesized that results would closely mirror previous work demonstrating that there was variability in provider perceptions and attitudes, with both advantages and limitations and concerns endorsed and described.

Methods

Procedure

The present study is part of a larger initiative to understand mental health provider experiences with, and perceptions of, TMH with a specific emphasis on the impact of the COVID-19 Pandemic and its role in TMH adoption and utilization. A comprehensive survey was designed for this study in order and has been described in previous work [8]. This study was reviewed by the Human Subjects Committee at VA Connecticut Healthcare System and determined to be exempt from further review. This study was conducted in compliance with the standards of the Internal Review Board of the Veterans Affairs Healthcare System.

Recruitment for the study occurred using targeted email invitations and a snowball sampling technique. Recruitment emails were sent to professional listservs for mental health providers both within and outside of the VA system. The survey was open for approximately one month and responses collected between from late May 2020 to June 2020 with this timeframe intentionally selected to capture practice modifications associated with COVID-19. Mental health providers who chose to participate in the study were directed to a secure online data collection platform (Qualtrics, Provo, UT), where written informed consent was provided prior to survey administration.

Measures

A brief demographic questionnaire was given to participants to collect personal and professional information. Demographic information was not required and included fields such as gender identification, racial/ethnicity, sexual orientation, age, relationship status, living situation (others in the home), and state of residence/employment. Professional information included discipline, employment setting, full or part-time work status, licensed or trainee status, and years of employment/practice.

Telemental Health Survey

This self-report survey was created by the study authors, who are experienced in survey design. The survey consisted of two parts: provider attitudes and perceptions of TMH prior to the Pandemic (Part 1; a retrospective assessment) and during the Pandemic (Part 2). Providers completed quantitative questions designed to assess their agreement with statements about the importance, necessity, and efficacy of TMH as a modality of care. Open-ended responses were included throughout assessment phases in order to glean more information into provider attitudes and perceptions of TMH. Providers were asked to describe and elaborate on any positive or negative feelings about TMH they had endorsed as part of the survey both in the retrospective analysis section (Part I) and as currently experienced during the Pandemic (Part II). A complete copy of the survey is available upon request.

Data Analysis

Quantitative data was analyzed using the statistical program SPSS [30]. All responses were screened for missing data prior to running analyses. Cases with more than 5% missing data were excluded from subsequent analyses. Descriptive statistics, including frequency data and mean scores, were utilized to describe the sample. Qualitative responses to any of the four open-ended questions were sufficient for inclusion in qualitative data analysis. Of the 1,448 respondents who completed the larger survey, 934 provided qualitative responses and were included in the qualitative coding analyses.

Qualitative Coding

All narrative data utilized in the study was analyzed using Consensual Qualitative Research (CQR; [31]) a rigorous and iterative team-based coding paradigm for qualitative data. Processes outlined in the CQR manual were closely adhered to, as were subsequent recommendations for utilization [32]. The goal of CQR is to independently review narrative data and then to work as a team to find consensus about coding domains and classification. The coding team was stable and consisted of three psychologists and one doctoral candidate, all of whom had received training in the methodology and had previous experience with qualitative data analysis. Three members served as a primary coders, and one psychologist served as an auditor. The latter reviewed the coding from a more distal perspective to check the quality and catch any groupthink that might be occurring on the primary team. The entire team reviewed and discussed coding disputes until consensus was achieved.

In reviewing narrative responses, coders work to develop domains (topic areas) and core ideas (the major points of each domain). Categories are then coded and quantified to describe trends or consistencies in the data. Domains were developed based on the structure of survey questions. Core ideas were derived from participant responses, based on an initial review of a subset of qualitative data, and then refined and modified as needed during the coding process. Consistent with recommendations for CQR utilization, the coding team worked to identify and discuss any biases or preconceptions prior and throughout data analysis. All members of the coding team had previous and personal experiences with TMH to varying degrees, with most of the team reporting previous TMH utilization prior to the onset of the COVID-19 Pandemic. The general expectations of the research team were that both positives and negatives about TMH would be identified by respondents.

Results

Participants

A sample of mental health providers (n = 934) were included in the present qualitative study and were part of a larger mixed methods study [8]. Most respondents were psychologists (68.4%) followed by social workers (23.8%), psychiatrists (3.3%), nursing/APRNs (1.7%) or other types of provider (3.1%). Most worked for the Veterans Health Administration (79.7%), followed by private practice (10.7%), and the remaining 9.6% worked in other hospital or clinic, academic medical centers, research, or community mental health settings. The majority endorsed working full-time (91.0%) and 8.7% identified as a trainee in their respective field of study. Participants represented all regions of the U.S. Most participants identified as White or Caucasian (87.9%), 3.7% as Black or African American, 3.1% as Asian or Asian American, 0.8% as American Indian or Alaskan Native, 0.1% as Native Hawaiian or Pacific Islander, 4.5% as mixed race/other, and 6.0% as Hispanic/Latinx. The majority of the sample self-identified as female (74.5%) and the mean age was 44.8 years old (SD = 12.5 years; age range = 23–87 years). Most participants reported living with others (82.5%) and over a third of providers reported that they were currently balancing work responsibilities with caring for family members.

Major coding domains were determined by the structure of the survey questions, Positive Feelings about TMH (Pre-Pandemic and During the Pandemic) and Negative Feelings about TMH (Pre-Pandemic and During the Pandemic). Each domain and the resulting core ideas are presented below and in Tables 1–3 below, which provide more in-depth information about operational definitions for each core idea and offer representative quotes for each section.

Table 1

Pre-Pandemic Positive Feelings about Telemental Health

THEMES	#	OPERATIONAL DEFINITION	REPRESENTATIVE QUOTES
Familiarity with TMH	132	Response references previous utilization of and familiarity with TMH as a modality (e.g., indicates that they were already using TMH in some capacity prior to the Pandemic)	Q1: TMH is a regular aspect of my day in providing care to patients Q2: I had always wanted to use telehealth and did so Q3: I used TMH twice monthly and was working to build up my clinical load
Serves a General Purpose	404	Response acknowledges that there are general benefits to using TMH and offers examples for how TMH serves a purpose in behavioral health care (e.g., increasing access to care)	Q1: I think TMH increases access to care and is a vital part of our practice as psychologists Q2: It is very convenient and allows services to be offered to people who would otherwise be very inconvenienced or not have any access to mental health care Q3: I appreciate its role in increasing access to care, particularly for more rural patients and those with transportation or mobility limitations
Positive Outcomes / Efficacy	73	Response references experiencing positive outcomes from delivering TMH services, describes TMH services as a success, or makes explicit statements about the efficacy of TMH or TMH demonstrating effectiveness	Q1: TMH allows for similar treatment outcomes to in-person therapy Q2: For the most part, I have had great success with telehealth Q3: I was thinking that the quality of TMH was not going to be as clinically productive as in-person work; however, since I have been using TMH, my experience thus far has proven otherwise
Other	114	Response includes narrative information that did not fall under existing core ideas	Q1: TMH is useful for some people at some times Q2: I thought TMH could be a helpful first step to attending in-person therapy, or a better option than nothing for certain groups of people Q3: TMH is a different kind of intimacy with the veterans… I see them in their homes, more relaxed usually, without the stressors or driving, parking, or being in a strange situation

Table 3

Negative Feelings about Telemental Health During COVID-19

THEMES	#	OPERATIONAL DEFINITION	REPRESENTATIVE QUOTES
Technology Problems (Pre-Pandemic)	157	Response acknowledges the role of technological difficulties in creating negative impressions of TMH, including poor connectivity, audio/visual quality issues, or lack of reliable WIFI to support clinical work	Q1: We have had technology problems with the platform mostly in relation to poor connectivity and loss of calls Q2: While I believe it is a valuable clinical tool, issues with connectivity (audio and visual quality) have been significant and quite frustrating Q3: Connectivity and technical reliability has been fair to poor in rural areas
Technology Problems (Pandemic)	127		Q1: I am frustrated by how often I experience technological glitches during sessions Q2: Connectivity issues have interfered with the effectiveness of several of my sessions Q3: Technical problems or limited technology literacy can make TMH difficult… it sometimes overrides the focus of treatment sessions and it becomes about logistics and technology instead of treatment
Concerns about Effectiveness (Pre-Pandemic)	153	Response includes specific concerns about the effectiveness of TMH as a modality of care generally or in regards to a specific type of service (e.g. assessment, neuropsychological evaluations), or general statements about TMH being ineffective or not as effective as face-to-face care	Q1: I did not feel that telehealth could be even close to face-to-face contact in terms of effectiveness Q2: I felt that services wouldn’t be as effective due to the depersonalization of using technology Q3: I believed that TMH was a less effective way to do therapy and should be avoided in all cases unless absolutely necessary (e.g. rural communities, agoraphobia, disability)
Concerns about Effectiveness (Pandemic)	125		Q1: I am glad I can continue to support my clients, but in-person [treatment] is much more effective Q2: For many clients, TMH is useful for a brief check-in or case management. For most of my clients, TMH is a completely ineffective delivery system for psychotherapeutic interventions Q3: I don’t think that I can provide the same level of behavioral observation to inform my clinical judgment, particularly for new patients
Negative Feelings about TMH (Pre-Pandemic)	256	Response references problems or challenges with TMH, aspects of TMH that are disliked, or limitations of TMH (things that feel different or harder; perceptions about what is lost)	Q1: It was difficult to create a therapeutic environment via TMH Q2: The separation is impersonal and counterproductive to optimum therapy and treatment Q3: I felt it was not intimate enough to create any real deep change or relational healing… I felt it was very informal and detached
Negative Feelings about TMH (Pandemic)	108		Q1: There are limitations with connecting with patients via video – something is lost Q2: There are observed differences between being able to be in the room with the full scope of nonverbal/behavioral observations and the “good enough” space through a video screen Q3: It removes the personal presence of the patient and therapist. I feel much less connected to my new TMH patients… I feel like it cheats the patients of the emotional presence of the provider. I experience a slight devaluation of therapy through this medium
Negative Emotions from TMH (Pre-Pandemic)	58	Response acknowledges negative emotions about TMH (worry, anxiety) or negative personal impact (feeling overwhelmed, exhausted or frustrated)	Q1: Felt overwhelmed by the thought of doing therapy from a distance Q2: I liked the idea of TMH, but found the logistics of incorporating it into my practice overwhelming Q3: Discomfort / Anxiety
Negative Emotions from TMH (Pandemic)	64		Q1: I personally dislike it and find it a much bigger energetic drain… it is less enjoyable than in-person care Q2: I have some anxiety about the system not working Q3: Frankly, it is exhausting to use hour after hour. I am a people person and miss being with people in the same location
Preference for Face-to-Face Care (Pre-Pandemic)	133	Response explicitly includes a stated preference for face-to-face care or includes statements about how TMH feels inferior in at least one way	Q1: Given the choice, I would always opt for face-to-face visits for my clients Q2: I preferred in-person appointments in almost all cases Q3: I strongly prefer providing in-person services… Tele-sessions diminish the quality of the therapeutic process
Preference for Face-to-Face Care (Pandemic)	99		Q1: TMH does not, in my opinion, replace live, in person encounters Q2: I very much prefer to meet with patients directly as I feel a stronger connection and am energized by it Q3: I still much prefer face-to-face interactions and find TMH to restrict the therapeutic environment and bond
Systems Issues (Pre-Pandemic)	93	Response references systemic difficulties or challenges that impacted the experience with TMH (e.g. burdensome scheduling, lack of available tech support) or general systemic concerns (e.g., privacy and security when using the internet)	Q1: It was not as easy to schedule my telehealth appointments as other visits Q2: It appeared difficult to use, [we] had no training on how to use it, were not given information on resources or contacts to assist with TMH services if needed Q3: The biggest con for me is the time it takes to coordinate the visits through the VA system. The scheduling issues are really burdensome and significantly affect many providers’ willingness to do telehealth
Systems Issues (Pandemic)	54		Q1: There is no way to contact tech support, just a lot of FAQs and online articles Q2: I have concerns about confidentiality due to technology being used and the possibility of limited privacy on the patients end Q3: The paperwork required here to document the visit was excessive. There are extra steps needed to book the appointment and then to send the video link, all of which are the responsibility of the clinician
Other (Pre-Pandemic)	89	Response includes narrative information that did not fall under existing core ideas	Q1: I understood that it was not in the patient’s best interest to use teletherapy. There were also ethical, legal, and financial restrictions adding to my refusal to use it Q2: I associated TMH with “big business” corporate groups that recruit providers and farm them out across the country… they pursue profit over quality Q3: The biggest problem is some veterans no taking this type of intervention seriously… some try to multi-task. We need to be doing a better job of educating and training the veterans on how to be a good patient in this new context
Other (Pandemic)	54		Q1: It brings up access to care issues related to having stable internet and a computer or smart phone Q2: I think it is the only option right now to be able to provide services. It is not my preference, or my patient’s preference, but it is what it is Q3: Low SES patients do not have the same level of access (e.g., less sophisticated equipment, slower internet speeds, no scanner/printer)

Pre-Pandemic: Positive Feelings about Telemental Health

A total of 752 individual responses included at least one positive code (80.5%). Three major core ideas emerged for pre-pandemic positive feelings about TMH: Familiarity with TMH (n = 132, 14%), Serves a General Purpose (n = 404, 43%), and Positive Outcomes and Efficacy (n = 73, 8%). Responses that did not fit under these themes were categorized as Other (n = 114, 12%). Table 1 provides the operational definitions used for coding as well as three exemplar quotes representative of each core idea. A minority of respondents provided personal information about the perception of telehealth for themselves, including “I have personally received therapy through TMH and found it to be very effective and a more efficient use of my time,” “It is best for my work life balance,” and “I am more relaxed… I can’t be physically assaulted.”

During the Pandemic: Positive Feelings about Telemental Health

A total of 591 responses included at least one positive code (63%). Five major core ideas emerged from the data: Positive Feelings about TMH (n = 182, 19.5%), Serves a General Purpose (n = 174, 19%), Pandemic Specific Purpose (n = 103, 11%), Positive Outcomes and Efficacy (n = 126, 13.5%), and Equivalent to Face-to-Face Care (n = 26, 3%). Responses that did not fit under these core ideas were categorized as Other (n = 127, 13.5%). Table 2 includes operational definitions for each core idea and three representative quotes.

Table 2

Positive Feelings about Telemental Health During COVID-19

THEMES	#	OPERATIONAL DEFINITION	REPRESENTATIVE QUOTES
Positive Feelings about TMH	182	Response includes at least one specific positive thought or feeling about TMH as a modality of care (e.g., indicates acceptance or liking of TMH services)	Q1: TMH is an excellent addition to psychological services already available Q2: I have now seen and experienced the benefits of TMH Q3: I think this is the future and it’s here now. My patients love it. It’s so much more convenient. We talk about how we didn’t know how it would be but we ended up loving it. I will continue with it
Serves a General Purpose	174	Response acknowledges that there are general benefits to using TMH and offers examples for how TMH serves a purpose in behavioral health care (e.g., increasing access to care)	Q1: TMH is a critical tool that enhances access to care Q2: Telehealth has allowed many more people to participate in appointments who otherwise would have missed and likely been a no-show Q3: I can reach a wider base of clients, including those with transportation issues, childcare or work conflicts, mobility issues, and anxiety issues related to driving or public transportation
Pandemic Specific Purpose	103	Response acknowledges the purpose of TMH in the context of the COVID-19 Pandemic (e.g., keeping patients and providers safe; offering an option for continuity of care)	Q1: TMH has been invaluable in the continued delivery of services during the era of social distancing and health concerns Q2: This is a great way for us to continue to provide services during the pandemic Q3: I have been appreciative that TMH has allowed me to see my patients throughout the pandemic. For a lot of my patients, this was their only ‘human’ contact outside of their family for the week
Positive Outcomes / Efficacy	126	Response references experiencing positive outcomes from delivering TMH services, describes TMH services as a success, or makes explicit statements about the efficacy of TMH or TMH demonstrating effectiveness	Q1: I was surprised how effective it appears to be with my patients Q2: TMH is very effective with the majority of patients Q3: I have been able to have extremely productive sessions. It has proven to be much more effective and user friendly than I anticipated
Equivalent to Face-to-Face Care	26	Response references the equivalence, or similarity, of TMH to face-to-face care (e.g., favorably compares TMH with in-person care)	Q1: In nearly all situations, telehealth is just as effective as in-person care Q2: Once a patient has made the adjustment, teletherapy visits are very similar to traditional [in-office] therapy appointments Q3: It has the same effect as face-to-face… once we start, we forget that we are not face-to-face in an office setting
Other	127	Response includes narrative information that did not fall under existing core ideas	Q1: There were several trainings that helped with general set up… we were generally prepared form a training perspective Q2: I am working from home and that is a plus Q3: It’s a brave new world. Not sure what it will do to all of us psychologically. Right now it seems very nice

A subset of respondents referenced benefits while also including some qualifiers, such as “Telehealth has turned out to be as good as face-to-face for somewhere between 50–80% of my work.” Some respondents alluded to ways that treatment had to shift due to the virtual format, such as “Modifications can be made to allow patients to receive evidence-based treatments and experience positive outcomes.” Several respondents continued to address fringe benefits to virtual care, with statements like “My no-show rate is the lowest it has ever been,” and “It is less expensive for my practice in terms of office rental.”

Negative Feelings about Telemental Health

In the Pre-Pandemic data, a total of 630 responses had at least one negative code (67%). During the Pandemic data yielded 486 responses that included at least one negative code (52%). The six core ideas derived from the data, and their operational definitions, were identical across pre-pandemic and pandemic narratives: Technology Problems (Pre-Pandemic n = 157, 17%; During Pandemic n = 127, 13.5%), Concerns about Effectiveness (Pre-Pandemic n = 153, 16%; During Pandemic n = 125, 13%), Negative Feelings about TMH (Pre-Pandemic n = 256, 27%; During Pandemic n = 108, 11.5%), Negative Emotions from TMH (Pre-Pandemic n = 58, 6%; During Pandemic n = 64, 7%), Preference for Face-to-Face Care (Pre-Pandemic n = 133, 14%; During Pandemic n = 99, 10.5%), and Systems Issues (Pre-Pandemic n = 93, 10%; During Pandemic n = 54, 6%). An Other category (Pre-Pandemic n = 89, 9.5%; During Pandemic n = 54, 6%) was utilized to capture responses not otherwise coded. Table 3 provides operational definitions and exemplar quotes for each core idea.

Across phases, respondents identified Concerns about Effectiveness specific to assessment generally as well as in terms of performing specific clinical services, such as neuropsychological evaluations. Sample statements include “I perform neuropsychological evaluations, and while my impressions of TMH are generally very favorable, I have reservations about certain aspects of assessment through TMH” and “It greatly limits and causes concern for validity and standardization of neuropsychological tests we can administer.” Several respondents identified Negative Feelings about TMH due to the patient being in their home environment, such as “Patients have been more relaxed… but sometimes too relaxed; some didn’t seem to view this as a doctor’s appointment (e.g. patients lying in bed/falling asleep, one without his shirt on, one was smoking cigarettes, one patient baked a cake, one was doing dishes).” There was also a minority of very negative sentiments, among them “It is dehumanizing and depersonalizing,” “TMH is not a substitute for psychotherapy,” and “It is a necessary evil.” Not all providers enjoyed working from home, as evidenced in this Other comment “I really dislike working in my home and not being able to separate work/home.” One respondent also articulated a larger sociocultural view on TMH and virtual life more generally “As a nation we are far too involved with communication through technology. Due to the image-conscious and artificial nature of social media, and the loss of reality orientation related to video games… it seems a questionable means of connecting with other people to nurture change.”

Global Codes

Participant responses were also examined holistically to determine a global code for describing the overall valence of the response. Global codes were coded as either Positive (Pre-Pandemic n = 256, 27%; During Pandemic n = 284, 30%), Negative (Pre-Pandemic n = 309, 33%; During Pandemic n = 145, 15.5%), Mixed (Pre-Pandemic n = 273, 29%; During Pandemic n = 205, 22%), or Neutral (Pre-Pandemic n = 96, 10%; During Pandemic n = 52, 5.5%). Table 4 provides operational definitions and representative quotes for each core idea.

Table 4

Global Codes

CODE	#	OPERATIONAL DEFINITION	REPRESENTATIVE QUOTES
Positive (Pre-Pandemic)	256	Responses were generally positive in nature with at least one specific positive impression noted and the absence of major negative sentiments	Q1: TMH is an excellent modality… I think TMH increases access to care and is a vital part of our practice as psychologists Q2: I have used telehealth at previous agencies and believe in the model strongly Q3: I found it to be a really effective method to provide mental health services and reached a greater number of people in rural areas and those with disabilities
Positive (Pandemic)	284		Q1: It is good enough for my practice and effective and convenient Q2: Despite some challenges with the rapid transition to telehealth, I am incredibly excited about the potential to reduce logistical barriers to accessing mental health services Q3: I have come to enjoy offering individual and group sessions via TMH. Many patients are enjoying it as well, and we will likely continue meeting via TMH after precautions are lifted
Negative (Pre-Pandemic)	309	Responses were generally negative in nature with at least one specific negative impression noted and the absence of major positive sentiments	Q1: I felt it was too impersonal with too many technical problems Q2: I did not recommend it broadly because it seemed inferior to in-person care Q3: I felt less authentic as a therapist
Negative (Pandemic)	145		Q1: For most of my clients, TMH is a completely ineffective delivery system for psychotherapeutic interventions Q2: [TMH] is no substitute for face-to-face contact Q3: It is not equivalent to face-to-face in ways that have not been measured by researchers. Rapport is harder to establish. Time is wasted dealing with technical issues. It is physically taxing to perform services via video
Mixed (Pre-Pandemic)	273	Responses included both positive and negative sentiments with at least one specific example provided for each, or explicitly referenced having a mixed perception	Q1: I have mixed feelings. It is good to use when there is nothing else available Q2: I was aware there were potential benefits in terms of access to care, but I struggled with how to handle potentially dangerous situations like suicidality Q3: I saw its utility for those who could not make it into sessions due to geography, etc. However, I did not feel it helped to cultivate a strong therapeutic relationship
Mixed (Pandemic)	205		Q1: It has been a lifesaver to keep offering some kind of service during the pandemic, but I still believe I could have been much more effective with the option of more traditional psychotherapy Q2: I have mixed feelings. It can be very helpful/effective with patients when not requiring structured interventions… But it gets more challenging when implementing tools/handouts/techniques. It’s doable, but seems less effective Q3: When it works, it is great. When it doesn’t work, it is annoying and disruptive to the session
Neutral (Pre-Pandemic)	96	Responses were generally neutral in nature or had an ambivalent tone, and did not include overtly positive or negative sentiments	Q1: I didn’t have strong feelings about it one way or another Q2: I was willing to try it, but had no preconceived notions about it Q3: So I was, and still am on the fence. I have no firm feelings either way
Neutral (Pandemic)	52		Q1: I guess I am just generally ambivalent about it Q2: I don’t have a strong opinion about it Q3: I have neutral feelings on TMH due to the situation the world is in

Overall, participant responses represented a wide range of attitudes and beliefs about TMH, ranging from the overwhelmingly positive “I am completely sold on telemental health. This has been a positive experience for patients as well as for me. It has been easier than I ever anticipated, and I will continue to use this in the future” to the very negative “Please don’t suggest that we are transitioning to this dumpster fire of a model.” Several Positive responses included mention of additional benefits to virtual care such as increasing treatment engagement and attendance, “This has been great. My no-shows and late cancellations have significantly dropped, as patients do not have to worry about traffic, sitters, etc.” There was more variability in the Negative responses, with some demonstrating stronger negative views such as “Virtual care is not psychotherapy,” or “It is mediocre, frustrating, and demanding.”

The Mixed responses offered important nuance and qualification to both positive and negative sentiments. One respondent stated, “I have found the platform of TMH to be more useful and productive that initially thought, however, for my style of clinical work, I still find in person clinical work to allow for more therapeutic depth.” A difference in clinical presence was somewhat of a common theme in responses, with responses such as “It has a benefit for certain situations (rural, driving distance, health issues preventing coming to the clinic). It is effective in still providing therapy; however, it does feel [like] a piece is missing… lacking the in-room presence… it is a good option for some situations.” A number of responses reflected the general statement of TMH being a great option for some, but not all, patients or situations. Examples include “It works for some, but not all, patients,” and “I find it to be effective with existing clients but less so when forming a therapeutic relationship, and especially uncomfortable in complex and/or high-risk situations”.

Discussion

The current study provides an important follow-up to previous work detailing changes in provider attitudes and perceptions over time [8]. The open-ended narrative approach provided an opportunity to expand upon and further understand previously reported quantitative metrics, offering helpful explication about provider perceptions and notably, their concerns about TMH and its overall clinical utility. Taken together, these studies offer a useful snapshot about current attitudes towards TMH during an ongoing global pandemic and provides an opportunity to examine how necessary utilization has resulted in shifting provider views.

It is of paramount importance that we assess and understand the full range of provider experiences and attitudes across aspects of clinical practice, including specific treatment protocols and modalities of service delivery. A true scientist-practitioner model requires the integration of both scientific evidence as well as clinical expertise, and we would do well to work collaboratively with our academic and clinical colleagues in the continued implementation and expansion of TMH, the spirit of which has been promoted in the evidence-based treatment literature more broadly [33]. Provider attitudes and willingness to employ a treatment or model are known variables that impact utilization [23, 25, 34] and overall implementation success.

Existing literature largely demonstrates satisfaction with TMH across patients and providers [35], though pre-pandemic research was largely limited to self-selecting samples who had already adopted TMH in their practice. Most of the research has focused on satisfaction as the main metric of evaluation, rather than explicitly asking about efficacy or effectiveness, with almost no direct exploration of negative or ambivalent perceptions. In an earlier study, our data found that while providers endorsed TMH as important, necessary, and effective, their overall attitudes were fairly mixed, with a substantial portion indicating they held at least some negative feelings about the modality [8].

Overall, qualitative data revealed a more nuanced picture of attitudes and experiences with TMH and reflected wide variability in provider reactions to TMH as a modality of clinical care. Pre-Pandemic responses were evenly split across global codes, with narrative data categorized into positive, negative, and mixed responses. There were less overtly negative categorizations during the pandemic, which is consistent with previously reported quantitative shifts in attitude [8]. A minority of providers displayed relatively neutral or ambivalent responses, not sharing sentiments either clearly in favor of or against TMH as a modality.

Advantages that touched on both access and accessibility as well as beliefs about clinical efficacy were described across response phases. Many providers saw TMH as a valuable clinical service in terms of reducing barriers to care and offering an effective method for psychotherapeutic intervention. Specific populations were identified that had more potential to benefit from virtual care, including those in rural communities, those with logistical or economic barriers to attending in person mental health appointments, and those with disabilities or other mobility/transportation limitations. A subset of providers described no notable difference between TMH and face-to-face care, experiencing the modality as equivalent and free of major limitations or concerns. Fringe benefits were also identified, including easier access to patients’ significant others, important clinical data available from viewing patients in their natural home environment, a reduction in cancelled or no-showed appointments, and personal benefit obtained from the ability to work remotely/from home. Many respondents also reflected on a key value of TMH during the pandemic: a safe way to continue providing care to patients.

However, many providers also endorsed and described negative attitudes and perceptions of TMH both prior to and during the pandemic. Biases towards TMH were identified, including feeling a sense of distance and detachment with TMH relative to in-person care. Participants articulated concerns about creating and maintaining a therapeutic alliance and intimate therapy environment. Some respondents spoke about the lack of a felt presence in virtual care as compared to face-to-face care, and the negative impact on interpersonal or relational change. Others felt that TMH was adequate and acceptable but simply acknowledged preferring in-person care and offering reasons for why they would choose face-to-face over TMH when given the option. Separate from beliefs about efficacy, several respondents acknowledged disliking TMH and experiencing negative emotions associated with its use, such as discomfort, anxiety, or feeling exhausted or overwhelmed by the demands of virtual care.

Important concerns about clinical effectiveness were described by a number of providers, and a theme that TMH was “good enough” but not equivalent to in-person care emerged with relative frequency. Providers described feeling less efficacy in a virtual format and indicated that their patients would not receive the maximum benefit possible through more traditional psychotherapy. There were also distinctions made for specific types of services and populations. Concerns about neuropsychological assessment were prevalent, highlighting issues of standardization, norms, test administration and interpretation. TMH was also seen as more useful for follow-up appointments and “check-ins” rather than initial appointments that focus on assessment, diagnosis, and rapport building. Several challenges specific to more structured treatments emerged, particularly those that require the regular transfer and collaborative utilization of materials like worksheets and other written tasks. The imperfection and unpredictability of technology was a salient barrier for many, with references to the negative impact on clinical care when technology failed or was otherwise unreliable/unstable. Several respondents referenced TMH being relatively seamless when it functioned well but severely limiting patient care when problems occurred. Finally, systems concerns and barriers existed, with reference to the scheduling and administrative tasks of TMH being more burdensome and time-consuming for providers.

It is important to note that much overlap was present in positive and negative responses, and the majority of providers held complex, balanced, and nuanced views of TMH. Many respondents were able to acknowledge and reflect on both advantages and limitations, emphasizing ways that TMH was useful as well as sharing concerns or negative views. One of the most common themes across questions and type of response was the sense that TMH works well in some situations for some patients, but it will not be the right or most effective choice in all settings or populations. The thoughtful perspectives of clinical providers likely reflect the reality that there are both advantages and disadvantages to TMH, as with any clinical modality or treatment, and what works for some patients will not work for everyone. It will be critically important to consider the unique and individual impacts of TMH on a given patient or situation as we move forward with TMH expansion and as it continues as a routine part of clinical care.

The present study extends previous work and offers additional support to the importance of assessing and understanding provider perceptions and attitudes towards treatment methods and modalities. Working collaboratively and addressing the concerns of providers implementing and utilizing TMH in diverse clinical settings will increase the likelihood of effective implementation and a thoughtful approach to TMH throughout the COVID-19 Pandemic and beyond.

Limitations

Study data was collected very early in the COVID-19 Pandemic, which likely had an effect on perceptions and responses. Additional longitudinal data will be necessary in order to more fully understand the effects of the Pandemic and the permanence of perceptions and attitude shifts. Pre-Pandemic responses were retrospective in nature, which introduces some potential for misremembering or other bias. It is likely that the actual necessity of TMH may have influenced provider perceptions during data collection. Being or feeling “forced’ to adopt a new modality and to do so quickly with little choice has the potential to introduce bias or otherwise affect attitude towards implementation. While an established and rigorous coding paradigm was used for qualitative data analysis, interpreting narrative data always involves inherent subjectivity. The study sample consisted primarily of psychologists who identified as White females and were VA employees; thus the results may not generalize broadly to the mental health profession. The VA has promoted TMH use nationally across its healthcare systems, and thus VA providers may have different experiences with TMH than those in other settings. While efforts were made to obtain a broad and diverse clinician sample, more representation from underrepresented racial and ethnic groups and clinicians in other settings (e.g., community clinicians) would have been preferable. Despite these limitations, the study nevertheless offers an important and deeper exploration into provider perceptions and experiences with telemental health early in the COVID-19 Pandemic. Future research should focus on obtaining additional information about provider perceptions and attitudes, as well as explore the lingering and after-effects of the Pandemic on TMH attitude and utilization.

Clinical Applications and Recommendations

Taken together with previous work [8], the COVID-19 Pandemic appears to have had a notable impact on experience with, and perceptions of TMH as a method of clinical service delivery. Results from the current study show that, while providers identify many advantages and positive feelings about TMH, concerns also exist about the appropriateness of TMH ubiquitously and the efficacy and utility of TMH across clinical contexts and individuals. Rather than being dismissed or challenged, provider experiences in real world practice should be evaluated and incorporated into future service delivery models as well as long-term policy impacts and clinical protocols. The field will do well to carefully assess the ideal conditions and optimal uses of TMH as we move forward and should work to identify the most thoughtful and effective way of utilizing TMH in the future. Healthcare systems should develop guidelines, in collaboration with provider and patient stakeholders, that take into account patient and provider preferences, recommendations, and concerns. Healthcare systems need to work to identify and address systemic barriers, such as access to high-functioning technology, broadband internet, and available technical support. We know that technology and connectivity problems have a negative impact on clinical care during TMH delivery, including a detrimental effect on the working alliance [36], reinforcing the importance of having reliable and seamless technological platforms available for use. Scheduling systems and virtual care features need to be further developed and utilized in a way that minimizes burden to patients and providers and ensures appropriate capability to address safety concerns, clinical emergencies, and other important aspects of care (e.g., privacy, security, the transfer of information and materials, measurement-based care). We also encourage readers to consider the potential for blended care models and flexible implementation of TMH. Rather than looking at TMH as an all-or-nothing option for an individual clinical situation, clinicians should consider the following: 1) In what ways may TMH be helpful or limiting in this particular situation and 2) Is there a place for TMH in this individuals care (whether that be a regular utilization of TMH or a blended care model that incorporates both in-person and TMH components).

We want to emphasize the importance of collaboration as we move forward with the “new normal” of telemental health care. The current study is a useful step in helping us move towards more communication across stakeholders and offers a deeper dive into the current perspectives of our mental health workforce, which has much to offer those who set and regulate policy and clinic guidelines. We recommend that healthcare administrators, leaders, and public officials carefully consider the reactions and concerns of our mental health workforce and engage in a process of collaboration and bidirectional feedback to ensure TMH continues to expand and sustain itself in a thoughtful and maximally effective way. As a field, there is much we do not know about the conditions in which TMH is appropriate and when this might not be clinically indicated or preferred. Working together, we are more likely to produce the most positive clinical outcomes we can for the diverse patient populations we serve.

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 12 KB)