Patient and provider experiences of telemental health during the COVID-19 pandemic in a New York City academic medical center.

OBJECTIVE: We aimed to evaluate patient and provider experiences with telemental health (TMH) at an academic outpatient psychiatry department in New York City during the first wave of the COVID-19 pandemic.
METHODS: Patients and providers completed online surveys evaluating their experience with TMH during the first wave of the COVID-19 pandemic. Surveys were distributed to 1,178 patients and 287 providers from July 2020 through October 2020.
RESULTS: 42.5% of providers and 21% of patients responded to the survey. The majority of patient and provider respondents rated the quality of phone and video visits as "equally good" or "somewhat worse" than in-person visits, while the majority of respondents were "somewhat satisfied" or "very satisfied" with video visits. Patients and providers preferred a hybrid model for future care. Common barriers to TMH included privacy, technical difficulties, and wi-fi access.
CONCLUSIONS: Patients and providers appeared willing to exchange some degree of quality for satisfaction with TMH. This study did not demonstrate with statistical significance any specific patient populations that would benefit more or less from TMH, suggesting that TMH may be a successful model for diverse patient populations. Our results suggest that providers, payors, and regulators should facilitate hybrid care delivery models that incorporate TMH beyond the pandemic.

Introduction

Decades of research on telemental health (TMH) have supported its use in improving access to mental health care without clearly compromising quality (Berryhill et al., 2019; García-Lizana and Muñoz-Mayorga, 2010; Hilty et al., 2015; Sucala et al., 2012). In pre-pandemic research, patient satisfaction with TMH tended to be positive, while provider opinions about TMH improved with use (Cowan et al., 2019). Despite the accumulation of favorable evidence for TMH, there have been significant barriers to TMH clinical implementation, including patient and provider expectations and preferences for in-person care, regulatory obstacles, concerns regarding the appropriateness of virtual services in addressing specific disorders or populations, lack of access to available technology, and difficulties with virtual service reimbursement (Dorsey and Topol, 2016; Wind et al., 2020).

In March 2020, the COVID-19 pandemic forced clinic closures and, in turn, required patients, providers, and healthcare systems alike to adopt telehealth across the country. This transition came with a forced relaxation of HIPAA and regulatory limitations and flexibility in insurance reimbursement for telehealth (Kinoshita et al., 2020). There is a growing body of TMH literature validating positive treatment outcomes for telepsychiatry and equivalency to face-to-face treatment (Bulkes et al., 2022; de Las Cuevas et al., 2006; Hubley et al., 2016; O'Reilly et al., 2007). Still, more information is needed to assess the virtual equivalency of different modalities and psychotherapies (Markowitz et al., 2021). Explorations in factors affecting quality, satisfaction, accessibility, and outcomes have proliferated since the beginning of the pandemic. A recent review on TMH during the COVID-19 pandemic highlighted the challenge of regulations and standard of care, effectiveness, privacy and confidentiality (Abraham et al., 2021). However, the growing consensus is that TMH is a reasonable modality of health care delivery, and that patients and providers are more accepting of TMH since the pandemic.

Over the course of a two-week time period in the epicenter of the initial surge of the COVID-19 pandemic, our large, urban academic medical center outpatient mental health department in New York City transitioned into a fully-functioning virtual clinic. We devised a survey to simultaneously collect patient and provider feedback and to better characterize perceptions of satisfaction and quality as distinct elements.

We surveyed patients and providers on two hospital campuses to understand their evolving experiences with TMH. We were specifically interested in perceived quality of TMH and satisfaction with TMH, as these are poorly understood proposed barriers to widespread TMH use. Since TMH can be a more affordable and accessible means of service provision, this study aimed to better understand patient and provider perspectives on TMH to help inform future practices.

Methods

Instruments

Two surveys (seen in “Supplementary Materials”), one for patients and one for providers, were created using Qualtrics software. To develop the survey, the team consulted with a group who had administered a similar survey in the Weill Cornell Medicine Department of Medicine (Sinha et al., 2020). The surveys included questions capturing demographic information and questions regarding patient and provider experiences with TMH. Questions regarding quality of care and satisfaction were scored using a 5-point Likert scale. Questions that pertained to patient and provider reasoning for preference of mental health care delivery were multiple choice with the option to rank five reasons, as well as an “Other” option where participants could free text an answer. The surveys were formated according to Qualtrics specifications to maximize readability and accessibility across mobile and desktop platforms.

Participants

The target population included patients who were receiving mental health services and their providers across the outpatient hospital programs of the New York City and Westchester County campuses of the NewYork Presbyterian Hospital -Weill Cornell Medical Center. Respondents from both patient and provider pools comprised the final study sample.

Procedures

Each participant received a unique link to a survey that could not be accessed once the survey was submitted, making this a closed survey that did not allow for multiple submissions. Both the patient and provider surveys took an estimated 8–10 min to complete. An informed consent statement was included in the initial email and at the beginning of the Qualtrics survey that explained the purpose of the study and notified participants that participation was voluntary and not associated with identifying information. The survey period was July 2020-October 2020. There was no financial compensation offered for completing the survey and no advertising was done for the study. The study and its associated surveys were approved by the Weill Cornell Medicine IRB committee.

Statistics

Responses were de-identified and anonymous when collated by Qualtrics such that no personal information, like IP addresses or cookies, was collected or stored in Qualtrics. Incomplete surveys were analyzed on an item-by-item basis. Descriptive statistics were used to characterize sample respondents. Categorical comparisons were completed using Chi Square tests of significance.

Results

Patient sample characteristics

The survey was distributed to 1176 patients, with 247 respondents (21% response rate) completing the survey out of 299 that were started (83% completion rate). Patient demographics are presented in Table 1 . Most patient respondents were Caucasian (N = 164, 79%), which is overrepresented compared with estimated combined patient enrollment (62%). Black/African American respondents were underrepresented compared with clinic enrollment, 8% versus 10%. There was an even distribution of patients with commercial insurance (N = 101, 47%) and those with public insurance (N = 107, 50%). Most patients spent 1 hour or less to commute to their appointment (85%), the cost of transportation to in-person visits was on average $10 or less for more than half of the clinic patients (56%), and it took most (85%) 1 hour or less to commute to their appointment.

Table 1

Patient and provider demographics.

Outpatient respondents N = 247	Total responses (%, N)
Gender	214
Male	31 (66)
Female	69 (148)
Age	219
18–45	38 (83)
46–65	37 (82)
>66 Years old	25 (54)
Race	207
Black	8 (16)
Latinx	8 (16)
White	79 (164)
Other	5 (11)
Insurance	215
Private	47 (101)
Public	50 (107)
None/Unknown	3 (7)
Income	167
Up to 40k	42.5 (71)
40–85k	27.5 (46)
>85k	30 (5)
Provider respondentsN = 122	Total responses(%, N)
Years in practice	92
0–5 years	25 (23)
6–10 years	17 (16)
11–15 years	18 (17)
16–20 years	10 (9)
21+ years	29 (27)
Clinical setting	143
Adult population	56 (80)
Partial hospital or day program	17 (24)
Child/adolescent	27 (39)

Provider sample characteristics

The survey was distributed to 287 providers, with 122 (42.5% response rate) respondents; 108 (88.5%) responders completed the survey. Provider experience levels ranged from early career to experienced clinicians, presented in Table 1. The majority of providers (56%) treated the adult outpatient clinic population, though many providers worked in more than one clinical setting.

Patient telemental health usage and barriers

The majority of patient respondents (N = 233, 94%) endorsed using telehealth during the COVID-19 pandemic to see their mental health provider, most via video (47.6%) or a mix of phone and video (N = 85, 36.5%) while 15.9% reported use of phone only.

Among the minority of respondents who were not using telehealth (N = 13, 5.3%), reasons included: not having adequate Wifi/internet (N = 1), Privacy concerns (N = 1), preference for in-person visits (N = 2), stable psychiatric status (N = 4), difficulty communicating due to hearing impairment (N = 1) and other reasons unrelated related to telehealth (N = 4). For the individuals that endorsed phone use only (N = 37, 15.9%), reasons included: only being offered phone sessions (N = 19), difficulties with software or did not have hardware available for session use (N = 36), and privacy concerns (23) (e.g., internet privacy, private space at home or discomfort with provider seeing home).

Patient respondents reported infrequent technical problems; 95% endorsed sometimes, rarely or never having technical issues during telehealth with their providers, and 32% reported someone was able to assist with resolving the problem “very frequently.”

Provider telemental health usage and barriers

Providers were seeing most of their patients on video (80%), or phone (20%), and less than 1% in person during the first six months of the COVID-19 pandemic. Providers most frequently endorsed the main reason for not providing video sessions was that patients lacked video enabled hardware (N = 43 ranked this within their top 3), followed by patient inability to download video applications on their devices (N = 29 ranked this within their top 3) and patients not having good or adequate data plans to accommodate video calls (N = 29 ranked this within their top 3).

Provider concerns related to using TMH were that it may compromise quality of care, followed by concerns that providers would not connect as well with their patients using virtual visits.

An equal number of providers endorsed that privacy or boundary issues related to use of telehealth were never or rarely an issue (N = 52) as endorsed that these issues were sometimes or frequently an issue (N = 51). Most providers endorsed never/rarely (N = 35) or sometimes (N = 56) experiencing technological issues during visits, while a minority of providers (N = 11) indicated that they frequently or very frequently experience such difficulties.

Of note, removing child and adolescent providers did not statistically change the results given the low number of respondents, and therefore we were unable to statistically compare results of these providers to those of adult treaters.

Quality and satisfaction

Quality and Satisfaction with Telemental Health are reported in Tables 2 a, b and 3 a, b.

Table 2

Patient reported quality and satisfaction with TMH compared with in-person visits.

	Visit quality (%, N)
	Phone N = 121	Video N = 184
a
Much worse	6 (7)	3 (5)
Somewhat worse	24 (29)	23 (42)
Equally
Good	57 (69)	61 (112)
Somewhat better	6 (7)	7 (12)
Much better	7 (9)	7 (12)
b
	Visit satisfaction (%, N)
	Phone N = 118	Video N = 183
Very dissatisfied	6 (7)	5 (10)
Somewhat dissatisfied	11 (13)	8 (14)
Neither satisfied nor dissatisfied	10 (12)	14 (25)
Somewhat satisfied	29 (34)	23 (43)
Very satisfied	44 (52)	50 (91)

Table 3

Provider reported quality and satisfaction with TMH compared with in-person visits.

	Visit quality (%, N)
	Phone N = 79		Video N = 103
a
Much worse	4 (3)		0 (0)
Somewhat worse	70 (55)		30 (31)
EquallyGood	25 (20)		69 (71)
Somewhat better	1 (1)		0 (0)
Much better	0 (0)		1 (1)
b
	Visit satisfaction (%, N)
	PhoneN = 79	VideoN = 103
Very dissatisfied	6 (5)	1 (1)
Somewhat dissatisfied	43 (34)	12 (12)
Neither satisfied nor dissatisfied	19 (15)	10 (10)
Somewhat satisfied	25 (20)	37 (38)
Very satisfied	6 (5)	41 (42)

Patient satisfaction and quality

There were no significant differences in the proportion of respondents who endorsed greater quality or satisfaction of TMH compared to in-person visits based on insurance status or gender.

There were no significant differences in satisfaction or quality of phone-based TMH by age group, but significantly more respondents in the 18–45 age group, versus other age groups, reported that the quality of video visits was as good or better than in person (x2=5.07, p = 0.02).

Data for TMH quality and satisfaction by race was limited. Qualitatively, black respondents rated phone visit quality better or the same as in-person visits proportionately higher than other races (N = 8, 100%). White respondents also endorsed experiencing better or the same quality (x2=4.34, p = 0.03) and satisfaction (x2=5.2, p = 0.02) with phone visits as with in-person visits, compared to other race groups.

Respondents who earned less (<$40 K/year) were more likely to find phone visits to be higher-quality and to report higher satisfaction with phone visits than respondents with higher incomes (x2=8.36, p = 0.004 and x2=7.2, p = 0.007).

Prior to the pandemic, 85.3% of patients endorsed traveling less than an hour to attend their visits. There was no difference in subjective quality of phone or video visits based on travel distance to in-person appointments prior to the pandemic. Satisfaction of phone visits was greater or equal to in-person visits for those who commuted more than 30 min prior to the pandemic (x2=6.67, p = 0.01). There were no differences in satisfaction by travel distance prior to the pandemic.

Provider quality and satisfaction

70% of provider respondents reported that the quality of care via video visits was equally good or much better than in-person visits and were somewhat or very satisfied with video visits overall (78%). In contrast, most providers found the quality of phone visits to be somewhat or much worse than in-person visits (73%), while the remainder reported that phone call quality was equally good or better. A greater proportion of providers endorsed greater quality of video compared to phone sessions (Table 3a; x2 (1, N = 182) =33.6, p<0.01).

78% of providers were somewhat or very satisfied with video visits and were more likely to be satisfied with video than phone (32%) visits (Table 3b; x2 (2, N = 182)=40.4, p<0.01). More providers were somewhat or very dissatisfied with phone visits (49%), while some reported that they were neither satisfied nor dissatisfied (19%).

Patient preferences

For patient respondents who endorsed that they preferred telehealth over in-person visits, the top reason was “no or less travel” (N = 67 ranked in top 3), followed by “easier to reschedule” (N = 41 ranked in top 3), and “less expensive” (N = 41 ranked in top 3). For those who preferred in-person visits, the top reasons endorsed included: “easier to talk openly” (N = 85 ranked in top 3), “more used to in-person treatment” (N = 72 ranked in top 3), and “better quality care in person.” (N = 49 ranked in top 3).

Provider preferences

The top provider reported reason for preferring in-person sessions versus TMH was that in-person visits allowed for greater ease of establishing rapport with patients (N = 46 ranked in top 3), followed by feeling that they provide better quality of care in person (N = 31 ranked in top 3), and also that they like seeing their colleagues (N = 29 ranked in top 3). Fewer providers overall preferred TMH over in-person, but of those who did, most reported that “no or less travel required” was the main reason for TMH preference (N = 26 ranked in top 3).

Compared to the pre-COVID-19 era, most providers (62%) indicated that they are much more willing to use telehealth for patient visits, while a small minority (5%) reported that they would be somewhat or much less willing to conduct patient visits via telehealth.

Future preferences

Most patients were “neutral” (N = 75) or “agree/strongly agreed” (N = 76) that they have a greater likelihood of attending future appointments via TMH versus in person. Patient respondents preferred a future hybrid care delivery model, averaging half TMH and half in-person appointments. Provider respondents, similar to the support endorsed by the patients, preferred a future hybrid care delivery model, averaging half TMH and half in-person appointments.

Discussion

Over the past couple of decades, innovations in technology have expanded to allow for the digital delivery of healthcare. Even with supporting data highlighting improved access to care, lower costs, and comparable patient satisfaction, there have been significant barriers in widespread adoption of virtual healthcare (Fortney et al., 2015; Jennett et al., 2003; Mann et al., 2020; Portnoy et al., 2016; Ramaswamy et al., 2020). The most frequently cited challenges have been perceptions of lower quality care and technophobia, as well as lack of equipment and infrastructure to accommodate telehealth services (Langarizadeh et al., 2017). In March 2020, the COVID-19 pandemic disrupted existing mental health services in the vast majority of countries and served as a catalyst to push patients, providers, and healthcare systems to adapt to telemedicine as a means of safe healthcare delivery (Singh and Sagar, 2022). Our study, conducted during the first 6 months of the COVID-19 pandemic, focused on patient and provider perceptions of TMH. Patents and providers were asked to rate the quality of TMH as compared to their traditional in-person visits, their satisfaction with TMH, and how interested they would be in continuing to utilize TMH when in-person visits were safe again.

Our results show that both patients and providers are generally satisfied with TMH, despite their perception that the quality of care can be somewhat worse compared to in-person care. Patient respondents experienced TMH as more accessible, convenient, and flexible compared with in-person care, while also eliminating costs and transportation time. Providers tended to prefer in-person care to TMH, but video visits required less travel, and offered fewer patient cancelations and more flexibility. Both patients and providers reported few issues with video-based communication technology.

Although TMH improved access to care and reduced costs associated with care, patients reported issues securing private space for sessions and concerns about internet privacy. Many providers felt that TMH made it challenging to establish rapport with patients, with privacy and boundary issues both affecting the quality of care delivered. However, despite these issues with TMH, both patients and providers reported a willingness to use TMH in the future.

This study is, to our knowledge, the first to demonstrate that both patients and providers are willing to sacrifice some measure of perceived quality of treatment for the benefits offered by TMH, including improved accessibility and convenience of care.

Importantly, this study focused explicitly on assessing self-reported quality and satisfaction without assessing subjective change in symptoms or tracking scores on symptom-related questionnaires. As a result, both patient and provider assessments of quality and satisfaction likely take into consideration factors related to the patient-provider alliance, rapport, overall comfort, and convenience, in addition to perceptions of symptom trajectory. Future studies may consider assessing objective measures of efficacy in comparison to subjective assessments of quality.

Our findings of positive patient and provider experiences with TMH during the COVID-19 pandemic, along with the preference for integrating TMH in the post-pandemic world, were consistent with emerging literature in the early course of the pandemic (Guinart et al., 2021, 2020; Sasangohar et al., 2020; Serhal et al., 2020). General telehealth studies have found that both patients and providers have high levels of satisfaction with the use of telehealth and are willing to use it in the future, finding it to be comparable in quality of care delivery, with increased flexibility and lower cost (Andrews et al., 2020; Malouff et al., 2021). Within psychiatry specifically, TMH offers an opportunity to exchange verbal and non-verbal communication without masks, which have been reported to negatively impact the quality of the psychiatric evaluation (Dondé et al., 2021).

Existing literature has found patient satisfaction with telehealth to be strongly correlated to access to care (Serhal et al., 2017; Torales et al., 2022). In our cohort, patients who earned a lower income, commuted a further distance, and had higher commute costs, were more likely to be satisfied with TMH services. Our research shows that TMH in urban areas is an acceptable care delivery model. Other studies have identified favorable patient and provider attitudes toward TMH, but similarly showed that a large percentage of respondents desired returning to in-person care and that hybrid options should be considered (Guinart et al., 2021, 2020). While many other studies have documented patient and provider satisfaction with TMH, none to our knowledge have compared this data with perceived quality of/satisfaction with care delivered (by providers) and care received (by patients). There is little data regarding patient and provider interest in the use of TMH in the future and the interest in hybrid care models.

The finding that mental health patients and providers may be willing to exchange some degree of care quality for satisfaction should be a significant consideration for policymakers, specifically as it pertains to coverage and reimbursement. Hospitals and healthcare providers should continue to advance TMH quality and access to care via improving upon technical difficulties and privacy issues, advocating for universal internet access, monitoring outcomes data, and investing in restructuring current systems to allow for the hybridization of healthcare.

Limitations

Our surveys had several limitations. First, our survey was sent to clinic patients and providers across two campuses and therefore our sample size was not randomly selected. Further, 10% of our sample providers primarily treated child patients who were not contacted to respond to our survey. Our sample was less racially diverse than the clinic and the community population which may have concealed the experiences of specific populations with TMH. The reasons for this particular response bias require further examination. The surveys also did not inquire about patient diagnoses or types of therapies delivered, and therefore we do not have a sense if certain patient pathologies or certain treatment modalities are more or less adaptable to a virtual format, which would be an area of future research.

The survey was distributed via email, and it was therefore subject to selection and response biases resulting from responders with access to technology, at least at the level of email. Mailed or telephone surveys may have better accessed the populations with poor access to technology, and results may have been more generalizable with higher response rates. However, the TMH barriers that exist for patients who have no consistent Wifi, smartphones/laptops, or adequate space will likely not be adequately addressed solely at the clinic level and will instead require greater policy or government involvement. Lastly, we surveyed patients and providers during a time of rapidly changing clinical protocol including expansion of video capabilities in the clinic, and perceptions of TMH may have changed over the course of the study period as the pandemic has persisted. However, we believe that technology limitations and technical difficulties have improved over time which likely would improve both patient and provider perception of care quality. We hope to be able to repeat this survey in the future to see how TMH perceptions have changed over time.

Conclusion

As the COVID-19 pandemic called for social distancing, outpatient mental healthcare transitioned to TMH as the primary form of healthcare delivery. Patients and providers alike were mostly satisfied with TMH; patients appreciated the convenience and accessibility and providers valued flexibility and efficiency. The COVID-19 pandemic provided a unique opportunity for patients and providers to acclimate to new modes of healthcare delivery. We have learned that although there was significant initial resistance to TMH, the opinions and preferences of both patients and providers were adaptable and TMH has proved to be a good alternative health care delivery model. To reduce mental health inequities and improve our care delivery models, future research must identify barriers to TMH and better identify which populations would benefit from improved access to mental healthcare via TMH versus those for which in-person care should be prioritized.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Declaration of Competing Interest

None of the authors of this manuscript have any conflicts to disclose.