Telehealth care before and during COVID-19: trends and quality in a large health system.

Objective: COVID-19 accelerated telehealth use to ensure care delivery, but there is limited data on the patient perspective. This study aimed to examine telehealth visit uptake before and during COVID-19 and correlates of patient satisfaction and interest in future telehealth visits. Materials and
Methods: This was a cross-sectional observational study between October 2019 and April 2020. Participants included patients who completed satisfaction surveys following telehealth visits.
Results: A total of 8930 patients completed the satisfaction survey using 4-point Likert Scales. Multivariable, hierarchical, cumulative logit models were constructed to examine correlates of satisfaction with quality of care and interest in future telehealth visits. Most patients were satisfied with the patient portal, video quality, and instructions (92.7%-96.8%). Almost half reported saving 1-2 h (46.9%). Correlates positively associated with quality of care and interest in future telehealth visits were ease of patient portal (odds ratio [OR], 1.43, 95% confidence interval [CI], 1.30-1.58; OR, 1.56, 95% CI, 1.41-1.73, respectively), video quality (OR, 1.62, 95% CI, 1.50-1.75; OR, 1.26, 95% CI, 1.16-1.37, respectively), instructions (OR, 5.62, 95% CI, 5.05-6.26; OR, 1.80, 95% CI, 1.62-2.01, respectively), and time saved (>4 h: OR, 1.69, 95%,CI, 1.22-2.34; OR, 3.49, 95% CI, 2.47-4.93, respectively). Being seen after the COVID-19 surge in telehealth (OR, 0.76, 95% CI, 0.63-0.93) or by providers with higher visit volume (OR, 0.71, 95% CI, 0.60-0.85) was associated with lower interest in future telehealth visits. Conclusions: Patients expressed relatively high satisfaction levels with telehealth. Better technical quality, quality of instructions, and greater time saved were associated with higher satisfaction ratings. To maintain interest in future telehealth use and improve the patient experience, we must enhance the quality of telehealth delivery platforms and instructions provided to patients.

INTRODUCTION

Telehealth care prior to the COVID-19 pandemic was projected as a promising way to deliver care for a wide audience in the next decade. There had been increasing uptake in targeted areas such as mental health care, diabetes management, telestroke, and healthcare delivery in rural populations. Prior to COVID-19, approximately 61% of healthcare institutions and 40%–50% of all hospitals in the United States used some form of telehealth. Immediate benefits of delivering care through telehealth, such as relatively low cost, the ability to offer remote monitoring for chronic diseases, time saved, and easy access, were driving the early adopters of this approach to care delivery.

Many technological, cultural, regulatory, financial, and health system challenges remained, however, and prevented universal uptake., The COVID-19 pandemic led to many stay-at-home orders and significant closures of outpatient facilities, accelerating the adoption of telehealth and leading to vast increases in video and phone visits and decreases in in-person visits., The systemic barriers to telehealth implementation, mainly payment and reimbursement, were addressed by the Centers for Medicare and Medicaid Services (CMS)., Furthermore, Office for Civil Rights decided that penalties would not be imposed for noncompliance of the Health Insurance Portability and Accountability Act (HIPAA) regulations, permitting providers to see patients through non-HIPAA compliant software or technologies. Such removal of policies and reimbursement barriers was critical in making telehealth more available soon after the start of the pandemic, helping to ensure care delivery across specialties for those who were not infected with the COVID-19 virus while minimizing exposure and infection., As the pandemic ensues, telehealth remains an important part of health care delivery. Currently, each state dictates which telehealth services are reimbursed and how. As of May 2020, all states cover primary care services through Medicaid and 31 states cover maternity services. As of September 2021, 18 states allow reimbursement for telephone services. CMS has also included behavioral health telehealth services to be covered under Medicare through 2023.

Given the increase in telehealth utilization for care delivery during the COVID-19 pandemic and onwards, it is important to understand the patient experience so that telehealth delivery can be optimized in a patient-centered way that aligns with the needs and preferences of patients. Toward this goal, we used data from satisfaction surveys administered to patients who participated in telehealth visits in a large US health system before and during the COVID-19 pandemic to examine: (1) the uptake of telehealth visits by provider specialty and (2) patient satisfaction with telehealth care. We additionally studied potential indicators of patient-rated quality of telehealth care and interest in future telehealth visits, including technical barriers, patient socio-economic status, and provider familiarity with telehealth. Understanding the relationship between these factors and how patients perceive the quality of telehealth care will be key for improving the patient telehealth experience.

MATERIALS AND METHODS

Study design and population

This cross-sectional observational study was conducted within a large academic healthcare system which had an established telehealth program prior to the COVID-19 pandemic. Through this telehealth program, video visits were possible using MyChart, the patient portal of the Epic (Epic Systems, Madison, WI) electronic health record (EHR) system, which allows patients to connect with physicians across various specialties throughout the entire healthcare system.

Data were obtained from the hospital EHR from October 18, 2019 through April 30, 2020, using convenience sampling. As part of a quality improvement initiative by the health system, an automatic mailing process through vendor Rx Health sent out a text message invite with a link to a purposely designed 7-item survey to all patients who participated in a video telehealth visit. The link was sent out the same day as the visit and patients could click on the link and submit their answers electronically through their own device (smartphone, tablet, or computer). No exclusion criteria applied. As surveys were anonymously completed and no personal health information or personally identifiable information was collected, this research study received IRB exemption from Yale New Haven Health System.

Measures

A 7-item patient satisfaction survey was administered. All items were answered on a 4-point Likert scale ranging from 1 = strongly disagree to 4 = strongly agree. Items concerned how easy it was to use the MyChart patient portal, the quality of the video and audio, the quality of telehealth care compared with an in-person visit, satisfaction with the instructions on telehealth received the day before and 15 min before the visit, interest in future telehealth visits, and likelihood of choosing a provider who did versus did not offer video visits. In addition, patients were asked how much time they saved by having a telehealth versus in-person visit (< 1 h; 1–2 h; 2–4 h; and > 4 h). Survey items can be found in Supplementary Table S1.

Other variables included in the analyses were provider (denoted by a unique anonymous identifier), clinic location (47 towns in Connecticut and 4 in New York), provider specialty type, and date of visit. Income level for the location in which care was provided was derived from the 2019 US Census Bureau data, which was used as a proxy for socio-economic status of the service area. Visit counts per provider served as a proxy for provider familiarity with telehealth. Finally, following visual inspection of the telehealth visit distribution over time (Supplementary Figure S1), a “COVID-19 Surge” variable was created to depict the surge in telehealth utilization caused by the COVID-19 pandemic. The start of this COVID-19 Surge was set to March 22, 2020, marking the beginning of the first full week of expanded video visits at the health system.

Statistical analyses

All variables were categorical and were summarized for the overall sample as frequencies and percentages. Income level data (low, medium, and high) and provider telehealth visit counts data (low, medium, and high) were analyzed as tertiles. The distribution of volume of responders to telehealth surveys by month was plotted. A breakdown of volume before and after the COVID-19 surge was summarized by provider specialty type.

Correlates of the pre-specified outcomes: (1) patient perception of quality of care during a telehealth visit compared to a traditional in-person visit; and (2) patients’ interest in future telehealth visits were derived from 2 multivariable, hierarchical, cumulative logit models with a random effect for provider specialty using generalized linear latent and mixed modeling. This methodology allowed us to provide a single odds ratio of cumulative probabilities for the association between 1 predictor and a summarized odds for each combination of preferences for our outcome categories (eg, strongly agree vs agree vs disagree vs strongly disagree). The independent variables in these analyses were ease of use through MyChart, video and audio quality, instructions before the visit, income level, before versus after the COVID-19 surge, provider visit count, and time saved. For the model assessing interest in future telehealth visits, the quality of care variable was additionally added. Sensitivity analyses were done with random effects for provider identification (each provider was assigned a unique ID number) and patient identification (each IP address was assigned a unique ID number).

To mitigate detected multicollinearity between the telehealth survey variables (see Supplementary Tables S2 and S3 for correlations and Variance Inflation Factor values, respectively), 2 approaches were applied. In the first approach, we used median centering to standardize the survey variables. In the second approach, we retained the variable for instructions before the visit as this was deemed the most actionable and eliminated the variables for MyChart and video and audio quality from the models.

Analyses were performed as complete case analysis and using STATA version 13 (StataCorp LP, College Station, TX, USA). P values less than or equal to .05 were considered statistically significant.

RESULTS

Using convenience sampling, a total of 83 632 electronic invitations were sent out, and 10 500 surveys were completed, a yield that is consistent with this type of sampling. Responses from 8 research visits, 15 unidentified clinic visits, and 1547 visits with missing data were excluded for a final analytic sample of 8930 responses (see Figure 1 for an overview of our analytic cohort). Of the final sample, 695 responses were completed by individuals who filled the survey multiple times. There were 1358 unique provider identifiers from 18 different specialties. The level of missingness was minimal and ranged from 0% to 7.74% per survey item, with the variable, “likelihood of choosing a provider who offered telehealth in the future” having the highest rate of missingness.

Figure 1.

Flowchart of analytic cohort.

An overview of the frequency of survey responses for telehealth visits per month is provided in Figure 2. There were 13 responses for visits between October 2019 and February 2020, with the number of responses increasing dramatically for the months of March and April, each having 2387 and 6530 responses, respectively. Figure 3 provides an overview of the proportions of survey responses by provider specialty relative to the total number of surveys, with internal medicine, hematology–oncology, urology, family medicine, and neurology being in the top 5 before the COVID-19 surge and internal medicine, hematology–oncology, cardiology, primary care, and endocrinology in the top 5 after the COVID-19 surge. Visits with the specialties of geriatrics, pain medicine, podiatry, genetic counseling, COVID-19, and physical therapy and rehabilitation had less than 1% of the total visits and were therefore grouped together into an “other” category. The largest increases in survey response volume occurred for cardiology (P < .001) and “other” (P = .023), while the largest decreases were seen in urology (P < .001), family medicine (P = .012), neurology (P = .008), and sleep medicine (P = .041).

Figure 2.

Overview of patient survey responses to video health care visits by month. Bar heights indicate frequency of survey responses for the given time period.

Figure 3.

An overview of the percentage of responses by provider specialty type relative to the total number of surveys before and after the telehealth surge in utilization. *P < .05, **P < .01, ***P < .001.

Figure 4 provides a descriptive overview of patient satisfaction results for the sample. Overall, patients had positive experiences using technology for telehealth, with the majority agreeing or strongly agreeing that MyChart made telehealth easy (23.8% agree, 72.5% strongly agree), the video and audio quality were good (28% agree, 64.8% strongly agree), and instructions before the visit were satisfactory (25.8% agree, 71% strongly agree). Almost 90% of patients agreed or strongly agreed that the quality of care they received was good and that they were interested in future telehealth visits (89.9% and 88.9%, respectively). Almost half of the patients saved 1–2 h (46.9%), and an additional 10.4% and 2.4% saved 2–4 h, and more than 4 h, respectively, by having telehealth visits. Income tertiles were ≤$70 941, $71 368–92 969, and ≥$94 446. The estimated median household income was $76 360 with an interquartile range of $41 142–99 094. Provider visit counts between October and April by tertiles ranged from 1 to 2 visits in the low-volume group, to ≥7 visits in the high-volume group. The median number of visits with survey responses per provider was 13, with an interquartile range of 6–22 visits. The majority of visits (74.8%) were done by high-volume providers (74.8%) while the low- and medium-volume providers accounted for 8.9% and 16.4% of the visits, respectively.

Figure 4.

Descriptive overview of patient survey responses for the overall sample. (A) Patient satisfaction levels. Survey questions in above order: The MyChart App made it easy; The video visit picture and audio quality were good; My family member or I received the same quality of care during our video visit as an office visit; My family member or I was satisfied with the video visit expectation instructions from the provider or provider’s office; My family member or I am interested in using video visits for future appointments; My family member or I would be more likely to choose a provider who offered video visits. (B) Patient time saved. Survey question: How much time did you save by having a video visit (accounting for wait time, etc.)?

Results examining correlates of patient ratings for quality of care are detailed in Table 1. There was a positive correlation between satisfaction with quality of care received, satisfaction on the ease of the MyChart telehealth application, ratings for video and audio quality, ratings for instructions before the visit, and time saved. In the reduced model, satisfaction level for quality of care was positively correlated with ratings for instructions before the visit and with time saved.

Table 1.

Overview of correlates for patient-rated satisfaction with quality of care

	My family member or I received the same quality of care during our video visit as an office visit
	Including all median centered variables		Keeping only instructions
Variable	OR (95% CI)	P values	OR (95% CI)	P values
The MyChart App made it easy	1.43 (1.30–1.58)	<.001
The video visit picture and audio quality were good	1.62 (1.50–1.75)	<.001
Satisfaction with video visit expectation instructions	5.62 (5.05–6.26)	<.001	9.23 (8.45–10.09)	<.001
Time saved
<1 h	1 [Reference]		1 [Reference]
1–2 h	1.07 (0.97–1.17)	.187	1.08 (0.98–1.19)	.102
2–4 h	1.32 (1.12–1.55)	.001	1.32 (1.13–1.56)	.001
>4 h	1.69 (1.22–2.34)	.001	1.76 (1.28–2.42)	.001
Income tertiles
High (≥$94 446)	1 [Reference]		1 [Reference]
Low (≤$70 941)	1.07 (0.95–1.20)	.271	1.07 (0.96–1.20)	.226
Medium ($71 368–92 969)	0.93 (0.83–1.04)	.215	0.95 (0.84–1.06)	.352
Provider volume tertiles
Low (1–2 visits)	1 [Reference]		1 [Reference]
Medium (3–6 visits)	0.88 (0.73–0.106)	.188	0.91 (0.75–1.10)	.315
High (≥7 visits)	0.94 (0.79–1.12)	.499	0.98 (0.83–1.17)	.853
COVID-19 surge
Before March 22, 2020	1 [Reference]		1 [Reference]
On/after March 22, 2020	0.96 (0.79–1.16)	.666	0.97 (0.80–1.18)	.79

Notes: Results of the hierarchical, multivariable cumulative logit model are presented as cumulative ORs and 95% CIs. The model including median-centered variables is presented as well as a reduced model, keeping satisfaction with the instructions before the visit only.

OR: Odds Ratio; CI: confidence intervals.

In analyses concerning factors associated with patients’ interest in future telehealth visits (Table 2), the following factors were identified: ease of the MyChart patient portal, video and audio quality, satisfaction with quality of care received, ratings for instructions before the visit, and time saved were associated with a higher interest in future telehealth visit use. Higher provider visit count and being seen after the COVID-19 surge were associated with a lower interest in future telehealth visits. For the reduced model, leaving in the variables for instructions before the visit, time saved, income tertiles, and provider visit count replicated our initial results (Table 2).

Table 2.

Overview of correlates for patient-rated likelihood of future use of video telehealth visit

	My family member or I am interested in using video visits for future appointments
	Including all median-centered variables		Keeping only instructions
Variable	OR (95% CI)	P values	OR (95% CI)	P values
The MyChart App made it easy	1.56 (1.41–1.73)	<.001
The video visit picture and audio quality were good	1.26 (1.16–1.37)	<.001
Satisfied with video visit expectation instructions	1.80 (1.62–2.01)	<.001	2.58 (2.35–2.83)	<.001
Time saved
<1 h	1 [Reference]		1 [Reference]
1–2 h	1.51 (1.38–1.66)	<.001	1.53 (1.39–1.68)	<.001
2–4 h	2.54 (2.15–3.00)	<.001	2.53 (2.14–2.99)	<.001
>4 h	3.49 (2.47–4.93)	<.001	3.50 (2.48–4.94)	<.001
Income tertiles
High (≥$94 446)	1 [Reference]		1 [Reference]
Low (≤$70 941)	1.00 (0.89–1.12)	.986	1.00 (0.90–1.12)	.226
Medium ($71 368–92 969)	0.94 (0.84–1.05)	.273	0.95 (0.85–1.07)	.352
Provider visit volume tertiles
Low (1–2 visits)	1 [Reference]		1 [Reference]
Medium (3–6 visits)	0.78 (0.64–0.94)	.01	0.80 (0.66–0.96)	.018
High (≥7 visits)	0.71 (0.60–0.85)	<.001	0.73 (0.62–0.87)	<.001
COVID-19 surge
Before March 22, 2020	1 [Reference]		1 [Reference]
On/after March 22, 2020	0.76 (0.63–93)	.007	0.77 (0.64–0.94)	.01
Quality of care during video visit compared to an office visit	4.19 (3.87–4.54)	<.001	4.51 (4.16–4.88)	<.001

Notes: Results of the hierarchical, multivariable cumulative logit model are presented as cumulative ORs and 95% CIs. The model including median-centered variables is presented as well as a reduced model, keeping satisfaction with instructions before the visit only

OR: odds ratio; CI: confidence intervals.

DISCUSSION

This study provides a unique view of the patient experience with telehealth care before and during the COVID-19 pandemic. In a large US healthcare system, the majority of patients who responded to a telehealth visit satisfaction survey expressed relatively high satisfaction with the quality of care received relative to a traditional in-person visit. Satisfaction with the quality of the technology, instructions before the visit, and time saved were important correlates of both patient perceived quality of care and interest in continuing telehealth care. In contrast, high provider visit count and being seen after the COVID-19 surge were associated with decreased interest in engaging with future telehealth visits.

Prior studies have assessed patient satisfaction with telehealth visits for the evaluation of individual care programs before the COVID-19 era. More recent literature describing the role of telehealth during the pandemic has focused on infrastructure, technology, and integration. The current study brings new insights regarding the patient perspectives and factors that may interact with patient preferences for telehealth care as the COVID-19 pandemic unfolded. These insights could be instrumental for designing sustainable telehealth care pathways in a manner that enhances the patient experience beyond the current public health crisis. In particular, our findings of the high degree of patient satisfaction with telehealth can help guide policies supporting telehealth care, such as that the policies made by the CMS during the pandemic remain unchanged. Our study also gives insight into quality standards for telehealth visits, such as tailoring telehealth for populations that may benefit the most (eg, time savings), improving video quality and internet coverage, and developing ways to make telehealth more patient-centered rather than focusing on increasing volume of visits. In addition, our study demonstrates the feasibility of collecting these quality metrics at the health system level.

The pandemic led to dramatic changes in the volume of telehealth visits over time. The number of survey responses related to telehealth visits in March 2020 was 1000 times, and those in April 2020, 3000 times greater than the number of monthly visits between October 2019 and February 2020. In addition to the need to provide care despite a complete shutdown of ambulatory care and the need to continue creating revenue, the increase in telehealth was also partially due to specialties adopting telehealth that had not traditionally done so prior to COVID-19. Despite the rapid adaptation to telehealth, we found that patient satisfaction with telehealth was very high, as previously reported with another health system. However, it is important to note that being seen after the COVID-19 surge was associated with a relatively lower interest in future telehealth visits than those being seen before the surge. It is possible that these results are due to self-selection bias, as most respondents after the surge in telehealth may have been more reluctant to using telehealth but had no other option at the time. Future research should aim to understand the reasons behind this decrease in interest in telehealth. For example, future work should explore whether this is potentially reflecting a desire to return to “normal” pre-COVID-19 times, including more in-person visits. Furthermore, understanding the factors that contribute to this lower interest could inform the design of more personalized telehealth care.

Patient-level factors linked with satisfaction and interest in future telehealth visits were consistent with previous studies. Patients have reported convenience, such as saving time and not having to miss work, as an important factor in liking telehealth., In the current study, patients who reported saving more than 2 h with telehealth were more likely to endorse satisfaction with their quality of care. Ratings for the quality of the video technology, the ease of using the technology, and instructions before the visit were also highly positively correlated with satisfaction level and interest in future telehealth visits. As these are all potentially modifiable factors, further investments in patient-centered instruction and navigation materials to enhance telehealth experiences are needed.,,,,, Previous strategies to address these barriers have included “technological liaisons” to help clinical teams set up software and troubleshoot, medical students setting up telehealth software for patients, and online self-guided tutorial videos to deliver telehealth instructions.

For provider-level factors, we noted that patients seen by providers with higher volumes were less likely to be interested in future telehealth visits. As provider volumes of telehealth visits increased, patient interest in future telehealth visits decreased over time. This suggests that increased provider familiarity with telehealth may not necessarily translate into higher perceived quality of the visits, and the role of time constraints and time spent with the patient should be explored. In addition, it is possible that patients seen by these providers may have had more complex medical problems that required multiple appointments, leading to lower satisfaction by patients as their issues were not resolved in a single visit.

The following factors are important to consider. The sample was a convenience sample and only represented a fraction of the total number of patients seen by telehealth and of the overall volume of visits conducted. As such, we do not know how the current findings would generalize to the larger patient group that received care by telehealth. We also acknowledge the possibility of small effect sizes in our findings and future studies should identify other factors that may impact satisfaction related to future telehealth use. Other limitations to this study included a lack of detailed patient profile information, such as patient demographics, due to the anonymous nature of the study. We were also unable to measure factors regarding important elements of the patient experience, including the quality of the therapeutic relationship during telehealth care visits, length of visit, and patient wait time in virtual waiting rooms. The survey was also restricted to patients seen by video telehealth visits; those who were seen by audio visits alone—eg, due to lack of video capability—were not included. This touches upon the wider issue of access and representation, and as the delivery of care using telehealth grows, gaps in access and representation should be a top priority., Initiatives that have increased access to telehealth have included provision of video-enabled tablets to veterans for telehealth visits and development of self-monitoring and self-management digital tools for patients with limited access to mental health care. In addition, the patient population in this study utilized MyChart for the telehealth visits. As such, it is difficult to assess whether their responses were related to telehealth in general or to the specific telehealth platform (MyChart). Furthermore, while socio-economic status was not a significant finding in this study, it is important to note that this measure had limitations (ie, the use of clinic locations rather than patient home address to assess household income; a relatively high median income). Future work should focus on evaluating how socio-economic status is exactly related to telehealth care use and related satisfaction levels. Future studies should also consider soliciting open-ended feedback from patients to evaluate previously unconsidered factors that patients deem as important to their use and evaluation of the telehealth experience. These studies should also seek input from patients with visits in languages other than English to help improve services for a diverse community. Finally, the study period only included responses from 2 months since the start of the pandemic. As these responses may not reflect patient sentiments in subsequent months of the pandemic, it would be beneficial to obtain new data now that telehealth care has been available for over 2 years.

CONCLUSION

The COVID-19 pandemic required patients and providers to rapidly adopt and adapt to telehealth care delivery. While this transition was forced out of necessity to protect patients and slow rates of transmission, our study underscores that a large patient volume is highly satisfied with this method of healthcare delivery and that rapid expansion is possible. As telehealth will remain an important mode of care delivery while the COVID-19 pandemic is ongoing, and will likely play a part in overall care delivery beyond, it will be crucial to understand factors that contribute to the patient experience and invest in a more patient-centered and more inclusive delivery model of telehealth. Several modifiable factors that contribute to patient satisfaction were identified and these may represent targets for testing in new models of telehealth care delivery.

FUNDING

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

AUTHOR CONTRIBUTIONS

All authors contributed to conception and design of this study, critical revisions of the manuscript, approved the final manuscript, and agreed to be accountable for all aspects of this work. PL and ML performed analysis and interpretation of the data.

SUPPLEMENTARY MATERIAL

Supplementary material is available at JAMIA Open online.

CONFLICT OF INTEREST STATEMENT

Dr Kim G. Smolderen reports grant support from Cardiva, and she is a consultant for Optum Labs LLC. Dr Carlos Mena-Hurtado serves as a consultant for Abbott, Boston Scientific, COOK, and Medtronic. None of the other authors have any conflicts of interest to report.

Click here for additional data file.