Drivers of variation in telemedicine use during the COVID-19 pandemic: The experience of a large academic cardiovascular practice.

BACKGROUND: COVID-19 spurred rapid adoption and expansion of telemedicine. We investigated the factors driving visit modality (telemedicine vs. in-person) for outpatient visits at a large cardiovascular center.
METHODS: We used electronic health record data from March 2020 to February 2021 from four cardiology subspecialties (general cardiology, electrophysiology, heart failure, and interventional cardiology) at a large academic health system in Northern California. There were 21,912 new and return visits with 69% delivered by telemedicine. We used hierarchical logistic regression and cross-validation methods to estimate the variation in visit modality explained by patient, clinician, and visit factors as measured by the mean area under the curve.
RESULTS: Across all subspecialties, the clinician seen was the strongest predictor of telemedicine usage, while primary visit diagnosis was the next most predictive. In general cardiology, the model based on clinician seen had a mean area under the curve of 0.83, the model based on the primary diagnosis had a mean area under the curve of 0.69, and the model based on all patient characteristics combined had a mean area under the curve of 0.56. There was significant variation in telemedicine use across clinicians within each subspecialty, even for visits with the same primary visit diagnosis.
CONCLUSION: Individual clinician practice patterns had the largest influence on visit modality across subspecialties in a large cardiovascular medicine practice, while primary diagnosis was less predictive, and patient characteristics even less so. Cardiovascular clinics should reduce variability in visit modality selection through standardized processes that integrate clinical factors and patient preference.

Introduction

The COVID-19 pandemic accelerated the adoption of telemedicine, resulting in the rapid deployment of virtual care delivery models in healthcare systems across the country.[1-4] This was supported by the Centers for Medicare and Medicaid Service (CMS) and subsequently private payers, who have offered payment parity across visit modalities (in-person, video, and telephone). By April 2020, nearly half (43.5%) of Medicare primary care visits were provided through telemedicine compared with 0.1% in February 2020.[5]

Cardiovascular disease is the leading cause of death globally, and it is therefore critical to understand the role of telemedicine in cardiovascular care delivery.[6] Previous studies have examined patterns of telemedicine use in cardiovascular practices during the pandemic, indicating persistent use of telemedicine with varying utilization patterns across subspecialties.[7] As telemedicine was adopted throughout the pandemic, clinics had to determine which visits should be conducted over telemedicine and which should be in-person. Visit modality could theoretically be driven by patient factors, clinician factors, visit factors, or some combination of all three. For example, certain patients may require in-person visits because they do not have access to the technology required for telemedicine. Other patients may require care that is not well-suited for a telemedicine visit, particularly when an in-person physical exam or test is required. Finally, clinicians and health systems may have preferences for specific visit modalities.

A number of studies have demonstrated disparities in access to telemedicine based on sociodemographic characteristics.[8-12] Prior clinician experience with telemedicine is associated with a twofold increase in confidence treating acute conditions via telemedicine.[13] Characteristics of clinic specialties may further contribute to the variation in telemedicine utilization. For example, telemedicine adoption at one integrated health system varied from 3.2% (dermatology) to 98.3% (psychiatry) of visits.[14] While the effects of various factors have been explored independently, few attempts have been made to compare their effects simultaneously within a specialty. One cross-sectional study across 1652 primary care and specialty care practices of adult patients seen at an integrated health system during the first few months of the pandemic found that practices and clinicians drove more of the variation in phone versus video usage than patient characteristics.[15] However, they did not explore how these factors drive the variation in in-person versus telemedicine usage.

In our study, we sought to understand the contribution of patient, clinician, and visit factors in explaining variation in in-person versus telemedicine use across cardiology subspecialties at Stanford Health Care (SHC)'s outpatient cardiology clinics during the first year of the COVID-19 pandemic.

Methods

Setting

SHC is a large academic health system located in the California Bay Area. Prior to the COVID-19 pandemic, most SHC cardiovascular outpatient visits were conducted in-person. In 2017, SHC implemented CardioClick, a telemedicine program providing video follow-up visits in one preventive cardiology clinic.[16] In 4 March 2020 the governor of California declared a state of emergency in light of rising COVID-19 cases and the first death in the state attributed to the disease. In the following weeks, SHC rapidly deployed telephone and video visits to allow for the continued delivery of outpatient services. To facilitate rapid scaling of telemedicine, mandatory training was created to convey the key principles of conducting successful video visits. Clinicians could complete video visits from clinic workstations or from secure SHC-provided laptop computers. Patient access to video visits was enabled through the SHC MyHealth mobile application, which allowed visits to be completed from a personal smartphone, tablet, or computer workstation.

Data source

We performed a retrospective cohort study of consecutive outpatient cardiology visits at SHC from 1 March 2020 to 28 February 2021. The data obtained from the electronic health record system included patient demographics, clinician information, and visit characteristics, including primary visit diagnosis. This study was deemed exempt from review and informed consent as a quality improvement initiative by the Stanford Institutional Review Board. The dataset analyzed for this study includes protected health information that cannot be shared publicly, so supporting data are not available.

The primary outcome was the visit modality (in-person vs. telemedicine). We considered 14 different predictor variables, including 8 patient, 3 clinician, and 3 visit factors. The eight patient characteristics included sex, age, self-reported race and ethnicity, insurance type, preferred language, whether an interpreter was needed, and the natural logarithm of distance between the patient's home ZIP code and the clinic ZIP code (logarithmic scale used to normalize the data). The clinician factors included the clinician seen, the clinic site, and the clinician type (physician vs. advanced practice provider). Finally, we included the primary diagnosis associated with the visit, whether it was a new or return patient visit, and the intended duration of the appointment (treated as a categorical variable with discrete lengths of time) as the 3 visit factors. We created separate datasets and models for the following specialties: general cardiology, electrophysiology, heart failure, and interventional cardiology. We excluded data points for a given categorical variable that appeared less than 10 times within a subspecialty dataset.

Statistical analysis

We used hierarchical univariable and multivariable logistic regression models to assess the relative explanatory power of factors associated with telemedicine use. We ran multivariable models for the subsets of clinician factors (clinician seen, clinic site, clinician type), visit factors (primary visit diagnosis, new vs. return visit, intended visit duration), patient factors (sex, age, race, ethnicity, insurance, interpreter needed, and distance), and all 14 factors combined. We tested univariable models for each of the 14 factors independently as well. For each model, we ran fivefold cross-validation to estimate the mean area under the curve (AUC) value based on the held-out test sets. A higher mean AUC (closer to 1) suggests that the model's variables were more predictive of visit modality. By comparing the results of models with different predictor variables included, we can infer which variables explained the greatest proportion of variation in visit modality.

Additionally, for each primary visit diagnosis within a subspecialty for which there were at least five distinct clinicians who each saw a minimum of five new or return patients, we calculated the fraction of visits that were conducted via telemedicine for each clinician. This data was used to visualize the variation in clinician telemedicine use across common diagnoses within each subspecialty. Data cleaning and modeling were conducted in R (version 4.1.2) and data visualization was completed using Tableau Desktop (version 2021.4.2).

Results

The study included 21,912 new and return patient visits between 1 March 2020 and 28 February 2021 with 69.0% delivered by telemedicine. The volume of visits and the distribution of modalities varied significantly across subspecialties: general cardiology (9813 visits, 71.7% telemedicine), electrophysiology (5236 visits, 94.0% telemedicine), heart failure (4540 visits, 46.4% telemedicine), and interventional cardiology (2323 visits, 45.4% telemedicine). Across subspecialties, patients’ average age ranged from 60.3 to 69.8 years and the fraction of female patients ranged from 36% to 47%. Around 10% of patients were Hispanic/Latino, 13% to 22% were Asian, and 3% to 7% were Black. The breakdown of patients by insurance type varied significantly across the subspecialties (Table 1).

Table 1.

Characteristics of cardiology clinic visits during the first year of the COVID-19 pandemic.

Visit characteristicsa	General cardiology	Electrophysiology	Heart failure	Interventional cardiology
Modality
Total visits	9813 (100%)	5236 (100%)	4540 (100%)	2323 (100%)
In-person	2778 (28.3%)	316 (6.0%)	2434 (53.6%)	1268 (54.6%)
Telemedicine	7035 (71.7%)	4920 (94.0%)	2106 (46.4%)	1055 (45.4%)
New versus return
New	2517 (25.6%)	1163 (22.2%)	462 (10.2%)	444 (19.1%)
Return	7296 (74.4%)	4073 (77.8%)	4078 (89.8%)	1879 (80.9%)
Primary visit diagnoses
Distinct count	113	55	68	33
Clinician factors
Distinct clinicians	39	15	18	10
Physicians	34	8	15	6
Adv. practice providers	5	7	3	4
Distinct clinic sites	6	3	2	4
Intended visit duration
30 min	7075 (72.1%)	1721 (32.9%)	3560 (78.4%)	1453 (62.6%)
40 min	0 (0%)	2268 (43.3%)	14 (0.3%)	82 (3.5%)
60 min	2603 (26.5%)	1153 (22.0%)	948 (20.9%)	767 (33.0%)
Other (20, 40, 45, 90, 120)	135 (1.4%)	94 (1.8%)	18 (0.4%)	21 (0.9%)
Age
Mean (Std. Dev.)	60.33 (16.9)	64.44 (16.1)	63.9 (16.1)	69.77 (13.9)
Sex
Female	4629 (47.2%)	2135 (40.8%)	1665 (36.7%)	965 (41.5%)
Male	5184 (52.8%)	3101 (59.2%)	2875 (63.3%)	1358 (58.5%)
Race
Asian	2170 (22.1%)	881 (16.8%)	603 (13.3%)	410 (17.6%)
Black	357 (3.6%)	153 (2.9%)	308 (6.8%)	57 (2.5%)
White	4698 (47.9%)	3090 (59%)	2510 (55.3%)	1355 (58.3%)
Other	2588 (26.4%)	1112 (21.2%)	1119 (24.6%)	501 (21.6%)
Ethnicity
Hispanic/Latino	987 (10.1%)	536 (10.2%)	558 (12.3%)	208 (9%)
Non-Hispanic/Non-Latino	8088 (82.4%)	4478 (85.5%)	3784 (83.3%)	2030 (87.4%)
Preferred language
English	8533 (87%)	4608 (88%)	4058 (89.4%)	2087 (89.8%)
Non-English	1280 (13%)	628 (12%)	482 (10.6%)	236 (10.2%)
Interpreter needed
No	8765 (89.3%)	4721 (90.2%)	4125 (90.9%)	2121 (91.3%)
Yes	1003 (10.2%)	49 (0.9%)	415 (9.1%)	202 (8.7%)
Insurance
Private/Other	2129 (21.7%)	1103 (11.2%)	861 (8.8%)	373 (3.8%)
Managed care	4155 (42.3%)	1731 (17.6%)	1452 (14.8%)	637 (6.5%)
Medicaid	76 (0.8%)	51 (1%)	81 (1.8%)	0 (0%)
Medicare	3316 (33.8%)	2314 (44.2%)	2087 (46%)	1292 (55.6%)
Distance to clinic (miles)
Median	17.1	22.5	23.4	22

We excluded data points for a given categorical variable that appeared less than 10 times within a subspecialty dataset. Not all values within the categorical variables are presented. Demographic data represents visits, not the patient populations. Distance is the distance from the patient ZIP code to the clinic ZIP code.

Within each subspecialty, the logistic regression models based on clinician seen had the highest mean test AUC scores among the univariable models, followed by those based on primary visit diagnosis (Table 2). The univariable models based on patient factors and other clinician and visit factors explained comparatively little of the variation in visit modality. At the multivariable level, the models that included all 14 predictor variables had the highest mean test AUC scores for each subspecialty, ranging from 0.854 for general cardiology to 0.726 for electrophysiology (Table 3). Clinician seen and primary visit diagnosis were more predictive of visit modality than all eight of the patient factors combined. For example, in general cardiology, the model based on the clinician seen had a mean AUC of 0.83, the model based on the primary diagnosis had an AUC of 0.69, and the model based on all patient characteristics combined had an AUC of 0.56. The relatively small differences between the mean AUCs of the 14 variable models and the models based solely on the clinician seen suggest that individual clinician practice patterns are likely the main driver of visit modality selection.

Table 2.

Relative explanatory power of univariable models associated with telemedicine use within cardiology subspecialties (mean test AUC of logistic regression models from cross validation).

Model predictor variablea	General cardiology (N = 9813 visits)	Electrophysiology (N = 5236 visits)	Heart failure (N = 4540 visits)	Interventional cardiology(N = 2323 visits)
Univariable clinician factor models
Clinician seen	0.832	0.684	0.679	0.808
Clinic site	0.676	0.518	0.519	0.724
Clinician type	0.519	0.542	0.539	0.487
Univariable visit factor models
Primary visit diagnosis	0.689	0.632	0.637	0.763
New versus return visit	0.525	0.519	0.521	0.520
Intended visit duration	0.551	0.535	0.542	0.652
Univariable patient factors models
Sex	0.517	0.525	0.522	0.566
Interpreter needed	0.520	0.514	0.514	0.519
Preferred language	0.532	0.519	0.524	0.519
Race	0.511	0.512	0.517	0.524
Insurance	0.540	0.525	0.530	0.591
Ethnicity	0.513	0.522	0.522	0.494
Age	0.535	0.537	0.533	0.630
Distance to clinic	0.507	0.550	0.551	0.508

The dependent variable was the visit modality (in-person vs. telemedicine, including video and phone visits). Cross-validation included five folds. Distance to clinic is the natural logarithm of the miles between the patient ZIP code and clinic ZIP code.

AUC: area under the curve.

Table 3.

Relative explanatory power of multivariable models associated with telemedicine use within cardiology subspecialties (mean test AUC of logistic regression models from cross-validation).

Multivariable modela	General cardiology (N = 9813 visits)	Electrophysiology (N = 5236 visits)	Heart failure (N = 4540 visits)	Interventional cardiology(N = 2323 visits)
All clinician factors	0.842	0.684	0.679	0.809
All visit factors	0.715	0.642	0.650	0.775
All patient factors	0.561	0.573	0.577	0.642
All factors	0.854	0.726	0.727	0.844

The dependent variable was the visit modality (in-person vs. telemedicine, including video and phone visits). Cross-validation included five folds. Clinician factors include clinician seen, clinic site, clinician type. Visit factors include primary visit diagnosis, new versus return visit, intended visit duration. Patient factors include sex, age, race, ethnicity, insurance, interpreter needed, and distance. All factors include the 14 clinician, visit, and patient factors listed previously.

AUC: area under the curve.

There was significant variation in telemedicine use across clinicians within each subspecialty, even among visits with the same primary diagnosis (Figure 1). For example, among general cardiologists seeing patients with the primary diagnosis of palpitations, five clinicians used telemedicine for less than 50% of their visits, while 20 clinicians used telemedicine for a majority of their visits. This included two clinicians who used 100% telemedicine and one who used no telemedicine. Figure 1 visualizes the variation in telemedicine use for one select diagnosis for each subspecialty. For more complete results for all diagnoses, see Appendices A and B.

Figure 1.

Variation in telemedicine use across clinicians in the same cardiology subspecialty for visits with the same primary diagnosis. Results are limited to clinicians with a minimum of five new/return patient visits.

Discussion

In this study, we sought to understand the influence of various factors on telemedicine use at a large academic cardiovascular center during the first year of the COVID-19 pandemic. The individual clinician seen was the best predictor of visit modality, followed by primary visit diagnosis. Patient characteristics and other visit factors explained only a small proportion of the variation in visit modality. This is consistent with previous research, which has demonstrated that clinician preference plays a more significant role than patient preference in determining visit modality.[15] Additionally, we found significant variation in telemedicine use across clinicians within subspecialties, even among patients evaluated for the same primary visit diagnosis, which supports the predominant role of clinician preference in determining visit modality. To our knowledge, this study represents the first evaluation of drivers of visit modality selection and variation in telemedicine use across clinicians within cardiology subspecialties.

The benefits of telemedicine are compelling. Telehealth programs have the potential to improve patient outcomes and satisfaction, enable operational efficiency, reduce the brick and mortar costs of in-person care, and expand the reach of specialized care.[16-20] Furthermore, the flexibility to offer services remotely may provide a better work-life balance for clinicians. It may also decrease clinician burnout, as telemedicine visits tend to be shorter in duration than in-person visits, giving clinicians more time to complete their visit notes.[21]

Allowing clinician preference to dictate visit modality, however, can compromise the quality of care, especially when in-person visits are clinically indicated but their availability is limited. For example, a telemedicine visit may be insufficient for a patient with symptoms of exacerbated heart failure, potentially delaying clinical recognition and management. On the other hand, a follow-up visit for dyslipidemia may be well suited for telemedicine. Patients who are not offered this option due to clinician preference alone would incur unnecessary costs to attend an in-person visit which may affect patient engagement with clinical care. For example, telemedicine visits were associated with greater program completion and engagement when compared to in-person visits.[16] Moreover, implicit clinician bias can contribute to inequitable virtual care access among historically marginalized groups, further widening health disparities.

Additionally, variation in health care utilization is an established driver of healthcare costs. In clinically appropriate situations where telemedicine would be adequate, a clinician's preference to choose an in-person visit may affect healthcare costs. Importantly, variation in practice patterns also limits the ability of health systems to predict health care resource needs, such as staffing and exam room allocation, which can result in operational inefficiencies.

Telemedicine use for outpatient visits will likely continue beyond the COVID-19 pandemic. As Thomas et al highlight, the key requirements for its long-term sustainability include developing a skilled workforce, empowering consumers, reforming funding, improving the digital ecosystems, and integrating telehealth into routine care.[22] Variation in telemedicine use attributable to clinician preference is complex and may be due to cultural beliefs, clinician training, clinician behaviors, or individualized care based on unmeasured patient factors.[23-24] Further evidence on the clinical and operational outcomes of telemedicine is needed to identify situations where telemedicine can be used effectively and potentially protocolized.

Limitations

The results of this study should be interpreted in the context of some limitations. Our study demonstrated significant variation in clinician use of telemedicine; however, the factors that drove these variations are unclear to the extent that some of these variations may be a result of unobservable differences between clinicians’ patient panels. The single-center, observational design of this study limits the generalizability of this study, and it is unclear if the same factors would hold the same predictive values prospectively. Furthermore, this study was conducted at a single academic center, and the findings may not be generalizable to other settings. We did not have access to individual clinician characteristics or patient preferences on visit modality, which limited our ability to investigate the contributions from these factors to visit modality selection.

Conclusions

At this large academic cardiovascular center, there was significant variation in telemedicine use among clinicians with the same diagnosis. Individual clinician practice patterns were more predictive of visit modality selection than clinical visit factors or patient characteristics. Future research should investigate the drivers of variation in telemedicine use in other settings and explore differences in clinical and operational outcomes between telemedicine and in-person visits.

Appendix B.

Table of primary visit diagnoses corresponding to Appendix A.

Subspecialty	Appendix A row number	Primary visit diagnosis	Clinician Telemedicine Fraction
			Mean	Standard Deviation	Clinicians
General cardiology	1	Aortic valve stenosis, etiology of cardiac valve disease unspecified	0.392	0.464	6
	2	Atrial fibrillation, unspecified type (CMS-HCC)	0.325	0.629	20
	3	Atypical chest pain	0.366	0.687	5
	4	Bicuspid aortic valve	0.313	0.374	5
	5	Bradycardia	0.411	0.633	5
	6	Cardiomyopathy, unspecified type (CMS-HCC)	0.169	0.752	7
	7	Chest pain, unspecified type	0.271	0.698	25
	8	Coronary artery disease involving native coronary artery of native heart without angina pectoris	0.245	0.718	31
	9	Dilated aortic root (CMS-HCC)	0.447	0.629	6
	10	Dyslipidemia	0.247	0.764	30
	11	Dyspnea, unspecified type	0.261	0.643	8
	12	Essential hypertension	0.235	0.719	32
	13	Hypercholesterolemia	0.113	0.903	10
	14	Mitral valve insufficiency, unspecified etiology	0.368	0.586	9
	15	Mixed hyperlipidemia	0.245	0.827	9
	16	Palpitations	0.272	0.697	25
	17	Paroxysmal atrial fibrillation (CMS-HCC)	0.210	0.721	14
	18	premature ventricular contraction (PVC)	0.213	0.804	9
	19	S/P TAVR (transcatheter aortic valve replacement)	0.384	0.482	5
	20	supraventricular tachycardia (SVT) (CMS-HCC)	0.235	0.703	8
	21	Syncope, unspecified syncope type	0.241	0.776	9
	22	Tachycardia	0.328	0.661	7
	23	Thoracic aortic aneurysm without rupture (CMS-HCC)	0.293	0.267	6
Electrophysiology	24	Atrial fibrillation, unspecified type (CMS-HCC)	0.100	0.929	13
	25	Atrial flutter, unspecified type (CMS-HCC)	0.172	0.920	6
	26	Bradycardia	0.089	0.960	5
	27	Cardiac pacemaker in situ	0.039	0.964	12
	28	Fitting and adjustment of cardiac pacemaker	0.398	0.580	6
	29	implantable cardioverter defibrillator (ICD) in place	0.029	0.984	8
	30	Nonischemic cardiomyopathy (CMS-HCC)	0.091	0.938	5
	31	Palpitations	0.103	0.924	11
	32	Paroxysmal atrial fibrillation (CMS-HCC)	0.037	0.965	12
	33	Persistent atrial fibrillation (CMS-HCC)	0.031	0.981	9
	34	premature ventricular contraction (PVC)	0.047	0.967	12
	35	supraventricular tachycardia (SVT) (CMS-HCC)	0.069	0.946	13
	36	Syncope, unspecified syncope type	0.070	0.957	11
	37	Typical atrial flutter (CMS-HCC)	0.095	0.937	8
	38	ventricular tachycardia (VT) (CMS-HCC)	0.041	0.968	13
Heart failure	40	Acute on chronic systolic congestive heart failure (CMS-HCC)	0.264	0.348	6
	41	Atrial fibrillation, unspecified type (CMS-HCC)	0.266	0.471	6
	42	Cardiomyopathy, unspecified type (CMS-HCC)	0.119	0.516	9
	43	Chronic diastolic heart failure (CMS-HCC)	0.136	0.401	7
	44	Chronic systolic heart failure (CMS-HCC)	0.212	0.474	10
	45	Coronary artery disease involving native coronary artery of native heart without angina pectoris	0.235	0.428	10
	46	Dilated cardiomyopathy (CMS-HCC)	0.215	0.533	8
	47	Dyslipidemia	0.254	0.644	5
	48	Essential hypertension	0.278	0.473	12
	49	Heart failure, unspecified HF chronicity, unspecified heart failure type (CMS-HCC)	0.219	0.330	5
	50	Ischemic cardiomyopathy	0.118	0.361	6
	51	Nonischemic cardiomyopathy (CMS-HCC)	0.170	0.490	9
	52	Paroxysmal atrial fibrillation (CMS-HCC)	0.263	0.380	5
Interventional cardiology	53	Aortic valve disorder	0.147	0.191	5
	54	Chest pain, unspecified type	0.261	0.664	5
	55	Coronary artery disease involving native coronary artery of native heart without angina pectoris	0.213	0.632	8
	56	Essential hypertension	0.306	0.524	5

Results are limited to diagnoses with at least five clinicians who had a minimum of five new/return patient visits.