Satisfaction with modes of telemedicine delivery during COVID-19: A randomized, single-blind, parallel group, noninferiority trial.

BACKGROUND: Little is known about satisfaction with different modes of telemedicine delivery. The objective of this study was to determine whether patient satisfaction with phone-only was noninferior to video visits.
METHODS: We conducted a parallel group, randomized (1:1), single-blind, noninferiority trial in multispecialty clinics at a tertiary academic medical center. Adults age ≥ 60 years or with Medicare/Medicaid insurance were eligible. Primary outcome was visit satisfaction rate (9 or 10 on a 0-10 satisfaction scale). Noninferiority was determined if satisfaction with phone-only (intervention) versus video visits (comparator) was no worse by a -15% prespecified noninferiority margin. We performed modified intent-to-treat (mITT) and per protocol analyses, after adjusting for age and insurance.
RESULTS: 200 participants, 43% Black, 68% women completed surveys. Visit satisfaction rates were high. In the mITT analysis, phone-only visits were noninferior by an adjusted difference of 3.2% (95% CI, -7.6% to 14%). In the per protocol analysis, phone-only were noninferior by an adjusted difference of -4.1% (95% CI, -14.8% to 6.6%). The proportion of participants who indicated they preferred the same type of telemedicine visit as their next clinic visit were similar (30.2% vs 27.9% video vs phone-only, p = 0.78) and a majority said their medical concerns were addressed and would recommend a telemedicine visit.
CONCLUSIONS: Among a group of diverse, established older or underserved patients, the satisfaction rate for phone-only was noninferior to video visits. These findings could impact practice and policies governing telemedicine. Published by Elsevier Inc.

Introduction

The availability of new technologies and interest in making medical care more accessible and convenient has spurred the development of remotely-delivered care (e.g., telemedicine). The COVID-19 pandemic2, 3, 4 has led to a significant shift to home-based telemedicine in many medical specialties.5, 6, 7 Home-based telemedicine uses phone and/or videoconferencing to deliver care to patients in their own homes,2, 3, 4 , and these visits are variably reimbursed. Because avoiding travel decreases COVID-19 transmission risks, telemedicine has been recommended for socially and medically vulnerable groups,10, 11, 12 such as those residing in rural areas, the elderly, those with comorbidities, and other groups at higher risk of COVID-19 complications.14, 15, 16, 17 However, many patients lack access to video capacity, , which may create significant inequities in receiving care, particularly if the video component of telemedicine is associated with better care quality and patient satisfaction. Phone-only visits, which may be preferred by patients who do not have the required technology for video visits or who may have lower digital literacy, may mitigate health disparities. Despite their potentially higher reimbursement than phone-only visits, video visits may offer few if any tangible medical care advantages over phone-only visits for certain medical conditions, and patients may be agnostic about the relative value and satisfaction with one form of telemedicine compared to another.

Higher patient satisfaction is positively associated with adherence to treatment protocols, which is notable given the high individual and societal costs associated with treatment non-adherence. Thus, the goal of our study was to determine if patient satisfaction with phone-only visits was noninferior to video visits at a large tertiary referral center in the South, a region which is home to many socioeconomically disadvantaged older individuals.

Methods with statistical considerations

Study design and participants

We conducted a parallel, single-blinded, noninferiority randomized patient-level clinical trial comparing satisfaction rates for two telemedicine delivery methods: phone-only (intervention group) and video visits (standard group). The two modes of telemedicine were compared to ensure that the less advanced telephone communication was not inferior to videoconferencing by a pre-specified margin of 15%.

Patients who had established care at participating clinics (at least one face-to-face visit with their physician in the past 12 months) were recruited and randomized to have their next scheduled routine clinical follow-up either via a phone-only or a video visit. The study was performed at the University of Alabama at Birmingham (UAB). Participants were enrolled from three clinical practices, including three physicians in rheumatology, five in cardiology, and three in family medicine, all of whom agreed to collaborate in this project and were blinded to which of their patients participated in the study.

Participants were eligible if they: 1) were scheduled for a routine in-office follow-up visit; 2) were age 60 years or older or had public insurance (i.e., Medicare/Medicaid); and 3) had videoconferencing capability. We restricted our study population to individuals 60 years of age or older or those with public insurance because of perceived concerns about their comfort and facility in using videoconferencing technology in these socially vulnerable groups.

Human subject protocols and consent procedures were approved by the UAB's Institutional Review Board (IRB). We obtained a waiver of informed consent given the study's minimal risk, the uncertain superiority of one form of telemedicine visit over another, the high probability of participants modifying their behavior due to their awareness of being observed (i.e., Hawthorne effect), and to mitigate participants’ burden. After the completion of the study procedures, previously blinded participants were debriefed about their study participation.

Blinding and randomization

Participants were randomly assigned to phone-only or video groups in a 1:1 ratio using computer-generated lists of random numbers implemented in REDCap. Patients, physicians, and investigators were blinded to the randomization assignment.

Interventions

The primary study goal was to compare visit satisfaction rates between phone-only and video groups. Our study protocol required that the research team contacted eligible patients by phone 2-3 weeks prior to a routinely scheduled in-office follow-up to invite participation in telemedicine visits. For each pre-appointment call, to ensure that all potential participants are contacted in a reproducible and standardized manner, research assistants followed an IRB-approved phone script that informed patients of the date and time of the scheduled appointment, that their appointment would be conducted by phone or a video call, confirmed the patient's best contact number, and verified their ability to participate in a video call by conducting a test video call using the videoconferencing platform in use for telemedicine services at our medical center. Only those patients who were able to successfully complete the test video call were subsequently randomized. Our study protocol was similar to the usual local telemedicine clinical protocols during the COVID-19 pandemic. We assessed participants’ videoconferencing capacity by conducting a test video call. Then the randomization procedure occurred. Participants were scheduled to have a phone-only or a video visit based on their randomly assigned group. Similar to the usual clinical procedures on the day of their appointment, physicians contacted patients by a phone or a video call. However, because physicians being blinded to patients’ study participation and because of prevailing institutional practices, a patient assigned to the video group could have a phone-only visit (e.g., due to videoconferencing difficulties) and a patient assigned to the phone-only group could have a video visit (e.g., if the physician decided that a video visit was needed).

Covariates

We collected participant data via phone surveys 36-72 hours post-visit to minimize recall bias and via electronic medical record (EMR) review. The survey collected information on self-rated health (excellent to poor response scale), hearing/vision impairment, education, health literacy, transportation difficulties, income, and employment status. Demographic characteristics (e.g., age, sex, race) were captured from the EMR. Area deprivation index (ADI), which measures neighborhood disadvantage, was derived using the patients’ residence zip code.

Outcomes

The primary outcome of interest was the patient's satisfaction, assessed using the validated visit satisfaction scale from the Consumer Assessment of Healthcare Providers and Systems (CAHPS)®30 survey. The patients were asked: "Using any number from 0 to 10, where 0 is the worst visit possible and 10 is the best visit possible, what number would you use to rate your telemedicine visit?”. Scores of 9 or 10 were prespecified to be grouped as satisfied and the satisfaction rate (proportion of satisfied participants) was calculated. Secondary outcomes included whether medical concerns were addressed at the visit, willingness to recommend a telemedicine visit, and an assessment of preference for the next visit type using the following question: “If you had a choice, what type of visit would you prefer as your next visit with [insert provider's name]?” with ‘telemedicine visit by phone’, ‘telemedicine visit by video’, or ‘in-person visit’ as possible choices. Exploratory outcomes included telemedicine acceptability (Telemedicine Perception Questionnaire [TMPQ] score, range 17 - 85, higher values are better), perceived autonomy support (healthcare climate questionnaire [HCCQ], range 0-15, higher values are better), perceptions about telemedicine relative to an in-person visit, and aspects of patient-physician communication (CAHPS survey). ,

Sample size and statistical analysis

We chose a noninferiority design based on the expectation that video visits might be preferred, that demonstrating noninferiority of satisfaction with phone-only visits would provide evidence for a patient's perception about phone-only visits and support continued patient access to and potential reimbursement for phone-only visits. We tested the noninferiority of phone-only versus video visits for the primary outcome of satisfaction rate. We assumed that the satisfaction rate for the home-based video visit (the superior group) would be 65%, slightly lower than the 68.5% satisfaction rate with facility-based telemedicine observed in a previous report. We established a ‘noninferiority margin’ so that the satisfaction rate with phone-only visits would be 15% lower than with video visits, the null hypothesis. The 15% noninferiority margin was chosen based on the assumption that less than 15% satisfaction rate difference in favor of video visits would be clinically irrelevant in declaring one mode of telemedicine superior to the other. We determined that a sample size of N = 100 per group would provide 72% power to reject the null hypothesis that phone-only visits were inferior to video visits by 15% or more when both were equivalent to each other with a 5% alpha significance level.

Variables were summarized using mean and standard deviation (SD) or median and interquartile range (IQR), as appropriate. Two sample t-test, chi-square test (Fisher Exact, if appropriate) examined differences between variables. Logistic regression evaluated the primary outcome, accounting for residual differences between groups and the potential differential effects of age versus insurance status. Because the fidelity of the intervention is key in noninferiority designs, where nonadherence to intervention can bias the assessments away from the null hypothesis by making the groups more similar than they may be in practice, we analyzed outcomes of individuals randomized who completed the post-visit surveys using both a modified intent-to-treat (mITT, as assigned by randomization and receiving a telemedicine visit) and per protocol (i.e., receiving the assigned visit type) analyses. We performed a sensitivity analysis that also included participants who partially completed post-visit surveys but who answered the visit satisfaction question. We conducted another sensitivity analysis that included the entire randomized population. Using a tipping point analysis, we imputed missing data using the satisfaction rate consistent with the null hypothesis (65% and 50% for video and phone-only groups, respectively) and assuming worst-best case scenario (assigning missing data as ‘not satisfied’ for phone-only and ‘satisfied’ for video group).

We explored heterogeneity of treatment effects using pre-specified subgroup analyses defined by race, sex, lack of transportation, income, and employment status. All analyses were performed using SAS 9.4 or R 4.0.4.

Results

Study participants and characteristics

The patients were screened for eligibility between May 28 and November 5, 2020 and the telemedicine visits were conducted between June 4 and December 2, 2020. Study enrollment was stopped when the pre-specified number of participants who fully completed surveys was reached (N=200). As seen in the CONSORT (Consolidated Standards of Reporting Trials) diagram (Fig. 1 ), a total of 2,800 people receiving routine care in the three clinical areas were assessed for eligibility. Of the 1,267 potentially eligible patients identified, 269 were randomized to receive either a video or a phone-only visit; 229 participants (85.1%) attended these visits. There were no significant differences in reasons for not participating in a telemedicine visit between groups (p = 0.12). A total of 200 participants (87.3%), including 96 assigned to video group and 104 assigned to phone-only group, completed surveys on average 2.7 (3.4) days post-visit. This population defines the modified intent-to-treat (mITT) analysis.

Fig. 1

CONSORT diagram showing patient enrollment, randomization, and follow up.

Overall, participants who completed the surveys were predominately women (N=136, 68%), 86 (43%) were Black with a mean age in the early sixties consistent with the inclusion criteria, and the majority had at least some college education (N=148, 74%) (Table 1 ). Compared to participants in the video group, those in the phone-only group were slightly younger and fewer had a Medicare plan as their medical insurance. The median (Q25, Q75) ADI national ranking was 68 (40, 85) indicating that a majority of participants were living in socio-economically disadvantaged neighborhoods.

Table 1

Demographic characteristics of participants who completed phone surveys and were included in the modified intent to treat analysis; p  <  0.05 in bold font.

	Telemedicine Video (N=96)	Telemedicine Phone (N=104)	p
Age, years, median (Q 25-Q 75)	66.75 (62.10-72.45)	62.25 (53.60-68.45)	0.001
Age group No. (%)			0.074
<65 years	39 (40.6)	63 (60.6)
≥65 years	57 (59.4)	41 (39.4)
Sex, female, No. (%)	67 (69.8)	69 (66.3)	0.71
Race, No. (%)			0.97
White	51 (53.1)	56 (53.8)
Black	42 (43.8)	44 (42.3)
Other	3 (3.1)	4 (3.9)
Insurance plans, No. (%)			0.03
Medicare	25 (26.0)	16 (15.4)
Medicaid	65 (67.7)	71 (68.3)
Other*	6 (6.2)	17 (16.3)
Specialty, No. (%)			0.13
Cardiology	28 (29.2)	18 (17.3)
Family medicine	39 (40.6)	51 (49.0)
Rheumatology	29 (30.2)	35 (33.7)
Outcome assessment timing, days, mean (SD)	2.88 (3.59)	2.58 (3.31)	0.54
Past telemedicine experience, yes, No. (%)	65 (67.7)	70 (68.0)	1
Device used for the telemedicine visit
Smartphone No. (%)	90 (93.8)	96 (92.3)	0.90
Computer/laptop, No. (%)	3 (3.1)	3 (2.9)
Tablet, No. (%)	1 (1.0)	1 (1.0)
Health status, excellent or very good, No. (%)	70 (72.9)	69 (66.3)	0.48
Transportation difficulties, No. (%)	7 (7.3)	14 (13.5)	0.23
Education, some college or more, No. (%)	73 (76.0)	75 (72.1)	0.64
Health literacy, inadequate†, No. (%)	23 (24.0)	20 (19.2)	0.52
Employment status, unemployed‡, No. (%)	73 (76.0)	83 (79.8)	0.64
Annual income, No. (%)			0.40
Low, < $29,999	14 (14.6)	21 (20.2)
Medium, $30,000-79,999	28 (29.2)	29 (27.9)
High, > $80,000	16 (16.7)	10 (9.6)
Prefer not to answer	38 (39.6)	44 (42.3)
Area deprivation index (ADI) ranking, state decile, median (Q 25-Q 75)§	5.00 [2.00, 8.00]	5.00 [2.00, 7.75]	0.95
Area deprivation index (ADI) ranking, national percentile, median (Q 25-Q 75) ‖	67.00 [41.00, 85.00]	69.00 [40.00, 84.75]	0.95

Viva, Blue Cross Blue Shield, United Health Care, Tricare; †Inadequate health literacy grouped the following answers: “Somewhat”, “A little bit”, and “Not at all”; ‡Employed is full-time, part-time, or temporary work; §State decile from 1 (least disadvantaged) to 10 (most disadvantaged); ‖National percentile from 1 (least disadvantaged) to 100 (most disadvantaged), missing for 9 participants.

Modified intent-to-treat analysis

Because there were statistically significant differences for age and insurance between groups, analyses were adjusted for age and insurance. The age- and insurance- adjusted difference in the overall visit satisfaction rate for the phone versus video group was 3.2% with 95% CI -7.6% to 14%, which did not contain -15% establishing noninferiority. The unadjusted satisfaction rate difference between phone-only and video groups was 6.5% (95% CI, -4.3% to 17.2%) (p  <  0.0001) (Fig. 2 ). Thus, phone-only visits were not inferior to video visits for visit satisfaction rate (p  < 0.0001). The satisfaction rates were higher than anticipated in both groups (78.1% for video vs 84.6% for phone-only) and not significantly different, (p = 0.32) (Table 2 ).

Fig. 2

Satisfaction (scores of 9 or 10) rates (95% confidence intervals) from modified intent-to-treat (mITT), per protocol, and sensitivity analyses; NI, noninferiority margin.

Table 2

Patient experience with telemedicine phone or video, modified intent-to-treat (mITT) and per protocol analyses; N (%) represented unless otherwise stated.

	Modified Intent-to-treat			Per Protocol
	Telemedicine Video (N=96)	Telemedicine Phone (N=104)	p	Telemedicine Video (N=66)	Telemedicine Phone (N=79)	p
Primary Outcome
Satisfaction rate, score ≥ 9	75 (78.1)	88 (84.6)	0.32	56 (84.8)	66 (83.5)	1
Satisfaction, median (IQR)	10 (9, 10)	10 (9, 10)	0.26	10 (9, 10)	10 (9, 10)	0.52
Secondary Outcomes
Preference for next visit			0.65			0.35
Telemedicine, same type	29 (30.2)	29 (27.9)		27 (40.9)	27 (34.2)
Telemedicine, different type	13 (13.5)	19 (18.3)		4 (6.1)	10 (12.7)
In-office	54 (56.2)	56 (53.8)		35(53.0)	42 (53.2)
Would recommend telemedicine			0.44			0.62
Yes, definitely	69 (71.9)	74 (71.2)		51 (77.3)	58 (73.4)
Yes, somewhat	21 (21.9)	25 (24.0)		12 (18.2)	17 (21.5)
No	6 (6.2)	3 (2.9)		3 (4.5)	2 (2.5)
No answer	0 (0.0)	2 (1.9)		0 (0.0)	2 (2.5)
Medical concerns addressed			0.22			0.09
All	80 (83.3)	94 (90.4)		53 (80.3)	71 (89.9)
Most	12 (12.5)	9 (8.7)		10 (15.2)	8 (10.1)
Some	4 (4.2)	1 (1.0)		3 (4.5)	0 (0.0)
Exploratory
Telemedicine compared to office visit			0.67			0.73
Telemedicine better	20 (20.8)	23 (22.1)		13 (19.7)	21 (26.6)
No difference	22 (22.9)	30 (28.8)		19 (28.8)	24 (30.4)
Office visit better	49 (51.0)	48 (46.2)		32 (48.5)	32 (40.5)
No answer	5 (5.2)	3 (2.9)		2 (3.0)	2 (2.5)
Satisfaction with physician (0-10 scale), score ≥ 9	92 (95.8)	102 (98.1)	0.61	63 (95.5)	77 (97.5)	0.84
Visit is convenient, yes	84 (87.5)	94 (90.4)	0.67	58 (87.9)	74 (93.7)	0.36
Perceived autonomy support, yes, HCCQ* ≥ 7	77 (80.2)	80 (76.9)	0.69	57 (86.4)	61 (77.2)	0.23
Telemedicine acceptability, TMPQ†, median [Q 25, Q 75]	65.0[5.25, 68.00]	65.0[59.00, 69.00]	0.20	65.5[55.75, 68.00]	65.0[59.00, 69.00]	0.36

*HCCQ, healthcare climate questionnaire; †TMPQ, telemedicine perception questionnaire, higher values are better.

When we examined the subgroups of sex, race, employment status, ADI, transportation availability, we found no heterogeneity of treatment effects by these characteristics (Fig. 3 ).

Fig. 3

ORs and 95% CIs for satisfaction (scores of 9 or 10) rate in the phone-only versus video group for age-adjusted modified intent to treat analysis (mITT) and unadjusted subgroup analysis; OR, odds ratio, CI, confidence interval.

Per protocol analysis

Out of the 200 participants who completed post-visit surveys, 145 people had their randomized type of telemedicine visit (79 in the phone group [74.5%]; 66 in the video group [68.8%]). This population was included as the per protocol analysis. There were no significant differences in the sociodemographic characteristics of participants in the per protocol analysis. In this per protocol analysis, the unadjusted satisfaction rate difference between phone-only and video groups was -1.3% (95% CI, -12.6% to 10%), which did not contain the inferiority boundary of -15% (p = 0.01) (Fig. 2). After adjusting for age and insurance, the satisfaction rate in the phone-only group was lower than in the video group by -4.1% (95% CI -14.8% to 6.6%). The 95% lower bound confidence limit was -14.8%, which did not include the -15% noninferiority limit, establishing noninferiority.

Sensitivity analyses

In a sensitivity analysis that also included 10 participants who partially completed post-visit surveys, the phone-only visits remained noninferior to video visits with respect to the satisfaction rate in each group. The tipping point analysis imputation using the anticipated satisfaction rates confirmed noninferiority (77.0% phone-only vs 73.1% video) as did the imputation using worst-best case scenario (66.7% phone-only vs 81.3%).

Secondary outcomes

In the mITT population, the proportion of participants who indicated they preferred a telemedicine visit of the same type for their next visit was similar (30.2% in the video vs 27.9% in the phone group, p = 0.78) (Table 2). In addition, the proportion of individuals who would definitely recommend telemedicine (71.9% video vs 71.2% phone-only group, p = 0.44) and the proportion of participants who believed that their concerns were addressed during the visit (83.3% video vs 90.4% phone-only group, p = 0.22) were similar. Interestingly, 20 (20.8%) participants in the video group and 23 (22.1%) in the phone-only group indicated that their telemedicine visit was superior to an office visit (p = 0.67). Telemedicine acceptability measured by the TMPQ score was high in both groups (65 [55.25, 68] phone vs 65 [59, 69] video, p = 0.2).

Discussion

In a randomized clinical trial, we found that the visit satisfaction rates (grouping satisfaction scores of 9 and 10, 0-10 scale) of established patients in cardiology, family medicine, and rheumatology clinics at a large multiple specialty clinic affiliated with an academic institution were high for both types of telemedicine visits and that the satisfaction rate with phone-only visits was not inferior to video visits. Indeed, the visit satisfaction rates we observed with phone-only telemedicine trended slightly higher than those for video visits. This finding may support some tendency in the older Medicare and younger Medicaid population to be more comfortable with phone-only rather than video visits, even among those who have video conferencing capability. The overall satisfaction with either modality was high and argues for reimbursement of both types of service at least and until the ease of use of videoconferencing by these populations becomes greater. To our knowledge, this is the first randomized trial to evaluate whether telemedicine phone-only visits are noninferior to video visits for visit satisfaction rate.

Satisfaction with a healthcare service, as assessed using the CAHPS® measures, influences whether a patient is likely to use that service in the future and thus, it is important to evaluate whether new healthcare services, including the recent reliance on telemedicine for chronic disease care, meet patients’ needs. Because some patients may lack broadband internet access, newer telecommunication equipment, or digital literacy required for videoconferencing, our finding that phone-only visits are not inferior to video visits in terms of visit satisfaction rate is encouraging. Moreover, our results suggest that patients without access to these technologies do not forgo desired services. While a phone-only visit significantly limits the extent of the physical examination and the subtle non-verbal cues afforded to clinicians by a telemedicine video are missing, our results show that from patients’ point of view both types of telemedicine visits were associated with high satisfaction rates and were favorably experienced by patients who undergo periodic evaluations in rheumatology, cardiology, and family medicine clinics.

The major strength of our study is the use of a study design that used randomization and blinding to minimize the potential for biases from either the providers or patients. Additional key strengths to our study are that we also targeted older and socioeconomically disadvantaged populations, recruited a large proportion of minority population (over half of the participants did not identify as white race) in whom healthcare disparities are well recognized, and, because the use of a waiver of consent, minimized selection bias of nonparticipation due to the consent process. Despite its strengths, our study has some limitations. We did not collect information about specific reasons or diseases evaluated during the telemedicine visit and it is possible that not all clinical conditions render themselves suitable for delivering/receiving care via different formats of telemedicine. We collected outcomes via telephone interviews, and our estimates of patient satisfaction may be subject to a social desirability bias. However, the participants in both groups were blinded to the intervention and data collected by the same research assistants; therefore, it is unlikely that this social desirability bias would be differential between the two groups. As observed in past studies that used similar measures of visit satisfaction, , the patient satisfaction scale we deployed demonstrated a ceiling effect. Since we compared the proportion of patients with high satisfaction rather than a continuous satisfaction rating, this issue is unlikely to affect either the reliability or validity of our findings. Nevertheless, this observation underscores the need for development of novel instruments to assess patient satisfaction that may be less prone to ceiling effects. Our results are not generalizable to the adult population who do not have access to video capacity, because we excluded these individuals in an attempt to ensure that each participant had equal opportunity to see their respective physicians using either technology. Our results are also limited to a small but important spectrum of medical specialties and a limited number of clinicians. Our per protocol analysis relied on a patient's report of the type of visit (phone-only vs video) they had because neither clinician notes, nor billing data, captured the type of telemedicine visit conducted, which could not be independently adjudicated by the research team. As such recall bias or information bias may have affected these results, but our sensitivity analyses suggest this would not have changed the findings.

Spurred by the pandemic, home-based telemedicine has garnered substantial increased attention from patients, healthcare professionals and administrators, insurers, and policy makers. Our findings provide added data on patients’ acceptance and satisfaction with different types of telemedicine in populations of concern, which can inform clinical, regulatory, and administrative context of telemedicine and related reimbursement policies for medical care of patients with chronic diseases during and beyond the COVID-19 era.

Declaration of Competing Interests

The authors do not have any conflict of interest to disclose related to this work.