Patient Satisfaction for Telehealth Physical Therapy Services Was Comparable to that of In-Person Services During the COVID-19 Pandemic.

BACKGROUND: The COVID-19 pandemic has transformed the practice of physical therapy (PT), from a model based on in-person delivery to one that includes telehealth. However, research is needed to assess value, patient satisfaction, and clinical outcomes in the delivery of telehealth PT. PURPOSES: We sought to compare patient satisfaction with in-person and telehealth PT and to determine the factors-such as age, gender, or insurance payer-that contributed to patient satisfaction scores.
METHODS: Patients between the ages of 18 and 90 years who received outpatient PT at our institution were texted a patient satisfaction survey. Surveys were sent to patients who received in-person PT from January 1 to May 8, 2020, and to patients who received telehealth PT from April 21 to May 8, 2020. Patients' age, gender, insurance payer, and written comments were also collected. Kruskal-Wallis tests were used to compare survey scores between groups. χ-squared tests were used to compare categorical patient characteristics between groups.
RESULTS: In total, 12,345 surveys were sent out. We included completed surveys from 1147 patients in the analysis. Of these, 33% identified as male and 67% as female, mean age was 60.3 years, 58% had commercial insurance, 36% had Medicare, 3% had Medicaid, 2% had no-fault/worker's compensation, and 1% had international insurance. The question of satisfaction was answered by 1074 patients. No statistical difference in satisfaction was seen in age or gender groups. Satisfaction differed by insurance type, but when analysis excluded patients with international insurance, there was no difference between groups. In-person PT patients reported higher satisfaction in achieving treatment goals, as rated on a 1-to-5 scale (4.7 ± 0.6), than telehealth PT patients (4.6 ± 0.6), although it is unclear if this result is clinically meaningful. There was no significant difference in the remaining follow-up questions.
CONCLUSION: We found no overall difference in patient satisfaction between those receiving in-person PT and those receiving telehealth PT. This was true for initial and follow-up visits, with the exception of two categories: the "ability of scheduling an appointment" and "progress towards attaining your treatment goals." However, the clinical significance of these two exceptions is questionable. Ongoing research is indicated to evaluate the efficacy of telehealth PT while identifying the best patients and conditions for use during non-crisis times. Future studies should determine the effect of telehealth PT on patient-reported outcome measures, function, and value. © Hospital for Special Surgery 2020.

Introduction

Coronavirus disease 2019 (COVID-19) has caused unprecedented changes in the healthcare industry. To provide continuity of patient care across all conditions, telehealth rapidly replaced in-person visits during the spring of 2020. A 2019 survey found 66% of US consumers expressed interest in accessing care through telehealth, yet only 8% had done so [8]. Barriers included lack of reimbursement models, difficulty with obtaining multiple state licensures, and overall lack of comfort with technology [3]. However, the COVID-19 pandemic and the social distancing that resulted forced these barriers to be quickly overcome. Across the nation, institutions reported significant increases in telehealth visits to keep patients and providers safe [2, 6, 9].

On March 20, 2020, New York State issued shelter-in-place orders, mandating closure of all non-essential businesses and cancellation of all elective surgeries. Many outpatient physical therapy (PT) clinics temporarily closed, leaving patients unable to continue in-person therapy. To ensure continuity of care, the rehabilitation department at Hospital for Special Surgery (HSS) determined that post-operative patients (those who had surgery less than 3 months prior) could continue to receive in-person PT and everyone else could receive telehealth PT. (The HSS telehealth rehabilitation program was based on one established in 2018 for post-acute total hip and total knee replacements [4].) Based on previous experience and infrastructure, HSS rehabilitation quickly transitioned care delivery to support the needs of patients during the COVID-19 pandemic. As of May 8, 2020, the department had completed 7048 telehealth visits.

The advantages of telehealth include convenient access to high-quality care, safety for the patient and provider, and reduced burden on patients who worry about taking time off of work and dealing with travel, parking, and child care [2]. However, to our knowledge, there are few studies that analyze patient satisfaction with telehealth PT [7]. As its use continues to grow, this is an important topic to explore.

This study was designed to answer three primary questions. (1) Was there a difference between patient satisfaction for in-person PT evaluations and telehealth PT evaluations? (2) Was there a difference in patient satisfaction for in-person PT follow-up visits and telehealth PT follow-up visits? (3) What was the level of patient satisfaction for those who began with in-person PT and converted to telehealth PT during the pandemic? Secondary questions were as follows: which factors contribute to patient satisfaction or dissatisfaction with rehabilitation services? Was there a difference in patient satisfaction based on decade of life, gender, or insurance payer?

Methods

This study was approved by the HSS Institutional Review Board and data was obtained from a retrospective review of patient satisfaction surveys. Patients included in this study were those between 18 and 90 years of age who completed outpatient PT in person at an HSS outpatient rehabilitation location or remotely through telehealth. Pediatric patients (younger than 18 years of age), occupational therapy patients, and those seen at corporate affiliate locations were excluded. Responses were reviewed from patients who received in-person PT from January 1, 2020, to May 8, 2020. Additionally, we reviewed surveys from patients who received telehealth PT from April 21, 2020, to May 8, 2020. The dates of surveys collected differ between the two groups because telehealth PT was launched on March 25, 2020, in response to the COVID-19 pandemic as an alternative way to deliver care and maintain social distancing. Surveys were first distributed for telehealth PT on April 21, 2020.

As per standard of care, all patients who received outpatient PT services were sent a Q-Review Survey (Quality Reviews, Inc., New York, NY, USA) to rate their satisfaction with the care they received. The questionnaire was sent as a web link via text message with an explanation of its purpose. Four surveys were automatically generated and sent to the patients based on their visit type (evaluation or follow-up) and mode of PT delivery (in-person or telehealth) at the following time points: 1 h after evaluation (in-person or telehealth); 30 days after beginning treatment (in-person or telehealth as of April 21, 2020); and after every 10 visits (in person only prior to April 21, 2020). In total, 12,345 surveys were sent; 1147 were completed.

Items were scored on a 5-star rating, similar to a Likert scale of 1 to 5 [1]. A score of 5 indicates “most satisfied” and 1 indicates “least satisfied.” There was one exception to this 5-star rating system: the question, “How likely are you to refer Hospital for Special Surgery to your friends and family?,” usually used to measure consumer loyalty, we used to measure overall satisfaction with PT services; it was graded on a 10-point scale with 10 indicating “most likely to refer” and 1 indicating “least likely to refer.”

Patients were also given the opportunity to include comments. Only patients who received telehealth PT would receive the question, “How would you rate your confidence that your rehab needs are being or were met through telehealth?” (Online Resource 1).

Summary statistics of the study population included reporting of means and standard deviations for continuous variables and frequencies and percentages for discrete variables. Comparison of patient demographic data between survey groups was conducted using one-way analysis of variance for continuous data. χ-squared tests were used to compare discrete demographic variables between groups. Kruskal-Wallis tests were used for the analysis of all survey items between in-person and telehealth respondents. Sub-group analysis compared the response of patients who had converted from inpatient at evaluation to telehealth at follow-up with the response of patients who did not experience a change in the type of care received. Comparison of overall satisfaction was assessed between patient characteristics. Satisfaction was defined as a 9 or 10 on the satisfaction question of the survey. Mann-Whitney U tests and χ-squared tests were used to compare continuous and categorical characteristics, respectively, between patients who were satisfied and those who were not. All tests of multiple comparisons were adjusted using Bonferroni technique. Statistical significance was set to an α equal to 0.05. All analyses were performed using SPSS, version 23.0 (IBM Corp., Armonk, NY, USA).

Results

Of the 1147 surveys received, 600 were for in-person PT evaluation, 133 were for telehealth PT evaluation, 310 were for in-person PT follow-up, and 104 were for telehealth follow-up appointments. A subset of 61 patients began care with in-person PT but were converted to telehealth PT services when, due to the pandemic, in-person visits became possible for essential care only.

Age range of the overall study population was between 18 and 90 years with a mean age of 60.3 (± 15) years. Women made up 67% of the total population (n = 770). Table 1 reports the patient characteristics among all survey types. No statistical difference was found between individual ages (p = 0.167) or as age groups (p = 0.081) between survey type. Roughly two-thirds of the respondents in each group were female; the proportion of women in each group was not significantly different between survey groups (p = 0.637). No statistical differences were found in insurance type between survey groups (p = 0.178).

Table 1

Subject characteristics by survey group and overall

Characteristic	In-person eval		In-person follow		Telehealth eval		Telehealth follow		Total		p value
	N	Freq. (%)	N	Freq. (%)	N	Freq. (%)	N	Freq. (%)	N	Freq. (%)
Age (mean, SD)a	600	59.3 (15.6)	600	60.9 (14.3)	600	61.9 (14.5)	600	62 (13.7)	600	60.3 (15.0)	0.167
Age group
< 20	600	6 (1%)	600	0 (0%)	600	1 (1%)	600	2 (2%)	600	9 (1%)	0.081
20–29	600	23 (4%)	600	6 (2%)	600	3 (2%)	600	0 (0%)	600	32 (3%)
30–39	600	60 (10%)	600	29 (9%)	600	7 (5%)	600	6 (6%)	600	102 (9%)
40–49	600	60 (10%)	600	26 (8%)	600	15 (11%)	600	9 (9%)	600	110 (10%)
50–59	600	106 (18%)	600	69 (22%)	600	23 (17%)	600	17 (16%)	600	215 (19%)
60–70	600	177 (30%)	600	85 (27%)	600	33 (25%)	600	40 (38%)	600	335 (29%)
70+	600	168 (28%)	600	95 (31%)	600	51 (38%)	600	30 (29%)	600	344 (30%)
Gender
M	600	196 (33%)	310	108 (35%)	133	38 (29%)	104	35 (34%)	1147	377 (33%)	0.637
F	600	404 (67%)	310	202 (65%)	133	95 (71%)	104	69 (66%)	1147	770 (67%)
Insurance type
Commercial	592	354 (60%)	308	173 (56%)	133	70 (53%)	103	60 (58%)	1136	657 (58%)	0.178
Medicare	592	197 (33%)	308	118 (38%)	133	57 (43%)	103	42 (41%)	1136	414 (36%)
Medicaid	592	26 (4%)	308	6 (2%)	133	4 (3%)	103	0 (0%)	1136	36 (3%)
Worker’s comp/no fault	592	12 (2%)	308	8 (3%)	133	2 (2%)	103	1 (1%)	1136	23 (2%)
International	592	3 (1%)	308	3 (1%)	133	0 (0%)	103	0 (0%)	1136	6 (1%)

Age reported as mean and standard deviation

Only one question received significantly different responses from in-person and telehealth patients. This was, “How would you rate the ability to schedule an appointment?” The mean score for in-person responses was 4.6 ± 0.8. Telehealth patients reported higher satisfaction in the same question, reporting a mean score of 4.8 ± 0.6 (p < 0.001) (Fig. 1). All other questions resulted in no statistical difference between in-person and telehealth patients. At follow-up, in-person PT patients reported higher satisfaction in achieving treatment goals (4.7 ± 0.6) compared with telehealth patients (4.6 ± 0.6) (p = 0.004). There were no significant differences in the remaining follow-up questions (Fig. 2).

Fig. 1

Comparison of evaluation questions between in-person versus telehealth patients. Error bars indicate standard deviations. The asterisk indicates statistically significant difference between groups.

Fig. 2

Comparison of follow-up questions between in-person versus telehealth patients. Error bars indicate standard deviations. The asterisk indicates statistically significant difference between groups.

The question of satisfaction was answered by 1074 patients (90%). Of these, 95% (N = 1016) reported being satisfied (scored 9 or 10 on question) while the remaining 5% (N = 58) reported a score of 8 or lower (Online Resource 2). No statistical difference was found in the mean response of overall satisfaction between the four survey groups (p = 0.771). At evaluation, there was no statistical difference in satisfaction score between the in-person group (9.7 ± 1.2) versus the telehealth group (9.8 ± 0.6, p > 0.999). Similar satisfaction scores were found at follow-up with both in-person patients and telehealth patients reporting a satisfaction score of 9.8 ± 1.1 (p > 0.999). The 61 patients who were converted from inpatient to telehealth PT reported a satisfaction score at follow-up of 9.9 ± 0.7. Compared with those who did not convert to telehealth (9.7 ± 1.1), there was no difference in their scores (p = 0.069) (Fig. 3).

Fig. 3

Mean scores of overall satisfaction of in-person, converted, and telehealth patients at evaluation and follow-up. Error bars indicate standard deviations. The asterisk indicates comparison of patients converted from non-converted patients (9.7) versus in-person to telehealth (9.9).

No statistical difference was found in overall satisfaction between groups on age (p = 0.217), gender (p = 0.771), or study (p = 0.771). However, insurance type was significantly associated with overall satisfaction (p = 0.001); patients with international insurance reported the lowest overall satisfaction score (6.8 ± 4.3), significantly lower than all other insurance types (p < 0.001 for all multiple comparisons) (Table 2). Because there may have been some misunderstanding of the survey due to language or cultural differences, insurance type was analyzed with these patients removed. When the international patients are excluded from this comparison, there is no significant difference in insurance type between patients who were satisfied and those who were not (p = 0.578).

Table 2

Comparison of patient characteristics between patients satisfied versus patients not satisfied

Characteristic	Satisfiedb		Not satisfied		p value
	N	Freq. (%)	N	Freq. (%)
Age (mean, SD)a	1016	60.3 (14.8)	58	59.5 (15.6)	0.601
Age group
< 20	1016	6 (1%)	58	1 (2%)	0.260
20–29	1016	30 (3%)	58	0 (0%)
30–39	1016	93 (9%)	58	5 (9%)
40–49	1016	96 (9%)	58	8 (14%)
50–59	1016	192 (19%)	58	15 (26%)
60–70	1016	307 (30%)	58	11 (19%)
70+	1016	292 (29%)	58	18 (31%)
Gender
M	1016	333 (33%)	58	20 (34%)	0.788
F	1016	683 (67%)	58	38 (66%)
Insurance type
Commercial	1007	587 (58%)	58	36 (62%)	0.009
Medicare	1007	362 (36%)	58	19 (33%)
Medicaid	1007	33 (3%)	58	1 (2%)
Worker’s comp/no fault	1007	22 (2%)	58	0 (0%)
International	1007	3 (0%)	58	2 (3%)
Responder group
In-person eval	566	532 (94%)	566	34 (6%)	0.780
In-person follow	293	278 (95%)	293	15 (5%)
Telehealth eval	124	119 (96%)	124	5 (4%)
Telehealth follow	91	87 (96%)	91	4 (4%)

Age reported as mean and standard deviation

Satisfaction defined as responding a 9 or 10 on the question “How likely are you to refer Hospital for Special Surgery to your friends and family?” Scores of 1–8 defined as not satisfied

Discussion

The COVID-19 pandemic limited the ability to provide in-person outpatient PT, but the implementation of telehealth allowed patients to receive care remotely, while remaining in compliance with shelter-in-place mandates. Our study found high levels of patient satisfaction with telehealth PT that generally did not differ from in-person care. For the initial visit, telehealth patients reported more satisfaction than in-person patients for the question concerning “ability of scheduling an appointment.” Patients receiving PT in-person did report higher satisfaction with “progress towards attaining your treatment goals” than those receiving care through telehealth.

This study has several limitations. First, data on telehealth PT was collected over a period of 3 weeks compared with 18 weeks for data on in-person PT. Because less telehealth data was collected, an uneven weight was attributed to telehealth responses. Future studies with larger sample sizes will allow for an adequately powered study to show statistical equivalence. Second, from March 13 to May 3, 2020, the in-person group was limited to immediate post-operative (less than 3 months from date of surgery) patients due to state mandated orders to hold all non-essential medical services. Before March 13 and after May 3, this group was not restricted and included all non-operative and post-operative patients. The telehealth group covered all non-operative and post-operative (any time duration from date of surgery) patients for the entire duration. Future studies are needed to explore patient satisfaction based on diagnosis and post-operative phase. Third, the rollout of our telehealth system and the training involved led to some inconsistencies in visit duration. While all evaluations were scheduled for 45 min for both in-person and telehealth PT, telehealth follow-up PT visits were less consistent, initially scheduled for 45 min but later transitioning to 30 min—the same time for an in-person PT visit. This change in the length of the session may have affected patient satisfaction, as is evidenced in the slightly higher satisfaction rating telehealth patients gave for the time spent with their therapist. Fourth, all respondent groups reported high levels of satisfaction, and it is unclear if only satisfied patients responded to the survey. Given that our data was collected during the COVID-19 crisis and the group with the highest satisfaction was the one that converted from in-person to telehealth PT, it is possible that fear of contagion was a contributing factor and led to a response bias, overestimating the level of satisfaction. Response bias may be reduced if this study is repeated when the crisis is over and patients have the option to choose between telehealth or in-person care.

All groups reported extremely high rates of satisfaction, even on follow-up among patients who began with in-person care and converted to telehealth. When compared with patients who did not convert, there was no statistical difference in satisfaction scores. High patient satisfaction across all telehealth groups demonstrates that it is a practical way to deliver care to patients when in-person care is not available. The only area where the in-person group reported statistically significant higher satisfaction was when they were asked about having achieved their treatment goals, but this difference is of questionable clinical significance as both the in-person and telehealth groups reported high satisfaction (4.7 and 4.6, respectively).

Our findings of high patient satisfaction with telehealth PT are consistent with previous authors who have reported similar results for care delivered virtually, although the contributing factors may differ [4, 5, 7]. Kruse et al. reported patient satisfaction with telemedicine to be correlated to improved outcomes, preferring telehealth for ease of use, low cost, improved communication, and decreased travel time [5]. In a retrospective case series, Fisher et al. described subjective reports of patient satisfaction with telehealth physical therapy following joint replacement surgery [4]. A study by Moffet et al. found no difference in patient satisfaction between those receiving telehealth PT and those receiving in-person PT; furthermore, the researchers found that satisfaction strongly correlated with improvements in gait and stair climbing in patients after total knee arthroplasty [7].

Several other factors may have contributed to high patient-reported satisfaction with telehealth PT in our study. A statistically higher rate of satisfaction concerning the courtesy and friendliness of the staff was interesting and may be attributable to the nature of a telehealth visit—it enables one-on-one time with the therapist for the entire session. There are significantly fewer demands on a therapist’s attention during a telehealth visit, whereas during an in-person visit, the therapist may be attending to more than one patient at a time. Additionally, patients who had a telehealth evaluation reported statistically significant higher satisfaction in the ability to schedule an appointment (Fig. 1). This difference may simply reflect the increase in the number of open slots on therapist schedules since, during the initial stages of shelter-in-place, outpatient volume was decreased. While the difference was statistically significant, it may not be clinically meaningful as the scores on average were 4.6 and 4.8 for in-person and telehealth PT, respectively.

When looking at insurance type and satisfaction, the only group associated with a statistically significant lower satisfaction was international insurance. The low ratings may have been due to a language barrier. All respondents who scored a 3 or below were called by rehabilitation management for follow-up. The patients who had international insurance reported difficulty understanding how to properly respond to the survey and were, in fact, pleased with PT services. This has been a learning opportunity as the survey is available only in English. Validated patient-reported outcome measures have gone through this exercise to adjust the language to account for cross-cultural words, idioms, or phrases. Of additional interest is that patients in older demographics (60 to 69 and over 70 years) reported high rates of satisfaction with telehealth PT, dispelling the notion that older patients not adapt to telehealth technology.

Finally, patients were allocated to telehealth involuntarily due to state directives to halt non-essential patient care while a shelter-in-place order was in effect. This could affect levels of satisfaction as patients may have been pleased to receive any PT under the unprecedented circumstances. Therefore, telehealth satisfaction rates during the COVID-19 pandemic may not reflect the rates of telehealth satisfaction during a non-crisis time. Future studies should evaluate similar questions after the pandemic to minimize recency bias that may have been present in our patients.

In conclusion, our study yielded early data demonstrating high levels of patient satisfaction with telehealth PT when in-person services were not available during the COVID-19 crisis. Telehealth has become a viable option to deliver PT, and as its use increases, ongoing investigation is needed to evaluate its efficacy.

(PDF 109 kb)

(PDF 50 kb)

(PDF 1224 kb)

(PDF 249 kb)

(PDF 1224 kb)

(PDF 1225 kb)

(PDF 1224 kb)