Preference for Telehealth Sustained Over Three Months at an Outpatient Center for Integrative Medicine.

BACKGROUND: Integrative medicine is a key framework for the treatment of chronic medical conditions, particularly chronic pain conditions. The COVID-19 pandemic prompted rapid implementation of telehealth services.
OBJECTIVE: We present outcomes of a complete and rapid transition to telehealth visits at an outpatient integrative medicine center in the Southeastern United States.
METHOD: Patients and administrative staff took surveys comparing telehealth to in-person visits within four weeks of our clinic's transition to telehealth and three months later. Beginning four weeks after the clinic's telehealth conversion in March 2020, patients who had a telehealth visit at the center completed a survey about their telehealth experience and another survey three months later.
RESULTS: Patient quality judgements significantly favored telehealth at baseline, B = .77 [0.29 - 1.25], SE = .25, t(712) = 3.15, p = .002, and increased at three months, B = .27 [-0.03 - 0.57], SE = .15, t(712) = 1.76, p = .079. Telehealth technology usability and distance from the center predicted patient ratings of telehealth favorability. Providers favored in-person visits more than patients, B = -1.00 [-1.56 - -0.44], SE = .29, t(799) = -3.48, p < .001, though did not favor either in-person or telehealth more than the other. Patient discrete choice between telehealth and in-person visits was split at baseline (in-person: n = 86 [54%]; telehealth: n = 73 [46%]), but favored telehealth at three months (in-person: n = 17 [40%]; telehealth: n = 26 [60%]). Overall, discrete choice favored telehealth at follow-up across providers and patients, OR = 2.69 [.1.18 - 6.14], z = 2.36, p = .018. Major qualitative themes highlight telehealth as acceptable and convenient, with some challenges including technological issues. Some felt a loss of interpersonal connection during telehealth visits, while others felt the opposite.
CONCLUSION: We report converging mixed-method data on the successful and sustained implementation of telehealth with associated policy and clinical implications during and beyond the COVID-19 pandemic.

Integrative medicine emphasizes a holistic, biopsychosocial model to treat the whole person. In the outpatient setting, integrative medicine employs an interdisciplinary team to address the physical, emotional, spiritual, functional, and social aspects of complex medical conditions. At our integrative medicine outpatient center within a large academic medical university in the Southeastern United States, nurse practitioners collaborate with psychologists, health coaches, physical therapists, acupuncturists, massage therapists, and movement instructors (yoga and Tai Chi) to deliver integrative care.

Historically, integrative medicine services have been delivered in-person with telehealth-delivery playing only a small role, typically limited to intervention trials.[1-4] Barriers to telehealth-delivery of services exist across patient and provider levels, including lack of payment models, delivery platforms, health and technological literacy, and provider buy-in.[5,6] Integrative medicine prioritizes an authentic, healing relationship between patient and provider, one barrier frequently cited is the concern that virtual modalities may harm rapport. Further, an interdisciplinary team often relies on being “under one roof” and having time to collaborate and co-create treatment plans.[7] At the same time, complex, chronic pain conditions represent one of the most common diagnoses for which patients seek integrative care at the outpatient center. In-person delivery of care poses numerous barriers for individuals with pain, limited mobility, and multiple comorbidities particularly those in rural, underserved areas. Telehealth is a historically under-utilized tool to address these needs.

The extant literature demonstrating initial acceptability and quality of telehealth-delivered care across a wide range of settings and specialties has only been accelerated by current world events.[8-14] The ongoing COVID-19 pandemic launched a complete transition to telehealth services in our clinic, like many others, removing historical barriers to its delivery such as poor reimbursement for virtual services and provider and administrative buy-in. Emerging literature shows successful implementations of telehealth services due to COVID-19 across diverse medical settings.[15-18] Recent chronic pain expert consensus guidelines emphasize the need for continuity of care, highlighting the importance of telemedicine during the COVID-19 pandemic.[19,20]

We did not know how the widespread implementation of telehealth in an integrative medicine clinic would impact the quality of care. To answer this question, we conducted a survey to better understand the experiences of patients, providers, and administrative staff. We entered into this project without a stated hypothesis. However, we discussed our biases in planning meetings. Due to the emphasis on relationship centered care in our clinic, we generally expected that telehealth services would be experienced as worse in quality than in-person services. We also expected any positivity in ratings of telehealth at baseline might deteriorate three months later. Overall, we thought telehealth services might be tolerated given the larger context of the COVID-19 pandemic, but would likely not be an acceptable replacement for in-person care. This paper presents a novel mixed-methods study describing the experience of multiple stakeholders during the rapid implementation in an outpatient integrative medicine setting.

Method

Participants

Patients

In this sample, the baseline survey was completed by 180 patients. Of patients who completed the baseline survey, 61 also completed the three-month follow up survey. For historical patient population descriptive data, we utilized an intake form completed by 3,218 representative new patients from July 2015 to March 2019. The mean age of our patient population is 46.98 (SD = 16.74). 24.7% of patients are male and 75.4% female. Clinic patients are 87.6% white, 7.9% Black or African American, 1.3% Asian, 0.5% American Indian or Alaskan Native, and 0.2% Native Hawaiian or other Pacific Islander, with 2.4% identifying their race as “other.”

Providers and administrative staff

Nineteen providers and seven administrative staff at the center completed the survey at baseline. At follow-up three months later, seven providers and five administrative staff completed the survey.

Procedure

We sent requests to fill out a survey to patients who had a telehealth appointment at the center from the time of telehealth transition (March 17, 2020) and for the next four months (n = 776 total). The first surveys went out four weeks after the conversion to telehealth, so the patients seen during those first weeks may have received their survey up to four weeks after their initial telehealth appointment. Subsequent patients having their first telehealth appointment in the clinic (for any service) received the survey within one week of their first telehealth visit. We distributed the provider survey to 25 providers who had telehealth appointments and 12 administrative staff. We re-sent the follow-up surveys at least three months following the initial survey to patients, providers and administrative staff. Surveys were developed in and hosted by REDCap.[21,22]

Materials

We constructed a mixed-methods survey to collect information about telehealth experience and attempted to maximize parallel questions between patients, providers, and administrative staff to allow for direct comparisons across stakeholders. This consisted of questions about service usage (patients only), custom items of judgements of telehealth, open-ended questions about telehealth experience, a standardized measure of telehealth usability and a survey about the impact of COVID-19 on their life.

Service usage

We asked respondents to report on the number of visits to Osher (any provider) in-person and via telehealth to date. We defined telehealth as follows: “healthcare visits that happen using technology that allows you and your provider to interact through audio and video over the internet. The center uses Zoom for telehealth visits.” We then asked them to select which providers on a list of disciplines they had experience with on telehealth. Following, we asked for the number of telehealth visits they have had at any point with providers outside of the center. Among these variables, we included whether the respondent resided within the same county as the center or not.

Modality preferences

We assessed patient preferences for telehealth or in-person visits by two parallel measures: continuous judgements and a forced-choice question. These two measures served as convergent perspectives on the overall issue of patient preferences.

Judgements of quality

We wrote Likert-type items asking respondents to communicate their experience of telehealth at the center compared to their previous experience of in-person visits at the center, or an “average” visit if they had no previous in-person visits at the center. To ground these ratings in relative comparison, we formatted the scoring of these items on an 11-point Likert centered on zero and ranging from –5 to 5. Zero anchored telehealth and in-person visits as the same on the dimension of interest on that item, positive values (1 to 5) indicated telehealth visits as better on that dimension greater degrees and negative values (–1 to –5) meaning in-person visits as better on that dimension. We asked respondents to respond to three items: overall experience of quality, quality of care, and quality connection with their provider or patients. The three attitudes were internally consistent as a group, Cronbach’s α = .90; 95% confidence interval (CI) = .87–.92.

Preference of visit today (if it were safe)

In order to assess preference, we asked participants the following question: “If it were safe to have an in-person visit today, would you choose to have your visit in-person or via telehealth?” This provided a discrete report of behavioral intentions in a realistic hypothetical scenario, providing a convergent measure to the continuous judgement ratings. This question was asked of patients and providers, but not administrative staff as they did not participate directly in medical visits.

Telehealth Usability Questionnaire (TUQ)

The TUQ is a 21-item questionnaire that assesses the usability of a given telehealth system.[23] The items were directly drawn from other validated measures to give a multidimensional, well-rounded picture of usability. In our sample, the full measure had excellent internal reliability (α = .97; 95% CI = .97–.98).

Coronavirus stressor survey

This survey was developed in response to the COVID-19 pandemic to index dimensions of stress specifically related to medical, relational, and social impact of the virus. This survey was unpublished. The survey asks a number of factual questions about potential stressors to oneself or “someone close to” the survey taker: ill, hospitalized, lost job, job requires exposure, increased responsibilities at home, difficulty with access to necessities of living. We summed the number of endorsements of oneself or of a close other to calculate a total COVID-19 stressor score.

In addition, three 5-point Likert style questions asked about overall amount of distress experienced related to the virus (1: no distress, 5: extreme distress), how many hours of exposure to news (1: none all, 5: more than two hours), and how much difficulty getting social support needed (1: no difficulty at all; 5: extreme difficulty). We treated these three Likert-style items independently as separate predictors in the model.

Free response

Patients, providers, and clinic staff answered the following question at baseline in a free-text format: “What stands out to you as the single biggest difference between telehealth visits and in-person visits?”

Data Analysis Strategy

Quantitative

We treated the two measures of modality preferences as complementary operationalizations and examined both independently. For each, we first assessed for two kinds of sampling bias to determine whether to account for them in the primary models: completion bias and no previous in-person visit at the center. For both questions, we worked from mixed effects modelling, which allowed for the modelling of random effects (e.g., timepoint or item) in order to examine fixed effects more accurately. We utilized both linear and logistic versions of these models as appropriate to the outcome variable. Fixed intercepts were used to test differences in outcome variables from zero, accounting for random effects and control variables. After exploring these primary models, we turned to exploring possible explanatory variables. We began with the Telehealth Usability Questionnaire as a predictor, proceeded with service usage variables, and ended with COVID-19 stressors. As these exploratory models progressed, we retained significant predictors of the outcome variable in following models. This allowed for a stepwise process that introduced new potential predictors accounting for previously discovered influential variables. All quantitative models were conducted in R[24]; MF) and independently verified in SPSS[25]; AGK).

Qualitative

Two study authors (HB and EF) independently reviewed the free-text responses taking note of emergent themes until thematic saturation was reached to inform a preliminary thematic framework. The two authors met to compare and discuss themes identified, and created a formalized structure of major and minor themes by which to code the entire dataset. The final version of the coding framework was reviewed and finalized with consultation from the primary investigators. The coders then independently coded the entire dataset using the formal framework and finally met to compare coding decisions, resolving discrepancies by consensus. The number of times each theme was identified was then tallied to inform the strength of that signal.

Results

Administrative Outcomes

Acupuncture and massage typically make up 28% of the visits that occur in the center. These visits were suspended during the initial three months of the pandemic shutdown since there was not a telehealth-based alternative available. All remaining kinds of visits (nurse practitioner, psychology, physical therapy, movement and all group classes) were transitioned to telehealth. Despite being unable to perform the 28% of visits comprised by acupuncture and massage, the center averaged 6% over budget for the final three months of the fiscal year and completed the year 4% ahead of budgeted visits due to the increased productivity of other disciplines.

At baseline, patients reported on average 3.04 (2.97) telehealth visits and 24.42 (SD = 80.08, Median = 10) previous in-person visits at the center (28.89%, n = 52, had no previous in-person visits at the center). Patients’ mean baseline TUQ score was 5.79 (SD = 1.30), averaging slightly below a recent sample of 100 otorhinolaryngology patients, T-Score = 46.6, 37th percentile.[26] At three months, patients’ mean TUQ score of 6.14 (SD = 1.16) was slightly higher than the mean of that same sample,[26] T-Score = 52.08, 58th percentile. Patients who completed the survey at both baseline and the three-month follow up scored significantly higher on the TUQ at follow up, t(49) = 2.35, p = .01.

According to the Coronavirus Stressor Survey, 39% of patients reported that neither they nor someone close to them experienced any of the pandemic-related stressful events in question, while 45.2% reported that they or someone close to them experienced 1–3 stressful events, 14.1% reported 4–6 stressful events, and 1.7% reported 7–9 stressful events.

Pre-Testing for Bias in Preference Measures

Follow-up completion bias

There were no group differences between participants who completed both the baseline survey and the three-month follow up and those who only completed the baseline survey found on scores on the TUQ, t(113.53) = 1.18, p = .24. Further, there were no group differences between those who completed both rounds of surveys and those who did not on average attitudes towards telehealth at baseline, t(128.04) = 0.76, p = .45. Overall, there did not appear to be a bias in sampling at follow-up based on mean-level differences of key variables at baseline.

No past in-person visit at the center

At baseline, some individuals (n = 51) began their treatment at the center after the complete transition to telehealth, thus did not have a previous in-person experience to compare against. These participants tended to judge telehealth services significantly higher than those who did have previous in-person visits at the center, B = .55 [0.16 – 0.95], SE = .20, t(712) = 2.76, p = .006. They also tended to choose telehealth over in-person in the forced choice question, though not significantly, OR = 1.88 [0.79 – 4.27], z(237) = 1.42, p = .16. Given the presence of some effects due to the presence of previous in-person visits, we included this dichotomous variable in all following models to account for potential bias.

Patient Modality Preferences

Patients rated telehealth as substantially better than in-person treatment across the three quality judgements accounting for the effect of having no previous in-person experience at the center, B = .77 [0.29 – 1.25], SE = .25, t(712) = 3.15, p = .002. There was likely either no change or even a slight increase in favorability of telehealth at three months, B = .27 [–0.03–0.57], SE = .15, t(712) = 1.76, p = .079. See Table 1 for a full report of these primary models of patient experience as well as final exploratory models of predictors.

Table 1.

Modelling Patient Preferences for Telehealth Visits Compared To In-person Visits.

	Judgements of Quality(positive = telehealth better than in-person)						Prefer Telehealth Visit Today(if equally safe)
	Primary model			Exploratory model			Primary model			Exploratory model
Predictors	B	SE	p	B	SE	p	OR	SE	p	OR	SE	p
Intercept	0.77	0.25	0.002	1.20	0.27	<0.001	0.72	0.24	0.165	1.34	0.40	0.459
3-month follow-up	0.27	0.15	0.079	0.12	0.14	0.367	2.48	0.42	0.029	1.80	0.49	0.231
Never in-person prior	0.55	0.20	0.006	0.45	0.19	0.018	1.84	0.43	0.155	1.21	0.53	0.721
TUQ score				0.73	0.06	<0.001				4.85	0.37	<0.001
Same county				–0.79	0.18	<0.001				0.28	0.49	0.009
Number of disciplines				0.28	0.07	<0.001
Random effects
σ2	2.43			1.41			3.29			3.29
τ00	2.53 Participant:Item			1.98 Participant:Item			1.55 Participant			0.89 Participant
	0.14 Item			0.16 Item
ICC	0.52			0.60			0.32			0.21
N	180 Participants			140 Participants			180 Participants			140 Participants
	3 Items			3 Items
Observations	718			566			241			190
Marginal R2/Conditional R2	0.011/0.529			0.258/0.705			0.036/0.344			0.520/0.622

At baseline, patients were about equally as likely to wish to continue with telehealth or continue with in-person if it were safe to do so (in-person: n = 86 [54%]; telehealth: n = 73 [46%]). At three months, these proportions shifted in favor of telehealth (in-person: n = 17 [40%]; telehealth: n = 26 [60%]). A mixed effects logistic regression model showed a large effect of timepoint allowing random intercepts by patients, OR = 2.48 [1.10 – 5.60], B = .91, SE = .42, z = 2.19, p = .029, such that patients were more likely to prefer telehealth over in-person at follow-up.

Exploring Predictors

We built a mixed-effects model assessing data from both timepoints, including the main effect of timepoint, and allowing for random patient-level effects. The TUQ predicted the judgements of telehealth relative to in-person. We then added service usage variables to the model in the next step. We found that residing in the rural areas outside of the urban county where our clinic is located, and the more kinds of disciplines experienced, the more favorable telehealth was rated. Preference for telehealth does not appear to be due to familiarity with the medium as the number of telehealth visits did not predict judgement. No COVID-related variables independently predicted judgements of telehealth relative to in-person visits. In the exploratory model, timepoint became less influential. It appeared the slight increase in telehealth favoriability at three months was better explained by changes in other variables, such as the TUQ. See Table 2 for a full report of the final exploratory model.

Table 2.

Thematic Analysis of Responses to “What Stands Out to You as the Single Biggest Difference Between Telehealth Visits and In-person Visits?”

Theme	Illustrative Quote	Signal Strength
Improved ease of completing visits	“In-person visits are anxiety-producing for I must secure transportation for a 60-mile one way trip”“Makes it easier to fit into my schedule. I don’t have to take my [children] with me or have someone miss work to watch them”	116
Telehealth as acceptable (adequate, equal, or superior replacement)	“They are identical in quality and much more convenient”“Quality of the visits without the stress of getting there”	77
Telehealth impact on interpersonal connection		73
Negative	“The human connection in-person is something I miss” “I will always prefer the in-person classes for all the softer, more subtle reasons that humans enjoy and thrive with human contact”
Positive	“I felt that the telehealth visits were very focused. There was nothing to distract either of us” “Being in my own environment I feel more comfortable and confident so I feel that I am being more my authentic self”
Technology difficulties	“Just seems like a lot could go wrong since my internet connection isn’t always reliable” “It’s difficult to know when to speak because of the time delay”	33
Perspectives on the translation to telehealth of movement/manual work and group classes	“For Tai Chi, you don’t feel the presence, unity of the other participants” “If you don’t have a big monitor it can be difficult to follow” “Being able to see myself on my screen is helpful and novel for me, as we do not have mirrors in the movement room”	25
Telehealth improves comfort	“It has removed so many layers of stress and anxiety, which in turn relieves physical pain” “Very comfortable in my own home knowing I can have my appointments done where I’m most comfortable” “Much easier with chronic pain not to have to get out and park and check in. I prefer doing teletherapy as much as possible!” “I have to do less tiring things to get to an appointment about having chronic fatigue. This makes it easier and also less likely that I’d have to reschedule an appointment or push myself too hard on a bad day”	20
Telehealth as a means to improve equity/ expand access	“The building is not set up for handicapped persons, [regarding] parking and accessible doorways” “I’ve not been able to get to a class in-person due to timing/commute/parking issues. Telehealth and quarantine have allowed me to finally attend!”	11

Note. Signal strength indicates the number of participants who communicated that theme.

Going through the same exploratory process, the TUQ and residing in the same county emerged as predictive of preference between telehealth or in-person visits, with those scoring higher on the TUQ and outside of the county more likely to choose telehealth (Table 1). Similar to the exploratory model of judgements of telehealth versus in-person visits, the effect of timepoint disappeared in the exploratory model, appearing to be better explained by these variables. Number of provider types was the other variable not internally replicated with this outcome measure from the final exploratory model of judgements.

Comparison Preferences Across Patients, Providers, and Staff

There were significant differences in judgements of telehealth to in-person treatment across different treatment roles at Osher (patient or provider). Providers judged telehealth worse than patients, B = –1.00 [–1.56 – –0.44], SE = .29, t(799) = –3.48, p < .001. While providers certainly did not share in the same favorable judgement of telehealth held by patients across timepoints,, they also did not appear to obviously prefer one modality over the other. Administrative staff were only asked their overall impression of the clinical process for telehealth. Their ratings were marginally more favorable to telehealth than the patients, B = 1.20 [–0.09 – 2.49], SE = .66, t(799) = 1.82, p = .07. See Figure 1 for a depiction of these estimated effects removing the increase in preference for telehealth at follow-up and the increase among those who had never been in-person at the center previously (patients only). Thus, Figure 1 presents conservative estimates of the preference for telehealth, underestimating the overall favorability of telehealth in the data.

Figure 1.

Modelled Ratings of Telehealth Visits As Compared To In-Person Visits by Stakeholder Group.

Note. This figure shows modelled estimates of participant-level ratings by stakeholder group controlling for timepoint and whether or not had a previous in-person visit at the center (patients only)with 95% CI. Staff only completed the overall judgement of quality of telehealth versus in-person visits.

A difference between providers and patients emerged in the forced choice question. Patients were more likely than providers to choose to have telehealth visits if it were safe to do so, OR = 4.03 [1.02 – 18.20], z = –1.98, p = .048. Overall, rate of selecting telehealth at follow-up increased across both providers and patients, OR = 2.69 [1.18 – 6.14], z = 2.36, p = .018.

Qualitative Findings

Participants described telehealth as an acceptable and adequate, at times equal or superior, alternative to in-person visits, especially during a pandemic (Table 2). Telehealth improved comfort for patients who described being able to hold their visits in the place they are most comfortable - at home. In the words of one patient, “It has removed so many layers of stress and anxiety, which in turn relieves so much physical pain…. Being in my own environment I feel more comfortable and confident so I feel that I am being my more authentic self.” Relatedly, there was a strong emphasis on the ways telehealth improved ease of completing visits through avoiding long commutes and difficulty parking while saving gas and time, removing the need for childcare, and the flexibility allowing folks to participate in more visits around their work schedule.

Shortcomings of telehealth as a visit modality were also well-described. Inevitably, internet connectivity and technical issues were described. Many participants commented on the loss of human connection and physical presence, including diminished interpersonal rapport (e.g., unable to read non-verbal communication) and community (e.g., loss of unity). In contrast, a minority of participants felt that telehealth actually improved their relational connection allowing deeper listening and better focus.

A small number of providers felt that the virtual modality was draining or exhausting, taking away the ritual and fulfillment that hands-on care provides. At the same time, providers commented on telehealth as an important mechanism by which to improve equity and remove barriers for folks who may not otherwise be able to access needed services: “I am happy to provide [it as] an option for people who [could] not receive care otherwise, or be in a lot of pain driving to us.”

Responses were mixed on the success of translation of physical movement or group classes (e.g., physical therapy, Tai Chi) to a virtual platform. While some described being unable to follow the movements as well on a small screen or commented on the loss of manual body work, others felt they could see both the instructor and their own movements better on video. Many participants felt that moving forward they would be happy for health coaching or psychotherapy counseling visits to continue via telehealth with manual therapies (physical therapy, massage, acupuncture) being offered in-person.

Discussion

We conducted a mixed-methods survey of patients, providers, and administrative staff on their experience of telehealth in our clinic after a rapid and complete implementation of telehealth services. In this report, patient experience favored telehealth over in-person visits across two sets of measures. Qualitative findings provide rich descriptive details on the ways patients experienced telehealth visits compared to in-person visits. Ease of completing visits was particularly compelling as the center, like many integrative medicine practices, cares for patients with chronic pain, chronic fatigue, or stress-related conditions for whom the comfort and safety of one’s own home offers an ideal environment for healing work. At three months, the favorability of telehealth over in-person visits remained stable and even expanded with the majority of patients indicating they would choose telehealth visits over in-person visits, if they were equally safe.

Across two converging measures of judgements and preferences, the usability of telehealth services and geographic proximity to the center predicted experiences favorable to telehealth for patients. Neither practice with telehealth through increased visits nor stressors related to COVID-19 pandemic predicted patient experience. Providers, on the other hand, were mixed in their experience of telehealth. They experienced it less favorably, but not better or worse than in-person visits. We compared administrative staff to these two groups in terms of overall judgement of clinical process, which strongly favored telehealth.

Limitations

Not all patients sent a survey completed one. This preserves the possibility that those filling this survey out had stronger positive or negative feelings about telehealth. This survey was conducted during the first months of the COVID-19 pandemic. Thus, it can only reflect the attitudes during that time. However, we did not find a relationship between preference for telehealth and pandemic related stress. An additional limitation was our lack of individual-level demographics in the dataset. Use of an unpublished survey on the impact of COVID-19 on stress limits the validity of this measure. While this survey is face valid, related results should be interpreted with due caution.

Policy Implications

Telehealth is a potential tool that, until recently, has not been widely utilized clinically nor supported systemically to address access needs. One unexpected impact of the COVID-19 pandemic was to categorically remove systemic barriers to telehealth (namely, parity in insurance reimbursement) and force clinics such as ours to rapidly implement technology-assisted modalities for patient care.

Our findings support the claim that telehealth-based treatment is acceptable, appropriate, feasible, sustainable, and at times, preferable. If barriers to telehealth return after the pandemic, patient preferences will likely not be found among them. Furthermore, policy that does not support telehealth for outpatient integrative medicine cannot do so under the name of patient preference, perceptions of quality, patient choice, or access. Given that the largest predictor of telehealth quality was the quality of the technology medium, our findings suggest that developing more reliable and easier teleconferencing technology and internet access becomes a matter for healthcare policy, particularly for those living outside of urban centers.