Experiences with Telemedicine for HIV Care During the COVID-19 Pandemic: A Mixed-Methods Study.

To characterize perspectives and experiences with telemedicine during the COVID-19 pandemic, we conducted a mixed-methods study in two HIV clinics in the US Northeast. Among surveyed patients with HIV (PWH) who had a telemedicine appointment (n = 205), 42.4% perceived telemedicine visits as useful during the pandemic. PWH and clinical staff identified benefits of telemedicine: (1) ability to engage and re-engage patients in care; (2) perceived patient-centeredness and flexibility; (3) opportunity to engage family and multidisciplinary care team members; and (4) opportunity to enhance telemedicine use proficiency through practice and support. Identified barriers included: (1) technical challenges; (2) privacy concerns; (3) loss of routine clinical experiences and interactions; (4) limited objective patient remote monitoring; and (5) reimbursement concerns. Efforts to optimize telemedicine for HIV care should consider strategies to improve technology support for PWH, flexible options to access care, additional platforms to allow patient remote monitoring, and appropriate billing and reimbursement methods.

Introduction

SARS-CoV-2 virus and the COVID-19 disease it causes has created an unprecedented global health emergency [1], resulting in a major transformation of the healthcare system. To ensure HIV care continuity during the pandemic, many clinics have rapidly deployed or extensively expanded telehealth. Defined as “the use of electronic information and telecommunication technologies to support and promote long-distance clinical health care, patient and professional health-related education, public health, and health administration,” [2] telehealth offers a variety of modalities (e.g., video or telephone-based communications, patient portal) and functions (e.g., medication adherence monitoring) [3, 4].

Telemedicine, a subset of telehealth focused on providing clinical services [2], has been applied in the context of HIV care to increase access to quality, timely healthcare; reduce patient travel burden; alleviate stigma; and improve cost-effectiveness [3, 5–7], with high patient satisfaction rates [8, 9], acceptability [10], improved medication adherence [11], and HIV viral suppression [12, 13]. Literature on the usage of telemedicine in HIV care during the COVID-19 pandemic is still evolving. Data captured early in the pandemic indicated positive reception among patients with HIV (PWH) toward telemedicine [14-16], with several challenges including patients' lack of access to technology and private space, the need for technical training among staff, and limited physical examination [15, 17, 18]. These studies are limited by their small numbers of patient participants and a single-site approach [10, 14–16, 18]. A recent policy statement by the Infectious Diseases Society of America and the HIV Medicine Association underscores the importance of developing interventions that promote digital health equity [19]; however, further data are lacking to support providers and programs’ understanding on how, exactly, to do so.

Thus, the goal of this exploratory study was to generate timely insights regarding experiences with telemedicine for HIV care from perspectives of patients and clinical staff at two urban settings where telemedicine was rapidly scaled up due to COVID-19. These findings may serve to directly inform efforts to optimize telemedicine in the context of HIV care.

Methods

Study Overview

Using a concurrent triangulation mixed-methods design [20], we conducted a survey and focus groups involving patients and clinical staff at two large urban HIV clinics in the US Northeast. We collected data from May 15, 2020 through August 11, 2020, shortly after the initial peak of the COVID-19 pandemic in these regions. Informed by the RE-AIM implementation science framework [21, 22], we broadly sought to understand how telemedicine was able to reach patients and its perceived effectiveness for delivering routine care to PWH. Patients were recruited to complete a one-time telephone-based survey to assess experiences with the pandemic and its impact on their health and access to care. In parallel, we invited clinical staff to participate in a virtual focus group and in a brief web-based survey to collect data on demographics and clinical roles. The study protocol was approved by institutional review boards at Yale University and SUNY Downstate Health Sciences University and was HIPAA compliant.

Settings, Participants and Procedures

This study was conducted in New Haven, Connecticut and Brooklyn, New York in the context of existing research collaborations [23, 24] focused on reducing tobacco use among PWH [23]. We therefore, actively recruited PWH with current tobacco use by electronic medical record (EMR) documentation. In addition, we recruited a subset of patients without current tobacco use from the New Haven-based clinic. Additional inclusion criteria were: (1) ≥ 18 years of age, (2) English-speaking, and (3) able to give informed consent. We sought to equally sample younger and older patients given our expectations that the COVID-19 pandemic may have differential impacts by age due to varying levels of social isolation, comorbidities, and technology access [25-28]. Clinical staff from these clinics were invited to participate to gain insights from a diversity of providers. All participants provided verbal informed consent for participation in research activities and were provided a $30 gift card for study participation.

During the study period, patients at the New Haven-based clinic who were due for a visit either made a telemedicine appointment through the Epic-based MyChart patient portal or were contacted by clinic staff to schedule a telemedicine visit. Clinicians at this site had the option to conduct telemedicine visits via a video conference link within MyChart or by directly calling the patient's phone number for an audio-only visit. At the Brooklyn-based site, telemedicine visits were similarly conducted in this flexible manner. Patients were contacted directly by clinic staff or could contact the clinic themselves to schedule a telemedicine appointment, which occurred either as a video visit through the DoxyMe platform or by phone for an audio-only visit.

Data Collection

We collected data from the two groups of patient and provider stakeholders in parallel; data collection took place between May 15, 2020 and August 11, 2020.

Patient Surveys

The “COVID-19 and PWH Survey” (see Supplementary Survey) was administered by telephone by research staff. Data were collected and managed using Research Electronic Data Capture (REDCap) system hosted at Yale University [29, 30]. The survey was informed by the work of Wolf and colleagues [31] and was modified based on multidisciplinary input from HIV clinicians, researchers, and a community partner. Validated measures were incorporated as applicable [32-34]. We collected data on several domains, including sociodemographic characteristics, COVID-19 related experiences and impacts, and experiences with and preferences regarding telemedicine. Participants were asked about recent completion of telemedicine visits based on experiences in the 2 months prior to survey date.

Outcomes of Interest

To assess patient perceptions of the utility of telemedicine, we examined responses to the following items: 1. “How useful have your medical appointments with telemedicine/telehealth been during the COVID-19 emergency?” and 2. “How useful do you think it will be to have medical appointments with telemedicine/telehealth after the COVID-19 emergency is over?” Participants were asked to rate responses using a 5-point Likert scale; responses were dichotomized as “useful” (“extremely” or “very”) vs. “not useful” (“moderately,” “slightly,” or “not at all”). Additionally, in post-hoc sensitivity analyses, we expanded the definition of “useful” to include “moderately” (“extremely,” “very,” or “moderately”) vs. “not useful” (“slightly” or “not at all”). Participants were then prompted to provide free-text comments that research staff documented verbatim.

Independent Variables

Participant characteristics included demographics (age, race, ethnicity, gender identity); use of and adherence to HIV medications; depression [33] and anxiety [34] symptoms; and telemedicine experiences. Most recent HIV biomarkers (CD4 cell count, presence of a detectable HIV viral load defined as > 50 copies/mL), and engagement in HIV care based on documentation of at least one HIV clinic visit in the 6 months prior to survey date consistent with current standards [35] were extracted from the EMR.

Clinical Staff Focus Groups

Using the cloud-based Zoom© videoconferencing platform, members of the investigative team conducted two focus groups with each site (n = 4 total). Participants were invited by local study team members via emails. Grand tour (i.e., broad questions) and follow-up probes were used to assess experiences using telemedicine during the COVID-19 pandemic to provide care to PWH (see Supplementary Focus Group Guide). Focus groups were digitally recorded and transcribed and then reviewed by a member of the research team for accuracy and de-identification. Upon focus group completion, participants were asked to complete a brief REDCap-based survey regarding their demographics and clinic role.

Data Analyses

Descriptive statistics were used to characterize participant characteristics and patient telemedicine experiences during the pandemic. Univariable and multivariable logistic regression models were built to examine associations between patient characteristics and patient ratings on how useful telemedicine visits were (among those who had completed a telemedicine visit during the pandemic only) and perceptions of all patient participants on how useful it will be to have telemedicine after the pandemic is over. Variables of interest were selected a priori to include in the multivariable models, namely age, smartphone ownership, and site. Additionally, variables that were significant at p < 0.05 in unadjusted analyses were also included in the multivariable models. Statistical analyses were performed using SAS version 9.4 software (Copyright© 2013, SAS Institute Inc., Cary, NC, USA). Open-ended responses to these items were analyzed using content analysis [36] (described further below).

We used a rapid assessment process [37] followed by an inductive process of iterative coding to identify themes using content analysis [36]. Members of the investigative team independently reviewed each transcript line-by-line to develop and refine the codebook and reach consensus on codes. Themes were then generated based on coded quotations and discussion by two investigators initially and then with input from the broader research team.

Data Integration

Analyses of survey data were done in parallel with qualitative focus group data. Themes developed based on focus groups were then merged with responses from the open-ended patient survey items.

Results

Quantitative Results

Patient Participants Sociodemographic Characteristics

Among a random sample of 755 patients who met the eligibility criteria, we attempted to contact 719 patients, among whom 40.2% (n = 289) were not successfully contacted and 20.4% (n = 147) declined participation. The most common reasons for declining were refusal (49%) and not enough time/too busy (25.2%). A total of 283 patients consented to participate. Missing responses (n = 10) were observed during data cleaning and therefore excluded from the analysis. The sociodemographic characteristics of the patients who completed all sections in the telephone survey (n = 273) are described in Table 1.

Table 1

Participant characteristics

Characteristics	Patients, N = 273*	Clinical staff, N = 23
Site, n (%)	273 (100%)	23 (100%)
New Haven, CT	173 (63.4%)	14 (61%)
Brooklyn, NY	100 (36.6%)	9 (29%)
Age, mean (SD)	52.1 (11.3)	49.2 (13.2)
Current gender, n (%)
Female	132 (48.4%)	20 (86.9%)
Male	141 (51.6%)	2 (8.7%)
Nonbinary, agender, genderqueer	0	1 (4.3%)
Race, n (%)
White	61 (22.3%)	10 (43.5%)
Black	178 (65.2%)	4 (17.4%)
Other	34 (12.5%)	9 (39.1%)
Hispanic, n (%)	43 (15.8%)	1 (4.3%)
Job role, n (%)
Behavioral health provider		4 (17.4%)
Physician		11 (47.8%)
Advanced practice practitioner		1 (4.3%)
Clinical pharmacist		1 (4.3%)
Nurse		2 (8.7%)
Other (e.g., program director)		4 (17.4%)
Providing direct services to patients, n (%)		20 (86.9%)
Able to work from home (telecommute), n (%)		20 (86.9%)
Using telemedicine to deliver care, n (%)		18 (78.3%)
If using telemedicine (n = 18), location to deliver telemedicine, n (%)
Home		3 (16.7%)
Clinic/hospital		4 (22.2%)
Both		11 (61.1%)
Occupation, n (%)
Employed	72 (26.4%)
Unemployed/disabled/retired/other†	201 (73.6%)
Annual household income, n (%)
$0 to $25,000	175 (65.3%)
$25,000 or more	93 (34.7%)
Housing status impacted by COVID-19, n (%)
No, lived in the same place	257 (94.2%)
No, still did not have a regular place to stay	5 (1.8%)
Yes, had to move but have a place to stay	9 (3.3%)
Yes, no longer have a place to stay	2 (0.7%)
HIV related variables
Engaged in recent HIV care, n (%)	241 (88.3%)
Undetectable HIV viral load‡, n (%)	206 (75.5%)
CD4 cell count, cells/mm3, median (IQR, Q1–Q3)	603.5 (532.4, 395.2–927.6)
Prescribed ART, n (%)	266 (97.4%)
Took ART at start of COVID-19 pandemic§, n (%)	271 (99.3%)
Missed any ART during COVID-19 pandemic, n (%)	51 (18.8%)
Substance use and mental health
Current tobacco use, n (%)
Every day	199 (72.9%)
Some days	34 (12.5%)
Not at all	40 (14.7%)
Unhealthy alcohol usea, n (%)	79 (28.9%)
Depressionb, n (%)
None to slight/mild	254 (93%)
Moderate/severe	19 (7%)
Anxietyc, n (%)
None to slight/mild	215 (78.8%)
Moderate/severe	58 (21.2%)
Number of people in household (including self), n (%)
1	99 (36.3%)
2	84 (30.8%)
3	44 (16.1%)
4 or more	46 (16.8%)
Owned a smartphone, n (%)	186 (74.4%)
Could use a smartphone for videoconferencing, n (%)	178 (96.2%)
Could use a smartphone for the internet, n (%)	181 (97.3%)
Had a telemedicine appointment in past two months, n (%)	205 (75.1%)

aAUDIT-C (Alcohol Use Disorders Identification Test)

bPROMIS Short Form v1.0—Depression 4a

cPROMIS Short Form v1.0—Anxiety 8a

*Numbers do not add up to 100% due to small amounts of missing data

†Two participants who indicated “other” provided additional information that they were students

‡ ≤ 50 copies/L

§Two participants indicated “sometimes” and 269 indicated “yes”

Patient Smartphone Access and Telemedicine Experiences

Among patient participants who owned a smartphone (74.4%, n = 186), the majority reported that they were able to use their smartphone for videoconferencing (96.2%) and accessing the internet (97.3%). Most patient participants had a telemedicine appointment (75.1%, n = 205) in the 2 months prior to the survey (Table 2). Among these patients, most had a telemedicine phone visit (85.9%) with fewer reporting telemedicine with video (7.3%) or both telephone and video (6.8%). The majority of telemedicine visits were with the participant’s HIV provider (i.e., MD, PA, APRN) (66.7%, n = 182). Other telemedicine visits were with a specialist/consultant (14.7%), behavioral health provider (i.e., social worker, counselor) (10.3%), nurse (4%), pharmacist (3.7%), and nutritionist (1.5%). With regard to reasons for telemedicine use, most patients (93.7%, n = 192) reported having a telemedicine visit in place of a routine face-to-face appointment. Others indicated their use of telemedicine to address other symptoms/urgent issue (5.4%) and possible COVID-19 symptoms (1%).

Table 2

Patient participant experiences and preferences for telemedicine use during the COVID-19 pandemic

Variable	Overall
IIa. Experiences with telemedicine among patients participating in a telemedicine visit during the pandemic, N = 206
Had telemedicine visit via…
Telephone	177 (85.9%)
Video	15 (7.3%)
Both	14 (6.8%)
Had telemedicine visit with a…a
HIV provider (MD, PA, APRN)	182 (66.7%)
Specialist/Consultant	40 (14.7%)
Pharmacist	10 (3.7%)
Nutritionist	4 (1.5%)
Nurse	11 (4%)
Behavioral health (social worker, counselor)	28 (10.3%)
Other†	10 (3.7%)
Reason for telemedicine visit
In place of a routine face to face appointment	192 (93.7%)
For possible COVID symptoms	2 (1%)
For other symptoms/urgent issue	11 (5.4%)
IIb. Preferences for telemedicine use among all patients, N = 273
Have any of your future medical appointments been rescheduled to telemedicine/telehealth?
Yes	57 (21.0%)
No	140 (51.5%)
Don’t know	75 (27.6%)
How important is it to be able to have routine medical appointments with telemedicine/telehealth during the COVID emergency?
Not at all important/Slightly important	61 (22.3%)
Somewhat important	39 (14.3%)
Important/Very important	173 (63.4%)
How important is it to be able to have urgent medical appointments with telemedicine/telehealth for symptoms (cough, fever, other) during the COVID emergency?
Not at all important/Slightly important	52 (19.0%)
Somewhat important	35 (12.8%)
Important/Very important	186 (68.1%)

aParticipants were asked to check all that apply

†Ten participants who indicated “other” had telehealth visits with primary care providers (n = 3), care navigators (n = 2), care coordinator (n = 1), pain medicine (n = 1), home attendant (n = 1), physical therapist (n = 1), and dentist (n = 1)

Overall, telemedicine visits were viewed favorably: 74.3% (n = 153) felt that their telemedicine visit was better than/just as good as a traditional (face-to-face in person) visit (Fig. 1), and 67.5% (n = 139) reported that they probably/definitely would recommend telemedicine to someone else (Fig. 2). Approximately two-thirds of patients felt that it was important to be able to have routine medical visits (63.4%, n = 173) or urgent medical appointments (68.1%, n = 186) via telemedicine during the COVID-19 pandemic (Table 2).

Fig. 1

Perceived quality of telemedicine visit compared to in-person visit by patients participating in telemedicine during the COVID-19 pandemic

Fig. 2

Perceived likelihood of recommending telemedicine to someone else by patients participating in telemedicine during the COVID-19 pandemic

Patient Perceptions Regarding Telemedicine Usefulness During and After the COVID-19 Pandemic

Among patients who engaged in telemedicine visits (n = 205), 42.4% of patients felt that telemedicine visits were useful during the pandemic (Table 3). In bivariate analyses, patients who had not recently been engaged in care were more likely to report that telemedicine was useful during the pandemic than patients who had recently been engaged in care (OR 2.64 [CI 1.05 to 6.60]; p = 0.04). In addition, patients who did not own a smartphone were less likely to report that telemedicine was useful during the pandemic than patients with a smartphone (OR 0.27 [CI 0.13 to 0.59]; p < 0.001). After adjusting for age, site, smartphone ownership, and recent engagement in care, not having a smartphone was associated with lower odds of finding telemedicine useful during the pandemic (OR 0.29 [CI 0.13 to 0.67]; p < 0.01). There was no significant difference in perception of telemedicine usefulness during the pandemic by age, site, or recent engagement in care (all p values > 0.05).

Table 3

Patient participant characteristics associated with telemedicine being useful during the COVID-19 pandemic, N = 205§

Variable	How useful have your medical appointments with telemedicine/telehealth been during the COVID emergency?		Unadjusted odds ratio (95% CI)	Adjusted odds ratio† (95% CI)
	Extremely/very	Moderately/slightly/not at all
Age§§	53.0 (10.1)	51.5 (11.7)	1.13 (0.88–1.46)	1.22 (0.91–1.63)
Current gender‡
Female	45 (51.7%)	55 (46.6%)	1.23 (0.71–2.14)
Male	42 (48.3%)	63 (53.4%)	Ref
Race
White	21 (24.1%)	27 (22.9%)	Ref
Black	57 (65.5%)	76 (64.4%)	0.96 (0.50–1.88)
Other	9 (10.3%)	15 (12.7%)	0.77 (0.28–2.11)
Ethnicity
Hispanic	12 (14%)	18 (15.3%)	0.90 (0.41–1.99)
Non-Hispanic	74 (86%)	100 (84.7%)	Ref
Annual household income
$0 to $25,000	60 (71.5%)	72 (61.5%)	Ref
$25,000 or more	24 (28.5%)	45 (38.5%)	0.64 (0.35–1.17)
Employment change due to COVID-19 outbreaka
Changed	16 (18.4%)	20 (16.9%)	1.10 (0.54–2.28)
Unchanged	71 (81.6%)	98 (83.1%)	Ref
Engaged in recent HIV care
No	14 (16.1%)	8 (6.8%)	2.64 (1.05–6.60)*	2.03 (0.74–5.61)
Yes	73 (83.9%)	110 (93.2%)	Ref	Ref
Undetectable HIV viral load
No	10 (11.5%)	17 (14.4%)	0.77 (0.33–1.78)
Yes	77 (88.5%)	101 (85.6%)	Ref
Missed any ART during COVID-19 pandemic
No	67 (77.9%)	102 (86.4%)	Ref
Yes	19 (22.1%)	16 (13.6%)	1.81 (0.87–3.76)
Owned a smartphone
No	10 (12.3%)	36 (34%)	0.27 (0.13–0.59)***	0.29 (0.13–0.67)**
Yes	71 (87.7%)	70 (66%)	Ref	Ref
Site location
New Haven, CT	48 (55.2%)	84 (71.2%)	Ref	Ref
Brooklyn, NY	39 (44.8%)	34 (28.8%)	2.01 (1.12–3.59)*	1.10 (0.55–2.22)

Statistically significant values are shown in bold

§One participant was excluded from analysis due to missing responses

§§In 10-year increments

*P value less than 0.05

**P value less than 0.01

***P value less than 0.001

†Adjusted for age, recent engagement in HIV care, smartphone ownership, and location

‡ Four participants indicated that their current gender was different from their assigned gender at birth

aThe survey question was “Has there been any change in your employment status due to coronavirus?”

Among all surveyed patients (n = 273), 27.5% indicated that telemedicine appointments would be useful after the COVID-19 pandemic was over (Table 4). In bivariate analyses, a 10-year increase in patient age was associated with higher odds of finding telemedicine useful after the pandemic (OR 1.31 [CI 1.02 to 1.69]; p = 0.03). Additionally, patients who were in Brooklyn, NY were more likely to endorse that telemedicine would be useful after the pandemic than patients who were in New Haven, CT (OR 1.96 [CI 1.14 to 3.38]; p = 0.02). In multivariable analyses, a 10-year increase in age remained to be significantly associated with higher odds of finding telemedicine useful after the pandemic (OR 1.38 [CI 1.05 to 1.81]; p = 0.03). Owning a smartphone and site were not associated with the odds of finding telemedicine useful after the pandemic (all p values > 0.05). In the sensitivity analyses (see Supplementary Table VI and Table VII), only age was associated with telemedicine being useful during the COVID-19 pandemic in bivariate (OR 1.52 [CI 1.13 to 2.06]; p < 0.01) and multivariable analyses (OR 1.59 [CI 1.13 to 2.22]; p < 0.01), but not after the pandemic.

Table 4

Patient participant characteristics associated with telemedicine being useful after the COVID-19 pandemic, N = 273

Variable	How useful do you think it will be to have medical appointments with telemedicine/telehealth after the coronavirus emergency is over?		Unadjusted odds ratio (95% CI)	Adjusted odds ratio† (95% CI)
	Extremely/very	Moderately/slightly/not at all
Age§	54.5 (9.7)	51.3 (11.7)	1.31 (1.02–1.69)*	1.38 (1.05–1.81)*
Current gender‡
Female	40 (53.3%)	91 (46.2%)	1.33 (0.78–2.27)
Male	35 (46.7%)	106 (53.8%)	Ref
Race
White	12 (16%)	49 (24.9%)	Ref
Black	58 (77.3%)	119 (60.4%)	1.99 (0.98–4.03)
Other	5 (6.7%)	29 (14.7%)	0.70 (0.23–2.20)
Ethnicity
Hispanic	11 (14.7%)	32 (16.3%)	0.88 (0.42–1.85)
Non-Hispanic	64 (85.3%)	164 (83.7%)	Ref
Annual household income
$0 to $25,000	50 (68.5%)	125 (64.4%)	Ref
$25,000 or more	23 (31.5%)	69 (35.6%)	0.83 (0.47–1.48)
Employment change due to COVID-19 outbreaka
Changed	9 (12%)	36 (18.3%)	0.61 (0.28–1.34)
Unchanged	66 (88%)	161 (81.7%)	Ref
Engaged in recent HIV care
No	10 (13.3%)	22 (11.2%)	1.22 (0.55–2.72)
Yes	65 (86.7%)	175 (88.8%)	Ref
Undetectable HIV viral load
No	13 (17.3%)	33 (16.8%)	1.04 (0.52–2.11)
Yes	62 (82.7%)	164 (83.2%)	Ref
Missed any ART during COVID-19 pandemic
No	55 (75.3%)	164 (83.2%)	Ref
Yes	18 (24.7%)	33 (16.8%)	1.63 (0.85–3.12)
Owned a smartphone
No	14 (20.3%)	50 (27.6%)	0.67 (0.34–1.31)	0.80 (0.39–1.66)
Yes	55 (79.7%)	131 (72.4%)	Ref	Ref
Site location
New Haven, CT	39 (52%)	134 (68%)	Ref
Brooklyn, NY	36 (48%)	63 (32%)	1.96 (1.14–3.38)*	1.78 (0.97–3.27)

Statistically significant values are shown in bold

§In 10-year increments

*P value less than 0.05

†Adjusted for age, smartphone ownership, and location

‡Four participants indicated that their current gender was different from their assigned gender at birth

aThe survey question was “Has there been any change in your employment status due to coronavirus?”

Clinical Staff Characteristics and Telemedicine Experiences

Four focus groups were conducted with 23 clinical staff, including physicians (47.8%), behavioral health providers (17.4%), nurses (8.7%), other clinical staff (17.4%), advanced practice practitioners (4.3%), and clinical pharmacists (4.3%). Their characteristics are also reported in Table 1. Most focus group participants reported that they were able to telecommute (86.9%, n = 20) and use telemedicine to deliver care during the pandemic (78.3%, n = 18). Among those who provided telemedicine services, more than half (61.1%) conducted the visits from both home and clinic/hospital.

Qualitative Findings Regarding Key Factors Affecting Telemedicine Utilization

Five major themes emerged from the qualitative analysis of the focus groups transcripts with clinical staff and two open-ended items in the patient survey asking about telemedicine usefulness during and after the COVID-19 pandemic that served to identify barriers and facilitators to implementing telemedicine for HIV-related care (see Supplementary Table V for illustrative quotes).

Theme 1. Telemedicine was Essential for Staying Connected During the COVID-19 Pandemic, Re-engaging Patients in Care, and Developing Patients’ New Skills to Maintain Continuity of Treatments

Clinical staff discussed how they perceived telemedicine as helpful to connect with patients and mitigate patients’ feelings of isolation and fears during the pandemic, and reduce no-show rates. Importantly, telemedicine during the pandemic offered an opportunity to re-engage individuals who were out of care as well as discuss new opportunities to maintain health that may have been previously de-prioritized. Telemedicine visits also allowed clinical staff to teach patients new skills, particularly self-administration of injectable medications, to avoid treatment interruptions. Patients similarly described the benefits of telemedicine including to keep their appointments with providers and address pandemic-related concerns.

Theme 2. Telemedicine Visits were Perceived to be Patient-Centered, Since Visits Occurred at Times and in Settings That were More Accessible for Patients, Though Lack of In-Person Interaction and Perceived Increased Workload Added Challenges for Clinical Staff

The ease of scheduling and flexibility were some of the main advantages of telemedicine expressed by clinical staff and patients. Clinical staff expressed the importance of meeting patients where they are through telemedicine, and patients underscored that telemedicine reduces burden of travel with minimal disruption of their daily routines. Clinical staff and patients described concerns about decreased in-person contact in telemedicine and limited non-verbal cues, especially in telephone visits, compared to face-to-face visits. In particular, behavioral care providers stated that limited non-verbal communication made it more difficult to assess patients' emotional presence. Some patients reflected on their telemedicine experience as impersonal and rushed. Additionally, while acknowledging the benefits of telemedicine for patients, some providers perceived added workloads and time demand to conduct telemedicine visits and follow-up.

Theme 3. Technical Challenges to Implementing Telemedicine were Common, Including Connection Issues, Lack of Access to Smartphones and Internet, and Inconsistent Use of Virtual Meeting Platforms, but were Overcome with Practice, Resource Allocation, and Innovative Solutions

Clinical staff and patients reported technology-related challenges as common barriers to telemedicine participation, including lack of access to the required technologies such as smartphones, and skills and confidence in using the telemedicine platforms. Among clinical staff, not having a dedicated work phone for telemedicine and inconsistent use of the videoconferencing platforms were described as challenges during the initial phase of telemedicine implementation. Providers also acknowledged that patients had varying levels of technology literacy and access. Connection issues during telemedicine visits caused discomfort and was a hindrance to telemedicine participation for some patients. To overcome the challenges, providers spoke about regular practice to develop proficiency and patient outreach to troubleshoot. These solutions were perceived to improve the quality of the telemedicine experience.

Theme 4. While Telemedicine Generated Privacy Concerns, it Also Facilitated an Opportunity to Engage Family Members and Other Members of the Care Team in Healthcare Delivery

Clinical staff reported challenges with ensuring privacy and confidentiality related to telemedicine describing their experience of conducting telemedicine visits from home and navigating disruptions in their living spaces. Some patients reflected on the difficulty of speaking about personal information during telemedicine visits. On the other hand, providers noted that they were able to more easily engage patients' family members and other members of the care team to deliver patient-focused care.

Theme 5. There was Consistent Support for Maintaining Telemedicine After the COVID-19 Pandemic, Particularly with Appropriate Reimbursement, Staffing Support to Help Patients Access Technology Effectively, Assessment of Patient Preferences, and Expansion to Incorporate Additional Platforms for Patient Remote Monitoring

Overall, clinical staff and patients expressed support for telemedicine use for HIV care after the COVID-19 pandemic is over. Providers believed that telemedicine has made physical and behavioral health care more accessible to patients. Provision of these services should continue with appropriate reimbursement and staffing support to assist patients in accessing the technology effectively. Patients and providers emphasized the importance of assessing patient preferences and needs for telemedicine or in-person visits to improve acceptability and care continuity. Patients expressed their appreciation to telemedicine use for routine visits with their providers, particularly for individuals who are older, with mobility issues, or unable to travel. However, patients would also like to have the option of in-person appointments, for example when physical exams were needed. Additionally, providers recommended incorporating innovative mobile technology that can enhance the quality of remote physical exam and monitoring.

Discussion

To our knowledge, this is the first study to explore multiple stakeholders' experiences and perspectives of telemedicine use in HIV care during the COVID-19 pandemic. In our sample, three-quarters of PWH in our study had participated in telemedicine visits during the pandemic, mostly conducted by telephone. Although the majority of those who participated in telemedicine visits felt that their telemedicine visit was better than/just as good as an in-person visit, less than half perceived telemedicine visits as useful during the pandemic and a smaller minority perceived that telemedicine would be useful after the pandemic. Factors associated with patient preference for telemedicine during the pandemic were lack of recent engagement in HIV care, smartphone availability, and site. Increase in patient age and location in Brooklyn, NY appears to be associated with more favorable perspectives regarding telemedicine usefulness after the pandemic.

Our study identified four main benefits of telemedicine, namely (1) ability to engage and re-engage patients in care and prioritize new and existing health concerns; (2) perceived patient-centeredness and flexibility; (3) opportunity to engage family and multidisciplinary patient care team members; and (4) opportunity to enhance telemedicine use proficiency through practice and support. Barriers to telemedicine use include (1) technical challenges; (2) privacy concerns; (3) loss of routine clinical experience and in-person interactions; (4) limited objective patient remote monitoring (e.g., blood pressure, medication adherence); and (5) concerns regarding reimbursement. We found that the majority of providers strongly endorsed maintaining telemedicine services beyond the pandemic. PWH also expressed support for telemedicine use after the pandemic, and underscored the importance of respecting patient preferences for telemedicine or in-person visits based on specific consultation purposes, for example scheduling in-person visits when physical exams are needed. In particular, we found that patients who had not been engaged in recent HIV care were more likely to perceive telemedicine as useful during the COVID-19 pandemic, and that patients in Brooklyn, NY were more likely to support that telemedicine was useful during and after the pandemic. While these findings may be driven by challenges with in-persons visits due to job inflexibility, distance to clinic, transportation barriers including reliance on public transportation, and stigma, they merit further investigation and suggest that telemedicine may offer an important alternative approach to in-person HIV care.

Our study additionally highlights that despite the relatively high number of smartphone ownership among PWH, video-enabled telemedicine visits occurred much less frequently than telephone-based visits. Video visits allows the exchange of visual information and non-verbal cues that are important in facilitating better provider-patient rapport and some aspects of physical examination [38]. However, such visits require high speed internet services and more complex setup that may present barriers to our sample of PWH who were older individuals and of lower socioeconomic status.

Our findings mirror other studies that examine telemedicine use for HIV care during the pandemic. One study in an HIV ambulatory clinic in Baltimore reported 70% of telemedicine visits were conducted by telephone [39], while another study in a Midwestern clinic opted to implement telephone-only telemedicine visits to alleviate technology access barriers among PWH receiving care during the pandemic [40]. In the rapidly evolving literature, researchers also examined sociodemographic factors affecting PWH participation in telemedicine visits during the pandemic, with varying results. Some studies reported older age (50–65 or > 65 years old), non-White, Medicaid insurance PWH were less likely to engage in a video visit [41], while others found no differences in age, gender, race, or ethnicity between PWH who completed in-person visits compared to those received telephone visits [40]. Differences in study designs, patient access to technology and digital health readiness, and telemedicine implementation policies, practices, protocols make it difficult to compare and generalize from these studies [42].

Future research to improve PWH engagement in telemedicine should consider equity-focused strategies to address the digital divide by assessing patients' technical readiness and providing programs to improve technology literacy and access. Some examples include considering the use of online patient portal as a determinant of patients’ readiness and capacity to use telehealth [41], conducting consistent assessment of patient characteristics (e.g., age, race and ethnicity, language, income, and educational level) to inform tailored interventions to promote equitable digital health access [19], and adapting a multilevel social-ecological framework in research and practice on digital health intervention planning, implementation, and evaluation [43]. Our study reveals several practice implications for facilitating telemedicine uptake that include increasing training for clinical and support staff to assist in connecting visits, developing policies and procedures that support remote patient monitoring (e.g., at-home sexually transmitted infection testing), providing financial support including reimbursement for telemedicine, and developing scheduling protocols to incorporate different visit types based on both patient's technical readiness and provider's assessment of patient’s medical needs.

Our study has some limitations. First, both sites were located in the US Northeast, potentially limiting generalizability of our findings. Second, due to sample size, we were unable to parse apart the telemedicine experience related to telephone vs. video visits, which could have provided further insights into future telemedicine efforts. In addition, given our modest sample size, we may not have been adequately powered to detect statistically significant associations between measured factors and outcomes of interest. Third, our sample was limited to English-speaking individuals and thus may not generalize to non-English-speaking patients who may be at greater risk for health care interruptions and challenges associated with the pandemic. Fourth, although we used validated measures where possible, we incorporated measurement tools that have not been validated to specifically assess the COVID-19 pandemic related experiences. Fifth, given the rapidly evolving context with the pandemic, our findings may not reflect current perspectives and experiences. Sixth, our sample was limited to PWH and may not generalize to individuals without HIV.

Conclusion

Our study provided timely insights into the telemedicine experiences for HIV care including barriers and benefits by engaging diverse stakeholders. Efforts to optimize telemedicine implementation for HIV care should consider strategies to improve technology support for PWH, patient-centered care options to maintain care, innovative platforms to allow remote monitoring, and establishing and implementing appropriate funding, billing and reimbursement methods to enable universal availability.

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 82 kb)