Impact of a Telehealth Program That Delivers Remote Consultation and Longitudinal Mentorship to Community HIV Providers.

Background.  To increase human immunodeficiency virus (HIV) care capacity in our region, we designed a distance mentorship and consultation program based on the Project ECHO (Extension for Community Healthcare Outcomes) model, which uses real-time interactive video to regularly connect community providers with a multidisciplinary team of academic specialists. This analysis will (1) describe key components of our program, (2) report types of clinical problems for which providers requested remote consultation over the first 3.5 years of the program, and (3) evaluate changes in participants' self-assessed HIV care confidence and knowledge over the study period. Methods.  We prospectively tracked types of clinical problems for which providers sought consultation. At baseline and regular intervals, providers completed self-efficacy assessments. We compared means using paired-samples t test and examined the statistical relationship between each survey item and level of participation using analysis of variance. Results.  Providers most frequently sought consultation for changing antiretroviral therapy, evaluating acute symptomatology, and managing mental health issues. Forty-five clinicians completed a baseline and at least 1 repeat assessment. Results demonstrated significant increase (P < .05) in participants' self-reported confidence to provide a number of essential elements of HIV care. Significant increases were also reported in feeling part of an HIV community of practice and feeling professionally connected to academic faculty, which correlated with level of program engagement. Conclusions.  Community HIV practitioners frequently sought support on clinical issues for which no strict guidelines exist. Telehealth innovation increased providers' self-efficacy and knowledge while decreasing professional isolation. The ECHO model creates a virtual network for peer-to-peer support and longitudinal mentorship, thus strengthening capacity of the HIV workforce.

In underserved areas, many obstacles hinder the delivery of comprehensive and high-quality healthcare services for individuals with complex chronic conditions, such as human immunodeficiency virus (HIV) infection. For persons infected with HIV who live in rural locations, barriers such as geographic isolation, economic hardship, stigma, and lack of confidence in local medical systems can cause delays in care that negatively affect health outcomes [1-9]. Healthcare practitioners in rural and other under-resourced regions also face substantial hurdles. Clinicians practicing in such communities are often strained by complex caseloads, infrequent access to colleagues for consultation, inadequate availability of specialists, and scarcity of behavioral health treatment services; in addition, time pressures and staffing constraints limit opportunities for continuing education [10-14]. These factors generate feelings of professional isolation and frequent provider turnover and may contribute to later adoption of new HIV therapies [9–12, 15–17]. Novel, technology-based strategies for provider support and education, such as distance telementoring, may reduce these barriers.

With the goal of strengthening provider capacity to deliver up-to-date, evidence-based HIV clinical care in the Pacific Northwest region of the United States, the University of Washington and Mountain West AIDS Education and Training Center ([MW AETC] formerly Northwest AETC) developed a collaborative, real-time, video-based HIV clinical consultation and mentorship program. The program, which began in 2012 with funding from the US Health Resources and Services Administration, is based on the Project ECHO® (Extension for Community Healthcare Outcomes) model developed by Dr. Sanjeev Arora [18, 19] at the University of New Mexico (UNM). Unlike telemedicine programs that connect a specialist provider to an individual patient, the ECHO model connects community providers simultaneously to both a multidisciplinary specialist team and peer providers from across a region to create a network for clinical support and education. Data from UNM's ECHO program demonstrated that community primary care providers who engage with this type of virtual support network achieve equivalent hepatitis C treatment outcomes compared with providers at an academic medical center, thus improving access to state-of-the-art hepatitis C care for patients in rural and underserved areas and prisons [18-21]. This innovative hepatitis C telementoring program was found to be cost effective, with an incremental cost-effectiveness ratio of $3700 [22]. The ECHO model has been replicated to support care in other fields, such as tuberculosis, diabetes, chronic pain, substance use disorders, dementia, transgender care, and others [23-29].

To our knowledge, there are no published evaluations of HIV ECHO programs. The goals of the current analysis, which comprises the first 3.5 years of MW AETC ECHO, are to (1) describe the structure and components of the program, (2) report the type of clinical problems for which community HIV providers sought support and remote consultation, and (3) evaluate changes in participating providers' self-assessed HIV knowledge and confidence to provide essential components of HIV clinical care. In this manner, we aim to assess the principal educational needs of HIV practitioners in rural and underserved areas and evaluate the provider-targeted effects of this community-based remote mentoring approach.

METHODS

Program Description

Following the Project ECHO® model [18, 19], the core component of the MW AETC ECHO program is a weekly, interactive video session, during which participants convene via video using cloud-based technology that can be accessed from any computer or mobile device (Figure 1). Participants connect in real-time to each other and to a multidisciplinary specialist panel at the University of Washington; the faculty panel includes experienced medical providers in infectious disease, psychiatry, addiction medicine, pharmacy, and social work. Community providers who join the program are encouraged to participate in the sessions weekly if their schedule allows and at least twice per month. The core faculty panel remains the same each week to foster mentorship relationships with participants.

Figure 1.

Snapshot of a University of Washington and Mountain West AIDS Education and Training Center (MW AETC) Project ECHO (Extension for Community Healthcare Outcomes) telehealth session; a community human immunodeficiency virus medical provider (large panel, left) is presenting one of their cases to a multidisciplinary team of academic specialists (large panel, right) and a network of their peers (surrounding smaller panels). Providers join the weekly session simultaneously via interactive video using a desktop, laptop, mobile device, or other video-conferencing system.

Each ECHO session begins with a concise (15-minute), didactic HIV update that focuses on clinically relevant topics, including federally funded practice guidelines, recent conference data, and other evidence-based best practices in HIV medicine. These presentations are given by local or national experts and are recorded, edited, and made available to participants and the public after each session. The didactic segment is followed by a brief question and answer period and then 1 hour of case consultations and discussion. Community providers present deidentified cases from their own patient panels for consultation from the faculty panel and for dialogue and ideas from the network of their peers. Providers are encouraged to present their cases again for follow-up once additional clinical information becomes available or if clinical changes occur. Participating providers attend ECHO sessions regularly, regardless of whether they are presenting a case, so that they can view the didactic talk, partake in the case discussions, contribute insight from their own practice, and learn from the expert panel and their peers.

After each session, participants who presented a clinical case receive written summaries of the panel's recommendations and key points from the discussion. All participants receive a follow-up e-mail after each session with links to resources or studies highlighted during the session. Participants also benefit from access to the specialty panel between sessions by e-mail or telephone for urgent clinical questions.

Tracking and Evaluation

Over the study period (February 2012 through August 2015), we prospectively tracked the types of questions for which community providers sought ECHO support through their clinical case presentations and organized these questions into 25 clinical categories. If a clinician presented a case with multiple questions, each of the questions addressed was counted separately; for example, one case could include a question about initiating antiretroviral therapy (ART), a question related to hepatitis C coinfection, and a question regarding substance use. If a provider presented the same case on multiple occasions for follow-up, each case presentation was counted separately.

Over the same study period, participating providers were asked to complete self-assessments of their HIV care knowledge and proficiency at baseline and at regular intervals. The survey included 23 questions related to confidence to perform essential components of HIV care, feeling part of a community of practice, and overall HIV knowledge. Questions related to confidence in various skill areas were based on a scale of 1 to 10 (1 = “cannot do,” 5 = “moderately can do,” and 10 = “highly can do”). Items related to feeling professionally isolated or feeling part of a community of practice were based on a scale of 1 to 5 (1 = “not at all,” 2 = “rarely,” 3 = “undecided,” 4 = “usually,” and 5 = “very much”), as were queries about HIV knowledge (1 = “novice,” 2 = “advanced beginner,” 3 = “competent,” 4 = “proficient,” and 5 = “expert”). At the start of the program, we asked participants to complete this assessment every 3 months; during the 1st year of the program, this was changed to every 6 months to reduce burden on participants, and during the 2nd year of the program the interval was changed to every 12 months.

Statistical Analysis

We compared means within survey items using a paired-samples t test to assess change over time and examined the relationship between each survey item and level of participation (number of sessions attended) using analysis of variance (ANOVA). For those survey items that were significantly associated with level of participation, we performed ANOVA to examine whether these items were significantly associated with length of time between first and last assessment.

RESULTS

The MW AETC ECHO program launched in February 2012 with a cohort of 11 clinical sites and 40 community HIV providers; thereafter, sites and practitioners joined on a rolling basis. By the end of August 2015, participation had expanded to 21 distinct clinical sites spread across 5 states (Figure 2).

Figure 2.

Geographic distribution of clinical sites participating in University of Washington and Mountain West AIDS Education and Training Center (MW AETC) Project ECHO (Extension for Community Healthcare Outcomes) as of August 2015; during the study period, a participating provider moved from Jerome, Idaho to Lewistown, Montana (data not shown).

Over the study period, we held 172 ECHO sessions, during which there were 553 case presentations (representing 313 unique cases). The mean number of participants per weekly session was 26.1 (standard deviation = 5.9). Table 1 shows characteristics of participants at the time of their enrollment in the program. Participants came from a variety of specialties (infectious disease, family medicine, internal medicine, pharmacy, nursing, and case management), although the majority were primary care practitioners. Approximately 40% of participants reported practicing in a rural location. Participants generally reported low-volume HIV-positive patient panels (a median of 19 patients for all participants and 18 for survey respondents) and few years of experience managing HIV (a median of 5 years for all participants and 4 for survey respondents). Over the study period, a total of 5302.5 continuing medical education or continuing nursing education credits were made available to participants.

Table 1.

Characteristics of MW AETC ECHO Participants at the Time of Their Enrollment in the Program (All Participants and Those Who Responded to at Least Two Self-Assessment Surveys)

Participant Characteristic	All Participants (N = 90)	Self-Assessment Respondents (N = 45)
Professional training/discipline
Physician	55.4%	61.2%
Pharmacist	13.6%	10.8%
Advanced nurse practitioner	10.3%	10.4%
Physician assistant	9.3%	9.3%
Nurse	5.1%	6.1%
Social worker	4.6%	2.2%
Other	1.7%	0.0%
Years experience treating HIV (median)	5.0	4.0
HIV-positive patient panel size (median)	19.0	18.0
Practice location
Rural	38.0%	39.2%
Urban or suburban	62.0%	60.8%
Program engagement (based on number of sessions attended)
Low (1–20 sessions)	14.2%	25.0%
Medium (21–60 sessions)	25.7%	39.0%
High (61–124 sessions)	60.1%	36.0%

Abbreviations: ECHO, Extension for Community Healthcare Outcomes; HIV, human immunodeficiency virus; MW AETC, Mountain West AIDS Education and Training Center.

The 553 case discussions during the ECHO sessions included a total of 1051 clinical questions, which were assigned to 1 of the 25 clinical categories that best matched the key questions addressed. The most frequent types of questions for which providers sought ECHO support and for which the panel gave recommendations included changing ART, evaluating acute symptomatology or laboratory abnormalities, and assessing or treating mental health issues (Figure 3).

Figure 3.

Types and frequency of clinical questions for which University of Washington and Mountain West AIDS Education and Training Center (MW AETC) Project ECHO (Extension for Community Healthcare Outcomes) participants sought remote consultation and support over the study period. The “other” category includes clinical questions that were posed infrequently during ECHO sessions and did not fit another specified category, such as end-of-life issues, transgender care, medicolegal questions, etc. Abbreviations: AIDS, acquired immune deficiency syndrome; ART, antiretroviral therapy; HIV, human immunodeficiency virus; LBTI, latent tuberculosis infection; MTB, Mycobacterium tuberculosis; OI, opportunistic infection.

Forty-five participating providers completed a baseline and at least 1 follow-up assessment (out of a total 90 participants over the study period, for a response rate of 50%). The number of days between first and last measure ranged from 96 days to 994 days; 2 participants' responses were only 96 days apart, and the majority (73%) were between 360 to 994 days (or approximately 1 to 3 years) apart.

Provider self-assessed confidence improved over time in several clinical skill areas (Table 2). Feelings of professional isolation decreased significantly, and providers reported that their sense of connection to MW AETC faculty, degree to which they feel comfortable serving as a resource to local peers, and amount that they feel part of an HIV community of practice increased significantly over time. Lastly, participants rated their overall knowledge of HIV care, which improved significantly.

Table 2.

Results of Self-Assessment Surveys Comparing Baseline and Most Recent Assessment for Providers Who Participate in MW AETC ECHO

Self-Assessment Domains and Survey Items	Pre-/Post Mean Scores	Paired Diff. of Meansa	P Value
Self-efficacy (scale 1–10)
Screen for HIV in the general population	8.56/9.07	0.51	.118
Counsel to reduce HIV transmission	7.18/8.98	1.80	<.001
Perform initial HIV-related history/physical	7.38/8.33	0.96	.001
Screen for viral hepatitis	7.62/8.56	0.93	.002
Screen for substance abuse	7.24/8.07	0.82	.004
Screen for mental health issues	6.56/7.82	1.27	<.001
Screen for sexually transmitted infections	8.24/8.33	0.09	.628
Select tests for monitoring HIV	7.42/8.24	0.82	.006
Interpret tests for monitoring HIV	8.18/8.27	0.09	.813
Evaluate exposures/advise regarding PEP	6.31/7.69	1.38	.002
Select an initial ART regimen	5.42/7.78	2.36	<.001
Manage common ARV side effects	6.00/7.27	1.27	<.001
Manage metabolic complications of ARVs	6.38/7.02	0.64	.054
Assess for drug-drug interactions	6.04/7.04	1.00	.004
Select salvage ART	6.13/6.24	0.11	.760
Manage opportunistic infections	5.76/6.93	1.18	<.001
Identify malignancies in persons with HIV	5.91/6.51	0.60	.023
Care for women of childbearing age with HIV	5.38/6.62	1.24	.006
Be a resource to other providers in region	5.44/6.53	1.09	.003
Community of practice (scale 1–5)
Degree of professional isolation	2.69/2.27	−0.42	.012
Feel part of an HIV community of practice	3.78/4.16	0.38	.016
Feel connected to MW AETC faculty	3.56/4.11	0.56	<.001
Knowledge (scale 1–5)
Overall knowledge regarding HIV care	2.89/3.22	0.33	.004

Abbreviations: ART, antiretroviral therapy; ARV, antiretroviral; Diff., difference; ECHO, Extension for Community Healthcare Outcomes; HIV, human immunodeficiency virus; MW AETC, Mountain West AIDS Education and Training Center; PEP, postexposure prophylaxis.

a Postparticipation assessment mean score minus preparticipation assessment mean.

Among survey respondents, the minimum number of sessions attended during the study period was 3 and the maximum was 124 (Table 1). Changes in 2 items were significantly associated with level of engagement in the program: (1) degree to which participants feel part of an HIV community of practice and (2) degree to which participants feel professionally connected to MW AETC faculty. Changes in 3 survey items were significantly associated with a longer time between first and last assessments: (1) counsel HIV-infected patients to reduce transmission to others, (2) manage metabolic complications of ART, and (3) degree to which participants feel part of an HIV community of practice.

DISCUSSION

This evaluation of the impact of MW AETC ECHO on community practitioners reveals that a structured, longitudinal telehealth mentoring program can accomplish 3 major goals: (1) increase community medical provider knowledge and confidence related to best-practice HIV medicine, (2) decrease feelings of professional isolation, and (3) engender local and regional HIV care champions. Achieving these goals enhances workforce development and expands the number of community practitioners equipped to provide high-quality HIV care. Furthermore, the model can be successfully implemented across a vast geographic region (the MW AETC ECHO program spans an area that encompasses over 25% of the US landmass).

Our primary finding is that participation in ECHO led to significant increases in self-assessed HIV knowledge and confidence to provide a number of essential components of HIV care. Several unique features of the ECHO model promote provider learning and self-efficacy. First, unlike traditional one-on-one consultations or ad hoc consultations, which usually occur by phone or e-mail, ECHO facilitates regular connections between academic faculty and community providers, allowing for structured, long-term mentorship tailored to provider needs. Second, unlike traditional didactic lectures or webinars, ECHO provides real-time, interactive discussion of cases from participants' own patient panels; thus, the discussions are highly contextualized, which fulfills key learning theory principles [30]. Third, providers join the sessions and contribute to the discussions even if they are not presenting a case, thereby creating a peer-to-peer network for clinical support and shared problem solving [29, 31, 32]. Participants also benefit from connections to faculty and peers for between-session support. In this manner, ECHO provides more than a consultation service; the program offers longitudinal mentorship and a clinical support network to a cadre of practitioners, which elevates provider knowledge and confidence and improves workforce capacity across a region.

The MW AETC ECHO program focuses on supporting low-volume HIV providers, many of whom practice in rural or semiurban areas; however, even providers in urban areas reported increased confidence and knowledge, so benefits of participation are not limited to rural practitioners. Providers in urban underserved areas may practice in locales with limited resources, poor access to mental health and other specialists, or few opportunities for continuing education [10-14]. Real-time access to a team that includes infectious diseases experts as well as an HIV-experienced psychiatrist, social worker, and pharmacist is one of the most important features of the program; providers benefit from easy access to multidisciplinary specialty advice as well as continuing education.

Studies have suggested that provider level of HIV experience affects patient outcomes, and a recent analysis determined that low-volume HIV providers achieve poorer performance on measures such as virologic suppression, mental health screening, and syphilis screening, compared with high-volume providers [33, 34]. Although we did not have access to patient-level data to assess these measures, our findings demonstrate that an ECHO telementoring program can improve provider confidence to initiate and manage ART, screen for mental health issues, and screen for sexually transmitted infections. An objective evaluation of performance measures for low-volume providers who are part of such a telehealth network will be an important next step. As other authors have suggested, involvement in a telehealth collaborative such as ECHO may be an efficient means of improving care quality and patient outcomes for low-volume or less experienced HIV providers and thus may reduce disparities in care between different practice settings [35].

Another noteworthy finding of the analysis is that participating providers reported significant decreases in feelings of professional isolation and increases in feeling part of a community of practice and feeling connected to academic faculty. This is consistent with evaluation data from UNM's hepatitis C ECHO, which found that participation in the program increased professional satisfaction [21]. Subjective feedback we have received from MW AETC ECHO participants repeatedly suggests that one of the most significant outcomes of participation is decreased feelings of professional isolation. Principal goals of the program are to foster professional networks and strengthen relationships between academic faculty and community physicians; our results reveal that a telehealth platform can indeed bolster these academic-to-community partnerships. The ECHO model is built on relationships, initiated with an ECHO team site visit to the clinician's practice before their participation in live sessions and then reinforced through weekly virtual interactions and as-needed consultations between sessions. The establishment of these relationships serves as a foundation for longitudinal clinical mentorship, oftentimes occurring over large distances; technology simply facilitates these mentoring relationships.

Although we were not able to objectively evaluate provider turnover with this analysis, we believe decreased feelings of professional isolation have potential to decrease provider burnout and turnover. This effect could increase provider stability and decrease recruitment and training costs for clinics in rural and underserved areas; further evaluation of this outcome is needed, especially given the challenge many clinics face in retaining primary care clinicians.

The ECHO telehealth model has potential to decrease systemic costs of care in other ways as well. For example, decreased need to refer patients long distances to academic medical centers can reduce transportation costs and shorten wait times for appointments and for specialist advice. We do not view ECHO as a replacement for subspecialty referral, but we do believe the program can improve efficiency and efficacy of the referral system. Subjective feedback from participants suggests that engagement in ECHO allows community providers to feel more confident knowing when and where to refer, how to complete the indicated medical evaluation before referral, and how to ensure that referrals are appropriate. Participating providers have also commented that engagement in ECHO leads to increased local patient recruitment, because patients gain confidence in the local medical system knowing that providers are actively linked to the academic medical center.

Participating clinicians also reported increases in their ability to serve as a resource to peers in their region; in this way, ECHO participants become local and regional care champions and the program strengthens the HIV care network. Our program is similar to a telehealth platform used by the US Veterans Administration (VA) medical system to support care for a variety of chronic medical conditions. According to an evaluation of the VA hepatitis C telehealth program, 75% of participating providers reported discussing information they learned from their telehealth case presentations with colleagues, and more than 40% reported helping a colleague care for a patient using knowledge learned during discussions of other participants' cases [31]. This propagation of knowledge learned in telehealth mentoring programs explains how every provider involved magnifies the reach and impact of such programs; ECHO creates links for rapid dissemination of knowledge and real-time idea exchange across large distances, and participants share best practices with their local peers.

The summary of types of questions for which participants presented cases during MW AETC ECHO sessions provides valuable insight into the clinical issues for which community HIV providers require the greatest degree of support and continuing education. By far the most common clinical topic for which providers sought ECHO support was changing ART. Although providers who participate in our program seem comfortable with the task of initiating ART, clinicians frequently sought guidance related to changing a patient's ART regimen. Because this topic arose so frequently, we adjusted the didactic curriculum accordingly and planned short talks related to switching ART. The ECHO curriculum is malleable and by tracking the types of cases presented it can be adjusted to meet the needs of the participants. The model allows ECHO faculty to stay attuned to the clinical challenges faced by participants. Data collected regarding the specific educational needs of community practitioners could be used to target other training interventions and, on a policy level, to direct resource allocation.

Other frequent topics for which community providers sought ECHO support included evaluation of acute symptomatology or laboratory abnormalities, care for mental health issues, diagnosis and treatment of opportunistic infections, management of low-level viremia, management of hepatitis C in persons infected with HIV, and issues surrounding adherence and engagement in care. For some of these topics, no strict guidelines exist to aid primary care practitioners. Participants often sought support for cases with complex medical as well as behavioral health issues and frequently commented that access to mental health and substance abuse services remains problematic in their communities. These data underscore that when designing an HIV telehealth program, it is crucial to involve a multidisciplinary panel that includes experts on mental health, substance use disorders, adherence support, and viral hepatitis coinfection.

Of note, although certain topics such as postexposure prophylaxis (PEP) and initiating ART did not arise frequently during case discussion, providers reported increased confidence managing these issues, which we attribute to the regular 15-minute didactic portions of the sessions, which deliver timely updates to the ART and prevention guidelines. Providers also reported increased ability to counsel patients regarding HIV testing and prevention, which are important measures for curtailing new HIV transmissions. We observed a recent increase in providers presenting pre-exposure prophylaxis (PrEP) cases to ECHO, and through a partnership with the Washington State Department of Health we are now incorporating a larger focus on PrEP in our ECHO sessions and a PrEP expert joins the sessions regularly.

CONCLUSIONS

This analysis of a telementoring program to support providers in community-based practice settings has several limitations, including the imperfect survey response rate, varying length of time between first and last self-assessment, variations in level of engagement of those surveyed, and reliance on provider self-reported outcomes. Despite these limitations, the evaluation demonstrates the beneficial effects of an HIV ECHO program and suggests that telehealth could be expanded to support workforce development and to help address disparities in HIV care access and quality. A critical component of the US National HIV/AIDS Strategy is to “Take deliberate steps to increase the capacity of systems as well as the number and diversity of available providers of clinical care and related services for people living with HIV [36].” We believe this analysis validates the ECHO model as a strategy to bolster HIV care capacity, provide continuing HIV education, and ensure a high quality of care across a broad geographic region. Future objective evaluations of the impact of such programs should include analyses of costs of care, provider job satisfaction and turnover, and objective patient-level clinical outcomes. To enhance the sustainability of these programs, advocacy is needed to expand funding resources as well as improve the regulatory and reimbursement structure for this type of novel healthcare support system [37-39].