A Multipronged Digital Response to Increased Demand for Telehealth Support and Training During the COVID-19 Pandemic.

In response to the COVID-19 public health emergency, the University of Kansas Center for Telemedicine & Telehealth (KUCTT) adopted a multipronged, digital strategy to address COVID-induced, high-volume telehealth inquiries in Kansas and sought to quickly disseminate rapidly evolving federal policy updates and foundational telehealth implementation guidance. Retrospectively, KUCTT examined participant engagement in three educational approaches (e.g., telehealth webinars, Project ECHO, brief instructional/informational videos) that were developed and delivered in real time to meet the specific and unique needs of healthcare administrators and providers due to the COVID-19-forced surge in telehealth utilization. KUCTT observed significant increases in telehealth educational engagement and website access in response to the COVID-19 telehealth surge and the multi-pronged digital educational strategy. From January to September of 2020, average attendance at non-COVID-19 ECHOs was 56.1 attendees while the average attendance for two COVID-19 ECHOs that occurred in March of 2020 was 225 attendees, a 300% increase in attendance. The University of Kansas Medical Center (KUMC) Telehealth website received triple the amount of page views in March and April of 2020 (n=1,559) compared to January and February of 2020 (n=526). Healthcare providers used and engaged with the educational programs in this fast-tracked, digital approach at greater rates when compared to pre-pandemic program and web data. This interest mirrors the COVID-19 telehealth surge and suggests that a multipronged approach was effective in disseminating rapidly evolving telehealth policy and defining essential elements of telehealth implementation.

Multiple studies have demonstrated that healthcare facilities and providers, both globally and in the USA, have seen large increases in provision of telehealth services due to the COVID-19 public health emergency (PHE) (Joshi et al., 2020; Parisien et al., 2020; Schulz et al., 2020; Thomas et al., 2020). Despite the promise of telehealth applications prior to the PHE, telehealth implementation and utilization were limited, even with increased availability of technology and telecommunication infrastructure (Gagnon et al., 2006). In response to the PHE, healthcare organizations and providers with various levels of telehealth experience and interest were challenged to rapidly adapt to online treatment modalities with little time to plan and prepare. Anecdotal information indicated that healthcare organizations, providers, educators, and their support systems scrambled to identify telehealth resources to support this significant undertaking. The following support request from a medical school training director illustrates a typical telehealth inquiry: “As you know, the health system is moving very quickly toward telehealth and no doubt there will not be the level of expertise and knowledge that you guys have. My thoughts: Everybody needs a short crash course.”

Similar to other well-established telehealth organizations, the University of Kansas Center for Telemedicine & Telehealth (KUCTT) was inundated with requests for telehealth support when COVID-19 was declared a PHE. KUCTT fielded requests for support of telehealth programs from a variety of stakeholders including health system administrators, behavioral health organizations, institutes of higher learning (i.e., graduate training programs), healthcare/behavioral health providers, and non-profit agencies as they planned to transition in-person patient care to telehealth modalities. Most of the telehealth support requests were received by emails to KUCTT’s leadership and KUCTT’s web-published contact email account. Support requests predominantly solicited guidance for three topic areas: (1) clinical training and support, (2) telehealth technology selection, and (3) rapidly evolving telehealth policies. Despite the notable availability of telehealth resources across the nation prior to the PHE in the form of national and regional telehealth resource centers, professional associations, and telehealth centers such as KUCTT, most available telehealth resources and training opportunities were designed to assist healthcare organizations and providers establish sustainable telehealth programs under normal circumstances, which is a comprehensive and time-intensive process. KUCTT focused on assisting local stakeholders with identifying the core essentials of rapid telehealth development and deployment by adopting strategies in response to stakeholder requests and quickly changing state and federal telehealth policies.

In response to these needs, KUCTT developed a multipronged, digital strategy to address COVID-19-induced, high-volume telehealth inquiries through a rapid cycle process of assessing and categorizing telehealth needs, designing and planning delivery methods, engaging stakeholders, and disseminating telehealth educational content and guidance. BTHO’s telehealth educational outreach approach is most consistent with the i-PARIHS implementation framework (Harvey & Kitson, 2015), wherein KUCTT's telehealth experts, resources, and educational approaches served as facilitators to stimulate action around telehealth education and implementation through (1) an examination of telehealth needs derived from the COVID-19 PHE (context), (2) expertise with telehealth (innovation), and (3) relationships with healthcare organizations and providers (recipients) to support the rapid deployment of telehealth lines of service (successful implementation) (See Fig. 1). This article describes KUCTT's approach in more detail, provides organizations’ responses to the digital programming, and highlights pre-post data regarding the interest and success of the strategy.

Fig. 1

Representation of the i-PARIHS model

Methods

KUCTT adopted the following principles to guide a strategic response to facilitate the dissemination of educational support in response to the significant demand for telehealth resources: (1) examine and understand the predominant needs of telehealth learners/requestors; (2) respond to telehealth needs and requests in a timely, individualized, and professional manner with a focus on connecting learners with existing and respected telehealth resources; (3) produce high-quality, educational materials that are sensitive to the learner’s level of experience and available time; (4) leverage KUCTT's existing resources and technologies to support educational outreach; and (5) encourage telehealth learners/requestors to incorporate quality improvement strategies to inform telehealth planning and sustainability beyond the PHE. In addition to traditional telehealth consults, KUCTT adopted and developed three educational approaches (e.g., telehealth webinars, Project ECHO, brief instructional/informational videos) to address these principles and facilitate learning and successful telehealth implementation across an array of stakeholder needs. These approaches were led by multi-disciplinary expert teams composed of physicians, advanced practice providers, behavioral health specialists, and administrators who developed program curriculums and video content that was guided by stakeholder requests for education and consultation. Telehealth experts partnered with the KUMC Continuing Education & Professional Development (CEPD) and reviewed stakeholder requests and participant feedback (i.e., participant evaluations, digital needs assessment for ECHO participants) to develop educational content and delivery models.

In alignment with the i-PARIHS framework (Harvey & Kitson, 2015), KUCTT's strategic response was grounded in a supportive, interactive problem-solving process with healthcare organizations and providers in the context of a PHE, which generated a recognized need for telehealth support and improvement. Through a rapid-cycle planning process, KUCTT reviewed and shared support requests to understand and prioritize needs; engaged with telehealth learners and requesters to clarify educational and access needs; assessed and measured response and feedback to digital educational approaches; and acted quickly to deliver and implement educational outreaches and learning opportunities (See Fig. 2). This review was not designed as an a priori research project, but rather a retrospective examination of stakeholder participation, engagement, and satisfaction to inform future generalizability and rapid-response educational efforts. Descriptions of the methods of development for each of the three educational strategies are outlined below.

Fig. 2

i-PARIHS Phases of Facilitation for the multipronged digital response

Educational Strategy 1: University of Kansas Medical Center (KUMC) Connect Telehealth Webinar Series

Prior to the PHE, KUCTT developed and produced a monthly webinar series that provided guidance on a variety of relevant telehealth topics including policy issues, applications, research, and innovation. This webinar series is promoted throughout the state and attracts consistent registration, attendance, and participation throughout the year. When COVID-19 emerged, this existing educational mechanism was leveraged to produce a fast-tracked, special webinar session titled Telehealth Options During COVID-19. The telehealth webinar was developed based upon the topic areas and urgency for information expressed by healthcare organizations and providers. The learning objectives for the webinar were developed in response to the most pressing telehealth questions and needs as identified by existing telehealth partners as well as healthcare organizations and providers that were exploring telehealth for the first time in response to the PHE. Due to the significant number of registrations and persistent demand for telehealth guidance and assistance, this webinar was delivered twice within a 2-day period.

Educational Strategy 2: KUMC Project ECHO (Extension for Community Healthcare Outcomes)

The KUMC Project ECHO hub is housed within KUCTT. Project ECHO was developed at the University of New Mexico as an innovative telementoring program and is recognized globally as an effective process to improve patient care outcomes. Project ECHO creates virtual learning communities by bringing together healthcare providers and subject matter experts through videoconference technology. Traditionally, Project ECHO sessions include brief didactic presentations and case-based learning, which fosters an “all learn, all teach” environment where multidisciplinary experts mentor participating healthcare presenters and all participants have the opportunity learn from each other through this interactive and engaged process (Arora et al., 2007; Zhou et al., 2016; University of New Mexico, n.d.). In response to the COVID-19 PHE, Dr. Sanjeev Arora, founder of the ECHO Institute, met with Project ECHO hubs and encouraged ECHO teams to leverage their ECHO resources to address local healthcare training needs. To accommodate this, he gave permission to amend the traditional case-based learning design in order to support the rapid dissemination of healthcare information and training.

Since its launch in 2015, the KUMC Project ECHO hub has produced 35 ECHO projects and has reached a broad audience locally, nationally, and internationally, hosting 193 individual TeleECHO sessions with a total reach of 5817 participants. KUCTT's access to respected telehealth experts and proficiency with Project ECHO delivery provided an opportunity to quickly ramp-up a telementoring approach to inform telehealth deployment that was accessible to a large community of learners.

Initially, the KUMC Project ECHO team developed a 4-session Telehealth ECHO plan to support healthcare organizations and providers with rapid telehealth deployment, which was a high demand support request in response to the PHE. Given the tremendous participant response to the initial Telehealth ECHO and requests to continue the learning collaborative, the KUMC Project ECHO team consulted with KUMC CEPD to facilitate a responsive telementoring plan to support ongoing telehealth implementation across a range of stakeholder groups that were experiencing a variety of telehealth demands and stressors. With guidance from KUMC CEPD and feedback from Telehealth ECHO learners, the KUMC Project ECHO team adopted a phased approach to delivering telehealth education through Project ECHO. The three phases of the Telehealth Project ECHO response are described below.

Phase 1 — Telehealth ECHO: Implementing Telehealth

In collaboration with KUCTT and local telehealth experts, the KUMC Project ECHO team quickly developed and launched a 4-session Telehealth ECHO series focused on assisting diverse healthcare organizations and providers prepare for the rapid development and deployment of telehealth programs and services to increase direct patient care during the COVID-19 PHE or other large-scale crises. The ECHO was delivered weekly in April 2020 and addressed learning objectives listed in Table 1. The ECHO specialty team was comprised of four telehealth experts including telehealth administrators (i.e., organizational director, chief executive officer), a physician, and a psychologist. Participants were encouraged to share systems issues and organizational challenges to promote the case-based learning component of the Project ECHO model. However, the Telehealth ECHO was modified to include an open forum question-and-answer segment to reduce the demand on participants to prepare and present cases.

Table 1

Learning objectives for the Telehealth ECHO: Implementing Telehealth

Learning Objectives
Objective 1. Identifying opportunities to use telehealth to address COVID-19 challenges for organizations and patient populations
Objective 2. Assessing organizational readiness to rapidly implement telehealth programs
Objective 3. Developing plans to implement and/or expand telehealth programs as part of organizational response to crisis situations
Objective 4. Identifying current reimbursement models available for telehealth patient visits during the COVID-19 pandemic
Objective 5. Educating staff and patients regarding best practices for telehealth
Objective 6. Modifying current systems and processes of patient management/care delivery for telehealth
Objective 7. Discussing the viability of sustaining telehealth beyond the COVID-19 pandemic

Phase 2 — Telehealth Lunch ECHO: Byte-Sized Learning

Phase 2 of the Telehealth ECHO series adopted a responsive approach to curriculum design and delivery based on feedback from participants who reported experiencing a high level of stress and fatigue associated with addressing COVID-19-related demands. The Telehealth Lunch ECHO transitioned from weekly to monthly sessions and included six sessions. The learning objectives for this ECHO series were based on pressing questions presented by the learning collaborative and included: discussing current state and federal telehealth policies relating to telehealth use and reimbursement; recognize the benefits and challenges of engaging in telehealth; describing the potential telehealth impact on the organization, the providers and the patients; developing strategies to identify and address the challenges associated with telehealth, and identifying possible information and data sources to use in practice improvement. The Telehealth Lunch ECHO maintained the open forum question-and-answer segment and did not place heavy emphasis on case-based learning. During Phase 2 of the Telehealth ECHO, healthcare organizations and providers adapted to telehealth implementation, and the learning collaborative attendance reduced in number. This resulted in a smaller but cohesive community of practice with a high level of engagement in the telementoring process.

Phase 3 — Telehealth ECHO: Sustaining Telehealth Beyond COVID-19

In September 2020, the KUMC Project ECHO team disseminated a brief needs assessment to all Telehealth ECHO registrants to measure the learning collaborative’s interest in participating in an additional phase of the Telehealth ECHO series that would focus on the fundamental aspects of developing and sustaining telehealth lines of service. The needs assessment found that the learning collaborative was interested and invested in learning strategies for maintaining telehealth lines of service beyond the COVID-19 PHE. The final phase of the Telehealth ECHO initiative will consist of a 9-session series that meets monthly from January through September 2021. The Sustaining Telehealth Beyond COVID-19 ECHO will return to the traditional Project ECHO model, and participants will be asked to share system-based cases to support the case-based learning model. The learning objectives will be developed in response to learning gaps identified by the learning collaborative through the needs assessment.

Educational Strategy 3: Telehealth Bytes Informational Videos

Telehealth Bytes are brief, animated videos developed by KUCTT to provide telehealth insights to assist with telehealth implementation and utilization. Telehealth Bytes were designed in response to frequently asked questions about relevant telehealth topics. The videos were based on general telehealth best practices and not intended to be comprehensive for specialty-specific implementation. Telehealth Bytes were originally conceptualized in 2019 as supplemental learning tools to complement existing graduate and post graduate educational offerings by providing telehealth training in brief, easy-to-digest segments.

In response to BTHO’s significant demand for telehealth training and support, the Telehealth Bytes initiative was redirected to an open-access, online video resource that was designed to respond to the most frequently asked questions. Understandably, healthcare providers faced numerous challenges in responding to the COVID-19 PHE. Many providers shared the perspective that they had limited time and mental capacity for new telehealth training. This support request highlights the telehealth training dilemma: “I wondered if you might possibly have something you could share with our faculty (e.g. video or script), maybe with examples of things one might encounter and clinical things they should think about or guidance. I would say that attention spans are limited these days; we get a lot of general referral to document that would take weeks to pour through. I know there will be a lot of learning by doing but any help would be much appreciated.”

KUCTT evaluated the telehealth topic areas and frequently asked questions in March 2020 that were driven by COVID-19 to inform the design and production of the Telehealth Bytes videos (see Table 2 for content areas). The Telehealth Bytes video scripts were developed by a team of telehealth experts and were delivered by a licensed psychologist with expertise in telehealth administration and practice.

Table 2

Telehealth Bytes content areas

Video Content Areas
Kansas State COVID-19 updates
Emergency start-up
Telehealth documentation
Orienting clients/patients to telehealth
Telebehavioral health: special considerations for the home
COVID-19 federal telehealth updates
Setting up a telehealth provider’s workplace
Informed consent for synchronous telehealth services
Telebehavioral health in a child’s home

Evaluation Aims

In the summer of 2020, KUCTT sought to evaluate the influx in engagement with telehealth resources by comparing participant and user data from COVID resources to non-COVID resources. The evaluation aims of this study are to (1) compare four COVID resources to similar non-COVID resources by examining participant and user data and (2) explore post-evaluation data to investigate knowledge gained by participants.

Measures

In an effort to determine if healthcare providers used and engaged with the educational programs at greater rates when compared to non-pandemic program and web data, a variety of data sources were used (see Table 3).

Table 3

Evaluation measures

Program	Type of data	Source of data
KUMC Connect Telehealth Webinar	Attendance Knowledge gained	Zoom reports Post-evaluation surveys
KUMC Project ECHO	Attendance	Zoom reports
KUMC Telehealth Webpage	Hits	Google Analytics
KUCTT YouTube Videos	Views	YouTube

Webinars and ECHOs

For the educational programs (KUMC Connect Webinar Series and the Telehealth ECHOs), attendance data was compared to attendance from non-COVID-19 webinars and ECHOs. These programs administered post-evaluation surveys to examine knowledge gained by participants.

Online Resources

To determine engagement with COVID-19-related website information, Google Analytics was used to examine differences in pre-COVID-19 traffic to page visits that occurred after March 15, 2020. Similarly, YouTube analytics measured engagement by investigating differences in viewership of COVID-19 content to content related to other subject areas.

Findings

Engagement

KUCTT found that across the three educational approaches, all experienced increased levels of engagement, measured by program registration and attendance and website data.

Findings for Educational Strategy 1: University of Kansas Medical Center (KUMC) Connect Telehealth Webinar Series

For the KUMC Connect Telehealth Webinar Series (two webinars offered in a 2-day period) total attendance was 437 (578 participants registered), which far exceeds the average of 51.8 attendees at other KUMC Connect Webinars.

Findings for Educational Strategy 2: KUMC Project ECHO (Extension for Community Healthcare Outcomes)

Record levels of attendance were documented for the COVID-19-related KUMC Project ECHO series in the spring of 2020. Between January and September, average attendance at non-COVID-19 ECHOs was 56.1 attendees while the average attendance for the two COVID-19 ECHO series (Telehealth ECHO: Implementing Telehealth and Telehealth Lunch ECHO: Byte-sized Learning) that occurred in March was 225 attendees (465 participants registered), reflecting an increase of 300% in attendance. The Telehealth ECHO: Implementing Telehealth (four sessions) were attended by an average of 165 professionals. See Table 4 for demographic data for program registrants.

Table 4

Demographics of online program registrants*

	KUMC Connect Telehealth Webinar Series (N = 578)	KUMC Project ECHO (N = 465)
County classification		Registrants were able to list multiple counties
Urban Semi-urban Densely settled rural Rural Frontier Out of state	299 (51.73%) 60 (10.38%) 61 (10.55%) 45 (7.79%) 28 (4.84%) 85 (14.71%)	174 (34.73%) 52 (10.38%) 119 (23.75%) 89 (17.76%) 67 (13.37%) Not recorded
Specialty
Administration Behavioral/mental health Family/internal medicine Nursing Public health Specialty care Other	22 (3.99%) 62 (11.25%) 24 (4.36%) 262 (47.55%) 3 (0.54%) 76 (13.79%) 102 (18.51%)	37 (8.37%) 66 (14.93%) 94 (21.27%) 21 (4.75%) 28 (6.33%) 86 (19.46%) 110 (24.89%)

*Demographic data for attendees was not collected

Findings for Educational Strategy 3: Telehealth Bytes Informational Videos

The KUCTT YouTube video channel analytics demonstrate a wave of interest in COVID-19-related topics. COVID-19-related topics accounted for over 52% (52.6) of all video views from January to July of 2020. This represents nearly 2500 (2489) views, and of the top ten most popular videos of 2020, 7 are COVID-19-related content (Table 5). YouTube analytics shows that 53.8% of traffic originates from the KUMC site. Another institution links to the videos as 23.4% of traffic originates from their site and the remainder of people who accessed the videos originated from email sites and apps (12.1%), social media sites (5.1%), other YouTube sites (3.3%) and Google searches (2.3%).

Table 5

Total BTHO YouTube video rankings from January to July 2020

Rank	Topic	Date uploaded	Views	COVID-19 related
1	Introduction of Zoom Cloud Meetings*	January 2015	562	No
2	Telehealth Bytes: Orienting Clients/Patients to Telehealth	March 2020	422	Yes
3	Telehealth Bytes: Telehealth Documentation	March 2020	354	Yes
4	Telehealth Bytes: Telebehavioral Health in the Home	March 2020	267	Yes
5	Telehealth Bytes: Telebehavioral Health in a Child’s Home	March 2020	223	Yes
6	Telehealth Bytes: Emergency Start-Up	March 2020	220	Yes
7	Telehealth Bytes: Informed Consent	March 2020	216	Yes
8	Introduction of Zoom Cloud Meetings – App Installation*	January 2015	207	No
9	Telehealth Bytes: Telehealth Workspace	March 2020	199	Yes
10	Telemedicine Consultation*	July 2019	165	No

*Video content developed and listed prior to the COVID-19 public health emergency

Findings for Website Sources

A review of website data reveals a similar trend of increased engagement. When compared to website traffic in January and February of 2020, the KUMC Telehealth page received triple the amount of page views in March and April (526 hits compared to 1559 hits). Website traffic increased in March, corresponding with the PHE and increased use of telehealth services by providers. Traffic peaked in April, but remained higher in May, June, and July than pre-COVID-19 traffic in January and February (see Table 6).

Table 6

Telehealth page views (average by month from January through July of 2020)

Month	Average page views
January	271
February	255
March	710
April	849
May	414
June	430
July	381

Knowledge Gained

While the engagement data reveals substantial interest in COVID-19 topics, the effectiveness of this fast-tracked approach is demonstrated by a multi-method data collection strategy that surveyed attendees of the educational sessions.

Post-series evaluation data collected from individuals requesting continuing education credit following the KUMC Connect Telehealth Webinar Series found that learning objectives were successfully met with high percentages of respondents indicating that their level of achievement in meeting the following learning objectives as good or outstanding. Items covered in this evaluation included the following: (1) describe the two main purposes of telehealth during COVID-19 (95%, n = 224); (2) identify currently available technology options (95%, n = 223); and (3) discuss policy and reimbursement revisions for using telehealth for COVID-19 (92%, n = 224).

Similarly, post-series evaluation data collected by KUCTT found that 95% of the Implementing Telehealth ECHO reported that they gained helpful knowledge from the series and 84% reported that they obtained helpful skills and techniques that will improve their professional practice.

A brief needs assessment survey was sent to the Telehealth ECHO registrants and attendees (N = 465) and was used to guide Phase 3 development and implementation. Of the 53 respondents, 96% replied that they are likely to attend future sessions. Respondents identified that it is important to them that they learn more about telehealth administration (100%), telehealth clinical issues (100%), and telehealth technology (92%). The Telehealth ECHO planning team met in November 2020 to develop the curriculum for Phase 3 of this project with the aim of delivering fundamental telehealth training according to the learning priorities identified by the learning collaborative. The first session of this ECHO series will guide participants through conducting a telehealth needs assessment, and the remainder of the sessions delve into the fundamental components that inform telehealth success and sustainability.

Discussion

As healthcare organizations and providers significantly increased telehealth service delivery during the COVID-19 PHE, urgent requests for telehealth support understandably increased. Telehealth adopters often expressed a need for condensed and easy to assimilate information to guide the rapid deployment of telehealth services. Experienced telehealth organizations like KUCTT were best positioned to respond to urgent telehealth educational needs and to deliver remote education. The success of KUCTT's multipronged, digital education strategy can be attributed to an ability to quickly adapt, mobilize, and disseminate existing telehealth resources and expertise. Educational strategies were developed to facilitate the adoption of telehealth innovations by healthcare organizations and providers during the COVID-19 PHE to improve healthcare outcomes for providers and patients alike. Approaches were devised to address the most relevant and pressing telehealth needs with a focus on providing assistance in a professional, timely, and convenient manner.

KUCTT's adoption of the i-PARIHS implementation framework contributed to the overall success of the multi-pronged, digital strategy in response to the high demand for telehealth education and support. KUCTT's telehealth experts, resources, and educational approaches served as facilitators to stimulate action around telehealth education and implementation by examining unique telehealth learning needs associated with the COVID-19 PHE and building upon relationships with healthcare organizations and providers. Adopting a rapid-cycle planning process allowed KUCTT to assess and measure feedback from learners to quickly inform, improve and deliver educational offerings consistent with the needs and preferences of telehealth learners. Examination of post-evaluation data, utilization/access data, and requests for on-going educational offerings indicate that KUCTT's strategic, educational approach was well-received and successful. The i-PARIHS implementation framework highlights and reinforces the primary lessons learned through the delivery of KUCTT's multipronged strategy. Expertise and resources can be disseminated in the most useful and efficient formats when the requestor’s context and specific learning needs are understood and continuously evaluated through engagement and feedback loops.

Telehealth associations, resource centers, and educators are encouraged to examine their existing resources as well as the contextual motivators for telehealth learning and support requests when considering pathways for supporting the telehealth needs of partner organizations, professional training programs, and individual providers. Efforts should be made to include mechanisms for engagement, quality improvement and consumer satisfaction to ensure needs are being addressing effectively and efficiently. KUCTT anticipates that learning preferences and needs will change over time as telehealth policy evolves and healthcare providers and organizations determine their telehealth futures and will continue to monitor and assess telehealth support requests as healthcare organizations and providers determine the future of telehealth lines of service. As educational and informational needs are defined, KUCTT will examine ways in which technology can be leveraged to advance the telehealth field.