Educating health professionals about COVID-19 with ECHO telementoring.

BACKGROUND: When the COVID-19 pandemic began, primary care clinicians had almost no knowledge regarding best practices COVID-19 treatment. Project ECHO developed a COVID-19 Infectious Disease Office Hours (Office Hours) program to respond to the needs of clinicians seeking COVID-19 information.
METHODS: This mixed-methods evaluation analyzed weekly post-session data and focus group results from the weekly Office Hours ECHO sessions during June 1, 2020- May 31, 2021.
RESULTS: A total of 1,421 participants attended an average of 4.9 sessions during the 45 Office Hours sessions studied. The most common specialties included: nurses= 530 (37%), physicians= 284 (20%), and 493 (34%) having other degrees. The participants stated that they were definitely (68.2%) or probably (22.0%) going to use what they learned in their work, especially vaccination information. Focus group results identified these themes: 1) quality information, 2) community of practice, 3) interprofessional learning, and 4) increased knowledge, confidence, and practice change.
CONCLUSIONS: This evaluation demonstrates that the Office Hours program was successful in bringing a large group of health professionals together each week in a virtual community of practice. The participants acknowledged their plans to use the information gained with their patients. This diffusion of knowledge from clinician to patient amplifies the response of the program, changes practice behavior and may improve patient care.

Background

The COVID-19 pandemic has had a devastating impact on the world. Over 200 million people have been infected and over 4 million have died. At the beginning of the pandemic, primary care clinicians across the globe had little knowledge about the transmission, diagnosis or treatment of COVID-19 or the public health measures required for their patients, communities and themselves.2, 3, 4 There was an urgent need to rapidly adapt to a changing environment and learn about route of transmission, protection from infection, management and treatment, infection control, and other public health measures. Due to the travel restrictions and requisite physical distancing, most primary care clinicians significantly increased their use of virtual learning methods to obtain the just-in-time information needed to care for their patients. ,

The University of New Mexico's Project ECHO (Extension for Community Healthcare Outcomes) telementoring program developed an innovative COVID-19 Infectious Disease Office Hours ECHO (Office Hours ECHO) program on March 23, 2020 to respond to the needs of New Mexico clinicians seeking best practices information regarding COVID-19. The Office Hours ECHO connects clinicians, public health practitioners and educators (aka “spokes”) with subject matter experts (SMEs) at the “hub” for 1 hour weekly sessions in which the spoke participants receive up-to-date information regarding the COVID-19 pandemic.

The goal of Project ECHO, a virtual telementoring network, is for SMEs to share best practices information and allow participants in rural and urban underserved communities around the globe to present case-based consultations and/or questions to the SMEs at the hub. There are currently 602 hubs throughout the globe in 45 countries running 1300 networks providing up-to-date content for more than 70 different medical, public health and education topics. Project ECHO telementoring is always virtual and synchronous, providing no-cost continuing education credits to the participants.

Methods

The Office Hours ECHO hub team consists of 6 members: the University of New Mexico's Chief of Infectious Diseases, a leading community primary care physician, an infectious disease epidemiologist from the New Mexico Department of Health (NM DOH), the medical director of Tricore Reference Laboratories, and 2 experienced Project ECHO physician facilitators. During the 60 minute Office Hours ECHO each week, the hub team provides up to 30 minutes of best practices information with or without a power point presentation. The hub team facilitators then lead a thirty minute Q/A session whereby spoke participants enter questions into the Zoom chat box and the hub team answers as many questions as time permits. When the hub team does not know the answer to a question the facilitators will often ask if any of the spokes have relevant information to share with the group.

Participant attendance

The sessions were held virtually using Zoom. Prior to the session, participants registered to receive the session link. As part of this registration process, participants provided city, state, and licensure information (MD, nurse, etc.). Demographics such as age, gender, and race/ethnicity were not collected as a part of the program. Once the session started, the Zoom platform recorded program attendance. This attendance information was linked to the registration data using email. If a participant's email could not be linked to the registration data (eg, they called in or used a shared link) then that participant was not included in the counts for licensure or state but was included in the total attendance count. The ECHO evaluation team calculated attendance by session, number of sessions attended by participant, and attendance by state. Attendance by state was mapped using Tableau.

Post-session evaluations

The Office Hours ECHO sessions provided free continuing education units (CEU) to participants. In order to receive credits, participants filled out an immediate post-session evaluation survey. The survey queried participants about their satisfaction with the session (did the session meet its objectives, was the content balanced and evidence-based, did they have the opportunity to ask questions, and what they will use from the session). Responses from the weekly surveys from June 1, 2020 to May 31, 2021 were aggregated and the ECHO evaluation team summarized number and percent of responses (not unique individuals). Open-ended responses to what participants plan to use from the session were grouped into the following categories: resources provided (eg, articles, websites, hotlines, tools); information on COVID-19 vaccinations; patient care (treatment, co-occurring disorders, and managing long-term sequela); prevention measures (masks, physical distancing, sanitization, and air filters); COVID-19 testing; viral variants; and viral transmission. eg, if a participant wrote in “Understanding longer term symptoms from COVID in 10% of the population and how to identify,” this was categorized as “Information on patient care.” Additionally, a number of participants wrote in some version of “I will use all of it” or “everything” without outlining specific aspects of the program, so this was made into its own category. The categories were not mutually exclusive and counts and percentages are for the number of responses rather than individuals (eg, if 1 individual said that they would use information on vaccines after 1 session and use the resources presented in the session after another session, this was counted as 2 responses).

Focus groups

In December 2020, participants who attended four or more sessions were invited to attend a focus group to better understand what aspects of the program were most impactful. A total of 18 people attended 1 of the 3 60 minute sessions. Participants were categorized as New Mexico (NM) or non-NM participants, and also categorized as either clinical (MD, DO, RN, etc.) or non-clinical (primarily educators). Focus groups were moderated by 2 trained ECHO research staff. Sessions were recorded, transcribed, and coded using Dedoose. Transcripts were coded separately by 2 ECHO evaluation team members and then categorized into broader themes. Participants were asked the following questions:

How has the Office Hours ECHO impacted your confidence in your ability to manage COVID-19?

What information do you feel is most useful for patients’ treatment?

How has this ECHO changed your practice and your work in regards COVID-19?

Do you feel this program is helpful in building a community of practice?

What has this program provided or not provided for boosting your own resiliency and self-care related to COVID?

Are the updates helpful? (For example the updates from the hub team members?)

Because this is not a typical ECHO format using case-based learning at every session, what are your thoughts on the sessions?

You may have noticed the digital librarian putting resources in the chat. In what ways is the librarian helpful or not helpful?

What keeps you coming back to the program?

Are there any recommendations for us to improve this Office Hours ECHO?

Digital librarian and session Q&A

A dedicated digital librarian team is also part of the Office Hours ECHO. They enter the evidence-based and peer-reviewed material presented during the entire session into the Zoom chat so that participants can obtain this information immediately or retrieve it later in the week on the Office Hours website.

During each session, participants are encouraged to enter their questions directly pertinent to the didactics being discussed and/or the Q/A being answered into the Zoom chat box. The hub team facilitators try to get as many questions answered as time allows. Most of the Q/A is addressed by the hub team facilitators asking the question for the spoke participant. Time permitting, the hub team facilitators will ask spoke participants to unmute their video so that the participant themselves can speak and ask, or clarify, their question.

This evaluation was approved by the University of New Mexico Institutional Review Board (#04 –341).

Results

Attendance

A total of 1,421 unique participants attended the 45 COVID-19 Office Hours ECHO sessions between June 1, 2020 and May 31, 2021. Attendance ranged from 133 in June 2020, peaked at 314 attendees in February 2021 and leveled out to 211 attendees in May 2021. While nearly half (N = 692, 48.7%) of participants attended a single session, a third (N = 431, 30.3%) of participants attended regularly (four or more sessions). Overall, participants attended an average of 4.9 sessions. A total of 284 (20%) attendees were physicians, 102 (7%) were nurse practitioners or physician assistants, 530 (37%) were nurses, 12 (1%) were physical/occupational therapists, and 493 (34%) had some other degree (bachelors, no degree, etc.). Compared to other professions, physicians and nurses were more likely to attend four or more times (N = 143, 33.2%, P <.01 and N = 128, 29.7%, P <.01, respectively). New Mexico had the highest number of attendees (N = 482, 33.9%). After NM, neighboring states (California, Arizona, Texas) had the highest attendance (Fig 2). Overall, there were attendees from 49 states and Puerto Rico. See (Fig. 1 A and 1B).

Fig 2

COVID-19 office hours ECHO attendance by state/territory, June 1, 2020 – May 31, 2021.

Fig 1

COVID-19 office hours ECHO attendance by session (A) and number of sessions attended by participants (B), June 1, 2020 – May 31, 2021.

Post-session survey

Generally, participants felt that the objectives of the sessions were met; the sessions delivered balanced, objective, and evidence-based content; and that there were excellent or very good opportunities to ask questions. Participants were probably (22.0%) or definitely (68.2%) going to use what they learned in their work. Participants were most likely to report using the sessions’ information on vaccination. Participants were least likely to report using the sessions’ information on special populations (eg, pregnancy, long-term care, pediatrics, etc.). See (Table 1 ).

Table 1

Office hours COVID-19 ECHO post-session evaluation survey responses, june 1, 2020 to May 31, 2021, N = 3,530 responses

	N (%)
	Poor	Fair	Good	Very good	Excellent
How well were the stated objectives met?	0 (0.0)	0.0 (0.0)	127 (3.6)	917 (26.0)	2,486 (70.4)
How well did the session deliver balanced and objective, evidence-based content?	0.0 (0.0)	3 (0.1)	100 (3.0)	860 (24.4)	2,567 (72.7)
Opportunities to ask questions were:	7 (0.2)	25 (0.7)	159 (4.5)	898 (25.4)	2,441 (69.2)
	Definitely Not	Probably not	Possibly	Probably yes	Definitely yes
Will you use what you learned in this session in your work?	25 (0.7)	93 (2.6)	481 (13.6)	777 (22.0)	2,149 (60.9)
What, if anything, do you plan to use from this session?
Response Category				N (%) of Responses
Information on vaccination				570 (27.8)
Information on patient care				592 (28.9)
Information on prevention measures (masks, physical distancing, etc.)				215 (10.5)
Information on testing				210 (10.2)
All of it/everything				200 (9.8)
The resources provided				147 (7.2)
Information on viral variants				116 (5.7)
Information on viral transmission				91 (4.4)
Information on special populations (pregnancy, long-term care, etc.)				88 (4.3)

In the 3 focus groups, 5 broad themes were identified describing participants’ experiences with the Office Hours ECHO sessions. The sessions: 1) provided quality information, 2) built a community of practice, 3) were impacted by healthcare workers fear of COVID-19, 4) provided an interdisciplinary ECHO format of training, and 5) increased knowledge and confidence, and changed practice. eg, there were 32 instances where participants shared examples of applying the information they learned in their practice. These instances were much more common among clinicians (N = 29, 90.6%) then among non-clinicians (N = 3, 9.4%). See (Table 2 ).

Table 2

Office hours COVID-19 focus group themed responses

Themes	Total	NM	Non-NM	Clinical	Non-clinical
ECHO provided quality information	N	N (%)	N (%)	N (%)	N (%)
Informative	37	15 (40.5)	22 (59.5)	20 (54.1)	17 (45.9)
Information is up to date	13	5 (38.5)	8 (61.5)	7 (53.8)	6 (46.2)
Information reduces fears	9	5 (55.6)	4 (44.4)	7 (77.8)	2 (22.2)
Corrected misinformation	8	3 (37.5)	5 (62.5)	2 (25.0)	6 (75.0)
Educated about vaccines	5	3 (60.0)	2 (40.0)	2 (40.0)	3 (60.0)
Liked evidence based information	2	0 (0.0)	2 (100.0)	1 (50.0)	1 (50.0)
Peer-reviewed and patient-centered resources	2	0 (0.0)	2 (100.0)	2 (100.0)	0 (0.0)
Attending the ECHO built community
Community of practice	3	2 (50.0)	2 (50.0)	3 (100.0)	0 (0.0)
Sense of support from ECHO	2	2 (100.0)	0 (0.0)	2 (100.0)	0 (0.0)
External factors impacting participants’ ability to use information
Lots of fears among health care workers	3	1 (33.3)	2 (66.7)	1 (33.3)	2 (66.7)
Lack of reliance on local DOH	1	1 (100.0)	0 (0.0)	0 (0.0)	1 (100.0)
ECHO Format
Like interdisciplinary ECHO format	19	11 (57.9)	8 (42.1)	9 (47.4)	10 (52.6)
Hub team members are knowledgeable	19	9 (47.4)	10 (52.6)	15 (78.9)	4 (21.1)
Suggestions for program improvement	12	2 (16.7)	10 (83.3)	11 (91.7)	1 (8.3)
Like digital library team/resources	8	6 (75.0)	2 (25.0)	6 (75.0)	2 (25.0)
Diffusion of Knowledge
Use information in practice	32	8 (25.0)	24 (75.0)	29 (90.6)	3 (9.4)
Participants pass on information to others	18	6 (33.3)	12 (66.7)	9 (50.0)	9 (50.0)
Use information to educate others	13	3 (23.1)	10 (76.9)	10 (76.9)	3 (23.1)
Increase confidence	12	6 (50.0)	6 (50.0)	10 (83.3)	2 (16.7)
Given respect and perspective for frontline workers	4	2 (50.0)	2 (50.0)	1 (25.0)	3 (75.0)
Use information make protocols	3	0 (0.0)	6 (100.0)	3 (100.0)	0 (0.0)

Digital librarian

During the 45 Office Hours ECHO sessions, the digital librarians provided links to 766 references, an average of 17 references per session. During each session, approximately half of the references corresponded to links from journal articles embedded in the SMEs presentations and half corresponded to sources cited by SMEs during the Q/A period. These included peer-reviewed studies, current guidance from the NM DOH, the Centers for Disease Control and Prevention (CDC), the World Health Organization (WHO), the National Institutes of Health (NIH) COVID-19 Treatment Guidelines, and nationally-recognized infection control and infectious disease organizations.

Questions and answers during the session

The Office Hours hub team answered 62 percent of all questions entered in the chat on average during each session. This may be an underestimation as many unanswered questions were duplicates or variations of the same question. The average number of questions entered during each weekly session was 21.9 (range: 7 –30). The unanswered questions were kept until the following week's session when the hub facilitators tried to incorporate the questions into the program.

Discussion

This evaluation demonstrated that the Office Hours ECHO participants are highly satisfied with the program in many ways: 1) the interdisciplinary and interprofessional ECHO format, 2) delivery of best practices information, 3) the bi-directional discussion between hub and spoke members, 4) the community of practice, and 5) the diffusion of knowledge. The spoke participants described that they benefitted from the knowledge gained regarding many COVID-19 topics including: vaccination, testing, masking and other public health prevention measures.

The largest group of clinicians represented were nurses, who accounted for 52% (N = 533) of unique participants. The next largest group of unique clinicians attending were physicians (N = 261, 18%). One possible reason for the high nursing participation is that they have been on the frontlines of the COVID-19 pandemic, in the outpatient, inpatient and community settings. In addition, nurses in the U.S outnumber physicians 10:3, with approximately four million active nurses in the US as compared to slightly fewer than one million active physicians. This difference in attendance may also reflect the need for this program within the nursing community. ,

The Office Hours ECHO had a large proportion of their participants register as non-clinician participants (N = 512, 36%). These participants, including educators, hospital and clinic administrative staff and public health practitioners may be attending these sessions to seek information to better inform their students, colleagues and constituents, for the same reasons that the clinician group has benefitted from the ECHO sessions. For the COVID-19 pandemic, educating non-clinical personnel and rethinking how clinical personnel might work more effectively is critical to increase awareness of public health measures vitally important to combatting this virus. , This program has created a novel and sustained virtual community of practice among primary care clinicians and non-clinical health professionals that continues today.

The Office Hours ECHO program has had a large number of unique participants that continue to attend this program almost every week- even though they have attended multiple sessions of the program already. In most ECHO programs, attendance tends to diminish over time as participants increase their knowledge and self-efficacy in the particular topic. Given the large amount of new information related to COVID-19, however, this Office Hours ECHO has had a steady increase in the average number of participants per weekly session. Additionally, it is quite rare for an ECHO program to have unique participants attend for so many sessions. The fact that 53 participants attended at least 21 sessions and 58 participants attended at least 31 sessions during the evaluation period (June 1, 2020-May 31, 2021) demonstrates the value of this program for so many clinical and non-clinical healthcare professionals.

Although the Office Hours ECHO was originally designed for NM clinicians, it quickly became a national COVID-19 telementoring program. Despite some of the information delivered being specific to NM, especially the NM DOH epidemiological data, the participants outside of NM stated (in the focus groups) that they are benefitting equally from the program content. This suggests that the Office Hours ECHO is filling an information gap by delivering just-in-time, evidence-based content relevant for a national audience of both clinical and non-clinical participants and critical for the pandemic response. Similar to a recent study by Bessesen, et al., the participants improved their knowledge and skills about infection prevention measures while participating in this educational program. The Project ECHO Chicago team led a COVID-19 telementoring program for health professionals working in skilled nursing facilities in the Chicago area. Although they studied a much more specific target audience, they found similar results as the Office Hours ECHO, with an increase in knowledge and skills gained regarding vaccines, infection prevention control and other public health measures for the participants.

Fidelity to the ECHO model is a concept that is important to maintain because the success of Project ECHO telementoring relies on certain key principles. Unlike a simple webinar, this Office Hours ECHO program fulfills the ECHO model criteria: 1) best practices information, 2) bidirectional learning= “all-teach, all-learn,” 3) case-based learning, 4) program evaluation, and 5) use of a virtual and synchronous platform. The hub team members made a conscious decision at the launch of this program not to include cases as a major component of each session due to the overwhelming amount of information that needed to be disseminated each week. Brief cases are still presented occasionally by spoke clinicians, however it is not the main part of the session.

There have been many innovative aspects to the Office Hours ECHO that have increased participants’ satisfaction with the program: 1) The digital librarian team brings just-in-time resources to every session. These resources are added to the Project ECHO Office Hours website within 1 to 2 days. It is quite possible that the digital librarian reinforces the notion of ECHO as a trusted source for evidence-based knowledge. , By entering the SME's articles and other resources into the Zoom chat immediately, the participants may be comforted by the knowledge that the hub teams’ information updates are peer-reviewed and evidence-based. 2) Thirty minutes of robust Q/A by the interdisciplinary hub team of infectious disease, primary care, epidemiology and laboratory science is provided to an engaged group of participants following the didactic session. 3) The 2 dedicated ECHO hub facilitators provide a degree of constancy and professionalism to the program that allows the SMEs on the hub team and the participants to feel comfortable and valued during each session.

Office hours ECHO limitations and suggestions for improvement

The weekly Office Hours program has several limitations. The most significant include: 1) not being able to answer every question submitted by participants due to limited time, 2) inability to include case-based learning in every session, and 3) inherent trade-offs of using a virtual telementoring platform instead of an in-person venue. The authors have begun to limit the time of the didactic presentations to allow ample time for more bi-directional questions and answers between the hub team subject matter experts and the participants. In addition, real, de-identified cases are being included whenever possible.

Conclusions

Due to the continuation of the pandemic and the robust participation of the Office Hours ECHO, this weekly telementoring program may be a needed educational forum for both clinical and non-clinical health professionals. This innovative COVID-19 telementoring program is providing just-in-time evidence-based information to both clinical and non-clinical health professionals, who are amplifying the response by diffusing their newly acquired knowledge to both patients and colleagues.