Stakeholder perceptions of the use of a rapidly deployed modified ECHO to train and prepare healthcare providers for COVID-19.

Background: Innovative approaches to deliver timely information to rural healthcare providers are necessary with the COVID-19 pandemic. Project Extension for Community Healthcare Outcomes (ECHO) is a telementoring program designed to provide practitioners in rural communities with opportunities to engage in specialty training. We examined participant perceptions of a rapidly deployed, single continuing education session to improve healthcare provider preparedness for COVID-19 in Idaho.
Methods: A modified Project ECHO session was developed to inform providers about emergency preparedness, treatment, testing, and resources for COVID-19. A post-session survey examined session impact and barriers on clinical practice.
Results: Respondents believed the modified ECHO session increased COVID-19 knowledge and would improve their clinical practice and preparedness. Respondents were satisfied with the session and identified content, interdisciplinary collaboration, and format as beneficial; perceived barriers for utilizing session information included a lack of relevance of content and clinical applicability, and time constraints. Conclusions: A rapidly deployed modified Project ECHO session was perceived as an effective mechanism to foster collaboration and relay information to promote best practices at the start of the COVID-19 pandemic. An established Project ECHO network may be useful to rapidly exchange knowledge and information during a health emergency.

Background

Lack of access to healthcare and other health disparities related to the rural nature of Idaho present challenges for addressing the Coronavirus disease (COVID-19) pandemic, despite the state’s low population density offering some protection from the spread of infectious disease. The first case of COVID-19 in Idaho was identified on March 13, 2020 and healthcare providers were faced with caring for patients with limited resources and information. Thus, the development of a sustainable, state-wide public health information program to provide timely information, resources, and scientific evidence for the rural healthcare work force was needed.

Project Extension for Community Healthcare Outcomes (ECHO) is a tele-mentoring continuing education program designed to provide practitioners in rural and underserved communities with opportunities to engage in otherwise unavailable specialty training. Project ECHO enhances provider knowledge, competence, and clinical skills for delivering high-quality care to patients through didactic lecture, patient case presentations, and interdisciplinary discussion over a web-conferencing platform. The ECHO model presents an ideal mode for information delivery during a pandemic—it is accessible, time-efficient, and allows for timely dissemination and discussion of information at state, national, or international levels. Therefore, in March 2020, the ECHO Idaho team was tasked to develop and deliver a modified ECHO infectious disease COVID-19 session. The purpose of this paper was to examine session attendance and participant perceptions of the rapidly deployed, single modified ECHO session on self-reported provider preparedness for COVID-19 in Idaho to inform future use of the ECHO network infrastructure.

Methods

Idaho primary care providers (PCPs) and healthcare stakeholders were invited to participate in a modified ECHO Idaho Infectious Disease COVID-19 (ECHO Idaho IDC) tele-education session. Participants were recruited via email and through an established network of PCPs, state health associations, university leadership, and stakeholder contacts. Staff targeted PCPs and other health-related professionals to participate.

The ECHO Idaho IDC session was led by two physicians with expertise in public health, with additional Idaho-specific information (e.g. Idaho lab testing procedures) presented by other experts within Idaho agencies. The session was conducted using Zoom (Zoom Video Communications, Inc., San Jose, CA), an online video-conferencing program and occurred on March 24, 2020 in a 90-min format: two didactic lectures (~30 min; “Overview of COVID-19 Epidemiology in the US and Idaho” and “Fundamentals of Pandemic Epidemiology in Clinical Practice: A Case Study in Long Term Care”), question and answer/discussion with the expert panel (~30 min), and didactic lectures repeated (~30 min). Session content was intended to improve emergency preparedness and patient treatment based on available evidence and lessons learned from other states’ COVID-19 experiences, while also providing information on testing and available resources in Idaho. The session objectives included understanding clinical care fundamentals during a pandemic, long-term and post-acute care setting attributes relevant to all clinical care delivered during the COVID-19 pandemic, mitigation strategies for clinical settings, and patient case discussion. Attendees were eligible to claim free continuing medical education (CME) credit or a certificate of completion.

Post-session survey

An electronic post-session survey designed to meet CME requirements and evaluate session content was developed in Qualtrics (Qualtrics, LLC, Provo, UT) and included five sections: (1) demographics; (2) CME credit items; (3) session impact on knowledge; (4) clinical practice and preparedness; and (5) session perceptions. Participants responded to several different item types: multiple choice, Likert scales (1 = strongly disagree, 5 = strongly agree; 1 = extremely dissatisfied, 5 = extremely satisfied), sliding scales (0 = not at all beneficial, 100 = extremely beneficial), and open-ended. The post-session survey link was distributed via Zoom chat during the session and the ECHO Idaho website. All individuals who attended the ECHO Idaho IDC session were invited to complete the post-session survey. Attendance and demographic information were tracked by ECHO Idaho staff using the iECHO database (ECHO Institute, University of New Mexico).

Data analysis

Attendee demographic information were extracted from iECHO. Post-session survey data were exported from Qualtrics to Statistical Package for the Social Sciences version 25 (SPSS Inc., IBM Corp, Armonk, NY) for analyses. Blank and incomplete response entries were removed; all other entries were included for analyses. Descriptive data were reported as mean ± SD for continuous variables, or as a percentage for categorical variables. Responses to open-ended questions were analyzed using a general inductive approach to reduce data to core meanings. Categories were revised until core themes emerged from the findings. Consistency checks were completed by a separate researcher to address issues of item trustworthiness.

Ethics approval

The project was certified exempt by the Institutional Review Board and all participants provided consent prior to completing the survey.

Results

Demographics

Total attendance included 649 individuals from 325 health organizations across 13 states and Washington, DC (Table 1). Idaho attendees (n = 286, 83%) represented 58 Idaho communities and were located across all seven public health districts (Figure 1); approximately 30% were in a rural area. There were 197 post-session survey entries (30.4% response rate) from the 649 attendees; 177 entries constituted completed surveys and were included in the analysis. Respondents were from 11 states and Washington, DC; the majority resided in Idaho (n = 156, 88%) representing 40 different Idaho communities and all of Idaho’s public health districts (Table 1).

Table 1.

Demographics of survey respondents (n = 177) and all session attendees (n = 649), shown as n (%).

Variable	Survey respondents n (%)	All attendees n (%)
Location
Idaho	156 (88.1)	544 (83.1)
Health district
District 1	16 (10.3)	41 (7.5)
District 2	25 (14.1)	116 (21.3)
District 3	12 (6.8)	34 (6.3)
District 4	57 (32.2)	239 (43.9)
District 5	10 (5.6)	27 (5.0)
District 6	20 (11.3)	51 (9.4)
District 7	16 (9)	36 (6.6)
Currently practicing	140 (79.1)	–
Rural designation	–	187 (28.8)
Profession
Physician	49 (35)	151 (23.3)
Physician assistant	36 (25.7)	61 (9.4)
Nurse practitioner	5 (3.6)	20 (3.1)
Pharmacist	4 (2.9)	18 (2.8)
Nurse	12 (8.6)	40 (6.2)
Social worker	6 (4.3)	26 (4.0)
Counselor	5 (3.6)	19 (2.9)
Health profession student	–	55 (8.5)
Other (e.g. health educator, policy maker)	21 (15)	200 (30.8)
Unknown	2 (1.4)	59 (9.1)
Primary practice setting
Private group practice	41 (23.2)	–
Academic medical center	7 (4)	–
Federally qualified health center	23 (13)	–
Community health agency	18 (10.2)	–
Rural health center	11 (6.2)	–
Other	38 (21.5)	–
Unknown	39 (22)	–
Primary practice location designation
PC-HPSA	32 (18.1)	–
MUA/MUP	27 (15.3)	–
Neither PC-HPSA or MUA/MUP	86 (48.6)	–
Unknown	32 (18.1)	–

MUA/MUP: medically underserved area and population; PC-HPSA: primary care health professional shortage area.

Figure 1.

Learner hours of contact of ECHO Idaho Infectious Disease COVID-19 attendees by city and separated by health district. Health districts are designated by number (1–7) and counties pertaining to each health district are displayed in differing shades of gray. One learner hour of contact represents 60 min of attendee participation.

Session impact

Respondents agreed or strongly agreed the session increased their COVID-19 knowledge and felt participation was impactful for clinical practice and preparedness (Table 2). Most respondents agreed the ECHO Idaho IDC session increased overall knowledge (n = 149, 88.2%), understanding of COVID-19 transmission (n = 145, 85.8%), where to find accurate COVID-19 information (n = 144, 85%), which individuals/patients are at higher risk for COVID-19 (n = 152, 90%), sanitation practices (n = 111, 65.7%), referral practices (n = 124, 73.4%), how to use PPE (n = 94, 55.6%), how to identify areas to separate individuals/patients with respiratory symptoms from others seeking care (n = 100, 59.2%), and best practice treatment options for individuals/patients diagnosed with COVID-19 (n = 106, 62.7%). Respondents also agreed that participation in the ECHO Idaho IDC session better prepared them to provide recommended care (n = 118, 70.3%), to change their work/clinical practice to meet current recommendations (n = 107, 63.7%), and to meet the needs of their patients (n = 104, 61.9%), community (n = 126, 75%), and state (n = 114, 67.9%). Most respondents reported session participation provided an increased sense of security (n = 111, 66.1%), a decreased sense of professional isolation (n = 118, 70.2%), the ability to safely care for themselves (n = 103, 61.3%) and calmly work with others through the pandemic (n = 126, 75%), and improved clinical practice (n = 93, 55.4%).

Table 2.

Responses to section 3 (n = 169) and section 4 (n = 168) “session impact” Likert scale items. Data are displayed as n (%).

The ECHO COVID-19 session increased my understanding regarding. . .	Strongly agree	Agree	Neither agree nor disagree	Disagree	Strongly disagree	N/A
Overall knowledge of COVID-19	49 (29.0)	100 (59.2)	10 (5.9)	6 (3.6)	4 (2.4)	–
Where to find accurate information about COVID-19	57 (33.5)	87 (51.5)	18 (10.2)	4 (2.4)	3 (1.8)	–
Transmission of COVID-19	48 (28.4)	97 (57.4)	18 (10.7)	4 (2.4)	2 (1.2)	–
Which individuals/patients are at higher risk for COVID-19	77 (45.6)	75 (44.4)	13 (7.7)	2 (1.2)	2 (1.2)	–
Sanitation practices regarding COVID-19	35 (20.7)	76 (45.0)	44 (26.0)	12 (7.1)	2 (1.2)	–
Referral information for patients, faculty, and staff who may have been exposed to COVID-19	41 (24.3)	83 (49.1)	31 (18.3)	13 (7.7)	1(0.6)	–
How to protect myself using personal protective equipment (PPE)	32 (18.9)	62 (36.7)	49 (29.0)	25 (14.8)	1 (0.6)	–
How to prepare alternative staffing plans for my work/clinical practice	27 (16.0)	53 (31.4)	57 (33.7)	29 (17.2)	3 (1.8)	–
How to identify areas to separate individuals/patients with respiratory symptoms from other individuals/patients seeking care	34 (20.1)	66 (39.1)	44 (26.0)	22 (13.0)	3 (1.8)	–
Best practice treatment options for individuals/patients diagnosed with COVID-19	34 (20.1)	72 (42.6)	40 (23.7)	17 (10.1)	6 (3.6)	–
As a result of my participation in this COVID-19 session. . .
I will change my clinical practice	12 (7.1)	57 (33.9)	63 (37.5)	8 (4.8)	2 (1.2)	26 (15.5)
I am better prepared to provide recommended care to individuals/patients	27 (16.1)	91 (54.2)	31 (18.5)	5 (3.0)	3 (1.8)	11 (6.5)
I feel an increased sense of security due to my increased knowledge	24 (14.3)	87 (51.8)	43 (25.6)	8 (4.8)	2 (1.2)	4 (2.4)
I am better prepared to meet the needs of my patients	24 (14.3)	80 (47.6)	39 (23.2)	5 (3.0)	3 (1.8)	17 (10.1)
I feel more able to safely care for myself	25 (14.9)	78 (46.4)	51 (30.4)	6 (3.6)	4 (2.4)	4 (2.4)
I am better prepared to meet the needs of my community	26 (15.5)	100 (59.5)	31 (18.5)	6 (3.6)	2 (1.2)	3 (1.8)
I feel capable to share recommendations with individuals/patients	31 (18.5)	109 (64.9)	23 (13.7)	1 (0.6)	2 (1.2)	2 (1.2)
I am better prepared to meet the needs of my state	26 (15.5)	88 (52.4)	45 (26.8)	6 (3.6)	1 (0.6)	2 (1.2)
I feel more able to calmly work with others through this pandemic	25 (14.9)	101 (60.1)	32 (19.0)	5 (3.0)	1 (0.6)	4 (2.4)
I am better prepared to change my work/clinical practice to meet current recommendations	20 (11.9)	87 (51.8)	43 (25.6)	6 (3.6)	1 (0.6)	11 (6.5)
I feel a decreased sense of professional isolation	34 (20.2)	84 (50.0)	33 (19.6)	12 (7.1)	2 (1.2)	3 (1.8)
My clinical practice will improve	22 (13.1)	71 (42.3)	50 (29.8)	5 (3.0)	1 (0.6)	19 (11.3)

Overall, the participants rated the session (using a sliding scale from 0 to 100) as greatly beneficial because it provided “real-time” access to COVID-19 expertise (n = 165, 87.4 ± 20.9), rapid access to COVID-19 training (n = 164, 83.0 ± 21.6), CME credit (n = 165, 84.1 ± 25.7), a recorded session (n = 165, 88.3 ± 20.7), did not require travel (n = 165, 92.9 ± 13.9), and helped develop a COVID-19 network of professional colleagues (n = 164, 77.1 ± 27.4). Additionally, the majority of respondents (91%) were satisfied with the session and indicated they would participate in future ECHO Idaho IDC sessions.

Three main themes emerged for beneficial aspects of the session (n = 142): didactic content (74.6%), ability to collaborate with a network of interdisciplinary providers (21.8%), and format of session (19.7%). Out of the 125 respondents who provided suggestions for improvement for future ECHO Idaho IDC sessions, five main themes emerged: effective in current format (36%), additional in-depth content (24%), session format issues (16%), additional sessions (11.2%), and inadequate response to questions (8.8%). Finally, of 11 responses provided for perceived barriers, two main themes emerged: relevance of content and clinical applicability (55%) and time constraints (36%). Examples of individual responses supporting each theme are depicted in Table 3.

Table 3.

Themes from open-ended questions and examples of corresponding responses.

Question	Theme	Responses
Most beneficial aspects (n = 142)	Didactic content (n = 106)	“Received up to date information from trusted sources as well as resource links”
		“Up to date information from our state”
		“Access to the state leaders in response to this event,” “the combination of state and local level overview combined with clinical expertise”
		“The access to professionals that are on the front lines of this virus”
		“Feeling like healthcare workers in Idaho are on the same page and being informed together of resources and updates regarding testing and PPE”
	Ability to collaborate with a network of interdisciplinary providers (n = 31)	“Great group of varied professionals”
		“Networking with colleagues”
		“Network of professionals in our state coming together”
		“Seeing so many people involved and wanting to be in this together”
	Session format (n = 28)	“Online, specific to Idaho, easy to access”
		“Online availability”
		“Easy access without travel”
Suggestions for future session improvement (n = 125)	Effective in current format (n = 45)	“Nothing, well presented”
		“Nothing, it was great”
	Additional in-depth content (n = 30)	“More information on PPE”
		“More practical advice on PPE and exposure control in the office”
		“More specific clinical information”
	Session format issues (n = 20)	“Faulty internet connection”
		“90 min was a little difficult to carve out of my schedule, 60 min would be preferred”
		“The chat box going off all the time is very distracting”
	Additional sessions (n = 14)	“Future sessions to focus on treatment”
		“Reoccurrence of meetings to continue knowledge, communication”
		“Would like to see follow-up session reviewing mitigation plans for those who lose work, insurance, need meds”
	Inadequate response to questions (n = 11)	“More answers”
		“Didn’t answer any of my big questions/concerns”
		“Would have liked more time for answering questions about lab testing and PPE”
Perceived barriers (n = 11)	Relevance of content and clinical applicability (n = 6)	“Relevance of topic”
		“Not clinically relevant”
		“More practical guidance on the next round of talks”
	Time constraints (n = 4)	“Time/work constraints”
		“Timing”

Discussion

We examined participant perceptions of a rapidly deployed, single, modified ECHO Idaho session on provider preparedness for COVID-19 to explore potential benefits of using a modified ECHO model for delivering training during a pandemic. Project ECHO and replication models have been effective for upscaling clinical skills of PCPs,[4,7] decreasing barriers for participation in educational opportunities, enhancing knowledge, and improving patient care. Our novel findings provide preliminary evidence that the delivery of a single, modified ECHO session may be an effective public health tool for quickly disseminating information and connecting professionals across communities to provide a coordinated, evidence-based response.

The ECHO Idaho IDC session reached a diverse healthcare professional sample across 13 states and Washington DC and over 50 communities (~30% rural) in the USA. In comparison, ECHO Idaho series’ typically average 26 participants per session. Although data is limited, Project ECHO has been used for collaboration and knowledge dissemination during the COVID-19 pandemic.[9,10] The Oregon ECHO Network successfully launched a rapid deployment COVID-19 Response ECHO session for healthcare providers, reaching 737 health professionals (25% practicing in rural areas). A weekly COVID-19 ECHO series in Oklahoma attracted upwards of 500 participants compared to typical Oklahoma ECHO sessions averaging 20 participants. The ECHO Idaho IDC session reached a substantial number of participants (n = 649), especially considering Oregon and Oklahoma have populations more than double that of Idaho, and greatly exceeded prior Project ECHO participation in Idaho.

Nearly all respondents reported their overall COVID-19 knowledge improved like previous research on ECHO Idaho programming and other ECHO literature.[11-13] Most respondents indicated their knowledge regarding locating resources, transmission, best-practices, PPE utilization, and sanitation for COVID-19 increased after participation. Respondents indicated feeling better prepared to face various pandemic-related tasks at the patient, community, and state level, and to change their practice based on session participation. Our findings are consistent with rural attendees of the Oregon COVID-19 Response ECHO, who indicated they would implement changes to their practice after attending the session. Future research should explore the impact participating in a single ECHO session has on outcomes related to clinical practice changes.

Providers treating COVID-19 patients on the frontline are at an increased risk for developing depression, anxiety, stress, and insomnia. Having a support system during a pandemic is crucial and may help decrease the risk for mental health disorders. Our respondents indicated feeling reduced isolation and enhanced community support (Table 2), which is consistent with previous Project ECHO findings.[4,16] The consistent findings suggest ECHO participation provides professional support and this may be beneficial for reducing anxiety, stress, and depression among providers.

Providing free training through an online platform for synchronous and asynchronous viewing reduces participation barriers. Importantly, the ECHO model is also compliant with pandemic interventions designed to “flatten the curve.” Most respondents (91%) indicated session satisfaction, specifically with accessibility and content, consistent with previously published data.[8,13] Open-ended feedback indicated learning about Idaho-specific COVID-19 data directly from experts provided streamlined, unbiased information that impacted understanding of COVID-19 (Table 3). Further, most respondents indicated they would participate in future ECHO Idaho IDC sessions providing support for the model during a pandemic.

Respondents identified barriers and areas for improvement related to session specifics, clinical relevance, and time constraints (Table 3). For example, a longer session to allow more in-depth answers was cited as being beneficial. Some concerns may be related to the limited time presenters had to prepare for the session; however, the rapid turnaround used in Idaho was consistent with other just-in-time ECHO COVID-19 sessions.[9,10] Additionally, respondents suggested a session occurring mid-day/during lunch and lasting 60 min would possess the fewest attendance barriers. Understanding participant preferences for session delivery may be useful for others who want to implement ECHO sessions quickly and maximize session attendance.

Limitations and future research

The true Project ECHO model consists of four main components: (1) amplification of information by leveraging technology, (2) evidence-based best practice information, (3) case-based learning, and (4) monitoring impact through outcomes. Similar to other just-in-time ECHO COVID-19 sessions, case-based learning was not emphasized in the traditional fashion. A case study specific to COVID-19 in Idaho was not used because there was only one confirmed COVID-19 case in Idaho and little evidence-based best practice information was available at the time. Further, the didactic presentations were delivered twice to maximize reach and attendance (e.g. Idaho has two time zones); hence, the session was considered a “modified” ECHO session. Although this may impact the fidelity of the session as it relates to the Project ECHO model, the decision was made to maximize the ability of Idaho providers and agencies to convene and coordinate a response that was as rapid and informed as possible. Previous literature has indicated a relationship between Project ECHO and other health agencies provides an important foundation for the rapid assembly of necessary professionals to collaborate in an emergency scenario.

The ECHO Idaho IDC session hosted the largest number of Idaho participants to date, yet the post-session survey was only completed by 30% of attendees. It is possible attendees did not complete the survey for multiple reasons: (1) attendees may not have been satisfied with the program; (2) partial attendees may have felt they could not provide ample feedback; (3) attendees may not have been willing to spend time filling out a survey given time constraints faced by clinicians; or, (4) attendees may have experienced response fatigue from numerous ECHO emails and survey requests. Thus, future research is warranted to explore reasons or barriers for ECHO participant survey completion.

Finally, it should be noted that we report outcomes from a single modified ECHO session at the lower tiers of CME evidence. The timing of an ECHO session with a rapidly developing novel disease makes assessment of certain outcome levels (e.g. patient health status changes) challenging. Future research examining the Project ECHO response to the COVID-19 pandemic or future health emergencies should aim to assess higher tier CME evidence (e.g. changes in patient or community health), as these outcomes are needed to better understand model efficacy.

Conclusion

The use of a single, rapidly deployed modified ECHO Idaho IDC session was well-received; respondents indicated session participation improved COVID-19 knowledge, clinical practice, and preparation. Respondents also reported high levels of satisfaction and perceived several benefits. The use of Project ECHO to connect providers across Idaho, a primarily rural state, in a rapid manner was an effective mechanism to foster collaboration and relay information during a pandemic. An established Project ECHO network may be a useful tool for rapidly exchanging knowledge and accurate information between public health experts and healthcare providers in rural areas during a health emergency.