ECHO Care of the Elderly: Innovative Learning to Build Capacity in Long-term Care.

BACKGROUND: Older adults are entering long-term care (LTC) homes with more complex care needs than in previous decades, resulting in demands on point-of-care staff to provide additional and specialty services. This study evaluated whether Project ECHO® (Extension for Community Healthcare Outcomes) Care of the Elderly Long-Term Care (COE-LTC)-a case-based online education program-is an effective capacity-building program among interprofessional health-care teams caring for LTC residents.
METHODS: A mixed-method, pre-and-post study comprised of satisfaction, knowledge, and self-efficacy surveys and exploration of experience via semi-structured interviews. Participants were interprofessional health-care providers from LTC homes across Ontario.
RESULTS: From January-March 2019, 69 providers, nurses/nurse practitioners (42.0%), administrators (26.1%), physicians (24.6%), and allied health professionals (7.3%) participated in 10 weekly, 60-minute online sessions. Overall, weekly session and post-ECHO satisfaction were high across all domains. Both knowledge scores and self-efficacy ratings increased post-ECHO, 3.9% (p = .02) and 9.7 points (p < .001), respectively. Interview findings highlighted participants' appreciation of access to specialists, recognition of educational needs specific to LTC, and reduction of professional isolation.
CONCLUSION: We demonstrated that ECHO COE-LTC can be a successful capacity-building educational model for interprofessional health-care providers in LTC, and may alleviate pressures on the health system in delivering care for residents.

INTRODUCTION

With the rapid growth of the aging population in North America, the demand for specialized long-term and geriatric care for clients with complex care needs is increasing.( Long-term care (LTC) homes, also referred to as skilled nursing facilities or nursing homes, are a residential option which include a broad range of services aimed to meet the needs of older adults with frailty, dementia, and other impairments who can no longer be cared for in the community.( In Ontario, Canada, the profile of residents served in LTC homes has changed significantly over the last 10 years, with increasing levels of acuity, frailty, and complexity requiring higher levels of care than in the past.( To qualify for admission, new residents must now present with significant physical and/or cognitive challenges. For example, approximately 90% of LTC residents in Ontario have cognitive impairment, including dementia, and approximately 80% of residents with dementia experience behavioural and psychological symptoms, including behaviours that present safety concerns.( The number of residents needing extensive support with activities of daily living, such as grooming, dressing, and eating, has also risen from 79% to 86% over the last five years, representing an additional 9,000 people needing significant assistance.(

As an increasing number of older adults enter LTC in the late stages of cognitive and physical decline, demands on point-of-care staff to provide additional and specialized care have intensified. Given the pressures on the provincial health system to address the needs of Ontario’s older adults, innovative solutions to improve quality of care in LTC are necessary. This includes education and training to optimize skills and knowledge of staff.( Continuing professional development programs focused on key topics in geriatrics that build capacity of staff and physicians in managing increased residents’ needs may lead to better quality of care. One program that has demonstrated the ability to build capacity in primary care providers (PCPs) across various medical specialities, including geriatrics, is Project Extension for Community Healthcare Outcomes (ECHO). Developed in 2003 at the University of New Mexico School of Medicine, Project ECHO® has consistently demonstrated improvements in treating chronic and complex health conditions globally.(

Project ECHO is an education program that uses video-conferencing technology to share knowledge through collaborative learning, and builds capacity in PCPs.( Project ECHO develops Communities of Practice using a Hub-and-Spoke model, which connects interprofessional teams of specialists at an academic centre (Hub) with PCPs (Spokes) particularly in rural, underserved areas. In the ECHO model, knowledge flows in multiple directions: from Hub specialists to PCPs, between PCPs, and from PCPs to specialists.( Project ECHO Care of the Elderly (COE) launched in 2018 at Baycrest, a Canadian geriatric research and health-care institution fully affiliated with the University of Toronto. In partnership with the North East Specialized Geriatric Centre at Health Sciences North in Ontario, Project ECHO COE was the first ECHO program in Canada to provide continuing education focused on caring for frail and medically complex older patients. Our previous ECHO COE programs successfully demonstrated improved knowledge and self-efficacy scores for participants.( Additionally, qualitative evaluation feedback indicated a need for an ECHO focusing specifically on care needs in LTC. In January 2019, ECHO COE-LTC launched in partnership with the Ontario Centres for Learning, Research and Innovation in Long-Term Care (CLRI).

The objectives of this study were to: 1) determine the effect of ECHO COE-LTC on participant satisfaction; 2) evaluate the effectiveness of ECHO COE-LTC on improving provider knowledge and self-efficacy; and 3) explore participants’ experiences of ECHO COE-LTC, including impact on practice, through semi-structured interviews.

METHODS

Study Design

This program evaluation of ECHO COE-LTC uses a mixed-method, pre-and-post test design to collect both quantitative and qualitative techniques for data analysis.

Study Participants

All participants were interprofessional health-care providers enrolled in the ECHO COE-LTC program from January to March 2019, and were recruited through emails and marketing booths at relevant conferences, as well as through the Ontario CLRI Listserv. Participants were eligible if they worked in LTC in Ontario and had access to the basic necessary technology required to join the online sessions. In order to promote interprofessional collaborative care, the attending physician, medical director, and/or nurse practitioner were encouraged to attend with their LTC team. In our program, we refer to participants at spoke sites as learning partners.

This study was approved by the Research Ethics Boards at Baycrest and Health Sciences North. All research-related activities complied with all relevant federal guidelines and institutional policies.

Educational Program

The ECHO COE-LTC program consisted of 10 weekly 1-hr-long sessions. The curriculum was developed through surveying Ontario LTC home medical directors, attending physicians, directors of care, and interprofessional team members (N = 116), as well as through previous ECHO COE programs (see Appendix A for curriculum). Each session was comprised of a 15-min didactic presentation on a LTC related topic, followed by a 5-min question and answer period. The remaining portion of the session focused on discussion of a de-identified complex case presented in a standardized manner by one of the learning partners. Following the case presentation, learning partners and Hub members had the opportunity to ask clarifying questions related to the case. The group then discussed both non-pharmacological and pharmacological suggestions for optimizing care. A trained Hub member facilitated each session, and at the conclusion of each session, summarized the suggested recommendations. These recommendations, along with relevant articles and tools for practice, are shared on a protected Community of Practice website for learning partners.

Measures

Demographics

Demographic information included age, sex, profession, years in practice, environment of practice setting, and percentage of older adults in their practice.

Outcome Measures

The Moore et al.( evaluation framework was used to assess the program. While Moore’s framework consists of seven levels, we focused on the first four levels as part of the initial program evaluation:

Level 1: Participation—number of participants;

Level 2: Satisfaction—weekly questionnaires assessing participant satisfaction with the setting and program delivery;

Level 3: Learning—knowledge test assessing participant declarative knowledge; and

Level 4: Competence—questionnaire assessing participant self-reported self-efficacy to perform a task.

Each week, participants rated their overall satisfaction with the session using a 5-point Likert scale ranging from 1=“Strongly Disagree” to 5=“Strongly Agree”.

At the end of the 10-week program, participants were invited to complete a post-ECHO feedback survey on the experience, including fulfillment of program expectations and topics most relevant to practice.

Knowledge was assessed using 14 multiple-choice questions developed by the didactic presenters with expertise in the field, and were reviewed by Hub members.

The self-efficacy questionnaire included 17 questions, which allowed participants to rate their perceived ability to perform a task from 0=“Not Confident” to 100=“Very Confident”. All questions were developed based on the program curriculum, and took into consideration the interdisciplinary professions of the learning partners.(

Focus Groups

Participants in the focus groups were asked to provide their feedback on their experience with ECHO COE-LTC, and their responses were probed for additional clarity and insight (see Appendix B for Interview Guide).

Surveys

Registered participants received an orientation package that provided information about the program, including commitments, and an opportunity to provide informed consent. Prior to the first session, all participants received an email containing a link to complete the pre-ECHO knowledge and self-efficacy surveys. At post-ECHO, the knowledge and self-efficacy surveys were repeated following the final session. In addition, participants were asked to complete a post-ECHO feedback survey and sign-up to participate in focus groups.

Immediately following each session, participants were given one week to complete a satisfaction survey. Quantitative data were collected using REDCap, a secure data collection platform.(

Data Analyses

Descriptive statistics were used to describe and summarize participant characteristics and satisfaction ratings. To explore overall differences in pre-and post-ECHO knowledge and self-efficacy, linear mixed effects models with a random intercept were carried out for mean score as the outcome and time point as the categorical independent variable with assumed unstructured or compound symmetry covariance where final model selection was based on the Akaike Information Criterion.( Comparison of knowledge and self-efficacy by profession (physicians vs. other health professionals) were done through Wilcoxon signed-rank tests for within group analysis and Wilcoxon rank sum tests for between-group degree of change. Effect size (ES) calculations were completed to measure the magnitude of differences. Statistical analysis was performed using SAS 9.4.( Focus group interviews were transcribed verbatim and de-identified, and a thematic analysis of the transcripts was conducted using QSR International’s NVivo 11 software (NVivo-QSR International Pty Ltd., Melbourne, Australia) (see Appendix C for details of thematic analysis).(

RESULTS

Demographics

The 69 participants represented various professions with the majority being nurses/nurse practitioners (Table 1). Participants were primarily females (n=53, 77%), and a cumulative proportion of 85.5% were between 30 and 59 years of age, with more than half (63%) of participants in practice for over a decade. Participants reported that an average of approximately 84% of their registered clients are aged 65 years or older. More than half of participants indicated that they received training in assessment, managing, or treating older adults.

TABLE 1

Demographics (N=69)

Participant Demographics
Professions, n (%)
Administrator (e.g., Director of Care, Educator)	18 (26.1 %)
Nurse	15 (21.7 %)
Nurse Practitioner	14 (20.3 %)
Attending Physician	9 (13.0 %)
Medical Director	8 (11.6 %)
Allied Health Professional (e.g., Social Worker, Occupational Therapist)	5 (7.3 %)
Age Group, n (%)
20–39 years	23 (33.3 %)
40–49 years	14 (20.3 %)
50–59 years	25 (36.2 %)
60+ years	7 (10.1 %)
Sex, n (%)
Female	53 (76.8 %)
Male	16 (23.2 %)
Years in Practice, n (%)
<4 years	13 (18.8 %)
5–10 years	13 (18.8 %)
>10 years	43 (62.3 %)
Practice Setting, n (%)
Suburban or urban	40 (58.0 %)
Rural	27 (39.1 %)
Both	2 (2.9 %)
Approximate % of Older Adults ≥65 years Registered as Patients, mean (SD)	84.3 (SD=21.0)

Outcome Measures

Satisfaction

Mean weekly satisfaction ratings are presented in Table 2. The average response rate was 60.1% and ranged from 49.3–68.1%. All participants who attended the weekly sessions felt that the sessions met the learning objectives.

TABLE 2

Overall mean satisfaction ratings

Survey Items	Mean (SD)
Overall, I was satisfied with the session.	4.3 (0.1)
This program content enhanced my knowledge.	4.1 (0.2)
The presenter(s) were clear and effective in delivering material.	4.4 (0.2)
I will be able to share knowledge gained from this session with others.	4.2 (0.1)
This session will enhance my clinical practice.	4.1 (0.2)
There was sufficient opportunity to interact with other participants.	4.2 (0.2)
The session was facilitated well.	4.4 (0.1)
I would recommend this session to others.	4.3 (0.1)

Rating Scale: 1=strongly disagree, 2=disagree, 3=neutral, 4=agree, 5=strongly agree.

At post-ECHO, of the 43 participants who completed the feedback survey, the majority of participants (86.1%) “Agree” or “Strongly Agree” that ECHO COE-LTC met their expectations.

Knowledge and Self-Efficacy

Of the 69 participants, 44 participants completed the pre-ECHO knowledge and self-efficacy surveys (64% response rate), of whom 14 (32%) were physicians and 30 (68%) were other health professionals. There were 39 participants who completed the post-ECHO knowledge and self-efficacy surveys (57% response rate), of whom 11 (28%) were physicians and 28 (72%) were other health professionals.

Overall, there was a significant 3.9% increase in mean knowledge scores from pre-ECHO to post-ECHO for all participants (p = .02, ES=0.38; Figure 1/Table 3), in which the observed increase in mean knowledge scores of physicians and other health professionals with complete pre-post scores was 5.7% (p = .19, ES=0.61) and 2.3% (p = .20, ES=0.29), respectively. The difference between the increase in knowledge between physicians and other health professionals was not statistically significant (p = .63; Table 4).

FIGURE 1

Change in knowledge score and self-efficacy rating from pre-ECHO to post-ECHO

TABLE 3

Overall change in knowledge and self-efficacy scores

		n	Mean Estimatea (Std. Error)	Estimated ESb	p value
Knowledge (Score in %)	Pre-ECHO	44	52.1 (1.7)
	Post-ECHO	39	56.0 (1.7)
	Difference		3.9 (1.5)	0.38	.02
Self-Efficacy (Rating from 0=not confident to 100=very confident)	Pre-ECHO	44	63.3 (2.2)
	Post-ECHO	39	73.0 (2.6)
	Difference		9.7 (1.7)	0.56	<.0001

Estimation method is restricted maximum likelihood, covariance structures for knowledge and self-efficacy were compound symmetry and unstructured, respectively.

Effect size calculation is based on the standard deviation at pre-ECHO (Knowledge SD=10.2, Self-Efficacy SD=17.4) where thresholds are as follows: 0.20=small, 0.50=medium, 0.80=large, 1.20=very large.

TABLE 4

Change in knowledge and self-efficacy scores stratified by profession

		Knowledge				Self-Efficacy
Professionsa		n	Mean (SD)	p valueb	ESc	Mean (SD)	p valueb	ESc
Physicians	Pre-ECHO	14	53.1 (10.4)			69.7 (12.7)
	Post-ECHO	11	57.8 (10.8)			79.8 (12.3)
	Difference Post-Pre	10	5.7 (9.4)	.19	0.61	10.2 (7.9)	.004	1.29
Other Health Professionals	Pre-ECHO	30	51.2 (10.3)			57.8 (18.2)
	Post-ECHO	28	55.9 (12.9)			67.8 (19.6)
	Difference Post-Pre	19	2.3 (7.9)	.20	0.29	8.0 (10.5)	.005	0.76

Physicians vs. Other Health Professionals: Knowledge Wilcoxon two-sample (rank sum) test p=.63. Self-Efficacy Wilcoxon two-sample (rank sum) test p=.66.

Wilcoxon signed-rank test for within-group paired differences.

Standardized response mean scale: 0.20=small, 0.50=medium, 0.80=large, 1.20=very large.

ES = effect size.

In terms of self-efficacy, there was an overall 9.7-point increase in self-reported ratings from pre-ECHO to post-ECHO (p < .001, ES=0.56; Figure 1/Table 3), wherein the reported increase in self-efficacy ratings for physicians and other health professionals with complete pre-post scores was 10.2 points (p = .004, ES=1.29) and 8 points (p = .005, ES=0.76), respectively. The difference in the degree of change in self-efficacy between physicians and other health professionals was not statistically significant (p = .66; Table 4).

Focus Groups

Nineteen individuals participated in the nine focus groups. Three common themes emerged from the transcripts.

Reflections on Experience

Overall, most participants reported a positive experience with ECHO COE-LTC. One participant stated that the program was “very informative and covers real-life situations that we’re dealing with on daily basis, and practical solutions to help us”. Many participants expressed appreciation for the interprofessional focus of ECHO COE-LTC through “different points-of-views” and “different professional insights”. Several participants described feeling “isolated” in LTC and noted that ECHO COE-LTC reduced isolation by providing a virtual community and affirming that others working in LTC are dealing with similar issues.

At the same time, a few participants expressed that, while they had a positive experience overall, they found the sessions to be either too “basic” or, conversely, too “technical” for their field of care. Despite this, most participants commented that they have, or would, recommend the program to others.

In terms of enablers to participation, the majority of participants commented that it was “easy to participate” because it was “very much geared towards participants”. Other participants reported the importance of the facilitator promoting participation in that they “asked open-ended questions and allowed time for us to respond”. One barrier reported was difficulty participating when the facilitator did not see the “Raise Hand” alert feature on Zoom technology.

Feedback on Delivery

Many participants reported a positive experience with the didactic presentation, commenting that it “was good to start the hour with that background,” “was nice to hear some of the specialists’ thoughts on how they would do things”, and that they “always get some added information and some knowledge from the didactic segments”. One participant reflected that the didactic presentations “would offer tools that could be utilized by anyone at any time, and they gave concrete solutions that anyone could use in dealing with the complex problems.” A few participants added that the short 15-min presentations are easy to share in team education sessions. However, other participants found the duration to be “very short” and “wish they had been longer”.

All participants commented on the case presentations with most stating that they were “very valuable”, “really thorough”, and “because I work in long-term care, I don’t believe that there was one case that somehow didn’t apply to our residents that I have had, or do have, or probably will have.” Many participants expressed appreciation that the cases “reiterated my questions and concerns” and “were all cases that we see”. Still, some participants felt that the case presentations were “over medicalized, with more focus on the medical aspects of the residents”. A few participants also expressed that it was sometimes difficult following along as discussions were taking place over both chat and video. Others commented that the cases had a “strict time limit” and that they would have liked “more time”.

Impact of ECHO COE-LTC

Most participants felt that ECHO COE-LTC was an effective way to stay up-to-date on evidence-based practices and learn new resources. Some participants also described that ECHO COE-LTC “reaffirmed practices” and that the access to specialists “was really beneficial”. One participant reflected:

“You can spend time searching up-to-date information until you’re blue in the face but sometimes the evidence doesn’t exist for what is the best course of action so then you go to expert opinion but […] it can be very difficult to access expert opinions. So I found [ECHO COE-LTC] helpful because there were a lot of experts.”

Many participants commented on integrating the knowledge they learned into their practice. They reported implementing screening tools, diagnostic assessments, medication modifications, and non-pharmacological interventions to deliver care. One participant commented:

“I’ve been able to take concepts, advice, ideas and be able to work them into clinical practice right away. It’s been really nice to have really usable information, that’s very practical, come at a time when you can start folding that into assessments and care of my patients.”

Another participant reported direct benefit of their learning to resident care:

“The chronic heart failure module definitely [was impactful] because at the time I had a patient who had been jumping in and out of hospital with exacerbations and taking some of those lessons [from ECHO COE-LTC] and applying them […] have been definitely beneficial.”

Most participants also described how they shared what they learned with their teams both to “educate other [LTC] homes”, and “to validate their own practices”. For example, they shared “tools” and “new information” for treatment plans. Those that have yet to share information cited time as the primary barrier.

DISCUSSION

This study suggests that ECHO COE-LTC is a feasible and effective capacity-building education program for interprofessional health-care providers working in LTC to improve their ability to care for frail, complex aging residents. Learning partners reported high weekly satisfaction ratings and that the program overall met their learning goals. Observed improvements in knowledge scores and self-efficacy ratings for ECHO COE-LTC align with other studies that have implemented the ECHO model across various medical specialities and have reported improvements in provider knowledge and self-efficacy.( In fact, our finding of improved self-efficacy is consistent with ECHO-Chicago, which piloted the use of ECHO in LTC to provide geriatric education to nurses and social workers.(

We provide the first evidence demonstrating that Project ECHO can be a successful approach for improving knowledge of care of the elderly for interprofessional health-care providers in LTC. The focus groups supported these findings through examples of impact of ECHO COE-LTC and provided additional insights on experience and feedback on program delivery. In particular, participants placed emphasis on the ability for ECHO COE-LTC to provide health-care providers with a platform to enhance their knowledge in the care of the elderly, and improve access to specialists and those with geriatric expertise within LTC. Furthermore, learning partners appreciated the focus on LTC where access to educational resources is different from other environments. Participants commented that ECHO COE-LTC could address the need for continuing education for those working in LTC by mitigating geographical barriers that can limit access to resources and education and lead to professional isolation. Specifically, learning partners appreciated having a virtual community where they could share the challenges and successes of providing care in LTC.

Although we highlight the strength of our mixed-method pre–post program evaluation, several limitations are worth noting. First, our data were only obtained from one province in Canada, and while it is unclear whether the results are generalizable to other geographies, other ECHO programs focusing on LTC (i.e., ECHO-Chicago) have demonstrated promising findings. Second, we had a small sample for focus groups; however, the themes that emerged were consistent with our previous ECHO COE programs. Lastly, we did not formally evaluate changes in provider practice or direct patient impact, although some of the responses in the focus groups did reveal some practice change. Future iterations of ECHO COE-LTC will include evaluation of provider practice change and patient-level outcomes.

CONCLUSION

This study supports the use of Project ECHO in building capacity for interprofessional health-care providers in caring for residents living in LTC. Overall, participating in ECHO COE-LTC was associated with high program satisfaction and improvements in provider knowledge and self-efficacy. Given these positive findings, future research should aim to build on the impact of Project ECHO through the evaluation of practice changes and patient-level outcomes.

Curriculum
Behavioural and Psychological Symptoms of Dementia (BPSD)
Wound Care
Acute Changes
Dementia
Congestive Heart Failure
End of Life / Palliative Care
Falls and Bone Health
Engaging Families in Care
Movement Disorders
Polypharmacy