Program Evaluation of SeeMe™: Understanding Frailty Together.

Background: Perley Health has implemented SeeMe™: Understanding frailty together (www.perleyhealth.ca), a comprehensive approach to care that integrates the assessment and management of frailty, with an emphasis on goals of care planning.
Methods: Program evaluation over the first year of SeeMe™ used a mixed-methods approach involving quantitative data from surveys, goals of care preferences, hospital transfers, and qualitative data from interviews.
Results: The SeeMe™ training is an effective way to educate nurses and physicians in long-term care about frailty. For residents with documented care preferences prior to SeeMe™, there was a 15% reduction in the number of residents who preferred to be transferred to hospital post-SeeMe™ implementation. There was no significant decrease in hospital transfers during the first year the program was introduced.
Conclusion: After the roll-out of SeeMe™, nurses, physicians, and families reported high satisfaction with the program, and nurses reported an increase in knowledge and confidence. There was also a reduction in the number of residents and families selecting to transfer to hospital. This suggests that the education from SeeMe™ influenced residents and families to choose less invasive interventions in the context of frailty and quality of life goals.

INTRODUCTION

Frailty is a progressive condition that reduces a person’s ability to recover from a health crisis. It is common among older adults in long-term care (LTC) and is a strong predictor of health outcomes.( With more support available in the community, seniors are able to age at home for longer which results in residents being admitted to LTC with more advanced disease, multiple comorbidities, and significant dementia, resulting in a frail population.( There are currently gaps in how frailty is assessed and managed, such as a lack of guidance on clinical interventions to address frailty.( This can lead to poorer health outcomes, such as decreased mobility and cognition and decreased quality of life.( Further research is needed to determine best practices when it comes to managing frailty.( This paper will illustrate that assessing and managing frailty using the SeeMe™ Program has been well-received by nurses and families, while improving the understanding of frailty and having a positive impact on resident and family-centred care.

Traditionally, LTC homes rely on medical models that were developed for younger and healthier people. These models do not typically include frailty as a key driver of decision-making, which is critical in determining if an intervention will help or cause harm.( Since frailty is strongly associated with a higher risk of mortality, hospitalization, poorer quality of life, and health-related costs,( this generates two potential problems. Firstly, a medicalized model focuses on a cure and sometimes includes interventions which may not be goal-concordant to residents who are frail. For example, contrary to common belief, a systematic review of the literature showed that hospitalization of nursing home residents did not decrease mortality.( In LTC, it is imperative that guidelines are relevant to the care of the frail older adult, as the outcomes may be different from those of non-frail individuals.( Secondly, failing to systematically assess frailty can cause difficulties when providing person- and family-centred care that is based on quality of life goals. Helping families and residents understand the potential consequences of various interventions is important so that they are comfortable and prepared to make decisions.

Currently, there are a variety of validated tools available to measure frailty, including the Frailty Index and gait speed.( The Clinical Frailty Scale (CFS) is another validated, judgement-based tool that evaluates specific domains in order to broadly categorize the degree of fitness and frailty.( Specific to LTC, the Inter-Resident Assessment Instrument (Inter-RAI) is currently integrated into facilities across North America and includes a comprehensive and complex suite of assessment tools.( Examples include activities of daily living (ADLs) scores and Changes in Health, End Stage Disease, Signs and Symptoms Scale (CHESS). Use of these scales could support a clinical diagnosis of frailty for a trained clinician. However, the scales typically focus on one dimension of frailty and do not offer a comprehensive approach to assessing frailty, which is an important consideration for residents in LTC who tend to be medically complex and multi-morbid. Using a combination of complimentary measurement tools, including the CFS, supports a comprehensive approach to frailty assessment. Further to accurate measuring, supportive frailty care for older adults must also involve clear communication and supportive decision-making with residents and families, especially when it comes to advanced care planning.(

Perley Health, a 450-bed LTC home in Ontario, Canada has created and adopted SeeMe™: Understanding together. It is a comprehensive frailty-informed approach to care that integrates the assessment and management of frailty and acute health events as part of standard clinical practice for residents living in LTC. It recognizes and assesses frailty as part of a person’s overall health, and supports residents and their families to make informed decisions about care options within the context of frailty.

This study evaluated whether a frailty-informed approach to care using SeeMe™ can be effectively implemented as part of standard clinical practice in a LTC home setting. More specifically, we wanted to examine the impact of the program on family, nurse, and physician understanding of frailty-informed care and their satisfaction with the education, training, and information provided. We also wanted to explore the effects of the program on resident goals of care and future health preferences, and how this aligns with the perceived experiences of residents and families when an acute health event occurs.

METHODS

Study Design

The SeeMe™ Program evaluation involved a mixed-methods approach, using a concurrent triangulation design( that included online surveys, a tool audit, an analysis of the rates of hospital transfers, and narrative interviews with families.

Intervention Studied

The first step of SeeMe™ is to understand the person’s degree of frailty by completing a Comprehensive Frailty-Informed Assessment (CFA). The CFA provides a detailed overview of different drivers of frailty, including the major drivers of cognition, function, and mobility, through data collected from clinical practice, including the Inter-RAI, and also assigns an overall Clinical Frailty Scale (CFS) score.( We chose the CFS as our measurement tool because of both its usability and its validity.( It is a short tool that takes into consideration the multiple drivers of frailty (activity, function, and disability) and complements data already collected from Inter-RAI; therefore, it does not require a lot of extra work on behalf of staff.

Following the assessment, a care conference is held between the resident and/or family and the interdisciplinary team to discuss the overall health status and considerations for future decision-making. During this meeting, potential treatment risks are considered in the context of frailty, and families are encouraged to discuss their goals, values, and preferences with the care team. Care decisions are documented using the Goals of Care (GOC) tool (Appendix A). The GOC tool was developed to help facilitate a more fulsome discussion than the process and tools that were in place prior to SeeMe™. These discussions help the care team support the resident and/or family in making informed decisions about the next steps of care if an acute health event arises.

The SeeMe™ Program was rolled out progressively at Perley Health, starting with two LTC units in February 2019, expanding to additional units every two months. By the end of 2019, all 12 LTC units within the home had adopted the SeeMe™ Program.

Recruitment and Sample Size

Participants included nurses (Registered Nurses and Registered Practical Nurses) and physicians working at Perley Health, and families of residents living at Perley Health during the program implementation timeframe of February 1, 2019 to January 31, 2020. Nurses and physicians were eligible to complete the survey if they had completed the SeeMe™ training, and were recruited through email by members of the evaluation team (AL, DA). Family members were eligible to participate in the survey if they had participated in a care conference during the program implementation timeframe. Recruitment for families was done by email. Surveys were emailed to 125 nurses, 10 physicians, and 348 family members.

Caregivers were also eligible to participate in an interview if their loved one had experienced an acute health event after January 1, 2019.

This study was approved by the Research Ethics Board at Carleton University (Protocol #112021). All research-related activities complied with all relevant federal guidelines and institutional policies.

Training for Nurses and Physicians

Nurses on designated units received one-on-one or small group training from a group of nurse champions (mentors) who had received prior training (with physicians, as described below) and were selected to lead the training in a mentorship role. Nurses were trained on aspects related to frailty and the SeeMe™ Program, such as the drivers of frailty, the tools used in the SeeMe™ Program, and how to explain frailty to a resident/family. Training also involved reviewing acute health event definitions, how to help families make informed decisions “in the moment” when an acute health event occurs, and nursing responsibilities related to sustaining the processes involved in SeeMe™. For more information on the train-the-trainer model, refer to Appendix B.

Training for physicians used similar modalities to the training for nurses and was provided by other physicians identified and recruited as subject matter experts in frailty. The two-day training for physicians and nurse champions (mentors) was accredited as a Continuing Medical Education (CME) event, and addressed the concept of frailty and its impact on clinical outcomes, assessment of frailty, and approaches to communication with families. There was also training on the logistics of delivering the program, such as the use of the tools and the SeeMe™ processes.

Education for Residents and Families

Residents and families were informed about the SeeMe™ Program at resident and family council meetings throughout the year. Printed resources describing SeeMe™ and the concept of frailty were sent to all residents and families and integrated into the admission materials for new residents (Appendix C).

Outcome Measures

The primary outcomes used in our evaluation included participant satisfaction and their knowledge about frailty. Specifically, we measured nurse and physician satisfaction with training, the tools used, and the care conference process. We also measured family satisfaction with the information provided, the care conference process, and the opportunity to identify future health and personal care preferences for the resident. To assess change in knowledge and confidence, we measured the impact of training on nurses’ knowledge and confidence in the delivery of frailty-informed care, including confidence in using the tools and applying the new processes associated with care conferences (i.e., facilitating conversations about goals of care).

The secondary outcomes for the evaluation were:

Change in GOC: An audit of the GOC tool to assess change in resident GOC before and after SeeMe™ Program implementation.

Hospital transfers: Hospital transfer data during the first year of the program was compared to the same time frame in the previous year for each unit, in order to determine if the SeeMe™ Program had an impact on the volume of hospital transfers.

Perceived alignment between GOC and interventions delivered and experience of SeeMe™: Interviews with families to assess experiences after an acute health event and overall experience with SeeMe™.

Data Collection

Surveys

Four survey tools were developed prior to the roll-out of the program and tailored to each group of participants— mentors, trainees, physicians and families— based on the primary program outcome measures (Appendices D–G). Nurses and physicians were invited to complete the survey immediately following the completion of their training, and it was emailed to participants by a member of the evaluation team. Families were emailed the link to the survey within one week of attending a care conference by a clerk from the admissions office. The completion of the one-time survey was voluntary. Participants consented to participate and were informed that their answers were confidential and they would not be identified.

The surveys for nurses and physicians were divided into two sections; the first section addressed the training, while the second section addressed the program tools. The survey for families consisted of a range of questions addressing their understanding of their loved one’s health status and satisfaction of the care conference process. All participants rated their satisfaction on a 5-point Likert scale ranging from ‘Strongly Disagree’ to ‘Strongly Agree’. Participants were also invited to provide general feedback/comments on their experience to inform program improvement.

Reminders were sent to nurses and physicians to encourage survey completion. As an incentive, nurses were presented with a certificate of completion once they completed the training and indicated they had completed the survey. Families were reminded about the survey at the care conference, and paper copies were made available as an alternative to the online version.

Audit Tool

A GOC audit tool was a table created to compare GOC preferences before and after SeeMe™ implementation (i.e., CPR, transfer to hospital). It also identified the person who completed the GOC tool documentation, and highlighted whether follow-up on the GOC was required (this was to ensure that staff were knowledgeable on the proper completion of the GOC tool). The GOC audit tool was populated using the GOC Tool (Appendix A) completed by two members of the evaluation team (AL, DA).

Interviews

The interviews were conducted by Master of Social Work (MSW) students from Carleton University as part of their research methods course (unpublished work), in collaboration with the evaluation team from Perley Health. This project was planned in conjunction with the overall program evaluation and was designed to address outcomes not examined through other data collection methods. The primary research question was: “What are the experiences before, during, and following an acute health event of Perley Health residents and their caregivers who have participated in the SeeMe™ Program?”

Participants were screened by Perley Health research staff in January and February 2020 and were eligible depending on whether they, as a resident, had experienced an acute health event, or if the person was a caregiver to a resident who had experienced an acute health event following the SeeMe™ Program roll out on January 1, 2019. A total of 42 residents experienced an acute health event during the designated time frame. The evaluation team at Perley Health contacted eligible participants to obtain consent to be contacted by the MSW students. Many residents and families were determined to be ineligible, for reasons such as death and inability to consent. Bereaved family members were also excluded from being contacted if the death of their loved one occurred within the last six weeks, out of respect for their loss. Time constraints were also a limiting factor in recruitment due to COVID-19 restrictions imposed in March 2020. This resulted in four residents and six families who were eligible to be contacted for an interview. When contacted, three residents and five families agreed to be interviewed.

Data Analysis

Descriptive statistics were used to describe and summarize the survey data, which were presented as frequencies and proportions.

Chi-square tests were performed to examine the change in the number of residents who preferred to transfer to the emergency department and receive CPR following the implementation of the SeeMe™ Program, as well as the observed number of transfers, where a p value of < .05 was considered statistically significant.

Qualitative data from interviews were transcribed and analyzed thematically with NVivo 12 software (QSR International (Americas) Inc., Burlington, MA) by MSW students using an iterative coding approach where each transcript was coded by at least two team members. Differences in coding were reviewed and discussed until consensus was achieved.

RESULTS

There were a total of 169 respondents to surveys across all groups: nurses (mentors and trainees), physicians, and families of current residents. The response rates for nurses, physicians, and families were 81% (n= 101/125), 40% (n= 4/10), and 16% (n=54/348), respectively. Key findings are found in Tables 1–4; full survey results for each group of respondents are found in Tables 5–8.

TABLE 1

Key satisfaction results from the SeeMe™ Evaluation Mentor and Trainee survey

Questions	Agreed + Strongly Agreed n	Total of all Responses n (%)
i) The Comprehensive Frailty-Informed Assessment summarizes relevant information related to the resident’s frailty level.	95	98 (97)
ii) The Comprehensive Frailty-Informed Assessment helped me understand the resident’s overall frailty status.	96	99 (97)
iv) The new care conference process enhances the resident and family-centred focus.	95	99 (96)
iv) The Goals of Care tool equips families and residents in making informed decisions about future health and personal care.	94	97 (97)
v) The Goals of Care tool is a helpful resource for communicating with families about acute health event management.	90	96 (94)

TABLE 2

Key satisfaction results from the SeeMe™ Evaluation Family survey

Questions	Agreed + Strongly Agreed n	Total of all Responses n (%)
i) The information provided during the care conference helped me understand the current health status of my family member.	51	54 (94)
ii) I now understand the importance of frailty when discussing the health of my family member.	47	54 (87)
iv) I was able to identify goals of care preferences for my family member.	51	54 (94)
v) I was able to identify future health and personal care preferences for my family member	52	53 (98)

TABLE 3

Key satisfaction results from the SeeMe™ Evaluation Family survey: responses from those who had participated in care conferences prior to the SeeMe™ Program being implemented

Questions	Agreed + Strongly Agreed n	Total of all Responses n (%)
i) Compared to previous care conferences, the new SeeMe™ Care Conference is better for understanding the current health status of my family member.	20	25 (80)
ii) Compared to previous care conferences, the new SeeMe™ Care Conference allows me to better explain what is important for my family member and their care.	20	25 (80)
iv) Compared to previous care conferences, the new SeeMe™ Care Conference is better for identifying goals of care for my family member.	19	25 (76)
v) Compared to previous care conferences, the new SeeMe™ Care Conference is better for identifying future health and personal care preferences for my family member was able to identify future health and personal care preferences for my family member.	19	24 (79)

TABLE 4

Key results from the SeeMe™ Evaluation Trainee survey (knowledge and confidence)

Questions	Agreed + Strongly Agreed n	Total of all Responses n (%)
i) After the mentor shift, I felt confident using the Comprehensive Frailty-Informed Assessment.	90	95 (95)
ii) After the mentor shift, I felt confident documenting in the Goals of Care Tool in Point Click Care.	85	95 (90)
iv) After the mentor shift, I felt confident applying the new processes associated with care conferences.	87	95 (92)

TABLE 5

All results from the SeeMe™ Evaluation Mentor survey

Questions	Strongly Disagree n (%)	Disagree n (%)	Undecided n (%)	Agree n (%)	Strongly Agree n (%)	Total n
1) As a SeeMe™ mentor, I felt equipped to facilitate the 1:1 mentor shifts.	0 (0)	0 (0)	1 (20)	1 (20)	3 (60)	5
2) The frailty-informed care weekend training in September 2018 was essential in equipping me as a SeeMe™ mentor.	0 (0)	0 (0)	0 (0)	2 (40)	3 (60)	5
3) The SeeMe™ mentor training session where tools were reviewed was essential in equipping me as a SeeMe™ mentor.	0 (0)	0 (0)	0 (0)	1 (20)	4 (80)	5
4) Support from the Collaborative Practice Leader was essential in equipping me as a SeeMe™ mentor.	0 (0)	0 (0)	0 (0)	2 (40)	3 (60)	5
5) The new care conference process enhances the resident and family-centred focus.	0 (0)	0 (0)	0 (0)	3 (60)	2 (40)	5
6) The Goals of Care tool equips families and residents in making informed decisions about future health and personal care.	0 (0)	0 (0)	0 (0)	3 (60)	2 (40)	5
7) The Goals of Care tool is a helpful resource for communicating with families about acute health event management.	0 (0)	0 (0)	0 (0)	2 (40)	3 (60)	5
8) The Comprehensive Frailty-Informed Assessment summarizes relevant information related to the resident’s frailty level.	0 (0)	0 (0)	0 (0)	2 (40)	3 (60)	5
9) The Comprehensive Frailty-Informed Assessment helped me understand the resident’s overall frailty status.	0 (0)	0 (0)	1 (20)	1 (20)	3 (60)	5

TABLE 6

All results from the SeeMe™ Evaluation Trainee survey

Questions	Strongly Disagree n (%)	Disagree n (%)	Undecided n (%)	Agree n (%)	Strongly Agree n (%)	Total n
1) The Introductory SeeMe™ Surge Module helped me understand the SeeMe™ framework.	2 (2)	0 (0)	2 (2)	45 (47)	47 (49)	96
2) The Registered Staff SeeMe™Surge Module helped me understand the purpose of the Comprehensive Frailty-Informed Assessment.	2 (2)	0 (0)	4 (4)	41 (43)	48 (51)	95
3) The Registered Staff SeeMe™Surge Module helped me understand the Goals of Care Tool.	2 (2)	1 (1)	3 (3)	43 (45)	47 (49)	96
4) The Registered Staff SeeMe™Surge Module helped me understand the new care conference structure.	2 (2)	1 (1)	4 (4)	43 (45)	45 (47)	95
5) After the mentor shift, I felt confident using the Comprehensive Frailty-Informed Assessment.	1 (1)	0 (0)	4 (4)	44 (46)	46 (48)	95
6) After the mentor shift, I felt confident documenting in the Goals of Care Tool in Point Click Care.	1 (1)	0 (0)	9 (9)	37 (39)	48 (51)	95
7) After the mentor shift, I felt confident applying the new processes associated with care conferences.	1 (1)	0 (0)	7 (7)	48 (51)	39 (41)	95
8) After the mentor shift, I felt confident independently administering the Montreal Cognitive Assessment.	1 (1)	0 (0)	6 (6)	48 (51)	39 (41)	94
9) The new care conference process enhances the resident and family-centred focus.	1 (1)	0 (0)	3 (3)	37 (39)	53 (56)	94
10) The Goals of Care tool equips families and residents in making informed decisions about future health and personal care.	1 (1)	0 (0)	2 (2)	39 (42)	50 (54)	92
11) The Goals of Care tool is a helpful resource for communicating with families about acute health event management.	1 (1)	0 (0)	5 (5)	36 (40)	49 (54)	91
12) The Comprehensive Frailty-Informed Assessment summarizes relevant information related to the resident’s frailty level.	1 (1)	0 (0)	2 (2)	37 (40)	53 (57)	93
13) The Comprehensive Frailty-Informed Assessment helped me understand the resident’s overall frailty status.	1 (1)	0 (0)	1 (1)	35 (37)	57 (61)	94

TABLE 7

All results from the SeeMe™ Evaluation Physician survey

Questions	Strongly Disagree n (%)	Disagree n (%)	Undecided n (%)	Agree n (%)	Strongly Agree n (%)	Total n
1) The Physician SeeMe™ Surge Module enabled me to understand the SeeMe™ Framework.	0 (0)	1 (25)	0 (0)	2 (50)	1 (25)	4
2) The Physician SeeMe™ Surge Module equipped me to use the Comprehensive Frailty-Informed Assessment.	0 (0)	0 (0)	1 (25)	1 (25)	2 (50)	4
3) The Physician SeeMe™ Surge Module equipped me to use the Goals of Care tool.	0 (0)	0 (0)	1 (25)	1 (25)	2 (50)	4
4) Support from the SeeMe™ Registered Staff Champions during care conferences helped solidify the team’s understanding of the new care conference process.	0 (0)	0 (0)	0 (0)	1 (25)	3 (75)	4
5) The new care conference process enhances the resident and family-centred focus.	0 (0)	0 (0)	0 (0)	4 (100)	0 (0)	4
6) The Goals of Care tool equips families and residents in making informed decisions about future health and personal care.	0 (0)	0 (0)	0 (0)	3 (100)	0 (0)	3
7) The Goals of Care tool is a helpful resource for communicating with families about acute health event management.	0 (0)	0 (0)	1 (25)	1 (25)	2 (50)	4
8) Discussing the Comprehensive Frailty-Informed Assessment at care conferences enhances the Goals of Care discussion.	0 (0)	0 (0)	0 (0)	2 (50)	2 (50)	4
9) The Comprehensive Frailty-Informed Assessment is helpful for understanding the resident’s frailty status.	0 (0)	0 (0)	2 (50)	0 (0)	2 (50)	4

TABLE 8

All results from the SeeMe™ Evaluation Family survey

Questions	Strongly Disagree n (%)	Disagree n (%)	Undecided n (%)	Agree n (%)	Strongly Agree n (%)	No Information Received n (%)	Total n
1) The information I received about the SeeMe™ Program before the care conference was helpful for understanding this new program.	0 (0)	1 (2)	10 (19)	24 (44)	11 (20)	8 (15)	-
2) The information provided during the care conference helped me understand the current health status of my family member.	0 (0)	1 (2)	2 (4)	26 (48)	25 (46)	-	54
3) The information provided during the care conference helped me understand how the health of my family member may evolve in the future.	0 (0)	0 (0)	4 (7)	32 (59)	18 (33)	-	54
4) I now understand the importance of frailty when discussing the health of my family member.	0 (0)	1 (2)	6 (11)	35 (65)	12 (22)	-	54
5) I was able to identify goals of care preferences for my family member (e.g., comfort/symptom management, managing illness, treatment of illness, extending life).	0 (0)	0 (0)	3 (6)	29 (54)	22 (41)	-	54
6) I was able to identify future health and personal care preferences for my family member (e.g., CPR, transfer to Emergency Department, surgery, medications, etc.).	0 (0)	0 (0)	1 (2)	30 (56)	23 (43)	-	54
7) The care conference allowed me to express what is important for my family member and me about their care.	0 (0)	0 (0)	1 (2)	28 (53)	24 (45)	-	53
8) During the care conference I felt comfortable asking questions.	0 (0)	0 (0)	1 (2)	19 (36)	33 (62)	-	53
9) During the care conference all my questions were answered to my satisfaction.	0 (0)	1 (2)	9 (17)	19 (37)	23 (44)	-	52
10) The information I received about the SeeMe™ Program was easy to understand.	0 (0)	2 (4)	7 (13)	25 (46)	15 (28)	5 (9)	54
11) I would recommend the SeeMe™ program to a friend or another family member.	0 (0)	1 (2)	15 (29)	24 (47)	11 (22)	-	51
12) Compared to previous care conferences, the new SeeMe™ Care Conference is better for understanding the current health status of my family member.	0 (0)	2 (8)	3 (12)	16 (64)	4 (16)	-	25
13) Compared to previous care conferences, the new SeeMe™ Care Conference allows me to better explain what is important for my family member and their care.	0 (0)	2 (8)	2 (8)	16 (64)	4 (16)	-	25
14) Compared to previous care conferences, the new SeeMe™ Care Conference is better for identifying goals of care for my family member (e.g., comfort/symptom management, managing illness, treatment of illness, extending life).	0 (0)	1 (4)	5 (20)	15 (60)	4 (16)	-	25
15) Compared to previous care conferences, the new SeeMe™ Care Conference is better for identifying future health and personal care preferences for my family member (e.g., CPR, transfer to Emergency Department, surgery, medications, etc.).	0 (0)	1 (4)	4 (17)	12 (50)	7 (29)	-	24

Primary Outcome Measures

Satisfaction—Nurses and Physicians

Table 1 shows the key survey results for nurse respondents related to program satisfaction. All survey results for mentors and trainees can be found in Tables 5 and 6, respectively.

All physician respondents (n= 4/4, 100%) agreed that the new care conference process enhances resident and family-centred focus and that the GOC tool helps families make decisions about future health preferences (n= 3/3, 100%; one person did not respond to this question) (Table 7).

Satisfaction—Families

Tables 2 and 3 shows the key survey results from families related to satisfaction. All survey results for families can be found in Table 8.

Knowledge and Confidence

Table 4 shows the key survey results from trainees related to knowledge and confidence.

Secondary Outcome Measures

1. Goals of Care

In the first year of the program, 349 GOC tools were completed. The audit revealed that 95% of residents with a GOC tool completed indicated they do not want CPR, while 83% of those residents indicated they do not want to transfer to the emergency department should an acute health event occur. Of these residents, 181 (52%) were new admissions with no previously documented levels of care, and 168 (48%) were existing residents with previously documented levels of care from conversations prior to SeeMe™. Of existing residents, after the SeeMe™ Care Conference, we found a 15% reduction in preference to transfer to the emergency department (χ21, N = 168) = 13.37, p = .0003) and a 5% reduction in preference for CPR (χ21, N = 168) = 5.53, p = .0187).

In the first year of the program, 206 Comprehensive Frailty-Informed Assessments were completed and the median CFS Score was 7 (range 4–8), indicating severe frailty.(

2. Hospital Transfers

A comparison of the number and proportion of hospital transfers by unit pre- and post-program did not show any statistically significant changes in hospital transfers during the first year of the program, with the exception of February 2019 (Table 9).

TABLE 9

Monthly % Change in Hospital Transfers Pre- and Post-SeeMe™ Implementation

Month	Hospitalized Pre-Program (2018–2019) %	Hospitalized Post-Program (2019–2020) %	Change %	Chi-square statistic χ 2	p value
February-19	8.8	1.3	−7.5	4.7368	.029523
March-19	7.5	1.3	−6.3	3.7348	.053289
April-19	6.9	3.1	−3.8	2.3684	.123812
May-19	1.9	1.9	0.0	0.0000	1.000000
June-19	4.0	2.2	−1.8	1.1796	.277447
July-19	2.2	2.7	0.4	0.0932	.760164
August-19	2.3	4.3	2.0	1.8610	.172507
September-19	3.4	3.9	0.5	0.1384	.709880
October-19	2.9	1.7	−1.2	0.1384	.709880
November-19	2.4	2.2	−0.2	0.0539	.816444
December-19	3.9	3.4	−0.5	0.1384	.709880
Jan-20	2.9	2.4	−0.5	0.1868	.665568

3. Perceived Alignment Between Goals of Care and Interventions & Overall Experience with SeeMe™

The MSW students conducted qualitative interviews with three residents and five families to learn more about their experience with SeeMe™ and the alignment between the GOC identified and the interventions that were delivered when an acute health event occurred. The interviews with residents were very brief, and the data was not comprehensive enough to analyze and include as part of the results.

Families who were interviewed reported an overall positive experience with the care conferences, with communication emerging as a prominent theme across all interviews. Families reported feeling satisfied with their experience developing GOC as part of the care conference process. The collaborative approach used by the interprofessional team helped families make informed decisions about GOC. One participant described discussing the decision with his mother to not receive CPR. This family member reported feeling happy that he had the chance to discuss his parent’s end-of-life wishes prior to the progression of dementia. Another person reported feeling confident with the information that the physicians provided about the risks and potential outcomes of decisions with respect to GOC, including decisions to provide comfort care if there was an acute health event that the resident may not recover from. One caregiver discussed how beneficial it was to have detailed information from the physician on the benefits and risks of different decisions when establishing GOC. The caregiver reported

He said, ‘It is really your call here, if you say no, here is the likely outcome and over time, what it would be in terms of end of life, and if you let him go for surgery here is the likely outcome, and again here are the risks and percentages of survival’, so I had that in my mind when I got the call from the head nurse that dad had fallen.

Another caregiver described how valuable it was for her and her family to have the information presented at the care conference in an accessible and meaningful way. Specifically, the caregiver described how the physician used an analogy of a canoe to describe her father’s frailty stating that

It’s funny because, now that’s our reference. The canoe is very stable now. And you know, on bad days, we say, ‘oh the canoe is almost tipping!’ So that was a very good reference for us. It made us visually imagine, you know?

This is in contrast with the feedback from one family member interviewed who reported feeling anxious and upset during the first care conference, as though their views were not being heard. The family member spoke to the care team about his concerns and reported a very positive experience in subsequent conversations, saying that they now felt like an essential member of the care team and was able to contribute knowledge about his parent’s care.

Of the families interviewed, there was perceived alignment between the GOC and the interventions that occurred following the acute health events. One caregiver described how, “In the care plan it said that if he falls and injures himself severely to call me, then I would make the call over the phone”. When the caregiver was informed that his father had a fall, the caregiver requested to speak with the physician, and

We had that discussion right there and then, and it was based on the discussion there that helped me make the decision, that dad really had a better than 50% chance of surviving this if he got the surgery.

This flexibility was noted by another family member, who said they appreciated being able to change their mind later if the circumstances changed.

DISCUSSION

The study suggests that the SeeMe™ Program can be effectively implemented in LTC and leads to improved understanding of frailty and the health status of residents by nurses and families. There was high satisfaction with the education, training, and information provided, and nurses reported feeling confident in delivering the program. Families valued the information provided about frailty and the communication that took place during the care conferences. Due to the small physician sample size, it is difficult to draw conclusions about the value of the program for physicians in LTC. Results demonstrated a significant reduction in the number of residents and families choosing to receive CPR and be transferred to hospital following program implementation. Although there was no control group, comments from the interviews suggest that the discussions that took place with the care team as part of the SeeMe™ Program influenced these preferences.

To our knowledge, SeeMe™ is the first frailty-informed care program that was developed and tailored to the LTC context in Canada. In addition to the frailty training being designed and situated in LTC, the program is unique because program elements are incorporated into existing workflow processes, such as the electronic health record and the annual care conference.

It is recommended to incorporate the assessment of frailty in complex health care decision-making so that interventions can be weighed against their likely outcomes and the patient’s wishes and values.( The SeeMe™ Program incorporates discussion about the resident’s frailty level which informs the goals of care conversation between the health-care team and the resident and family. Some of the key barriers to initiating goals of care discussions in nursing homes are identified as lack of education, and lack of involvement of the family and interdisciplinary team.( SeeMe™ seeks to address these barriers through its applied training approach and the ongoing communication between the interprofessional team and families.

There was a statistically significant decline in the preference to transfer to hospital and to receive CPR after the implementation of SeeMe™. This suggests that the frailty-informed education influenced residents and families to choose less invasive interventions in the context of frailty and quality of life. This is supported by the qualitative interviews where families expressed satisfaction with the care conference process, and valued the support and guidance provided when making goals of care decisions.

Although there was a decrease in the number of people wishing to transfer to hospital, there was no difference in the number of transfers before and after program implementation. There are plausible reasons that a decline in transfers was not observed. There are situations when hospital transfers are unavoidable; therefore, without assessing the reason for each transfer, it is difficult to evaluate whether this number could be further reduced. More data and a thorough analysis of the reason for hospital transfers would provide additional insight. It is also important to note that the study was not designed specifically to assess changes in the number of transfers.

Although a thorough review of the details surrounding the hospital transfers was out of scope for this evaluation, feedback through interviews with families suggested that GOC were respected when acute health events actually occurred. This aligns with the results of a systematic review of advance care planning (ACP) interventions for nursing residents, where studies demonstrated that when ACP takes place, the interventions that follow tend to be in line with the resident’s documented preferences.( Higher quality communication with clinical staff can contribute to reducing disparities between interventions and GOC.( The SeeMe™ Care Conference structure facilitates early communication about GOC, and families reported transparent communication about the state of the resident’s health. The SeeMe™ Program seeks to ensure that nurses are equipped to have discussions with families and to verify these preferences “in the moment”, when an acute health event occurs. The integration of real-time communication into the process is intended to increase the likelihood that treatment is in line with the resident’s GOC.

The evaluation revealed several positive benefits of the SeeMe™ Program including improving resident and family-centred care, promoting a better understanding of frailty and its impacts on health outcomes, and facilitating communication during acute health event management.

Although the study highlighted many benefits of the program, there are several limitations that are worth noting. The sample size was low for the mentor and physician survey, as well as for families interviewed. This makes it difficult to generalize their experience of the program for other providers and settings. Also, an objective assessment of trainee skills and competency would have strengthened the evaluation of the training. Another limitation to note is that many nurses had been exposed to education on frailty in the year prior to the implementation of SeeMe™. It is possible that this knowledge may have already been integrated into clinical practice prior to the evaluation period. It is also important to note that there was no control group in the study; therefore, it is difficult to say with certainty that the decline in requests for interventions were a result of the SeeMe™ Program. Finally, due to the frailty level and cognitive status of the residents, it was not feasible to include them in the evaluation; however, getting feedback first-hand from residents would have provided valuable insight. Future evaluation methods will incorporate more robust measurements to expand the applicability of the results.

It is also important to note some challenges that we were encountered during program implementation. The main challenge was related to staffing. Due to priority care needs at the point of care, training for nurses had to be cancelled at the last minute, resulting in logistical problems with rescheduling and tracking of nurses who had received training. Furthermore, since all training required backfill at the point of care, financial restraints need to be considered prior to implementation of such an intervention. The onset of the pandemic in March 2020 and the restrictions put in place led to challenges with recruitment for the qualitative interviews, resulting in fewer family interviews.

CONCLUSION

The SeeMe™ Program has been well-received by nurses and families, and the results of the evaluation demonstrate an improved understanding of frailty and a positive impact on resident and family-centred care. Interviews with families suggested a perceived alignment between GOC and resulting interventions during acute health events.

Next steps for program evaluation will involve further analyses of acute health events to determine actual alignment with stated GOC preferences. Incorporating a quality-of-life measure would also provide insight into the benefit of the program for LTC home residents. Given the potential health system impact on hospital capacity, while better aligning care with quality of life goals, the SeeMe™ Program is a model of care that other LTC homes may consider adopting.