The Avoidable Transfer Scale: A New Tool for Identifying Potentially Avoidable Hospital Transfers of Nursing Home Residents.

Background and
Objectives: Prior approaches to identifying potentially avoidable hospital transfers (PAHs) of nursing home residents have involved detailed root cause analyses that are difficult to implement and sustain due to time and resource constraints. They relied on the presence of certain conditions but did not identify the specific issues that contributed to avoidability. We developed and tested an instrument that can be implemented using review of the electronic medical record. Research Design and
Methods: The OPTIMISTIC project was a Centers for Medicare and Medicaid Services demonstration to reduce avoidable hospital transfers of nursing home residents. The OPTIMISTIC team conducted a series of root cause analyses of transfer events, leading to development of a 27-item instrument to identify common characteristics of PAHs (Stage 1). To refine the instrument, project nurses used the electronic medical record (EMR) to score the avoidability of transfers to the hospital for 154 nursing home residents from 7 nursing homes from May 2019 through January 2020, including their overall impression of whether the transfer was avoidable (Stage 2). Each transfer was rated independently by 2 nurses and assessed for interrater reliability with a kappa statistic.
Results: Kappa scores ranged from -0.045 to 0.556. After removing items based on our criteria, 12 final items constituted the Avoidable Transfer Scale. To assess validity, we compared the 12-item scale to nurses' overall judgment of avoidability of the transfer. The 12-item scale scores were significantly higher for submissions rated as avoidable than those rated unavoidable by the nurses (mean 5.3 vs 2.6, p < .001). Discussion and Implications: The 12-item Avoidable Transfer Scale provides an efficient approach to identify and characterize PAHs using available data from the EMR. Increased ability to quantitatively assess the avoidability of resident transfers can aid nursing homes in quality improvement initiatives to treat more acute changes in a resident's condition in place.

Translational Significance: The 12-item Avoidable Transfer Scale presents a new, refined, and manageable method for retrospectively identifying potentially avoidable hospital transfers (PAH) of nursing home residents. This provides a contrast to previous methods of identifying potentially avoidable hospital transfers, which use medical claims or root cause analyses, and it can ease the time burden of staff in identifying PAHs. The Avoidable Transfer Scale further evolves the science of identifying potentially avoidable hospital transfers of nursing home residents, by focusing on issues that could be influenced in the nursing home setting.

Avoiding unnecessary use of the Emergency Department and hospital continues to be a significant focus of research and program implementation efforts (1). These “potentially avoidable hospital transfers (PAHs)” have been used to measure quality in medical care and as targets for interventions to reduce inappropriate care, in part because they can be costly for insurers. Dual-eligible beneficiaries (those with both Medicare and Medicaid insurance coverage) make up the majority of nursing home residents and have at least 2 times the risk of avoidable hospitalizations compared with traditional Medicare beneficiaries (2–4). Reducing the incidence of PAHs can both improve the care of nursing home residents and reduce expensive and unnecessary procedures and medical visits (5,6).

Estimates are that between 23% and 60% of transfers of nursing home residents are potentially avoidable, even though the definition of “avoidable” varies between settings and agencies (7,8). Early studies of avoidable hospitalizations focused on identifying which hospital diagnoses are related to potentially avoidable hospitalizations. PAHs are spurred by conditions that either should not have occurred or those that can be managed in a community or nursing home setting (3). In nursing homes, the majority of PAHs are linked to one of 5 medical conditions: pneumonia, heart failure, urinary tract infection, dehydration, and asthma (4). Tying avoidable hospitalizations to diagnostic codes does not allow for clarification of the systemic and situational factors that may be contributing to a hospital transfer, avoidable or not (9,10).

Other than diagnosis-based avoidability, previous studies of PAHs of nursing home residents have found a number of associations with PAHs. Associations with avoidability include fevers, resident/family insistence, provider insistence (despite nursing staff reassurances), not attempting treatment in facility, dementia diagnosis, and less-severe baseline illness or change in condition (11–13). Other studies have found that areas for intervention to reduce PAHs of nursing home residents include improving communication, revamping care processes, clarifying goals of care, and securing on-site resources such as equipment, regular staff, and providers (10,14).

More recent scholarship has focused less so on claims-based and diagnosis-based definitions of avoidable, allowing for the expertise of clinical staff to identify the avoidability of a transfer through robust root cause analyses performed on site in nursing homes (9,11). INTERACT is a nursing home quality improvement project that has incorporated in-depth root cause analyses to evaluate PAHs (15,16). In addition to refining the definition of PAHs of nursing home residents, results from INTERACT and other studies have also identified potential areas for improvement in nursing home care that may help reduce avoidable transfers (8,10,17,18). However, root cause analysis is a time-intensive process and efforts are difficult to sustain within an individual nursing home.

Claims-based algorithms do not account for the range of clinical presentations and do not adequately identify intervention targets. Given the inadequacy of claims-based definitions of PAHs and that root cause analyses by embedded clinicians provides a richer, more detailed standard of identifying PAHs but are too cumbersome to sustain in daily clinical practice, this study seeks to develop a tool to better identify PAHs. We sought to develop a quantifiable method of PAHs identification that more efficiently incorporates the lessons of robust root cause analyses, to determine the avoidability of admissions and provide information to nursing homes for quality improvement. To address this need, we have developed and pilot-tested an instrument to rate the avoidability of transfers using information commonly available in the facility medical record. The innovation of this study is that it takes advantage of an evaluator’s clinical acumen but can be completed remotely via the electronic medical record (EMR). We report here the first stage of development of the Avoidable Transfer Scale, a new instrument to assess avoidability of a transfer of a nursing home resident.

Research Design and Methods

The Indiana University Institutional Review Board approved this study.

Stage 1—Instrument Development

An 8-year Centers for Medicare and Medicaid Services (CMS) Innovations Center initiative was launched in 2012 to reduce PAHs of nursing home residents that incorporated both root cause analyses and diagnosis-based identification of PAHs (19,20). There were 2 phases of the initiative with 7 regions in the United States included in the first phase and 6 regions included in the second phase. The Indiana-based site of the initiative, Optimizing Patient Transfers, Impacting Medical Quality and Improving Systems: Transforming Institutional Care (OPTIMISTIC), included placement of a registered nurse in 19 area nursing homes with nurse practitioner support during Phase 1 of the project (21). Phase 1 focused on implementing a clinical model to reduce PAHs and Phase 2 expanded the project to include enhanced reimbursement for managing 6 potentially avoidable conditions in nursing homes rather than transferring residents to the hospital (pneumonia, heart failure exacerbation, chronic obstructive lung disease or asthma exacerbation, electrolyte imbalance or dehydration, urinary tract infection, and new skin infection) (20,22).

OPTIMISTIC nurses supported the nursing home staff in improving advance care planning, managing clinical care when there is an acute change in condition of one of the nursing home residents, and root cause analyses of transfers of nursing home residents to the hospital. During Phase 2, OPTIMISTIC staff provided additional support and education about the 6 conditions in addition to their other duties. The original nursing homes retained the onsite support of OPTIMISTIC staff in addition to the enhanced payment model (clinical + payment facilities), and additional facilities were added with only the enhanced payment option (payment only facilities) (23). In both Phase 1 and Phase 2 of the demonstration project, the nurse assigned to a resident’s nursing home performed a root cause analysis when a resident transferred to the hospital.

To develop a more pragmatic approach to determining avoidable transfers, we set out to develop an instrument that enabled nurses who were unfamiliar with the resident’s transfer to assess them using only data in the facility EMR. From January 2019 through January 2020, a project team comprised of nurses, nurse practitioners, and physicians undertook intensive small and large group root cause analyses of select hospital transfers to refine their definition of an avoidable hospital transfer and identify areas for improvement in their respective nursing homes.

The review process proceeded from collaborative groups of 3–4 reviewers plus a group leader to a core team of 3 individuals who developed the final product (J.L.C., S.K., R.E.). The collaborative root cause analysis groups met monthly from June 2019 until December 2019 to review a total of 30 different cases and identify contributors to avoidable hospital transfers. All OPTIMSITIC clinical staff participated in these collaborative groups. Discussion leaders took notes and reviewed written responses to case reviews.

Once the data were collected from the collaborative groups, a smaller cohort of discussion leaders, 3 additional volunteer staff nurses, and the core pilot project team met twice to review the list of components and themes that contributed to avoidable hospital transfers. Evidence from the group root cause analyses and from an extensive literature review were used to identify thematic contributors to the avoidability of a PAH. Instrument items derived from the literature review include the 100–100–100 criteria, availability of needed services in the nursing home, evidence of advance care planning, and clinician decision making (11,12,24). Two overlapping components and 11 themes that were related to avoidability were identified from the literature review and the root cause analysis meetings. The components were patient level and systems level; the themes were staff skills, orders executed, diagnosis/workup attempted, communication (patient/care partners or family), communication (clinical care team), change in condition identification, facility resources, goals of care, provider comfort with managing patient in house, acuity of illness, and request by patient/care partners or family (Table 1). Patient-level components are those associated with a specific patient situation that led to treatment decisions regarding the change in condition prompting a hospital transfer. Systems-level components are those that are associated with the nursing home’s culture, practices, and resources that could affect any resident experiencing a change in condition. Twenty-seven individual questions were developed. Each was associated with either systems or patient level, or both, and with one or multiple of the 11 themes that the expert review team identified as essential to the construct of avoidability of an acute transfer from a nursing home.

Table 1.

Themes and Components Related to Avoidability

	Definition
Theme
Staff skills	Staff’s ability to perform required tasks to manage resident in house
Orders executed	Action taken on orders for resident prior to and as a result of the CIC
Diagnosis/workup attempted	Staff and/or PCP assessment of resident with CIC or orders to evaluate patient’s CIC
Communication (family and care partners or family)	Staff and/or PCP discusses CIC and other clinical issues with care partners/family and/or resident (or attempts to do so)
Communication (clinical care team)	Staff and PCP discuss CIC and other clinical issues with each other or attempt to do so
CIC identification	Staff and/or PCP identify CIC or take actions to identify CIC
Facility resources	Availability of resources in facility to care for resident.
Goals of care	Evidence that goals of care are known to facility staff/PCP when making recommendations and decisions about CIC
Provider comfort with managing resident in house	Provider’s comfort with keeping resident in facility for management of CIC. This may be provider-dependent or facility-dependent
Acuity of illness	Severity of illness contributes to management of CIC
Request by resident/care partners or family	Resident and/or care partners/family request where CIC is managed (either in nursing home or in hospital)
Component
Patient level	Components associated with a specific patient situation that led to treatment decisions regarding the CIC
Systems level	Components in the facility that could affect any patient with a CIC

Notes: CIC = change in condition; PCP = primary care provider.

Stage 2—Instrument Pilot Testing

To determine whether the instrument could be completed based on data available in the EMR, we pilot tested it on a set of transfers from 10 of 17 participating nursing homes with project nurses embedded in them. The 10 nursing homes were selected because they shared a common EMR. The cases for review were identified during the time period of May 2019 to January 2020. We selected the first transfer for a resident during the pilot time frame for a review. Each transfer review using the instrument was performed by 2 project nurses. The nurse reviewers were assigned to review cases in nursing homes they had not been embedded in as part of their regular OPTIMISTIC project duties. By ensuring that the reviewing nurses had not evaluated the transfers previously, we were able to perform a pilot test of the instrument.

Five project nurses conducted ratings using the preliminary 27-item instrument. All nurses had been trained as OPTIMISTIC intervention nurses and had participated in root cause analyses. Additionally, these nurses underwent additional training with the core pilot project team (J.L.C., S.K., R.E.) on how to use the instrument to evaluate hospital transfers of nursing home residents. Each transfer was reviewed independently by 2 nurses to evaluate interrater reliability. They exclusively used the EMR to review the transfers and answer the questions in the avoidability instrument. Additionally, the reviewers provided their overall impression of whether they thought each transfer was avoidable or not.

Data Analysis

Using a kappa statistic for chance-corrected agreement, interrater reliability was assessed for each avoidability instrument item and the question of overall avoidability. In keeping with the Landis and Koch guidelines for measurement of interrater reliability, items with a kappa greater than 0.20 were retained (25). Missingness of EMR-derived data was evaluated, though it was not used as a strict criteria to eliminate items. The score on the avoidability instrument was then compared with the overall rating of avoidable versus not avoidable.

Results

There were 154 cases reviewed. Due to an incomplete evaluation from one case review, only 153 cases were analyzed. Each case represents a unique nursing home resident transfer to the Emergency Department or hospital stay. The mean age of the residents was 75.8 years old (Table 2), the majority were female (58.4%), and the majority were identified as White (73.4%). Most had some degree of cognitive impairment on the Cognitive Function Scale (65.1%).

Table 2.

Patient Characteristics (N = 154 Patients)

	Overall (N = 154)
Age at closest minimum data set assessment
Mean (SD)	75.8 (11.7)
Range	37.0–100.0
Gender, n (%)
Male	64 (41.6)
Female	90 (58.4)
Race,*n (%)
White	113 (73.4)
Black	35 (23.6)
Missing	6 (3.8)
Marital status, n (%)
Never married	20 (13.0)
Married	35 (22.7)
Widowed	41 (26.6)
Separated	0 (0.0)
Divorced	30 (19.5)
Unknown	28 (18.2)
Congestive heart failure, n (%)
Yes	62 (40.3)
Chronic obstructive pulmonary disease, n (%)
Yes	63 (40.9)
Diabetes, n (%)
Yes	68 (44.2)
Cerebrovascular accident/stroke, n (%)
Yes	24 (15.6)
Hypertension, n (%)
Yes	130 (84.4)
Cancer, n (%)
Yes	5 (3.2)
Depression, n (%)
Yes	100 (64.9)
Activities of daily living score
Mean (SD)	19.1 (3.3)
Range	7.0–27.0
Cognitive Function Scale score, n (%)
1—cognitively intact	51 (34.9)
2—mildly impaired	33 (22.6)
3—moderately impaired	55 (37.7)
4—severely impaired	7 (4.8)
N—missing	8 (5.2)
CHESS score
Mean (SD)	1.4 (1.0)
Median (Q1, Q3)	1.0 (1.0,2.0)
Range	0.0–4.0

Notes: CHESS = Changes in Health, End-stage disease and Symptoms and Signs.

*Race was only reported as Black, White, or missing.

The nurse raters’ answers to the 27 items were evaluated, and all items with a kappa less than 0.20 were removed, resulting in 12 remaining items (Table 3; Items: 2, 3, 7, 8, 14, 21–27). Of the 12 selected items, 6 had unknown answers for more than a third of the items, indicating that the raters could not find the information in the nursing home EMR for the case being reviewed (Supplementary Material Sections 1 and 2). This would yield a 6-item instrument. The scatterplot of agreement between the first and second raters was plotted for the 6-, 12-, and 27-item versions of the Avoidable Transfer Scale, along with a nonparametric smoothed curve of best fit to illustrate the relationships and to examine the association of total scores between the 2 raters (Figure 1). The best-fitting curve was slightly closer to the 45-degree line of absolute agreement for the 12-item scale compared with the other 2 scales.

Table 3.

Instrument Questions and Kappa Among Multiple Raters—for All Items (Sample Size Depends on Agree/Disagree Responses, See Item Tables in Supplementary Material; N = 154)

Instrument Questions	N	Kappa	Missing, N (%)
1. A pertinent nursing assessment was completed (within 12 h) prior to transfer	143	0.104	11 (7.1%)
* 2. A thorough systems-focused assessment was completed by the nurse at the time the change in condition was noted	137	0.378	17 (11.0%)
*3. All physician/provider orders pertaining to this change in condition were completed	68	0.527	86 (55.8%)
4. Results of diagnostic testing were reported to the PCP or covering team as soon as available	27	0.000	127 (82.5%)
5. Warning signs of CIC were reported by staff to a supervisor	47	0.187	107 (69.5%)
6. Prior to transfer, staff reported CIC to patient’s primary care physician/advanced practice provider	116	0.127	38 (24.7%)
*7. Available diagnostic services were utilized to evaluate/diagnose the CIC	81	0.337	73 (47.4%)
*8. Available treatments were used to treat and stabilize patient	82	0.294	72 (46.7%)
9. Standard diagnostic tools (for nursing homes) were available in facility at time of CIC	86	−0.018	68 (44.2%)
10. Standard treatment tools (for nursing homes) were available at time of CIC	84	−0.016	70 (45.4%)
11. If there was a recent decline in function (past month), it was addressed appropriately	48	0.057	106 (68.8%)
12. In addition to code status documentation, goals of care have been established or addressed in the medical record	137	0.128	17 (11.0%)
13. Goals of care were known at the time of transfer	134	−0.033	20 (13.0%)
*14. Goals of care were honored at the time of transfer	95	0.501	59 (38.3%)
15. CIC was communicated among covering nursing staff	23	−0.045	131 (85.1%)
16. CIC was communicated between staff when changing shifts	15	−0.071	139 (90.3%)
17. There was no bias or personal feelings that led to the transfer	18	0.160	136 (88.3%)
18. The physician or advanced practice provider was notified of CIC when it occurred	128	−0.013	26 (16.9%)
19. The family or responsible party for the resident was notified of CIC when it occurred	130	0.123	24 (15.6%)
20. The physician or provider ordering or approving a transfer was familiar with the patient	35	0.000	119 (77.3%)
** 21. Hospitalization was clearly part of the resident’s goals of care	115	0.556	39 (25.3%)
** 22. Medical necessity at the time of the transfer favored hospital transfer	125	0.329	29 (18.8%)
** 23. In the 12 h prior to transfer, two of the three 100–100–100 criteria were met (temp > 100°F, heart rate > 100, systolic blood pressure < 100)	120	0.375	34 (22.1%)
*24. Family or responsible party for resident were involved in treatment plan for CIC	91	0.450	63 (40.9%)
*25. This question is concerning policies and procedures that are related to the CIC that prompted the hospital transfer. Policies and procedures to address the CIC were followed	50	0.369	104 (67.5%)
** 26. At the time of transfer, the resident’s condition could not be safely assessed in the facility	126	0.265	28 (18.2%)
** 27. At the time of transfer, the resident’s condition could not be safely treated in the facility	117	0.363	37 (24.0%)

Notes: CIC = change in condition; PCP = primary care provider.

*Items marked with one asterisk met the criteria for inclusion based on the kappa result.

**Items marked with 2 asterisks met the criteria for inclusion based on the kappa result and the number of missing.

Figure 1.

Plots of total scores—first rating by second rating (N = 154).

Scale scores were significantly higher for admissions judged by raters as avoidable compared to admissions rated as unavoidable for the full 27-item scale score (6.8 vs 3.7, p < .001; Table 4), the refined 12-point scale (5.3 vs 2.6, p < .001), and the 6-point scale (3.8 vs 1.9, p < .001). Based on these results, the 12-item version was chosen as the final scale.

Table 4.

Scores by Avoidability Rating of 2 Nurse Reviewers Answering “Was This Transfer Avoidable?” Versus the Avoidable Transfer Scale Score

	Avoidability Judgment			p Value
	No (n = 105)	Yes (n = 48)	Total (N = 153)
27-Point Avoidable Transfer Scale score				<.001*
Mean (SD)	3.7 (2.3)	6.8 (3.0)	4.7 (2.9)
Median (Q1, Q3)	3.0 (2.0,5.0)	6.0 (5.0,9.2)	4.0 (2.0,6.0)
Range	0.0–11.0	2.0–13.0	0.0–13.0
12-Point Avoidable Transfer Scale Score				<.001*
Mean (SD)	2.6 (1.6)	5.3 (2.4)	3.4 (2.2)
Median (Q1, Q3)	2.0 (1.0,3.0)	5.0 (4.0,7.0)	3.0 (2.0,5.0)
Range	0.0–8.0	1.0–11.0	0.0–11.0
6-Point Avoidable Transfer Scale Score				<.001*
Mean (SD)	1.9 (1.1)	3.8 (1.5)	2.5 (1.5)
Median (Q1, Q3)	2.0 (1.0,2.0)	4.0 (3.0,5.0)	2.0 (1.0,3.0)
Range	0.0–5.0	1.0–6.0	0.0–6.0

Notes: N = 153 patients because of one incomplete rating.

*p Value is from F-test in Linear Model ANOVA.

Items associated with both systems-level and patient-level components were included (Supplementary Material Section 3). Not all of the themes identified as relevant to avoidability were included in the final 12-point scale. No questions associated with the themes of communication with the clinical care team, identification of the change in condition, or request for transfer by the patient or family were included in the final 12-point scale.

Discussion and Implications

The instrument described here, the Avoidable Transfer Scale, combines the convenience of an EMR with the rigor and clinical relevance of root cause analyses, providing a new approach to identification of a potentially avoidable hospital transfer of a nursing home resident. Furthermore, the individual items from the scale can be used to identify areas for improvement in individual nursing homes. For example, Item 8 says “Available treatments were used to treat and stabilize the patient.” In a specific nursing home, if there is a trend for this question to be often marked incomplete, then this could prompt the nursing home to evaluate how familiar staff and providers are with treatments available in the building.

Three of 11 themes identified by the core pilot team were excluded from the final 12-item Avoidable Transfer Scale based on our review criteria. Communication between members of the clinical care team may be especially difficult to identify in the EMR because most clinicians do not document every conversation that they have with each other. Likewise, identification of the change in condition may not be well documented until the resident needs to be transferred to the hospital. Finally, given the evidence from published literature that patient or care partner requests for transfer are relevant to avoidability, it was unexpected that this theme would be excluded (8,11,18). Because all 3 of these issues were identified as relevant to avoidability by the OPTIMISTIC staff, additional questions that address these themes may need to be tested. In the future, improved documentation of these important themes may improve patient care by aiding in identifying avoidable transfers.

By using the Avoidable Transfer Scale, nursing homes may be able to not only retrospectively identify PAHs in their buildings but also identify issues common across transfers to target for future improvement efforts, thereby reducing the number of avoidable transfers. Leveraging information available in the EMR and codifying it into nominal data, future iterations of the Avoidable Transfer Scale can be refined so that it functions in an automated and scalable format.

We have developed an instrument to identify PAHs that does not require the participation of the staff at the nursing home where the transfer took place. However, it does require use of clinical judgment in the ratings, preserving the invaluable contribution of clinical acumen to PAH identification. The ability to implement quality improvement initiatives such as this should be the standard of care in nursing homes. Quality assurance and performance improvement is required of nursing homes that bill CMS for services and adequate staffing is one way to ensure this is performed well (26,27). Recent initiatives such as INTERACT and the CMS Initiative to Reduce Avoidable Hospitalizations Among Nursing Facility Residents have made strides in reducing avoidable hospital transfers, but future study is needed to further the science (15,16,20,22,28,29). As such, the Avoidable Transfer Scale represents an evolution toward more efficient and enhanced nursing home care.

Tools such as the Avoidable Transfer Scale offer opportunities to outsource quality improvement work. Recent scholarship on nursing home care has revealed that in emergency situations like the coronavirus disease 2019 pandemic it may be necessary to limit access to buildings or employ other infection mitigation strategies (30,31). Furthermore, the staffing crisis is significant, which can lead to staff with quality improvement duties reassigned to cover essential nursing home functional duties instead (32,33).

Although these results are promising, there are some limitations. In an effort to develop an instrument that could be widely used at low cost, we provided only 2 lecture-based trainings over the course of 7 years for the nurses. Better interrater reliability may be achieved by more extensive training and review of ratings to achieve standardization in ratings. This is the first stage of development of this instrument and in future studies of the Avoidable Transfer Scale other psychometric measures will be examined, such as test–retest reliability, factor analysis, and predictive validity. We may investigate individual patient and nursing home level factors to identify potential markers to further the tool’s potential for predictive validity. This may include exploring items from the Avoidable Transfer Scale and additional contributors. Pairing use of the Avoidable Transfer Scale with an initiative to improve documentation of each item may reduce unknown data, such as incomplete documentation about pretransfer clinical care. There is some evidence that facility factors (rural vs urban; percent Medicaid; star rating; private vs nonprofit; staff turnover rate) may affect quality of care and hospitalizations of nursing home residents (32,34). Furthermore, not all nursing homes have access to an EMR, which enhances the convenience of this tool, though is not necessary to use it. Finally, this study was limited to facilities that participated in a high-profile demonstration project in one metropolitan area and the Avoidable Transfer Scale should be tested in diverse settings. Based on these limitations, further refinement is needed prior to widespread use.

In conclusion, the Avoidable Transfer Scale shows promise as a tool that can be used to identify avoidable hospital transfers of residents of nursing homes through medical record review. The next step in development of this tool is to perform additional research on the instrument, including research on the excluded themes, to further refine the instrument. Expanded training and testing of this tool in other regions will establish its generalizability and relevance to improving nursing home care. Future iterations of the Avoidable Transfer Scale may aid in remote quality improvement initiatives via the EMR and machine learning could be leveraged to automate this process.

Click here for additional data file.