The predictive properties of frailty-rating scales in the acute medical unit.

BACKGROUND: older people are at an increased risk of adverse outcomes following attendance at acute hospitals. Screening tools may help identify those most at risk. The objective of this study was to compare the predictive properties of five frailty-rating scales.
METHOD: this was a secondary analysis of a cohort study involving participants aged 70 years and above attending two acute medical units in the East Midlands, UK. Participants were classified at baseline as frail or non-frail using five different frailty-rating scales. The ability of each scale to predict outcomes at 90 days (mortality, readmissions, institutionalisation, functional decline and a composite adverse outcome) was assessed using area under a receiver-operating characteristic curve (AUC).
RESULTS: six hundred and sixty-seven participants were studied. Frail participants according to all scales were associated with a significant increased risk of mortality [relative risk (RR) range 1.6-3.1], readmission (RR range 1.1-1.6), functional decline (RR range 1.2-2.1) and the composite adverse outcome (RR range 1.2-1.6). However, the predictive properties of the frailty-rating scales were poor, at best, for all outcomes assessed (AUC ranging from 0.44 to 0.69).
CONCLUSION: frailty-rating scales alone are of limited use in risk stratifying older people being discharged from acute medical units.

Introduction

The proportion of the UK population aged 65 and older will increase from 17 to 23% by the year 2035 [1], which is expected to lead to an increase in acute hospital care use. Older people accessing acute care are at an increased risk of adverse outcomes [2]. Interventions that improve outcomes for older people will be more cost-effective in selected populations than if just aimed at all older people [3]. For older patients presenting with medical emergencies, risk stratification tools have been used to identify those people at the highest risk of adverse outcomes [4]; however, the predictive properties of such tools are insufficient for clinical practice [5, 6].

An alternative approach to using risk stratification tools is to describe a clinical phenotype of older people who are at a high risk of adverse outcomes. Frailty is a state of loss of resistance to external stressors leading to extreme vulnerability and subsequent decline [7]. In community populations, frailty is associated with an increased risk of functional decline, death, hospitalisation and worsening mobility [8] so it is logical to test frailty scales to assess their predictive properties in acute care to see if they can be used as a method of identifying those at risk.

The aim of this study was to assess the predictive value of frailty-rating scales in older people presenting as medical emergencies.

Method

Study design

This was a secondary analysis of the Acute Medicine Outcome Study (AMOS)—a two-centre, observational cohort study involving older people discharged home from acute medical units (AMUs) within 72 h. Participants were recruited from the AMUs at Queen's Medical Centre, Nottingham, and Leicester Royal Infirmary, Leicester between January 2009 and November 2010. Patients were eligible if they were resident in the hospital catchment area, were ≥70 years and were expected to be discharged home from the AMU within 72 hours. Patients were excluded if they lacked mental capacity, unless a family consultee or the responsible AMU physician agreed. The selection process for inclusion can be seen in Supplementary data available in .

Baseline assessment

Data were collected either from patient records or by direct enquiry, including:

demographics including age, gender, height and weight;

Charlson comorbidity index [9] (a comorbidity score derived from a weighted list of medical conditions);

Geriatric syndromes (defined as the reported presence in the medical notes for that admission of any of the following: fall, reduced mobility, continence disorder, current pressure sore, memory problems or polypharmacy ≥4 prescribed medications);

list of prescribed medications;

Mini Nutritional Assessment [10] (a six item tool which is widely used to identify people at risk of malnutrition);

Folstein Mini-Mental State Examination [11] (MMSE—a standardised measure of global cognitive function);

Barthel ADL index [12] (a 10-item scale assessing basic activities of daily living, with possible scores ranging from 0 to 20);

EuroQoL-5D [13] (EQ5D—a standard measure of quality of life);

General Health Questionnaire-12 (GHQ-12—a 12-item screening tool for identifying minor psychiatric disorder ranging from 0 to 36).

Frailty-rating scales

Five frailty-rating scales were tested, including the models proposed by Fried (CHS model) [8], Ensrud (SOF model) [14], Rothman [15], Ávila-Funes [16] and a frailty index (FI) [17]. Frailty indices count the number of deficits present and describe these as a proportion of all deficits assessed; individuals with an FI of >0.25 are at an increased risk of adverse outcomes [18]. For the full list of variables used to construct the FI, Please see Supplementary data available in . Table 1 shows the original items for each scale, with the adapted items used in this study, and the cut-off score used to categorise participants into frail or non-frail comparisons.

Table 1.

Frailty-rating scales cut-offs, domains, original items and adapted items

Scale	Domain	Original Item	Adapted Item
CHS model frail = 3/5	Nutritional status	Weight loss >10 lbs unintentionally in the preceding year	Weight loss >5 kg in preceding year
	Strength	Grip strength, the lowest 20% scoring positive	Grip strength, the lowest 20% in this population scoring positive
	Energy	Self-reported exhaustion	Assessed from the question ‘Do you feel full of energy?’, where an answer of ‘no’ scored positive
	Mobility	Gait speed over 15 feet, the lowest 20% scoring positive	Gait speed 2.4 m walk, the lowest 20% scoring positive
	Physical activity	Kcals/week, lowest 20% scoring positive	Assessed by questions from EuroQol-5D, the lowest 20% scoring positive Mobility (confined to bed = 3, problems walking = 2, no problems = 1, Self-care (unable to wash/dress = 3, some help to wash/dress = 2, no problems = 1) and Usual activities (unable to perform usual activities = 3, some problems = 2, no problems = 1)
SOF model frail = 2/3	Nutritional status	Weight loss ≥5% between third and fourth examination	Weight loss >5 kg in preceding year
	Strength	Inability to rise from a chair five times without using his/her arms	Inability to rise from a chair five times without using his/her arms
	Energy	Assessed from the question ‘Do you feel full of energy?’	Assessed from the question ‘Do you feel full of energy’?, where an answer of ‘no’ scored positive
Ávila-Funes's instrument frail = 3/5	CHS model	See above	See above
	Cognition	MMSE and Isaacs set test, lowest 25% scoring positive	MMSE, lowest 25% scoring positive in this population
Rothman's instrument frail = 2/4	Mobility	Gait speed of >10 s to walk back and forth over a 10 foot course	Gait speed 2.4 m walk, the lowest 20% scoring positive
	Physical activity	Score <64 for men, <52 for women on the physical activity scale for the elderly	Assessed by questions from EuroQol-5D, the lowest 20% scoring positive Mobility (confined to bed = 3, problems walking = 2, no problems = 1, Self-care (unable to wash/dress = 3, some help to wash/dress = 2, no problems = 1) and Usual activities (unable to perform usual activities = 3, some problems = 2, no problems = 1)
	Nutritional status	Weight loss >10 lbs in past year (intentional and unintentional)	Weight loss >5 kg in preceding year
	Cognition	MMSE <24/30 indicating cognitive impairment	MMSE <24/30 indicating cognitive impairment

Outcomes

The outcomes were assessed at 90 days using hospital records and a postal follow-up questionnaire. Outcomes were mortality, institutionalisation (defined as a new admission to a care home or a move to a new care home as a result of an increased care need), unplanned hospital readmission and decline in function (defined as a decrease of two or more points on Barthel ADL). In addition, a composite outcome was constructed and defined as any of the outcomes of mortality, institutionalisation, unplanned hospital readmission and decline in function, at 90 days.

Statistical analyses

Descriptive statistics were drawn to characterise the baseline population. Relative risks (RRs) were calculated to compare the risk of poor outcomes in frail and non-frail participants. To assess the predictive properties of each scale, a receiver-operating characteristic curve reflecting the full range of values in each scale was plotted with the area under the curve (AUC) calculated for each scale and outcome. The AUC analysis was carried out using the full range of values in each scale—so the AUC reported for the CHS model reflects all five levels of the scale. An area >0.70 was used to show that a scale has an acceptable discriminatory value, 0.60–0.69 as poor and 0.50–0.59 as very poor [1]. A score of 0.50 indicates that the scale has no predictive value. In addition, the sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) were also calculated for the groups classified as frail versus non-frail using each scale. 95% confidence intervals (CIs) were reported for AUC and RRs. Statistical analyses were performed using Stata version 9.

Results

In total, 669 participants were recruited from two centres (Nottingham and Leicester) between 2009 and 2010. Two participants subsequently withdrew consent and their data could not be used. Of those remaining (667) in the study, 494 (83%) returned at least some outcome questionnaires at 90 days, and service outcomes such as readmissions and length of stay were available for 644 participants (23 withdrew). Demographics and clinical characteristics of the participants are shown in Table 2.

Table 2.

Baseline characteristics of the older participants

Variable	Patients
Age (median, inter-quartile range)	80 (75, 85)
Gender (n, female %)	386 (57.9)
Ethnicity (n, White %)	647 (97.0)
Residence (n, alone %)	309 (46.3)
Charlson rating (n, %)
None (0)	235 (35.2)
Mild (1–2)	335 (50.2)
Moderate (3–4)	79 (11.8)
Severe (≥5)	18 (2.7)
Number of medications (median, IQR)	6 (4, 9)
Number of geriatric syndromes (mean, SD)	1.3 (±0.8)
Cognition (n, %)
Normal (≥25/30)	523 (78.4)
Mild (21–24/30)	84 (12.6)
Moderate (10–20/30)	50 (7.5)
Severe (≤9/30)	10 (1.5)
EQ5D (median, IQR)	0.69 (0.36, 0.80)
Barthel ADL (median, IQR)	18 (17, 20)
General Health Questionnaire (median, IQR)	11 (8, 15)
Prevalence of frailty (n/N frail, %)
CHS model	110/613 (17.9)
SOF model	427/643 (66.4)
Ávila-Funes instrument	142/571 (24.9)
Rothman instrument	142/589 (24.1)
Frailty index	173/559 (30.9)

IQR, inter-quartile range; SD, standard deviation.

At 90 days following the index visit to AMU 36.7% (245/667) of participants experienced one or more adverse outcome. Overall, the risk of mortality was 5.1% (34/667), the risk of institutionalisation was 0.9% (6/667) and the risk of readmission was 26.7% (172/644). Of the participants who completed the follow-up questionnaire (494/667), the risk of functional decline was 20.2% (97/480). The predictive properties of the different frailty measures are shown in Table 3.

Table 3.

Predictive properties of frailty-rating scales for 90-day outcomes

Scale	Cut-off	Sensitivity	Specificity	PPV	NPV	AUC (95% CI) (full scale)	RR (95% CI)
Mortality (n = 667)
CHS model	≥3/5	25.8	82.5	7.3	95.4	0.61 (0.52, 0.70)	1.6 (0.9, 3.0)
SOF model	≥2/3	76.5	34.2	6.1	96.3	0.59 (0.50, 0.67)	1.6 (0.9, 3.1)
Ávila-Funes	≥3/6	50.0	76.5	10.6	96.5	0.68 (0.59, 0.76)	3.0 (2.0, 4.5)
Rothman	≥2/4	50.0	77.3	10.6	96.6	0.67 (0.57, 0.76)	3.1 (2.1, 4.6)
FI	>0.25/1.00	48.3	70.0	8.1	96.1	0.69 (0.60, 0.78)	2.1 (1.4, 3.1)
Readmissions (n = 644)
CHS model	≥3/5	22.9	84.6	35.0	75.3	0.52 (0.47, 0.57)	1.4 (1.0, 2.0)
SOF model	≥2/3	68.5	35.1	28.1	75.0	0.53 (0.48, 0.58)	1.3 (1.0, 1.7)
Ávila-Funes	≥3/6	31.5	78.5	34.6	76.1	0.55 (0.50, 0.60)	1.4 (1.1, 2.0)
Rothman	≥2/4	33.6	80.0	37.3	77.2	0.53 (0.48, 0.59)	1.6 (1.2, 2.2)
FI	>0.25/1.00	40.7	72.5	35.1	76.9	0.57 (0.52, 0.63)	1.5 (1.2, 2.0)
Institutionalisation (n = 667)
CHS model	≥3/5	0.0	81.9	0.0	99.0	0.57 (0.42, 0.72)
SOF model	≥2/3	60.0	33.5	0.7	99.1	0.44 (0.25, 0.63)	Null
Ávila-Funes	≥3/6	0.0	74.9	0.0	98.8	0.50 (0.36, 0.64)
Rothman	≥2/4	0.0	75.7	0.0	98.9	0.45 (0.30, 0.60)
FI	>0.25/1.00	40.0	69.1	1.2	99.2	0.55 (0.25, 0.85)	1.5 (0.5, 4.4)
Functional decline (n = 480)
CHS model	≥3/5	22.8	88.5	33.9	81.6	0.55 (0.49, 0.62)	1.8 (1.1, 2.9)
SOF model	≥2/3	69.1	36.4	21.5	82.4	0.56 (0.49, 0.62)	1.2 (0.8, 1.7)
Ávila-Funes	≥3/6	36.7	75.7	26.9	83.1	0.59 (0.52, 0.66)	1.9 (1.2, 2.8)
Rothman	≥2/4	29.3	84.6	32.0	82.8	0.59 (0.52, 0.66)	2.1 (1.5, 3.2)
FI	>0.25/1.00	33.7	84.5	35.4	83.5	0.57 (0.50, 0.64)	1.6 (1.1, 2.2)
Composite (n = 667)
CHS model	≥3/5	22.1	84.5	45.5	65.0	0.53 (0.48, 0.57)	1.3 (0.9, 1.8)
SOF model	≥2/3	70.2	35.8	39.1	67.1	0.54 (0.50, 0.59)	1.2 (0.9, 1.5)
Ávila-Funes	≥3/6	32.2	79.3	47.2	67.1	0.57 (0.52, 0.62)	1.4 (1.1, 1.9)
Rothman	≥2/4	33.2	81.1	50.0	68.0	0.55 (0.50, 0.59)	1.6 (1.2, 2.1)
FI	>0.25/1.00	39.5	74.0	46.8	67.9	0.59 (0.54, 0.64)	1.5 (1.1, 1.9)

PPV, positive predictive value; NPV, negative predictive value; AUC, area under the curve; RR, relative risk.

–, These are the scores for the dichotomised scale.

Four of the five frailty models significantly predicted death, but only poorly using AUC analysis. Three of the five models predicted readmissions, again only very poorly. The number of people who were institutionalised was too low for meaningful estimations. Four of the five significantly predicted functional decline, but again only very poorly. All five predicted the composite outcome, but again only very poorly.

Discussion

This study assessed the role of frailty-rating scales in predicting adverse health outcomes in AMUs in the UK. At 90 days, over a third of participants experienced one or more adverse outcome. The most common of these was readmission (26.7%) followed by functional decline (20.2%).

Although most of the scales performed better than chance in predicting a range of poor outcomes, none of them performed adequately, and most performed either poorly or very poorly. Those scales that included cognition (Ávila-Funes, Rothman and FI) had a better predictive accuracy for all outcomes bar institutionalisation.

Strengths of this study include a large sample size, with detailed individual health and service outcomes collated from two sites. Owing to initial constraints placed on the study by the ethical committee, we were unable to include a larger proportion of participants with cognitive impairment, so study population may not be representative of all older patients accessing AMUs. If we had included more higher-risk patients, this would have increased sensitivity and reduced specificity, but the effect on the AUC is uncertain. A sizeable proportion (17%) of those recruited declined ascertainment of clinical outcomes despite the protocol offering postal, telephone or face-to-face follow-up, and these patients tended to have more adverse characteristics than those in whom clinical outcomes were ascertained. Despite these two factors, the incidence of adverse outcomes was much higher than expected, which will have increased the power of the study. Thus, we believe that the estimates of the frailty scales to predict such adverse outcomes are broadly correct. It is unlikely that larger study would contribute significantly to our understanding of the role of these tools as the CI for each of the AUC scores are incompatible with good or excellent predictive ability.

A possible limitation of this study is that we did not use the exact published tool but adapted them as described so that data collected in this cohort study could be used. To minimise this problem, the scales were adapted as closely as possible to the published scales, reflecting the common domains used in frailty research. We believe that this process does not explain their poor performance. One final limitation is that the frailty scales might be better at predicting longer-term outcomes rather than these short-term ones.

The weak predictive properties of the frailty scales are consistent with the findings from other studies [6, 19], adding further doubt to the usefulness of risk stratification alone in the acute care context. The frailty level of a patient is one of many factors that contribute to a poor outcome, and currently, the presence of frailty as measured by these scales alone is not sufficient to predict outcomes.

Despite the poor predictive values, the scales are still better than chance, and should not be fully dismissed. All scales had a high NPV for mortality, institutionalisation and functional decline, indicating that those who are not classified as frail would not be likely to have the adverse outcome. The RR scores for nearly all of the frailty scales demonstrated that frailty was able to identify people at a higher risk of adverse events. This may provide an opportunity to use frailty-rating scales as a tool for exclusion considering the majority of the population was classified as non-frail and are unlikely to have an adverse outcome. An exception to this is the SOF model which classified the majority as frail, possibly related to the large proportion (78%) of people who could not rise from a chair unaided—none of the other frailty scales used this item. It is unlikely that available frailty scales alone will have adequate predictive ability to be used in standard day-to-day clinical management. Instead, further work is needed to identify a battery of indicators, including clinical judgement that might have a good predictive value when combined.

Frail older people are at an increased risk of adverse outcomes compared with those who are non-frail

Frailty-rating scales have limited clinical application due to their poor predictive properties in identifying those at risk of adverse outcomes

Ethical approval

Ethical approval was obtained from Southampton REC (reference 08/H0502/139) with research governance approvals being obtained in each centre.

Funding

The study was funded by the National Institute for Health Research (NIHR) and the University of Leicester. This article presents independent research funded by the National Institute for Health Research (NIHR) under its Programme Grants for Applied Research funding scheme (RP-PG-0407-10147). The views expressed in this article are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health.

Supplementary data

Supplementary data mentioned in the text is available to subscribers in