Falls screening and assessment tools used in acute mental health settings: a review of policies in England and Wales.

OBJECTIVES: There is an urgent need to improve the care of older people at risk of falls or who experience falls in mental health settings. The aims of this study were to evaluate the individual falls risk assessment tools adopted by National Health Service (NHS) mental health trusts in England and healthcare boards in Wales, to evaluate the comprehensiveness of these tools and to review their predictive validity.
METHODS: All NHS mental health trusts in England (n=56) and healthcare boards in Wales (n=6) were invited to supply their falls policies and other relevant documentation (e.g. local falls audits). In order to check the comprehensiveness of tools listed in policy documents, the risk variables of the tools adopted by the mental health trusts' policies were compared with the 2004 National Institute for Health and Care Excellence (NICE) falls prevention guidelines. A comprehensive analytical literature review was undertaken to evaluate the predictive validity of the tools used in these settings.
RESULTS: Falls policies were obtained from 46 mental health trusts. Thirty-five policies met the study inclusion criteria and were included in the analysis. The main falls assessment tools used were the St. Thomas' Risk Assessment Tool in Falling Elderly Inpatients (STRATIFY), Falls Risk Assessment Scale for the Elderly, Morse Falls Scale (MFS) and Falls Risk Assessment Tool (FRAT). On detailed examination, a number of different versions of the FRAT were evident; validated tools had inconsistent predictive validity and none of them had been validated in mental health settings.
CONCLUSIONS: Falls risk assessment is the most commonly used component of risk prevention strategies, but most policies included unvalidated tools and even well validated tool such as the STRATIFY and the MFS that are reported to have inconsistent predictive accuracy. This raises questions about operational usefulness, as none of these tools have been tested in acute mental health settings. The falls risk assessment tools from only four mental health trusts met all the recommendations of the NICE falls guidelines on multifactorial assessment for prevention of falls. The recent NICE (2013) guidance states that tools predicting risk using numeric scales should no longer be used; however, multifactorial risk assessment and interventions tailored to patient needs is recommended. Trusts will need to update their policies in response to this guidance.

Introduction

Falls are the most frequently reported patient safety incident [1]. Approximately 283,000 falls are reported every year in hospitals across England and Wales, with over 36,000 reported from mental health units and 38,000 from community hospitals [1]. Falls rates in mental health units for older people varied from 7.7 to 48 falls per 1000 bed-days, which is significantly higher than fall rates in community hospitals (4.5 to 12 falls per 1000 bed-days) and acute hospitals (4.3 to 13 falls per 1000 bed-days) [2]. There is an urgent need to improve the care of older people at risk of falls or who experience falls in acute older adult mental health settings as the rate of falls is higher in these settings than in other clinical settings [2]. Falls cost the UK National Health Service (NHS) more than £2.3 billion per year, and account for four million hospital bed-days in England annually [3]. In addition to the impact on healthcare costs, falls have significant human costs, including distress, loss of confidence and reduced quality of life [1]. The 2013 UK National Institute for Health and Care Excellence (NICE) guidelines recommend the use of multidisciplinary falls risk assessment of older people at risk of falls in both inpatient and community settings [4]. This extends previous guidance which did not include inpatients [5], and highlights the lack of evidence of effectiveness of multifactorial interventions for older people who are inpatients in specialist mental health units, how these interventions can be targeted at those at greatest risk, and the need for further research.

A range of falls risk assessment tools have been developed and tested in different clinical settings in order to identify older people who are at risk of falls, and to facilitate effective targeting of falls prevention interventions [6], [7]. There is scant evidence to support the use of any screening tool alone to predict falls, and most falls risk assessment tools have been found to discriminate poorly between fallers and non-fallers [7]. Use of these tools in settings/populations that differ from those for which they were developed is less successful in terms of effectiveness to predict falls; this compromises the validity of these tests and their wider application [7], [8]. It is not yet known which tools are most effective for use in acute mental health settings.

This review presents one element of a larger study exploring falls in acute mental health settings for older people [9]. The objective was to identify the range of tools recommended for operational use and included within the policy guidelines of individual NHS mental health trusts in England and health boards in Wales, and to explore whether these were sufficient to meet the NICE recommendations [4]. A further objective was to determine the predictive validity of these assessment tools in order to determine the effectiveness of their operational use.

Methods

All mental health trusts in England (n = 56) and healthcare boards in Wales (n = 6) were invited to supply their falls policies and other relevant documentation (e.g. local falls audits). Some policies were publically available on the Internet, but for others, the authors contacted the information governance team at each trust and requested the relevant documents. Non-clinical and environmental risk assessment tools for falls were excluded from the analysis as they do not assess patients’ clinical risk factors for falls.

In order to evaluate the comprehensiveness of the tools, a proforma was developed, using Microsoft Excel (Microsoft Corp., Richmond, WA, USA), to ensure a comprehensive approach and systematic data extraction across the policies. Information was extracted from the falls assessment tools as documented in the policies, the risk variables assessed within each tool were listed, and these were compared with NICE recommendations for the assessment of multiple risk variables [5]. Variables from the tools were compared with the 2004 NICE guidance as this was the guidance in place when the policies were collected; this was updated in 2013 [4].

An analytical review was undertaken to evaluate the predictive validity of each of the falls risk assessment tools outlined in these policies. The predictive validity of these tools was analysed by evaluating sensitivity, specificity, positive predictive validity (PPV) and negative predictive validity (NPV) (Table 1). A comprehensive literature search of the following healthcare databases was undertaken: EBSCO, PSychinfo, Nursing Index, MEDLINE, Pubmed and Cochrane Database of Systematic Reviews. The search strategy used the following keywords: falls, risk assessment tools, STRATIFY, MORSE, FRASE, FRAT, screening, predictive validity, elderly and older age. No limitations on year of publication were applied. Studies were included if a prospective investigation of the predictive properties of the tools outlined in the included policies had been conducted. Only studies published in the English language were considered for inclusion. In addition, studies were required to have reported predictive properties of these tools.

Table 1

Summary of predictive validity analyses of studies included in this review.

Study	Clinical setting	Tool	Positive predictive validity (%)	Negative predictive validity (%)	Sensitivity (%)	Specificity (%)
Kim et al. (2007) [13]	Acute ward	STRATIFY	2.40	99	55	75
Milisen et al. (2007) [14]	Acute geriatric ward	STRATIFY	18	93	67	59
Haines et al. (2006) [15]	Rehabilitation unit	STRATIFY	–	–	77	51
Vassallo et al. (2005) [16]	Acute medical ward	STRATIFY	28	91	68	66
Hill et al. (2004) [17]	Acute geriatric ward	STRATIFY	–	–	43	43
Papaioannou et al. (2004) [18]	Acute care medical unit	STRATIFY	–	–	91	49
Chiari et al. (2002) [20]	Elderly inpatient unit	STRATIFY	–	–	20	87
Oliver et al. (1997)(Cohort 1) [21]	Elderly inpatient unit	STRATIFY	62	98	93	88
Oliver et al. (1997)(Cohort 2) [21]	Elderly inpatient unit	STRATIFY	48	90	55	88
Vassallo et al. (2008) [12]	Geriatric rehabilitation unit	STRATIFY	30	85	82	34
Kim et al. (2007) [13]	Acute ward	MFS	6	100	55	91
Morse et al. (1989) [24]	Geriatric unit	MFS	10	99	78	83
Eagle et al. (1999) [26]	Rehabilitation and geriatric ward	MFS	38	81	72	51
O’Connell and Myers (2002) [27]	Acute aged care	MFS	18		83	29
Nandy et al. (2004) [22]	Community care	FRAT	57	86	42	92

STRATIFY, St. Thomas’ Risk Assessment Tool in Falling Elderly Inpatients; MFS, Morse Falls Scale; FRAT, Falls Risk Assessment Tool.

Sensitivity of a tool is determined by the percentage of patients who had a fall after being predicted to be at high risk, and specificity is determined by the percentage of patients who did not fall after being predicted to be at low risk [10], [11]. PPV is determined by the percentage of high-risk patients who went on to fall, and NPV is determined by the percentage of low-risk patients who did not have any falls [10], [11]. All relevant identified studies were included without considering their methodological qualities, as this was not within the scope of this review. The predictive properties of the tools from the included studies have been summarised in Table 1.

Results

Of the 62 potential falls policies, 44 were obtained from mental health trusts in England, and two were obtained from healthcare boards in Wales. Two trusts supplied their generic health and safety risk assessment policies, and when they were asked to supply their specific falls prevention policy for clinical use, one trust reported that they did not have a falls prevention strategy. Another trust reported that they were currently reviewing their policy so were not in a position to send this. Thirty policies were publically available on the Internet (trust websites).

Forty-two of the 46 falls policies recommended the use of specific falls risk assessment tools as part of their falls prevention strategy, predominantly the St. Thomas’ Risk Assessment Tool in Falling Elderly Inpatients (STRATIFY), Environmental Risk Assessment for Falls, Falls Risk Assessment Scale for the Elderly (FRASE), Morse Falls Scale (MFS) and various versions of the Falls Risk Assessment Tool (FRAT) (Table 2). Tools that focused solely on environmental factors (n = 7) for falls risk assessment were excluded from further analysis as these tools were not focused on identification of the risk factors for individual inpatients. Therefore, 35 policies were included in the analysis.

Table 2

Fall risk assessment tools outlined in mental health trusts’ policies in England and healthcare boards in Wales.

Risk assessment tools outlined in mental health trusts’ policies	Number of trusts’ policies
St. Thomas’ Risk Assessment Tool in Falling Elderly Inpatients	4
Falls Risk Assessment Scale for the Elderly	5
Falls Risk Assessment Tool	24
Environmental Risk Assessment for Falls	7
Morse Falls Scale	2

Predictive validity of falls risk assessment

Initially, 282 references were identified in the literature search. After screening the full text, 14 studies were found to meet the inclusion criteria and were included in this review. Ten studies investigated the predictive accuracy and effectiveness of the STRATIFY [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], and other studies reported the predictive accuracy of the FRAT [22], FRASE [23] and MFS [13], [24], [25], [26], [27]. The majority of studies had tested and validated these tools in acute medical, rehabilitation and geriatric wards for patients over 65 years of age [12], [14], [15], [16], [17], [18], [19], [20], [21], one study validated Part 1 of the FRAT in a community setting with an older population, but none of the tools outlined in the policies had been validated for use in acute inpatient mental health settings. All of the validation studies were published prior to 2008. The STRATIFY was originally developed in the UK, is the most widely used falls risk assessment tool in clinical practice [10], and has been subjected to more independent validation studies compared with the other tools adopted by the mental health trusts. The STRATIFY assesses five risk factors for falls (prior history of falls, gait instability, agitation, visual impairment and incontinence) to predict the severity of a patient's risk of falls (each risk scoring 1). Patients with a cut-off score of 2 or 3 are considered to be at high risk of falls [10]. Sensitivity of the STRATIFY ranged from 91% to 20% [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], specificity ranged from 87% to 34% [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], PPV ranged from 28% to 65%, and NPV ranged from 99% to 74% [12], [13], [14], [15], [16], [17], [18], [19], [20], [21].

The MFS was developed and validated in three different clinical settings including acute and long-term geriatric and rehabilitation care settings [24]. Only two policies included the MFS for falls risk assessment. The MFS includes six risk variables (history of falls, presence of secondary diagnoses, use of mobility aids, problems with gait and status of mental health) to identify the falls risk of hospitalised patients [24]. The total score for the MFS is 125, with three different cut-off scores to predict the severity of falls [24]. Scores <25, 25 to 50 and ≥51 indicate patients at low risk, medium risk and high risk, respectively [13]. Predictive accuracy of this tool varied between validation studies; sensitivity ranged from 55% to 83%, specificity ranged from 29% to 91% [13], [24], [26], [27], PPV ranged from 6% to 10%, and NPV ranged from 100% to 81% [13], [24], [26]. PPV and specificity are reported to be poorer than sensitivity and NPV of the MFS [13].

The FRASE was developed following a survey in Ireland [28], and this tool was adopted by five mental health trusts. Only one quasi-experimental study [23] that reported the predictive accuracy of the FRASE with receiver operating curve (ROC) statistics was identified. This study reported high inter-rater reliability for the FRASE with a correlation coefficient of 0.964 [23]. However, the FRASE was found to have poor predictive accuracy, with an ROC score of 0.560 for the prospective group [23].

Nandy et al. developed and validated the FRAT for use in primary care settings for older people [22]. The FRAT consists of two parts: Part 1 for identifying patients at high risk for falls, and Part 2 with guidance for further assessment and care planning. The five risk variables of Part 1 are: previous history of falls, four or more prescribed medications, diagnosis of stroke or Parkinson's disease, balance problems, and difficulty rising from a chair without using the arms. In the UK, one study [22] has evaluated the validity of Part 1 of the FRAT, and reported PPV of 57% for patients with three or more risk factors, specificity of 92%, NPV of 86% but poor sensitivity (42%) [22]. The FRAT is by far the most commonly used risk assessment strategy, used by 24 of the mental health trusts. Examination of the tools used found that there was variation in content. Only five of the 24 mental health trusts had adopted a validated version of the FRAT, 10 mental health trusts had devised their own version of the FRAT without predictive scoring to categorise the level of risk (low, medium and high), and the other nine mental health trusts had devised their own version of the FRAT but with tools predicting risk using numeric scales. One trust used a Rio (computerised patient record system) version of the FRAT for multifactorial falls risk assessment.

Comparison with NICE recommendations

The 2004 NICE guidance [5] recommends that 15 risk variables should be screened for when assessing patient falls risk, and these were compared with the falls risk assessment tools used by mental health trusts (Table 3). Falls risk predictive tools such as the MORSE, STRATIFY, FRASE and FRAT do not assess all the risk variables recommended in the NICE guidelines for falls prevention [4], [5]. The tools outlined in policies differed greatly in terms of the risk factors included in the assessments. For example, the STRATIFY only includes five risk factors, but the policies of four trusts included their own version of the FRAT which included all the multiple risk variables recommended in the NICE [5] falls guidelines, and also met the recommendations of the updated version of the NICE guidelines [4]. Analysis of the percentage of risk factors included in the falls assessment tools used by 35 trusts revealed that there were higher levels of inclusion for risk factors such as previous falls history (97%); balance, gait and mobility (100%); cognition (86%); vision (83%); medications (77%); and neurological problems (71%). However, lower levels of inclusion were found for risk factors such as environmental hazards (40%), osteoporosis (40%), urinary incontinence (49%), cardiovascular problems (26%), postural hypotension (37%) and fear of falling (34%).

Table 3

Summary of risk factors recommended by the National Institute for Health and Care Excellence (NICE) and included in the tools adopted by the policies of the mental health trusts.

NICE-recommended risk factors for multifactorial assessment	Trust polices including risk factors by tool (%)
Falls history	97
Gait/balance/mobility/strength	100
Osteoporosis	40
Fear of falling	34
Vision	83
Cognitive/confusion/mental health	86
Neurological problems	71
Urinary incontinence	49
Environmental hazards	40
Cardiovascular examination	26
Medication	77
Footwear	51
Postural hypotension	37
Alcohol intake	29
Nutrition	23

Discussion

This policy review found that screening tools form an important part of the strategy for the prevention of falls. However, only 11 trusts used standardised and validated tools such as the STRATIFY, MFS and FRASE, and 24 trusts used the FRAT to screen patients at risk of falls. Nine mental health trusts used their own version of the FRAT to predict risk (as low, medium or high) using numeric scales; however, these versions do not seem to have been subjected to any validation. This study summarised the predictive properties of the tools that are commonly adopted in the policies of mental health trusts (Table 1). The STRATIFY has been subjected to more validation studies compared with the other risk assessment tools. However, the predictive accuracy of the STRATIFY varied significantly in different settings and populations [10], [12], [13], [14], [15], [16], [17], [18], [19], [20], [21]. Sensitivity of these tools ranged from 20% to 93%, specificity ranged from 34% to 92%, NPV ranged from 80% to 100%, and PPV ranged from 2% to 62%. High NPV and moderate specificity of these tools may provide reassurance that these tools are good at identifying patients at low risk of falls; however, PPV of these tools was generally low, which implies that the interventions are poorly targeted if most patients are deemed to be at ‘high risk’, and this may reduce the significance of falls prevention programmes if staff perceive that too many patients are identified as being at high risk of falls [8], [10]. A recent systematic review suggested that the STRATIFY may not be an optimal tool for predicting high-risk fallers for effective falls prevention and management, as setting and population affect its performance [10]. Ideally, for effective operational use and to add value to clinical risk assessment, a risk prediction tool should have prospective validation for predictive validity (sensitivity, specificity, NPV and PPV), particularly in the clinical setting and in the population in which it is to be used, and should have narrow confidence intervals for these predictive validity properties [10]. The comparison of predictive properties of these validated tools (Table 1) across a range of studies shows that no single tool was more effective than the other tools for the assessment of falls risk, as predictive ability varied between clinical settings. None of the falls risk assessment tools adopted in mental health trusts had been validated or tested in acute mental health settings, so trusts have limited evidence on which to base their choice of tool.

NICE guidelines [4], [5] recommend that older people on psychotropic medications should have their medication reviewed and discontinued if possible to reduce their risk of falls; however, only 77% of trusts’ policies included medication review as part of falls risk assessment. This study found very low levels of inclusion of some important risk factors, such as fear of falling (34%), postural hypotension (37%), urinary incontinence (49%), environmental hazards (40%) and osteoporosis screening (40%). Consequently, there is a danger that staff using these less comprehensive and unvalidated tools could miss other risk factors for falls, thus reducing the effectiveness of falls prevention strategies. The National Patient Safety Agency's report on falls recommends multifactorial assessment and interventions for falls, as this may reduce the number of falls by 18% [1]. The NICE falls prevention guidance has been updated recently, and although much of the guidance remains the same, the guidance has now been extended to include inpatient settings [4]. Only four of the policies included in this review met the recommendations of NICE guidance. Trusts need to revisit and review their local policy in response to the most recent NICE guidance. Further work is needed to test and validate tools in acute mental health settings. To be of use to practitioners working in mental health settings, tools should account for the additional risks that mental health problems add to increasing falls risk.

To the authors’ knowledge, this is the first study to evaluate the falls risk assessment tools adopted in NHS mental health trusts’ clinical falls prevention policies in England and healthcare boards in Wales. However, this study only reviewed the falls policies, and did not explore actual clinical practice. Although a large proportion of falls policies (68%) were included, it was only possible to include two of the six healthcare boards in Wales. In addition, the literature review included studies without consideration of their methodological quality. The reported studies were conducted in different clinical settings and in a range of different populations.

Clinical implications and conclusion

Falls risk assessment is the most commonly used component of risk prevention strategies used in clinical practice, but most trusts’ policies examined in this study included unvalidated tools and even validated tools such as the STRATIFY, MFS and FRASE that are reported to have inconsistent predictive accuracy. Their positive predictive abilities are generally very low, which could lead to false reassurance, poorly targeted interventions and poor use of staff clinical time. This study raises questions about the operational usefulness of the tools, as none of these tools have been tested in acute mental health settings. The falls risk assessment tools of only four mental health trusts met all the recommendations of the NICE guidelines on multifactorial assessment for prevention of falls [5]. There were lower levels of inclusion of risk factors such as postural hypotension, cardiac problems, fear of falling and osteoporosis screening than other factors within the tools. Tools that fail to include the full range of risk factors do not provide staff with the opportunity to screen for and address risk factors for falls. The most recent NICE guidance [4] states that tools with a numerical predictor of risk should no longer be used; however, multifactorial risk assessment followed by multifactorial interventions tailored to the patient's needs is recommended. Mental health trusts need to review and update their policies in response to the new NICE guidelines for falls prevention in order to reduce patient falls.