Scales for predicting risk following self-harm: an observational study in 32 hospitals in England.

OBJECTIVE: To investigate the extent to which risk scales were used for the assessment of self-harm by emergency department clinicians and mental health staff, and to examine the association between the use of a risk scale and measures of service quality and repeat self-harm within 6 months.
DESIGN: Observational study.
SETTING: A stratified random sample of 32 hospitals in England. PARTICIPANTS: 6442 individuals presenting with self-harm to 32 hospital services during a 3-month period between 2010 and 2011. OUTCOMES: 21-item measure of service quality, repeat self-harm within 6 months.
RESULTS: A variety of different risk assessment tools were in use. Unvalidated locally developed proformas were the most commonly used instruments (reported in n=22 (68.8%) mental health services). Risk assessment scales were used in one-third of services, with the SAD PERSONS being the single most commonly used scale. There were no differences in service quality score between hospitals which did and did not use scales as a component of risk assessment (median service quality score (IQR): 14.5 (12.8, 16.4) vs 14.5 (11.4, 16.0), U=121.0, p=0.90), but hospitals which used scales had a lower median rate of repeat self-harm within 6 months (median repeat rate (IQR): 18.5% vs 22.7%, p=0.008, IRR (95% CI) 1.18 (1.00 to 1.37). When adjusted for differences in casemix, this association was attenuated (IRR=1.13, 95% CI (0.98 to 1.3)).
CONCLUSIONS: There is little consensus over the best instruments for risk assessment following self-harm. Further research to evaluate the impact of scales following an episode of self-harm is warranted using prospective designs. Until then, it is likely that the indiscriminant use of risk scales in clinical services will continue.

This study involved a large national sample of patients and services and it is likely that our findings will be generalisable to the rest of England and applicable to managed healthcare settings in other countries.

We gathered responses from staff closely involved in self-harm service provision, and obtained copies of the risk tools.

The study was observational in design and we cannot therefore infer causation.

We only considered repeat episodes of self-harm presenting to the study hospitals, rather than community episodes or episodes presenting to other centres.

Background

Hospital services play an important role in engaging people who self-harm and treat approximately 220 000 episodes of self-harm in England annually.1 Self-harm is associated with a high risk of suicide and other adverse outcomes,2 3 and good quality care has the potential to contribute significantly to suicide prevention.4 Psychosocial assessment is a key component of management.5 Clinical guidelines recommend a psychosocial assessment on presentation to the emergency department in order to gauge a person's mental state and willingness to remain for further assessment.6

Assessments by mental health staff typically take place at a later stage and involve a comprehensive evaluation of an individual's personal situation, history of self-harm, family history and mental state.7 8 The evaluation of risks (factors which may increase the likelihood of adverse outcomes) and patient needs (resulting, eg, from mental or physical illness or difficult social circumstances) are particularly important.9

Clinical assessments are social interactions between clinical staff and patients.10 11 Some large quantitative studies have suggested that the process may itself have a protective effect against future suicidal behaviour.5 12 Assessments may be most helpful when used as an opportunity for clinicians to engage with patients and when patients have their need for help legitimised by the process.10 Tailored management plans can be developed, laying the foundation for longer term care.7 9 In contrast, other studies found that clinical assessments may sometimes encourage clinicians to disengage with patients and therefore negatively impact the therapeutic relationship.11

Risk assessment tools consisting of brief checklists of key risk factors, symptoms or antecedents are often regarded as core components of psychosocial assessments, but evidence for their effectiveness is limited.9 13 14 Some guidelines advise against the use of these tools to determine clinical management.9 Others recommend that clinicians should only use scales that have undergone psychometric testing15—for example, tested for their reliability, predictive ability and diagnostic accuracy, as well as construct, internal and external validity.16

A small number of descriptive studies have investigated the use of risk assessment tools in UK mental health services.13 14 17 Risk assessment tools vary widely in their type, structure and content,13 14 17 with clinicians sometimes using locally developed and untested tools.15 17 The use of risk tools may also differ by care setting. For example, emergency department service pressures often necessitate rapid forms of assessment for physical risk, and scales are commonly used.18 19

There have been few studies on the use of risk scales across self-harm services. This is perhaps surprising given the high risk of suicide and other adverse outcomes in patients who self-harm. To our knowledge, no studies until now have investigated the association between the use of risk scales and service-level outcomes such as the quality of care or the rate of repetition of self-harm.

Using data from a study of 32 hospitals in England,20 we aimed to investigate the use of risk scales following self-harm in National Health Service (NHS) services. We included emergency departments and specialist mental health treatment settings in order to capture initial and subsequent risk assessments.

Specifically, the objectives of this study were:

To investigate the extent to which risk scales were used for the assessment of self-harm by emergency department clinicians and mental health staff.

To examine the association between use of a risk scale and service-level outcomes, such as service quality and repeat self-harm within 6 months.

Method

Recruitment and study procedure

For this observational study, we used data from a nationally representative stratified random sample of 32 hospitals in England.20 The methodological details are described elsewhere,20 but in brief, the sample was stratified to include small and large hospitals, and hospitals with low self-harm and high self-harm admission rates. The original sample was also stratified to ensure that four hospitals were selected within each of the eight former health regions. Only hospitals with an emergency department were included in the sample.

Half of the hospitals were teaching hospitals (16/32). Fifteen of the hospitals served urban areas, 7 rural and 10 urban and rural equally. The number of beds ranged from 300 to 1865. The hospital catchment population (based on hospital self-report) ranged from 120 000 to 750 000. Specialist self-harm teams (ie, any liaison psychiatric service with at least one member of staff located within the emergency department) were in place in the majority of hospitals (29/32). Most of the assessments were carried out by a mental health nurse, including mental health liaison nurses and specialist self-harm teams (3109/4075), followed by a psychiatrist (799/4075) and 167 by another mental health professional (eg, social worker). The proportion of self-harm episodes given a psychosocial assessment by mental health staff varied from 24% to 88%, with a median rate of 58%.20

Key emergency department and psychiatric staff (eg, consultants in emergency medicine and liaison psychiatry, mental health managers and mental health nurses) were interviewed at each site about self-harm service provision. As part of the interview, we asked about the methods used for assessing risk and for copies of the risk measures used. We were particularly keen to distinguish untested, locally developed instruments from published scales which had undergone some element of formal testing for their predictive ability. In the current study, we did not aim to compare the diagnostic accuracy of different instruments.

In each hospital, data were collected on consecutive presentations for self-harm over a 3-month period in 2010/2011. In total, over 6400 individuals were included. For the purposes of this study, self-harm was defined as ‘a deliberate non-fatal act whether physical, drug dosage or poisoning, carried out in the knowledge that it was potentially harmful and in the case of drug overdose that the amount taken was excessive’.21

The data collection process for self-harm episodes at each site was classified as a local audit and therefore individual patient consent was not required.

Outcomes

Self-harm service quality

A service quality scale based on the Royal College of Psychiatrists’ self-harm service standards was developed as part of previous work.20 22 Questions on the scale pertained to key aspects of service provision; for example, supervision arrangements for all staff involved in psychosocial assessments, 24 h access to specialist mental health services and availability of rooms for privacy and confidentiality. There were 21 items in total (see table 1 for all items) each scored 0–1 giving a potential scoring range from 0 to 21. Responses from staff at interview were used to complete the scales and higher scores indicated a higher overall quality of self-harm services.

Table 1

Service scale items with number and proportion of hospitals endorsing each item

Item number	Service scale items	Yes
		N	Per cent
1	Is there a protocol/guideline/aide memoire for staff in the A&E department for the immediate medical management of self-harm?	29	90.6
2	Is there a protocol/guideline/aide memoire for staff in the A&E department for the immediate assessment of risk and severe mental disorder for patients who self-harm?	28	87.5
3	Is there a designated self-harm specialist clinical service? (+A&E liaison)*	29	90.6
4	Is there a local specific planning/working group (of the team who undertake the psychosocial assessments) which meets at least once a year to plan/oversee the service for patients who self-harm?	22	68.8
5	Are there psychosocial assessment training sessions for new staff who are involved in the psychosocial assessment of patients?	30	93.8
6	Are there supervision arrangements in place for staff members (new and existing) who undertake psychosocial assessments?	23	71.9
7	Are there written guidelines/a checklist, to assist psychiatric clinicians in the psychosocial assessment of patients who self-harm?	27	84.4
8	Does the A&E department have 24 h access to a psychiatrist, psychiatric nurse or social worker who is able to undertake psychosocial assessments?	30	93.8
9	If yes to 8, is immediate (within 15 min) advice available over the telephone?	22	68.8
10	If yes to 8, is emergency attendance, when requested, available within 1 h?	7	40.6
11	Do regular (at least once a year) service planning/strategy meetings take place between the self-harm team/psychiatric service and the general medical service involved in the care of patients who self-harm?	31	96.9
12	Are rooms which allow for privacy and confidentiality available for conducting interviews with patients who self-harm either in or close to the A&E department?	21	65.6
13	Does a formal arrangement exist with Social Services to visit and offer advice to patients who self-harm who have significant social difficulties?	31	96.9
14	Can those admitted as inpatients remain in hospital until they have received a psychosocial assessment?	21	65.6
15	Is there a policy stating that a patient's GP should be contacted within 24 h of patient discharge from an A&E department?	26	81.3
16	Is there a policy stating that a patient's GP should be contacted within 24 h of patient discharge from a medical inpatient unit?	13	40.6
17	Are patients who self-harm routinely given printed material about local services, voluntary groups and how to obtain access to them?	6	18.8
18	Are there any formal links with non-statutory services (eg, self-help groups, the Samaritans)?	15	46.9
19	Has a system been set up for the monitoring of hospital attendance/discharge and referral of patients who self-harm?	13	40.6
20	Has there been any audit of the service for patients who self-harm in the past 2 years?	13	40.6

*Any liaison psychiatric service with at least one member of staff located within the ED.

A&E, accident and emergency; ED, emergency department; GP, general practitioner.

Repetition of self-harm

Individuals were followed up for 6 months after their index episode of self-harm. Repeat self-harm episodes leading to hospital presentation were identified through hospital databases. Presentations were linked to individuals through name, date of birth and NHS number.

Analysis

Our main analyses were descriptive. In order to investigate the association between use of risk scales and outcomes, we carried out additional analyses on aggregated hospital data. For this part of the analysis, we combined emergency department and mental health settings and compared hospitals using a tested tool (n=14) to hospitals not using a tested tool (n=18). This enabled us to capture the use of risk scales at any point along the care pathway. To test the robustness of the approach, we also carried out a sensitivity analysis based on emergency department assessments only. We initially used the Mann-Whitney U test to compare groups (services using and not using published risk scales) on overall service quality scores and repeat self-harm at 6 months.

Using negative binomial regression models, we then adjusted for risk factors previously shown to influence rate of repetition (history of self-harm, current psychiatric treatment, cutting as method, aged under 35 and gender).23 24 For risk factors which were continuous variables the sample was split into tertiles to facilitate clinical interpretation. One hospital was missing data on the outcome of repeat self-harm and was excluded from the analysis.

Statistical analysis was performed using SPSS V.2025 and Stata release V.12.26

Results

Risk and needs assessment by emergency department staff

In 28 of 32 (87.5%) hospitals there was a protocol or guideline for the immediate assessment of suicide risk for patients who presented with self-harm in the emergency department. Guidelines for the assessment of needs were available for patients who self-harm in 7 (21.7%) of the 32 emergency departments. There was a referral policy regarding who should be referred for assessment to the psychiatric service in 15 (46.9%) of the 32 emergency departments; 5 (15.6%) also had a written guideline or checklist to assist in deciding who should be referred.

Twenty (62.5%) of the hospitals reviewed had a specific triage procedure for patients who presented with self-harm. Triage is a formal dynamic process, performed at initial admission to the emergency department, whereby patients are prioritised on the basis of clinical urgency.27 Registered nurses typically assess physical needs, patient history and vital signs, evaluate if emergency interventions are necessary and assign a rating based on the urgency of the care required.27 In 18/32 (56.6%) hospitals there was a specific triage tool. The Manchester Triage tool27 was used in 14 (43.8%) of the hospitals, 3 (9.4%) used an in-house locally developed triage tool and 1 (3.1%) used the Mantis Triage tool.

Risk tools used by emergency department staff

Emergency department staff used a wide range of tools to assess risk following self-harm. Most commonly, they used locally developed structured proformas (n=13, 40.6%). Staff at 12 (37.5%) emergency departments used published scales (table 2); 8 (66.6%) of these were used in conjunction with locally developed proformas or other generic assessments. The SAD PERSONS scale28 was the most frequently used scale, being employed in 9 (28.1%) of the emergency departments. Two (6.2%) hospitals used solely the SAD PERSONS scale and clinical judgment of medical staff to assess risk and staff at 1 (3.1%) hospital reported using only clinical judgment to assess risk (table 2).

Table 2

Risk assessment tools used by emergency department and mental health staff (categories not mutually exclusive)

Description	Frequency of use by emergency departments	Frequency of use by mental health services
Published risk scales*
SAD PERSONS Scale27	9	2
Suicide Intent Scale29†	1	1
Pierce Suicide Intent Scale28	1	1
Beck Depression Inventory36	1
All other risk assessment tools
Structured pro forma (developed locally)	13	22
Galatean Risk Screening Tool (GRiST)	1
Emergency department mental health/suicide risk assessment form	3
Safe-risk pro forma	1
Functional analysis of care environments (FACE)	1
Emergency department doctors’ handbook	1
Mental health clustering tool	1
Threshold assessment grid (TAG)	2
Policy on intranet	1
Risk assessment based on the care programme approach (or using CPA forms)	3	6
CARSO: clinical assessment of risks to self and others	1	1
Rapid assessment and treatment tool	1
St George's tool	2
Computerised assessment system	1	2
Initial screening assessment form‡	1	1
Risk assessment matrix	2	1

*Published scales, which have undergone psychometric testing (eg, to evaluate reliability, predictive ability, diagnostic accuracy and construct, internal and external validity; Bossuyt et al, 2003).15

†Section of pro forma based on Suicide Intent Scale (mental health services).

‡Tool taken from Morgan.37

Risk and needs assessment by mental health staff

Overall, 31/32 (96.9%) mental health services reported that they carried out risk assessments following self-harm, and 29/32 (90.6%) reported evaluating service user's needs as part of their wider assessment process. Written guidelines were available to assist clinicians in their psychosocial assessment in 27/32 (87.5%) services.

Risk tools used by mental health services staff

A wide range of risk assessment tools were used by clinicians in mental health services. Clinicians most frequently reported using locally developed proformas for risk assessment (n=22, 68.8%). Three (9.4%) mental health services used published scales in their assessment of self-harm. One mental health service used the Pierce Suicide Intent Scale29 in conjunction with a locally developed proforma (3.1%). Two (6.3%) mental health services used the SAD PERSONS scale; one of these used the scale in conjunction with the Suicide Intent Scale30 and a locally developed pro forma (table 2).

Service quality score

Fourteen of the 32 (43.8%) hospitals used published scales as part of risk assessment in the emergency department or mental health setting. Overall, service quality scores ranged from 10.5 to 19, with a median score of 14.5 (IQR 12, 16). Hospitals which used scales had the same median service quality score as those hospitals which did not use scales (median score (IQR): 14.5 (12.8, 16.4) vs 14.5 (11.4, 16.0), U=121.0, p=0.90).

Repeat self-harm at 6 months

Hospitals which used published scales as a component of their risk assessments had a lower median rate of repeat self-harm at 6 months than hospitals which did not (median repeat rate (IQR): 18.5% (16.3, 20.8%) vs 22.7% (20.4, 25.2%), U=57.0, p=0.008). When we analysed the repetition data using negative binomial regression, the unadjusted incidence rate ratio was 1.18 (95% CI (1.00 to 1.37), p=0.039), indicating that services which did not use risk scales had rates of self-harm repetition that were almost 20% higher than services which used such scales.

However, when we adjusted for differences in casemix (by considering potential confounders such as the proportion of patients: with a history of self-harm; who used cutting as a method; who were in current treatment; who were women or aged under 35 years) the association between use of a risk scale and repetition was attenuated (IRR=1.13, 95% CI (0.984 to 1.29), p=0.08). Repeat self-harm was explained to a greater extent by casemix factors such as male gender (IRR=1.3, 95% CI (1.1 to 1.5), p=0.001 for hospitals in the highest vs lowest tertile for proportion of male patients) and history of self-harm (IRR=1.19, 95% CI (1.0 to 1.4) p=0.015 for hospitals in the highest vs lowest tertile for proportion of patients with a history of previous self-harm).

We conducted a sensitivity analysis to determine whether the effect of risk scales was specific to the emergency setting by using emergency department data only. The rate of repetition was lower for emergency departments that used a published scale (19% IQR (15.1, 21.1%) vs 22.3% (19.1, 25.1%)), although statistical evidence for a difference was limited (U=70, p=0.053).

Discussion

Main findings

In this study of 32 hospitals in England we have shown that a wide variety of tools and scales are used for the assessment of risk following self-harm. There is little consistency in practice. Many services use locally developed proformas or tools of questionable validity for risk assessments. Psychometrically tested scales are in use in about one-third of services, but their use varies between emergency department and mental health settings. Overall, the SAD PERSONS is the most commonly used scale. Our data are descriptive and are primarily based on service level (rather than individual level) outcomes. Therefore, the results should be interpreted cautiously, but our findings suggest that in services where scales are used, the incidence of repeat self-harm may be lower.

Strengths and limitations

The study was observational in nature and we cannot, of course, infer causation. The analyses are based on aggregate data over a 3-month period in 32 hospitals; therefore, generalisability may be an issue. However, we selected a random national sample of services and it is likely that the findings will be applicable to the rest of England. We believe our results will also be relevant to managed healthcare settings in other countries. Data on service characteristics and use of risk tools were collected through interviews, which may be subject to reporting bias. However, we interviewed the person most closely connected with services, and in all but one centre carried out interviews with emergency department and mental health staff as well as obtaining copies of the risk assessment tools themselves.

We specifically restricted ourselves to scales that had undergone psychometric testing with some assessment of their predictive accuracy for suicidal behaviour or self-harm. Our outcome measures were self-harm repetition at 6 months and self-harm service provision. We did not record community episodes of self-harm. It is possible that had we done so we would not have found the apparent association between use of a scale and self-harm repetition. It is possible that use of the scales may have influenced help-seeking behaviour rather than repetition itself. Neither did we record repeat self-harm which resulted in presentation to hospitals other than those in which the index episodes were identified.

Interpretation and future research

The wide range of tools in use among emergency department and mental health services suggests that there is little consensus over the best instruments for risk assessment. This probably reflects the poor evidence base and the lack of any ‘gold standard’ for the assessment of repeat self-harm. Staff at emergency departments employed a particularly wide range of tools, which may reflect the style of service provision in that setting and a ‘high risk’ approach to management.31 By contrast, mental health staff were less likely to use published risk scales. This could reflect a greater reliance on comprehensive psychosocial assessment.

Perhaps particularly concerning was the fact that locally developed tools were in widespread use in emergency department and mental health settings. This is consistent with other reports.13–15 17 Locally developed tools are generally not evidence based. They may have unclear scoring criteria and wide variations in structure, content and focus.13 14 They are used inconsistently across and within hospitals.32–34 The Royal College of Psychiatrists recommends that use of locally developed forms should be phased out altogether.34

Other services used published scales for risk assessment, but the evidence base for these is also weak. The psychometric properties of some scales have been investigated but their predictive value is very limited. This is perhaps due to the intrinsic properties of the scales themselves but also because of the comparatively low base rate of repeat self-harm (and particularly suicide). On this basis, the National Institute of Health and Care Excellence (NICE) guideline for the longer term management of self-harm recommended that scales should not be used in isolation to predict risks or determine clinical management.9 They could be used as an adjunct to assessment. Similarly, recent research has cast doubt on the use of the SAD PERSONS scale (the most commonly used emergency department instrument in our study) because of its consistently poor predictive ability.35 36

Interestingly, our study suggested that services which used risk scales may have had lower repetition rates than services which did not. It could be that use of the scales was a marker for more comprehensive services, but this was not borne out by our service quality scores. When we adjusted our models for possible confounders, the apparent protective effect of the use of scales was attenuated. Use of risk scales could also have reflected a more formal approach to assessment across the care pathway.

Alternatively, use of the scales may have influenced patients’ willingness to re-present as there is evidence from qualitative work that service users dislike ‘tick box’ approaches to assessment which may impair therapeutic engagement (as opposed to more holistic assessments which may be experienced as helpful).15 In other words, the lower rate of repetition in hospitals using tested scales may be because people have repeat episodes of self-harm in the community (which were unrecorded in this study), and may not return to the hospital service due to finding the assessment an adverse experience. A future study including follow-up of self-harm in the community might help determine the influence of this on subsequent help-seeking behaviour. Further research on the positive and negative impacts of these instruments following self-harm would be of benefit, perhaps using prospective or randomised controlled designs and ideally with a qualitative component seeking perspectives of staff and patients. Until then, it is likely that the indiscriminant use of risk scales in clinical services will continue.