Feasibility and acceptability of the 'Acutely Presenting Older Patient' screener in routine emergency department care.

BACKGROUND: risk stratification tools for older patients in the emergency department (ED) have rarely been implemented successfully in routine care.
OBJECTIVE: to evaluate the feasibility and acceptability of the 'Acutely Presenting Older Patient' (APOP) screener, which identifies older ED patients at the highest risk of adverse outcomes within 2 minutes at presentation. DESIGN AND
SETTING: 2-month prospective cohort study, after the implementation of the APOP screener in ED routine care in the Leiden University Medical Center.
SUBJECTS: all consecutive ED patients aged ≥70 years.
METHODS: feasibility of screening was assessed by measuring the screening rate and by identifying patient- and organisation-related determinants of screening completion. Acceptability was assessed by collecting experienced barriers of screening completion from triage-nurses.
RESULTS: we included 953 patients with a median age of 77 (IQR 72-82) years, of which 560 (59%) patients were screened. Patients had a higher probability of being screened when they had a higher age (OR 1.03 (95%CI 1.01-1.06), P = 0.017). Patients had a lower probability of being screened when they were triaged very urgent (OR 0.55 (0.39-0.78), P = 0.001) or when the number of patients upon arrival was high (OR 0.63 (0.47-0.86), P = 0.003). Experienced barriers of screening completion were patient-related ('patient was too sick'), organisation-related ('ED was too busy') and personnel-related ('forgot to complete screening').
CONCLUSION: with more than half of all older patients screened, feasibility and acceptability of screening in routine ED care is very promising. To further improve screening completion, solutions are needed for patients who present with high urgency and during ED rush hours.

Risk stratification tools for older patients in the emergency department (ED) have rarely been implemented successfully in routine practice.

This study evaluated the feasibility and acceptability of the Acutely Presenting Older Patient (APOP) screener in routine ED care.

With more than half of all older patients screened, feasibility and acceptability of screening in routine ED care was promising.

Barriers and facilitators found in this study could be used as guidance for implementation elsewhere.

Introduction

Risk stratification of older patients visiting the Emergency Department (ED) may help to deliver appropriate care, but few studies address the feasibility and acceptability of screening in clinical practice [1]. Older ED patients are at higher risk of various adverse outcomes compared with younger patients [2]. This is partly explained by non-specific disease presentation or the presence of comorbidities or cognitive disorders, which complicates their ED presentation, diagnosis and management [3-5]. Risk stratification can be used to identify patients at highest risk of adverse outcomes and allows targeted interventions to be applied for those who need it most [6] Although there are many risk-stratification tools reported in literature, widespread dissemination in routine clinical practice remains scarce.

The gap between research and practice needs to be bridged by focusing more on implementation outcomes [7,8]. Although tools can have the best validated predictive values, there will be no benefit for patients if tools are not used due to unsuccessful implementation in practice [9,10]. Only very few studies have yet focused on the feasibility of implementing risk stratification tools for older patients in the ED [1]. Understanding how tools are likely to be used in routine clinical practice is important to ensure that they are accepted by ED care providers, which increases the chance of successful implementation.

The ‘Acutely Presenting Older Patient’ (APOP) screener identifies older patients at the highest risk for functional decline and mortality and aids in the recognition of cognitive impairment [11,12]. The APOP screener is tailored for use in everyday ED practice and takes less than 2 minutes to administer directly at presentation [12]. The aim of the present study was to determine the feasibility and acceptability of the APOP screener in routine ED practice.

Methods

Study design and setting

A prospective cohort study was used to evaluate the feasibility of the APOP screener in routine care. This study was conducted in the ED of the Leiden University Medical Center (LUMC), The Netherlands [13]. In the ED, a triage-nurse first prioritises patients based on their disease severity, using the Manchester Triage System (MTS) [14]. Patients who bypass ED triage are patients eligible for thrombolytic therapy or with an indication for cardiac catheterization. The APOP screener was incorporated after routine triage from 1 March 2018 and evaluated during a 2-month inclusion period from 2 April to 3 June 2018. Acceptability of the screener was assessed with a questionnaire, which was sent out after the 2-month inclusion period. The questionnaire was analysed with both quantitative and qualitative methods. The medical ethics committee of the LUMC waived the necessity for formal approval, as the study closely followed routine care. The Netherlands Trial Register number: NTR7171.

Study participants

All consecutive ED patients aged ≥70 years during the 2-month inclusion period were eligible for screening and therefore inclusion. Because the APOP screener was incorporated in the routine care process after ED triage, we excluded patients who bypassed triage. Patients who were triaged to the immediate urgency level (MTS category ‘red’) were excluded, because the APOP screener was not developed and validated for this population.

ED triage-nurses, the main users of the APOP screener, were included to assess the acceptability of the screener.

Intervention

The APOP screener identifies the individual risk of 90-day functional decline and/or mortality and signs of impaired cognition for patients aged ≥70 years. The screener consists of nine questions and can be administered within 2 minutes [12]. We incorporated the screener at the end of the triage-form in the electronic health records (EHRs) of all older patients. Triage-nurses were instructed to screen all older patients after routine triage. The screening results were saved in the EHRs, visible for all care providers.

Implementation strategy

Before implementation, we executed pilot studies with triage-nurses to assess the barriers and facilitators of the APOP screener [12]. Because incorporation in the EHRs was experienced as the most important facilitator, we addressed this before implementation in routine care. We carried out a 1-month education program for all ED personnel to enhance awareness and explain the procedures of screening (see Supplementary data) [15,16].

Data collection

Feasibility of screening

The number of screened patients divided by the total number of older patients per day yielded the screening rate. Patient characteristics, collected from EHRs, were demographics (age, gender) and severity of disease indicators (arrival by ambulance, MTS triage urgency and chief complaint [14], Charlson comorbidity index (CCI) [17], and discharge destination). To measure organisation-related characteristics on a patient level, we used real-time prospective observations by medical students who were present in the ED 7 days a week (8.00 am–11.00 pm). Personnel was not informed about the reason for observation. We observed the number of personnel, the total number of ED registrations and the actual number of patients upon arrival time. Because our ED consists of 14 treatment rooms, we used this number as a cut-off point for the analyses. The ED length of stay (LOS) was measured by the subtraction of the ED arrival time from the departure time.

Acceptability of screening

To assess acceptability, triage-nurses were sent a questionnaire per email, including two reminders. The questionnaire consisted of multiple-choice questions and open textboxes (see Supplementary data). Five questions explored the opinions of nurses on the screener with 10-point Likert scales (1 meaning ‘totally disagree’ and 10 meaning ‘totally agree’) and two multiple-choice questions explored barriers of screening completion.

Outcome measures

Feasibility of screening was assessed by measuring (i) the screening rate and (ii) patient- and organisation-related determinants of screening completion. To assess acceptability, we collected opinions and experienced barriers of screening completion.

Data analysis

Data are presented as means with standard deviation (SD), medians with interquartile ranges (IQRs) or numbers with percentages. Patient- and organisation-related characteristics were compared between the screened and not screened patients with independent samples t-test, Mann–Whitney U test and χ2 test. In order to identify the determinants of screening completion, univariable and multivariable logistic regression analyses were performed with screening completion as the dependent variable and forced entry of patient- and organisation-related determinants as independent variables. Because of potential multicollinearity, we measured the severity of disease by including only arrival by ambulance and triage urgency. As organisation-related determinants, we included variables known at ED arrival: the number of patients upon arrival time, day of arrival and time of arrival. Results were presented as odds ratios (ORs) with 95% confidence intervals (CIs). A P-value <0.05 was determined as statistically significant. To assess acceptability, we calculated median grades and frequencies of answers from the questionnaire. Statistical analyses were performed using IBM SPSS Statistics version 25.

The qualitative input alongside the quantitative answers from the questionnaire was used to assess acceptability. We used the open textboxes and selected quotes that matched the answers.

Results

A total of 5,188 patients visited the ED during the 2-month inclusion period, of which 1,016 (19.6%) were ≥ 70-years old (see Supplementary data). We excluded 30 patients who bypassed triage and 33 patients who were triaged to the immediate urgency level. This resulted in 953 triaged older patients who were eligible for APOP screening and included in this study.

Table 1 shows the patient characteristics and organisation-related characteristics on a patient level for the total study population. The median age was 77 (IQR 73–82) years and 471 (49.4%) patients were male. Most patients were triaged as urgent (N = 443, 46.5%). The most common chief complaint was minor trauma (N = 276, 29.3%). The mean number of ED registrations per day was 83 (12), and for 295 (36.7%) patients, the number of ED patients upon arrival time was higher than 14.

Table 1

Patient characteristics and organisation-related characteristics on a patient level for the total study population

	N = 953
Patient characteristics
Demographics
Age, median (IQR)	77 (73–82)
Male, n (%)	471 (49.4%)
Severity of disease indicators
Arrival by ambulance, n (%)	293 (30.7%)
Triage urgency, n (%)
non-urgent (green and blue)	219 (23.0%)
urgent (yellow)	443 (46.5%)
very urgent (orange)	291 (30.5%)
Chief complaint, n (%)
Minor trauma	276 (29.3%)
Malaise	247 (26.2%)
Dyspnoea	96 (10.2%)
Abdominal pain	91 (9.7%)
Chest pain	75 (8.0%)
Loss of consciousness	41 (4.4%)
Major trauma	15 (1.6%)
Mental health problems	10 (1.1%)
Other	91 (9.7%)
CCI, median (IQR)	5 (4–7)
Destination, n (%)
Discharged home	488 (51.5%)
Admission	422 (44.5%)
Other	38 (4.0%)
Organisation-related characteristics
Number of ED personnel, mean (SD)	11 (1)
Number of ED registrations on arrival day, mean (SD)	83 (12)
Number of ED patients upon arrival time, n (%)
0–14 patients	508 (63.3%)
>14 patients	295 (36.7%)
Day of arrival, n (%)
Weekday	717 (75.2%)
Weekend	236 (24.8%)
Time of arrival, n (%)
Day (8–16 h)	506 (53.1%)
Evening (16–23 h)	326 (34.2%)
Night (23–8 h)	121 (12.7%)
ED LOS (minutes), median (IQR)	196 (133–265)

Missings: 23 personnel, 150 patients upon arrival time, 2 ED LOS.

Feasibility of screening

Of all 953 triaged older patients, 560 (59%) were screened during the 2-month evaluation. The absolute numbers and percentages of screened patients are shown in Figure 1. The total number of older patients ranged between 8 and 28 patients per day. The screening rate varied between 33 and 81% and was relatively stable during the 2-month period without showing a linear trend over time. The screening rate remained stable in routine practice over a longer period (see Supplementary data).

Figure 1

Screening in absolute numbers and screening rate over the study period. Absolute numbers and percentages of older patients screened in the ED during the 2-month inclusion period starting 1 month after the implementation of the APOP screener. Dotted lines are placed between Sundays and Mondays to indicate the weeks. The absolute numbers of older patients visiting the ED ranged between 8 and 28 patients per day. The screening rate varied per day between 30 and 82%.

Table 2 shows the patient- and organisation-related characteristics stratified by the completion of screening. Screened patients were older (78 vs. 77 years, P = 0.045), arrived less often by ambulance (27.3 vs. 35.6%, P = 0.006), were more often triaged urgent (51.1 vs. 39.9%) and less often triaged very urgent (25.5 vs. 37.7%) (overall P < 0.001) compared with patients who were not screened. Screened patients more often arrived at the ED when it was less busy due to a low amount of patients (0–14 patients) upon arrival (68.1 vs. 56.2%, P = 0.001) and screened patients had a longer median ED LOS (213 vs. 176 minutes, P < 0.001) compared with patients who were not screened.

Table 2

Patient characteristics and organisation-related characteristics on a patient level stratified by screening completion

	Patients screened	Patients not screened (N = 393)	P-value*
	(N = 560)
Patient characteristics
Demographics
Age, median (IQR)	78 (73–83)	77 (72–81)	0.045
Male, n (%)	279 (49.8%)	192 (48.9%)	0.769
Severity of disease indicators
Arrival by ambulance, n (%)	153 (27.3%)	140 (35.6%)	0.006
Triage urgency, n (%)			<0.001
non-urgent (green and blue)	131 (23.4%)	88 (22.4%)
urgent (yellow)	286 (51.1%)	157 (39.9%)
very urgent (orange)	143 (25.5%)	148 (37.7%)
Chief complaint, n (%)			0.004
Minor trauma	172 (31.0%)	104 (26.9%)
Malaise	130 (23.4%)	117 (30.2%)
Dyspnoea	70 (12.6%)	26 (6.7%)
Abdominal pain	55 (9.9%)	36 (9.3%)
Chest pain	43 (7.7%)	32 (8.3%)
Loss of consciousness	22 (4.0%)	19 (4.9%)
Major trauma	4 (0.7%)	11 (2.8%)
Mental health problems	8 (1.4%)	2 (0.5%)
Other	51 (9.2%)	40 (10.3%)
CCI, median (IQR)	5 (4–7)	5 (4–7)	0.943
Destination, n (%)			<0.001
Discharged home	303 (54.4%)	185 (47.3%)
Admission	247 (44.3%)	175 (44.8%)
Other	7 (1.3%)	31 (7.9%)
Organisation-related characteristics
Number of ED personnel, mean (SD)	11 (1)	11 (1)	0.803
Number of ED registrations on arrival day, mean (SD)	83 (12)	83 (12)	0.165
Number of ED patients upon arrival time, n (%)			0.001
0–14 patients	323 (68.1%)	185 (56.2%)
>14 patients	151 (31.9%)	144 (43.8%)
Day of arrival, n (%)			0.242
Weekday	429 (76.6%)	288 (73.3%)
Weekend	131 (23.4%)	105 (26.7%)
Time of arrival, n (%)			0.066
Day (8–16 h)	315 (56.3%)	191 (48.6%)
Evening (16–23 h)	179 (32.0%)	147 (37.4%)
Night (23–8 h)	66 (11.8%)	55 (14.0%)
ED LOS (minutes), median (IQR)	213 (150–283)	176 (115–234)	<0.001

*overall P-value between groups measured by χ2 for categorical values and Mann–Whitney U test for non-parametric variables.

Because of the hypothesised interrelationship between patient- and organisation-related characteristics, we analysed the characteristics that were independent determinants associated with screening completion (Table 3). In the multivariable model, patients had a higher probability of being screened when they had a higher age (OR 1.03 (1.01–1.06), P = 0.017). Triage urgency was associated with screening completion in a non-linear fashion (P = 0.003). Patients had a lower probability of being screened when they were triaged very urgent compared with urgent (OR 0.55 (0.39–0.78), P = 0.001) and when the number of ED patients upon arrival was higher than 14 (OR 0.63 (0.47–0.86), P = 0.003).

Table 3

Determinants of screening completion in older ED patients

	univariable	P-value	multivariable	P-value
	OR (95% CI)		OR (95% CI)
Patient-related determinants
Demographics
Age	1.02 (1.00–1.05)	0.032	1.03 (1.01–1.06)	0.017
Male	1.04 (0.80–1.35)	0.769	1.10 (0.82–1.47)	0.534
Severity of disease indicators
Arrival by ambulance	0.68 (0.52–0.90)	0.006	0.80 (0.57–1.13)	0.211
Triage urgency		<0.001[*]		0.003[*]
non-urgent (green and blue)	0.82 (0.57–1.14)	0.235	0.83 (0.57–1.20)	0.316
urgent (yellow)	ref	ref	ref	ref
very urgent (orange)	0.53 (0.39–0.72)	<0.001	0.55 (0.39–0.78)	0.001
Organisation-related determinants
Number of ED patients upon arrival time
0–14 patients	ref	ref	ref	ref
>14 patients	0.60 (0.45–0.80)	0.001	0.63 (0.47–0.86)	0.003
Day of arrival
Weekday	ref	ref	ref	ref
Weekend	0.84 (0.62–1.13)	0.242	0.83 (0.58–1.17)	0.285
Time of arrival		0.067[*]		0.138[*]
Day (8–16 h)	ref	ref	ref	ref
Evening (16–23 h)	0.74 (0.56–0.98)	0.035	0.77 (0.57–1.05)	0.094
Night (23–8 h)	0.73 (0.49–1.09)	0.12	0.24 (0.02–2.77)	0.251

* P-value testing whether the overall variable is statistically significant for categorical variables with more than two categories.

Acceptability of screening

In total 68 triage-nurses received the questionnaire, of which 34 (50.0%) nurses returned it. The questions exploring their opinions about screening are shown in Supplementary data. On a scale from 1 to 10, nurses graded the importance of identifying frailty in older patients using the APOP screener with a median of 8 (IQR 7–9). They graded the question ‘Do you think that the APOP program in its current form contributes to better care for the older patient in the ED?’ with a median of 6 (IQR 5–7). Some nurses indicated points for improvement (quotes 1 and 2).

Quote 1.

‘Good aim for the vulnerable older patient. Personally, I think it’s not going well yet, mainly due to the busy ED […] I do not yet have a positive experience with regard to APOP that it leads to improvement’.

Quote 2.

‘There’s still a long waiting time and length of stay in the ED; more than 4 hours; also for high risk screened patients. […] Because of the increased complexity in the ED, high workload and ED crowding, older patients do not receive yet the care they should receive regarding their high risk screening result.’

Figure 2 shows the answers of the question: ‘If you were unable to complete the APOP screener, what was mostly the reason?’ Some nurses experienced patient-related barriers, such as the patient was too ‘sick’ (N = 12 nurses, quote 3) or the patient ‘refused’ screening (N = 5, quote 4).

Figure 2

Experienced barriers of screening completion from triage-nurses working in the ED. Frequency of reported barriers of screening completion by 28 triage-nurses. Nurses were able to fill in multiple barriers. Patient-related barriers were ‘patient was too sick’ and ‘patient refused screening’. Organisation-related barriers were ‘the ED was too busy’ and ‘it took too much time to complete screening’. Personnel-related barriers were ‘screening was useless’, ‘screening questions were difficult to ask’ and ‘forgotten to complete screening’.

Quote 3.

‘[…] and if the patient is too sick or has to be seen by a physician immediately, it has less priority.’

Quote 4.

‘Patients say ‘those questions again’. So we see patients who have been asked these questions multiple times. It happened to me twice that patients refused.’

The most frequently reported barriers for screening completion were organisation-related: it was too ‘busy’ (N = 21, quote 5) and there was no ‘time’ to complete screening (N = 6, quote 6).

Quote 5.

‘During triage it is often too busy to complete the screening questions properly.’

Quote 6.

‘When it’s busy, screening takes too much time.’

Personnel-related barriers came from nurses who stated the screening questions as ‘difficult’ (N = 3, quote 7) and nurses who ‘forgot’ to complete screening (N = 6, quote 8). None of the nurses stated that screening was ‘useless’.

Quote 7.

‘I sometimes find the question with the months in reversed order difficult to ask.’

Quote 8.

‘If the APOP screener is not completed at triage, there is no reminder [in the system]. Because of this, I often forget to complete it when a patient arrives by ambulance.’

Discussion

The present study evaluated the feasibility and acceptability of the APOP screener in routine ED practice. The screener was completed in 59% of older ED patients, with a stable screening rate over time. Screening completion was associated with both patient characteristics—age and triage urgency and organisation-related characteristics—the number of ED patients. Moreover, screening was accepted by the users, who stated that it is important and useful. The experienced barriers of screening completion from triage-nurses were patient- (‘patient was too sick’), organisation- (‘ED was too busy’) and personnel-related (‘forgot to complete screening’).

The evaluated screening rate is somewhat higher compared with other risk-stratification tools used in the ED setting [1]. One feasibility study evaluating the Emergency Geriatric Screening tool found a screening rate of 43% [18]. Asomaning et al. showed that the Identification of Seniors At Risk (ISAR) tool could be administered in 52% of ‘eligible’ older ED patients [19]. However, in another study evaluating ISAR, the screening rate was 34% after implementation, followed by an increase toward 50% over the course of 7 months [20]. The observed screening rate of 59%, assessed 1 month after implementation in routine care, seems therefore acceptable compared with other studies.

Time to complete screening is an important determinant of feasibility and acceptability [1,21]. The 2-minute time to complete the APOP screener could therefore be an important facilitator of screening completion [12]. We believe that another facilitator was the incorporation of the screener in the EHRs, making screening a part of routine care procedures. The results of our study show an association between screening and ED LOS but do not show whether a longer ED LOS is caused by screening, whether screening is caused by a longer ED LOS or whether this association is caused by other unknown factors. The determinants of screening completion were both patient- and organisation-related. Firstly, patients had a higher probability of being screened with increasing age. This is probably because triage-nurses use their clinical judgement to indicate which patients are possibly vulnerable before they decide to complete the screener [21]. Secondly, we found that non-urgent and very urgent patients had a lower probability of being screened than urgent patients. We might need to improve the motivation of triage-nurses by explaining the importance of screening for these patients. However, for very urgent patients, medical care has priority and therefore a screening rate of 100% might be difficult to achieve. Although we recognise the importance to screen all older patients and identify those patients who are dying in order to deliver appropriate palliative care at the right time, the APOP screener was not validated for that use nor is it feasible. Thirdly, the number of patients upon arrival also had an impact on screening completion. This organisation-related factor could be changed by reducing exit blocks from the ED [22]. Importantly, overcrowding was most often experienced as a barrier of screening completion, because it results in less time or less priority to complete the screener. Priority can partly be determined by the experience of benefits of screening, because benefits are not always experienced by the users (shown by quotes 1 and 2), which might result in a ‘lack of outcome experience’, a known factor for non-adherence [23]. Although the importance of screening was graded high and screening was accepted by users, we should take the benefits for users into account in order to improve screening completion, i.e. by generating fast-track admissions for high risk screened patients with clinical indication for hospitalisation, or by generating other care pathways such as a geriatric evaluation unit or a specialised geriatric acute medical ward.

Screening older patients on their risk of adverse outcomes can help ED personnel to think about the differences between older patients on a regular basis. The identification of high-risk patients can be an opportunity to ensure targeted interventions that are started, and allows faster and more focused use of time, personnel and resources. In this way, risk stratification in the ED has the potential to improve outcomes for older patients. In the present study, we show that risk stratification with the APOP screener in routine care seems feasible and acceptable. More research will be needed to investigate feasibility in different hospitals and health care systems to generate guidance on how screening tools can be successfully implemented on a wide scale. The fact that the APOP screener recently has been implemented in the EHRs (HiX, Chipsoft) used by approximately half of all Dutch hospitals and has been put into routine use by several EDs throughout The Netherlands is very promising [24].

This study has several strenghts. Firstly, this is the first implementation study investigating feasibility and acceptability of screening older patients in routine ED practice on a large scale. Secondly, we used real-time observations of everyday practice to measure real-time barriers. Finally, the screener was implemented for an unselected population of older ED patients, which increases generalizability. Generalizability is, however, also a limitation of this study. Although the APOP screener was validated in four Dutch hospitals, this implementation study was done in one academic hospital. Nonetheless, we believe that the barriers and facilitators found in this study could be used as guidance for implementation elsewhere.

In conclusion, with more than half of all older patients screened, feasibility and acceptability of screening in routine ED care is very promising. To further improve screening completion, solutions are needed for patients who present with high urgency and during ED rush hours.

Click here for additional data file.