Does the implementation of a novel intensive care discharge risk score and nurse-led inpatient review tool improve outcome? A prospective cohort study in two intensive care units in the UK.

OBJECTIVE: To develop a clinical prediction model for poor outcome after intensive care unit (ICU) discharge in a large observational data set and couple this to an acute post-ICU ward-based review tool (PIRT) to identify high-risk patients at the time of ICU discharge and improve their acute ward-based review and outcome.
DESIGN: Retrospective patient cohort of index ICU admissions between June 2006 and October 2011 receiving routine inpatient review. Prospective cohort between March 2012 and March 2013 underwent risk scoring (PIRT) which subsequently guided inpatient ward-based review.
SETTING: Two UK adult ICUs. PARTICIPANTS: 4212 eligible discharges from ICU in the retrospective development cohort and 1028 patients included in the prospective intervention cohort.
INTERVENTIONS: Multivariate analysis was performed to determine factors associated with poor outcome in the retrospective cohort and used to generate a discharge risk score. A discharge and daily ward-based review tool incorporating an adjusted risk score was introduced. The prospective cohort underwent risk scoring at ICU discharge and inpatient review using the PIRT. OUTCOMES: The primary outcome was the composite of death or readmission to ICU within 14 days of ICU discharge following the index ICU admission.
RESULTS: PIRT review was achieved for 67.3% of all eligible discharges and improved the targeting of acute post-ICU review to high-risk patients. The presence of ward-based PIRT review in the prospective cohort did not correlate with a reduction in poor outcome overall (P=0.876) or overall readmission but did reduce early readmission (within the first 48 hours) from 4.5% to 3.6% (P=0.039), while increasing the rate of late readmission (48 hours to 14 days) from 2.7% to 5.8% (P=0.046).
CONCLUSION: PIRT facilitates the appropriate targeting of nurse-led inpatient review acutely after ICU discharge but does not reduce hospital mortality or overall readmission rates to ICU. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2017. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

Large retrospective cohort of patients for development of risk score which is effective in identifying high-risk intensive care unit (ICU) discharges.

Large prospective cohort of ward-based patient reviews incorporating the risk score and post-ICU ward-based review tool.

Retrospective data set potentially limits potential risk factors for readmission.

Lack of validation cohort to definitively demonstrate effectiveness of risk score in identifying high-risk ICU discharges.

A focus on short-term binary outcomes such as mortality may have missed potential benefits to important longer term morbidity outcomes.

Background

Despite progressive improvement in patient survival in the intensive care unit (ICU), approximately 10% of ICU discharges require readmission to the unit during the same hospital stay.1 Patients readmitted to ICU suffer longer lengths of stay, a higher mortality and increased costs compared with illness severity-matched index admissions.2

Specialist teams to review ward patients recently discharged from ICU can help detect and treat physical and psychological problems after discharge while providing continuity of care, specialist input and the potential for expedited intensivist intervention. While the implementation of routine post-ICU review may seem intuitive and is advocated by the Department of Health,3 the published data on the impact of these services on patient outcome are mixed.4–8 One explanation for this could be that often limited resources are not being targeted to those at greatest risk of deterioration. Identification of those patients at increased risk is challenging; multiple risk profiling models exist to assist clinicians in discharge decision-making9–15 for a variety of study populations and outcome criteria.16 While there is evidence these tools impact on clinician behaviour,17 little work has been done to use these risk models to target resources to the highest risk patients acutely after ICU discharge. Effective targeting of these resources might enhance their impact by improving survival following discharge from ICU, reducing ICU readmissions and hospital costs.

We have developed a clinical prediction model for death and readmission after ICU discharge in a large retrospective patient cohort and developed a tool to identify high-risk patients at ICU discharge. Additionally, we have modified the nurse-led acute post-ICU review on general wards after discharge from ICU to incorporate a new review tool. We have assessed the impact of this approach on allocation of nursing resources and patient outcomes in a 1-year prospective cohort of patients discharged from ICU.

Hypothesis and objectives

We hypothesise that risk profiling at the time of ICU discharge will assist the acute post-ICU review team to improve allocation of resources to higher risk patients. Additionally, we hypothesise that this will lead to an improvement in the composite outcome of ICU readmission and ward death within 2 weeks of discharge.

The objectives of this study were to:

Use local data from a large national data set to develop a risk score predictive of poor outcome (defined as death and/or readmission within 14 days of ICU discharge).

Integrate the risk score into an acute post-ICU ward-based review tool (PIRT) to be used by nurse-led review teams.

Determine prospectively whether the use of PIRT results in more appropriate targeting of acute post-ICU ward-based review.

Determine prospectively whether the use of PIRT results in improved outcomes following ICU discharge (primary objective).

Methods

Study location and context

Two ICUs at separate hospital sites in the Oxford University Hospitals NHS Foundation Trust were included in the study. One ICU is a mixed medical-surgical unit while the other covers predominately elective postsurgical patients with some haematology and oncology. Neurosurgical and cardiac patients are cared for in separate units and were not included in this study.

Acute post-ICU ward-based review

Ward-based review of patients acutely discharged from the two units in this study is performed by a dedicated team of specifically trained ICU nurses with oversight from a team coordinator and clinician as required. Clinical review and examination of inpatients on general surgical and medical wards is performed for the purpose of routine review of discharged patients, escalation of care and provision of advice to parent medical teams.

Study design

A representation of the project time flow is shown in figure 1.

Figure 1

Project time flow. PIRT, post-ICU ward-based review tool; POR, poor outcome risk.

Retrospective cohort (2006–2011)

Retrospective analysis of patient outcomes between 2006 and 2011 was performed. Routine ward-based review for all patients in this group was performed based on perceived clinical need at ICU discharge. The timing, duration and number of days of patient review performed were not protocolised and the decision to review a patient rested with either the discharging clinician or the nursing team.

Data on post-ICU patient reviews were captured and analysed for patients discharged since 2010.

Prospective cohort (2012–2013)

During the prospective intervention phase, at time of ICU discharge each patient underwent PIRT discharge risk scoring and was profiled to high or low risk of poor outcome. Ward-based patient reviews during this period were performed using the PIRT daily review tool.

The post-ICU review tool

PIRT discharge risk scoring used a simplified scoring algorithm derived from the poor outcome multivariate risk score (POR, see below) to facilitate accurate scoring, risk profiling and decision-making, especially out of hours. As well as the discharge risk score, the PIRT provides tools for daily review incorporating quantitative scoring of physiology, laboratory parameters and other post-ICU factors (online supplementary appendix 1). The daily review content provides a systems-based approach based in part on the Postoperative Mortality Score.18 Deranged physiology, blood results or clinical deterioration are scored on the PIRT to provide a simple summative daily review score of patient progress that could be used by staff to trigger senior reviews, prioritise the patient for further review or cease further ward reviews. There was no protocolisation based on PIRT discharge risk or daily scores nor did we impose formal thresholds for escalation or the decision to cease patient review to permit the nursing team flexibility to use the tool at their discretion.

No other changes were implemented that would affect discharge decision-making or readmission and no changes were made to the acute post-ICU resources available. The intervention was unblinded and all eligible ICU patients were included without randomisation. To ensure rigorous implementation of the PIRT model, those patients who received ward-based review but where the PIRT discharge risk score or form was partially incomplete were analysed per protocol as if they received no PIRT review (n=252).

Outcomes

The primary composite outcome was death or readmission to ICU within 14 days of ICU discharge following the index ICU admission. Secondary outcomes included death on the ward, early (within 48 hours) and late (between 48 hours and 14 days) readmission, the presence of a fully recorded patient review and daily PIRT review scores (prospective cohort only).

A recorded ward-based review episode was defined as either the presence of a complete digital record for the retrospective cohort or a fully completed PIRT document for the prospective cohort.

Data sources

The Intensive Care National Audit Research Centre (ICNARC) case mix programme19 incorporates data on patient demographics, comorbidities, hospital stay, ICU admission and outcomes. Data entry is performed by dedicated trained staff throughout the ICU stay and is locally validated.

All data recorded in the ICNARC database for every patient discharged from ICU during the time frame of interest were extracted. Patients still admitted to the ICU or who had been discharged less than 2 weeks prior to the cut-off date were excluded. The ICNARC records were manually reviewed and quality controlled. Any data fields with less than 95% completion or internal consistency for all patients in the time frame of interest were removed. Each patient episode was inspected for errors in data entry and where errors could not be conclusively corrected, the episode was excluded from analysis.

All acute nurse-led post-ICU reviews have been digitally documented since 2010. Data extracted from this database were analysed to provide simple metrics on timing and frequency of these reviews. For the prospective intervention cohort, each ward review implemented a physical PIRT form and these were audited manually to determine frequency and timing of review as well as the fidelity of form completion.

Study populations

Inclusion and exclusion criteria

All patients discharged from ICU alive and satisfying the inclusion and exclusion criteria were included in the study. ICU discharge was based on clinician assessment rather than to fixed criteria.

Exclusion criteria included discharge to another hospital trust or critical care unit outside the study, discharge for palliative care, self-discharge, repatriation and discharge with the expectation of dying. The presence of exclusion criteria was identified through the ICNARC data set. The presence of a Do Not Attempt Resuscitation order was not regarded as an exclusion criterion as this does not preclude readmission.

Over the period June 2006 to October 2011, 5734 patients were admitted to ICU and the survivors were routinely assessed at the time of discharge for the need for ward-based post-ICU review. Retrospective analysis of all 4212 eligible discharged patients in this time frame was performed for the primary outcome of interest and development of the POR score.

Analysis of ward-based reviews was limited to a subgroup of patients from this cohort (all those discharged in 2010 and 2011—1765 eligible discharges).

Prospective intervention cohort (2012–2013)

Between March 2012 and March 2013, 1315 patients were admitted to ICU and the 1028 eligible discharges were prospectively PIRT discharge risk scored and reviewed as deemed necessary using the PIRT tool.

Risk score development

Two risk scores were developed. The POR score was developed to provide weighting of discharge risk to permit accurate outcome analysis. The PIRT discharge risk score was used to inform the review team and implemented a simple additive score to stratify patients to high or low risk of poor outcome at the time of ICU discharge.

The POR score

The POR score was developed to permit accurate analysis through adjustment of patient outcomes for the severity of patient illness at ICU discharge. This was implemented as other risk scores are predominately based on the early phases of ICU admission or require data not included in the ICNARC database.

For patients discharged between June 2006 and October 2011, inclusive, univariate linear regression analysis was performed on the index ICU admission of each hospital stay using a range of factors for association with poor outcome (table 1). Factors included in the analysis were selected to be of relevance to the post-ICU population, recorded on the ICNARC database and readily available at the time of ICU discharge. Patients already discharged home or still in hospital (and not readmitted to ICU) at 14 days post-ICU discharge were analysed as ‘good’ outcomes to increase the relevance of the findings to the acute post-ICU period. Those patients readmitted and/or dead within 14 days of ICU discharge were analysed as ‘poor’ outcomes. Due to the different nature of the patient populations and service provision at the two hospital sites included in the study, the site of ICU admission was included in the model.

Table 1

Univariate analysis of poor outcome for 2006–2011 cohort

Characteristic	Percentage poor outcome n=356	Percentage good outcome n=3856	Significance
Demographics
Age group			<0.001
<55	29.5	39.9
55–64	22.2	19.9
65–74	28.9	22.2
75+	19.4	18.0
Sex (female)	37.6	40.1	0.471
Prior to ICU admission
Community CPR	0.3	1.8	0.217
Pre-ICU hospital stay 3+ days	27.5	15.5	<0.001
At ICU admission
Unplanned admission	63.7	68.3	<0.001
ICU type			<0.001
Surgical	29.5	17.0
Medical-surgical	70.5	83.0
Primary organ system pathology			<0.001
Cardiovascular	9.3	16.1
Genitourinary	7.3	9.1
Metabolic	2.0	5.5
Musculoskeletal	6.5	7.4
Neurological	2.8	7.4
Other	2.2	1.7
Respiratory	17.4	16.0
Gastrointestinal	52.5	36.6
Admission type			0.890
Medical	39.3	43.2
Surgical	60.7	56.8
Urgency of surgery			0.013
No surgery	39.3	43.7
Elective	34.3	32.1
Emergency	26.4	24.2
In-hospital CPR	2	2.4	0.034
Any CPR	2.5	4.6	0.394
During ICU admission
ICU length of stay 5+ days	24.7	18.9	0.001
Coma or stupor at 24  hours in ICU	16.3	17.0	<0.001
Composite organ support score ≥10	17.7	14.5	0.001
APACHE II score			0.002
Not recorded	6.5	5.8
1–9	8.7	12.3
10–19	52.8	52.4
20–29	27	25.9
30–39	4.8	3.4
40+	0.3	0.3
At ICU discharge
Timeliness of discharge			0.005
Not recorded	9.0	7.1
Fully ready	54.2	57.7
Delayed	30.6	31.8
Early	6.2	3.4
Care level at ICU discharge			0.053
Not recorded	9.0	6.5
Level 0	0.0	0.2
Level 1	18.5	22.6
Level 2	58.7	55.2
Level 3	13.8	15.5
Dependency at ICU discharge			0.001
None	9.3	7.1
Minor	75	81.7
Major	15.7	11.2

Composite organ support score derived from summation of levels of ICU organ support during total ICU stay.

Significance by univariate analysis. Where categories exist composite significance for the class is given.

All numbers as percentages.

APACHE II, Acute Physiology and Chronic Health Evaluation II; CPR, cardiopulmonary resuscitation; ICU, intensive care unit.

Multivariate logistic regression analysis of these risk factors was performed using backward elimination with main effects modelling with poor outcome as the outcome variable. Model stringency was set as entry criteria of likelihood ratio 0.05 and elimination at likelihood ratio 0.1. The final model incorporated only variables that were independently associated with poor outcome following ICU discharge (table 2). For categorical variables (age, primary admission pathology), ORs were determined relative to a category in each case.

Table 2

Multivariate analysis of poor outcome, 2006–2011 cohort

Category	Significance	OR (95% CI)
Demographics
Age*	<0.001	3.29 (2.44 to 4.44)
75+
65–74	<0.001	2.29 (1.67 to 3.08)
55–64	<0.001	1.82 (1.34 to 2.48)
Prior to ICU admission
Pre-ICU hospital stay 3+ days	<0.001	1.63 (1.27 to 2.09)
At ICU admission
Unplanned admission	0.032	1.33 (1.02 to 1.73)
In-hospital CPR	0.004	2.35 (1.31 to 4.24)
Site	<0.001	1.93 (1.47 to 2.54)
Admission†	<0.001	0.38 (0.26 to 0.56)
Cardiovascular
Genitourinary	<0.001	0.45 (0.29 to 0.69)
Metabolic	0.034	0.50 (0.26 to 0.95)
Musculoskeletal	0.461	0.84 (0.54 to 1.33)
Neurological	0.058	0.63 (0.39 to 1.02)
Other	0.806	0.91 (0.45 to 1.87)
Respiratory	0.132	0.79 (0.58 to 1.08)
During ICU admission
ICU length of stay 5+ days	0.017	1.37 (1.06 to 1.77)
At ICU discharge
Major dependency at ICU discharge	0.005	1.55 (1.14 to 2.11)

*Relative to patients under 55.

†Relative to gastrointestinal admissions.

CPR, cardiopulmonary resuscitation; ICU, intensive care unit.

The factors that were retained in this multivariate model for poor outcome were then used to generate the POR score based on multiplication of multivariate ORs that generated an individualised relative risk. For example, a 67-year-old patient admitted directly to ICU from the Accident and Emergency unit with community-acquired pneumonia, staying for 4 days on the ICU and with low dependency expectations at ICU discharge, would score 2.41 in the POR score (see worked example, table 3).

Table 3

Worked example of calculation of POR score based on multivariate ORs

Risk factor	OR (see table 2)
67 years old	2.29
Unplanned admission	1.33
Primary respiratory pathology	0.79
Short ICU stay	No increase in risk (OR 1.0)
Low discharge dependency	No increase in risk (OR 1.0)
Short hospital stay prior to ICU	No increase in risk (OR 1.0)
No CPR	No increase in risk (OR 1.0)
Final POR score	2.29×1.33×0.79=2.41

CPR, cardiopulmonary resuscitation; ICU, intensive care unit; POR, poor outcome risk.

The PIRT discharge risk score

The PIRT discharge risk score was developed based on the findings of the POR score combined with the experience of the post-ICU review team to adjust and weight some risk factors. This provided a simplified summative score based on comparable factors to the POR score (online supplementary appendix 1).

Bias

For the intervention cohort selection bias was addressed through the open enrolment of all eligible ICU discharges. We have assessed for evidence of selection bias retrospectively as part of the analysis. Performance bias is limited by the fact that, other than the implementation of PIRT, there were no other changes in review provision or resources and blinding was not necessary. No changes to ICU bed numbers or resources occurred during the study. Outcomes for all patients were analysed subsequent to the completion of the intervention phase to minimise detection bias and only quantitative, objective outcomes were used. As consent was not required for this study, attrition bias was limited to patients lost to follow-up; objective outcomes for all patients were recorded regardless of retention and patients were analysed on a per protocol basis.

Statistical analysis

Statistical comparisons were performed using the Student’s t-test (assuming non-equal variances) for continuous variables and either χ2 or Fisher’s exact test for categorical variables. Linear regression analysis was used for association between variables. All statistics were performed on SPSS V.20 (IBM).

Results

Retrospective study population (2006–2011)

Five thousand seven hundred thirty four patients were admitted during the period June 2006 to October 2011 of which 701 (12.2%) died on the ICU. Of the survivors 512 were excluded: 6 repatriations, 38 palliated patients, 42 patients on whom treatment was withdrawn, 7 self-discharges and the remainder (419 patients) excluded due to incomplete data. Of the 4579 patients remaining, 443 were readmitted, 64 of them twice and 17 three times or more. The final cohort of 4212 included index ICU admissions; 356 (8.5%) were readmitted with 131 (3.1%) early readmissions while 169 patients died in hospital within 2 weeks of leaving the ICU (4.0%).

The majority of readmissions (308, 86.5%) and ward deaths (113, 66.9%) occurred within 14 days of discharge. Median time to readmission was 4 days (IQR 2–9) and 7 days (IQR 3–19) for death on the ward after ICU discharge.

Table 4 shows outcomes and Acute Physiology and Chronic Health Evaluation II (APACHE II) scores for patients discharged between 2006 and 2012; baseline demographics between cohorts were comparable (see table 4 subtext). The APACHE II scores for patients included in study analysis were different between the retrospective and prospective study cohorts for both the 2006–2011 and 2010–2011 groups (unpaired t-test, P<0.01). However, risk model scores were not significantly different between the 2012 cohort and either the 2006–2011 (unpaired t-test, P=0.08) or 2010–2011 (unpaired t-test, P=0.91) groups.

Table 4

Included patient summary statistics by year

	Discharges by year*
	2006		2007		2008		2009		2010		2011		2012†
Included cohort size	185		769		762		731		956		809		1028
Age (years)	57±18		56±18		57±17		58±18		59±17		57±17		58±18
% Female	39.5		36.8		42.0		41.0		40.0		40.2		42.9
Early readmission	7	3.8%	24	3.1%	31	4.1%	30	4.1%	26	2.7%	13	1.6%*	40	3.9%
Late readmission	14	7.6%	32	4.2%	38	5.0%	43	5.9%	61	6.4%	37	4.6%	49	4.8%
Readmission	21	11.4%	56	7.3%	69	9.1%	73	10.0%	87	9.1%	50	6.2%	89	8.7%
ICU death‡	56	13.5%	128	13.2%	137	14.0%	124	11.4%	146	11.4%	110	11.1%	129	10.0%
Hospital death	10	5.1%	31	4.0%	27	3.5%	26	3.6%	47	4.9%	28	3.5%	34	3.3%
Median APACHE II	18 (13–24)		16 (12–21)		16 (12–21)		17 (12–23)		16 (12–21)		16 (13–20)		15 (11–18)§

Cohort size encompasses all index discharges from ICU within the study inclusion dates.

Age expressed as mean±SD, APACHE II as median (IQR).

*Univariate analysis of admission year (2006–2011) versus early or late readmission, ICU or hospital death not significant (P>0.05).

†P>0.05 for all comparisons 2012 vs 2006–2011 for age (unpaired t-test), sex (Fisher’s exact test, two tailed).

‡ICU deaths derived from whole population (prior to patient exclusions, other than those excluded due to lack of data on ICU outcome).

§APACHE II significantly lower in 2012 versus both 2006–2011 and 2010–2011 cohorts (for included study patients P<0.0001, unpaired t-test).

APACHE II, Acute Physiology and Chronic Health Evaluation II; ICU, intensive care unit.

Predictors of poor outcome

POR score

Of the 19 factors analysed (table 1), 13 generated a univariate significance of P<0.05 and eight were retained in the multivariate model. The only independent demographic factor in the multivariate POR score model (table 2) was increased age at admission. Unplanned admission, primary pathology site (gastrointestinal admissions at highest risk), site of ICU admission and length of hospital stay prior to ICU admission all increased the risk of poor outcome as did the length of ICU stay and the level of patient dependency on discharge. The need for cardiopulmonary resuscitation (CPR) at any point during the hospital admission increased the risk of poor outcome.

There is a positively skewed distribution of discharge risk scores in the development cohort (figure 2). Patients with a good outcome had lower POR scores (mean 2.52±1.95) than those with poor outcome (3.93±2.81). POR scoring correlated with outcome at 14 days (linear regression P<0.001, R2=0.216) in the development cohort (figure 3).

Figure 2

Distribution of poor outcome risk scores across 2006–2011 cohort.

Figure 3

Poor outcome frequency across deciles of risk score, 2006–2011 cohort.

Receiver operating characteristic curves (figure 4) demonstrated that POR scoring (area under the curve (AUC) 0.677) outperformed APACHE II scoring (AUC 0.573) in predicting the primary study outcome in those patients discharged between 2010 and 2011. There was no correlation between the POR score and the time post-ICU discharge that patients were either readmitted to ICU or died on the ward.

Figure 4

Receiver operator curves for risk scores, 2006–2011 cohort. APACHE II, Acute Physiology and Chronic Health Evaluation II.

PIRT risk score

A PIRT discharge risk score of 10 or more qualified as a ‘high’ risk discharge and within the population with a complete PIRT carried a positive predictive value of 16% and a negative predictive value of 89% (sensitivity 41%, specificity 70%). Within the 2010–2011 cohort both the PIRT discharge score and a PIRT discharge score of 10 or more correlated with poor outcome (linear regression, P=0.034 and P=0.047, respectively).

Acute post-ICU review and impact, 2010–2011 retrospective cohort

Ward-based review was achieved (and documented) for 50.9% of all eligible ICU discharges with a mean of 2.8±2.4 reviews per patient (median 2, IQR 1–4). POR scores for patients who were reviewed were lower (mean 2.23±1.75) than those patients who were not (3.35±2.46). There was an inverse correlation between the more unwell patients, as defined by the POR score, and the presence of ward-based review (linear regression, P<0.001, beta −0.253). As shown in figure 5, the presence of a routine post-ICU review did not have an impact on the incidence of poor outcome when controlled for patient POR score (linear regression P=0.260).

Figure 5

Overall ICU discharge outcomes 2010–2011 versus 2012 cohorts. ICU, intensive care unit; PIRT, post-ICU ward-based review tool.

However, in the patient population that received ward-based review, there was good correlation between the number of reviews and the severity of illness as assessed by the POR score (linear regression, P<0.001, beta 0.227) but outcome was not improved for those patients with greater numbers of reviews, when controlled for POR score (linear regression P=0.240) or APACHE score (P=0.498)

Prospective cohort (2012–2013) population and risk

Up to 1315 patients were admitted to ICU during the 1-year prospective cohort. Of these, 129 died on ICU (9.8%). Up to 183 patients were excluded, 8 due to discharge to palliative care, 1 on whom treatment was withdrawn, 1 self-discharge, 2 transfers to an alternative ICU outside the study and 171 due to incomplete data collection. Of the remaining 1132 patients included in the study, 111 were readmitted with 15 patients readmitted twice and 7 patients readmitted three times or more.

Prospective cohort (2012–2013) review

Of the 1028 index admissions eligible in the 2012 cohort, PIRT-based review was achieved (and documented) for 67.3% of all eligible discharges with a mean of 2.9±2.3 reviews per patient (median 2, IQR 1–4). Patients who were reviewed had higher POR scores (2.86±2.27) than those not seen following discharge (2.64±2.34). There was no correlation between POR score and the presence of PIRT-aided review (P=0.154). However, higher POR discharge risk score patients were seen a greater number of times (P<0.001).

Patients who were PIRT discharge risk profiled and reviewed on the ward had higher PIRT risk scores than those not reviewed (7.13±4.83 vs 1.22±3.09) but patients with higher risk scores were not more likely to be reviewed (P=0.674). PIRT discharge risk correlated with poor outcome (P=0.034, beta=0.081) and POR score (P<0.001). In the population that had the PIRT fully completed the PIRT daily review score on day 1 following ICU discharge was significantly associated with the POR score (P<0.001) and with poor outcome (P<0.001). In those patients who received two or more PIRT daily review scores, the gradient of the change in this score correlated well with poor outcome (P<0.001) even when controlled for POR score (P=0.001) and also correlated with the number of times seen (P<0.001).

The presence of PIRT-based review did not correlate with a significant improvement in overall outcome (figure 6) but was associated with reduced early readmission (P=0.039) and increased rate of late readmission (P=0.046), when adjusted for POR score. There was no significant reduction in deaths on the ward (P=0.738) and the overall rate of readmission was not altered (P=0.948).

Figure 6

ICU discharge outcomes, 2012 cohort. APACHE II, Acute Physiology and Chronic Health Evaluation II; ICU, intensive care unit; PIRT, post-ICU ward-based review tool.

Those patients receiving no ward-based review showed no difference in overall outcome between the 2010–2011 and 2012 cohorts, regardless of severity of illness (χ2; all risk P=1; low risk P=1; high risk P=0.482). Comparison of high-risk patients who were reviewed and not reviewed demonstrated no difference in any baseline demographics (age group, ICU site, sex) or clinical indices (pre-ICU hospital stay, ICU length of stay, unplanned ICU admission, CPR, medical or surgical status, dependency at ICU discharge and APACHE II score) indicating selection bias was not a major factor (P≥0.05 Pearson’s χ2test).

Discussion

Key results

Using a large retrospective cohort of patients from two mixed medical-surgical ICUs we have developed a risk score for outcome after ICU discharge that has moderate predictive value for death or readmission to ICU within 14 days of discharge. Incorporating a modified version of this risk score into a novel inpatient review tool improves allocation of post-ICU review resources to those in greatest need resulting in higher risk patients being seen more frequently. Integrating the PIRT into the work of the nurse-led post-ICU team is associated with a delay in readmission to ICU. A reduction in the rate of early readmission was offset by an increase in the rate of late readmission with no overall improvement in readmission or hospital mortality rates.

Limitations

Our use of the ICNARC database in a retrospective manner limits the breadth of available patient factors that could be included in the risk modelling. In particular, the study was unable to capture data on patients discharged from ICU with limitations on care set to preclude readmission since this information is not included in ICNARC. A bespoke retrospective or prospective data capture to expand the factors recorded would have enhanced the quality of the study but was beyond the resources available. Furthermore, with a broader scope of data collection we would be able to employ the use of other validated risk scores for ICU discharge and compare these to the POR score.

We used the PIRT discharge risk score to categorise patients into high and low risk at ICU discharge, rather than the POR score, despite the statistical superiority of the POR score and slightly higher association with outcome. Having trialled the POR as a discharge tool it was not practical in clinical use due to calculation errors negating its statistical superiority. The PIRT discharge score is more practical, has an excellent negative predictive value and associates well with poor outcome. We retained the POR score in this study as an accurate marker of patient risk at discharge for statistical analysis.

This study would have been improved through the use of an intermediate cohort of patients to validate the PIRT and POR risk scores without any changes to the post-ICU review system prior to the implementation of PIRT-directed inpatient review system. While this would have permitted prospective validation of the risk score and increased confidence in our risk model, the PIRT score had the desired impact of improving the distribution of ward-based review resources and as such was successful. We have adjusted patient outcomes using the unvalidated POR score as in this patient cohort it outperforms the APACHE II and is more relevant to the severity of patient illness at discharge, unlike other available risk scores.

Our results suggest that the impact of PIRT is likely to be on readmission rather than on outcome overall. However, the risk profiling method we have used is powered to identify patients at risk of readmission or death and as such we may have diminished its overall effectiveness. However, multivariate regression analysis of the data set using readmission as the end-point generates a similar set of risk profile characteristics and high-risk patient identification was good.

The percentage of patients with fully documented inpatient review in both the retrospective and intervention cohorts was low (50.9% and 67.3% of all discharged patients, respectively). Clearly some patients do not require routine ward-based review acutely after ICU; additional explanations include poor documentation, limited staff resources and issues with paper notes as well as some PIRT entries being discarded due to lack of completion.

The outcomes used as end-points in this study included death or readmission to ICU within 14 days of ICU discharge. These were selected as most likely to demonstrate a beneficial impact of our intervention. It would have been informative to investigate the impact of the PIRT on longer term morbidity and quality of life indices as well as these represent outcomes of significant importance to patients and families. However, this was beyond the scope of the study.

Interpretation

Analysis of the 2010–2011 cohort indicates that acute post-ICU ward-based review did not have an overall impact on outcome and was not effectively targeted to the most unwell patients. There is no clear explanation for why ward-based review prior to the use of the PIRT was poorly targeted and this finding perhaps highlights how difficult it can be to objectively determine discharge risk based on clinical indices and experience.

The multivariate analysis of patient demographic and ICU-related data revealed a number of prognostic factors that are relevant to discharge risk stratification. Elderly patients, unplanned admissions, patients with a long hospital stay before ICU admission or a long ICU stay, and patients with certain primary admission pathologies and whose inpatient stay is complicated by the need for CPR are at greatest risk of poor outcome after ICU discharge. These factors are well recognised in previous studies as increasing risk.14 16 20–23 The POR risk model developed from these factors had only moderate predictive power for poor outcome (AUC 0.677) but was sufficient to yield a significant improvement in targeting of ward review resources to higher risk patients through the PIRT risk score. Comparable studies16 have focused on a single outcome (death or readmission) and presented risk models with AUCs of between 0.66 and 0.92 which places the POR at the lower end of the published spectrum. However, compared with well-published models such as the Sequential Organ Failure Assessment score and simplified Therapeutic Intervention Scoring System, the POR score performed comparably.24

Targeting of post-ICU review teams to inpatients identified as ‘high risk of readmission’ at the time of ICU discharge has been advocated for some time.9 However, more appropriate targeting of the post-ICU resources in this study did not have an overall impact on outcome. While this is disappointing it is worth considering why this might be the case. The per protocol analysis might have led to dilution of perceived PIRT effect through improvements in outcome in patients who received ward-based PIRT review but due to incomplete documentation were subsequently analysed as not receiving a review. However, this is unlikely to have affected outcomes sufficiently to have eliminated a statistically significant impact. While the rate of review was higher in the intervention cohort, the number of reviews per patient was comparable between the two cohorts (P=0.281, unpaired t-test) suggesting that resources were not a limiting factor.

Implications for future research

The study is non-blinded and non-randomised and as such has a role primarily in developing hypotheses. However, it raises the interesting question of whether previous studies assessing the value of acute post-ICU review and assessment have been hampered by the poor targeting of resources. It is encouraging that the use of the PIRT improved targeting of post-ICU review which opens up the potential for larger scale studies and inclusion in future work looking at the allocation and stratification of both pre-ICU and post-ICU resources. The results and risk modelling are clearly specific to the ICU and population studied but are generalisable to other ICU populations with a similar intake spectrum and the principles of risk modelling and targeting of inpatient review can be extrapolated to a wide range of clinical discharge locations.

The concomitant increase in the late admission rate in the intervention cohort is likely to be due to PIRT-based review assisting in maintaining holistic ICU style care for the immediate postdischarge phase but being unable to prevent subsequent deterioration from secondary pathologies in the later postdischarge phase. It is well known that readmitted patients do less well than those on their primary admission, even when matched for their illness severity2 and it would be interesting in future studies to assess if patients readmitted following PIRT-based review have better outcomes than those readmitted after routine—or no—readmission.