Identification of the high risk emergency surgical patient: Which risk prediction model should be used?

INTRODUCTION: National guidance states that all patients having emergency surgery should have a mortality risk assessment calculated on admission so that the 'high risk' patient can receive the appropriate seniority and level of care. We aimed to assess if peri-operative risk scoring tools could accurately calculate mortality and morbidity risk.
METHODS: Mortality risk scores for 86 consecutive emergency laparotomies, were calculated using pre-operative (ASA, Lee index) and post-operative (POSSUM, P-POSSUM and CR-POSSUM) risk calculation tools. Morbidity risk scores were calculated using the POSSUM predicted morbidity and compared against actual morbidity according to the Clavien-Dindo classification.
RESULTS: The actual mortality was 10.5%. The average predicted risk scores for all laparotomies were: ASA 26.5%, Lee Index 2.5%, POSSUM 29.5%, P-POSSUM 18.5%, CR-POSSUM 10.5%. Complications occurred following 67 laparotomies (78%). The majority (51%) of complications were classified as Clavien-Dindo grade 2-3 (non-life-threatening). Patients having a POSSUM morbidity risk of greater than 50% developed significantly more life-threatening complications (CD 4-5) compared with those who predicted less than or equal to 50% morbidity risk (P = 0.01). DISCUSSION: Pre-operative risk stratification remains a challenge because the Lee Index under-predicts and ASA over-predicts mortality risk. Post-operative risk scoring using the CR-POSSUM is more accurate and we suggest can be used to identify patients who require intensive care post-operatively.
CONCLUSIONS: In the absence of accurate risk scoring tools that can be used on admission to hospital it is not possible to reliably audit the achievement of national standards of care for the 'high-risk' patient.

Introduction

The Royal College of Surgeons of England has identified that the delivery of emergency surgical care in England and Wales is currently suboptimal [1], with mortality rates reaching up to 25% [2]. The college has, therefore, outlined recommendations emphasising the need for early identification of patients who are at high risk of mortality. Patients who are predicted greater than 5% mortality should be operated on with a consultant present. Those who are predicted greater than 10% mortality should be reviewed by a consultant within 4 h of admission and managed with Level 3 care post-operatively [1,3].

The National Confidential Enquiry into Patient Outcome and Death (NCEPOD) report in 2011 entitled “Knowing the Risk” [3], however, accepts that there are a variety of risk scoring tests available (Table 1) [4], and that many are imprecise. In addition many of these systems require information that is not available when the patient is admitted to hospital which is the time recommended for risk assessment by the Royal College. Despite this, many surgical departments are using such risk soring tools pre-operatively based upon predicted operative findings. This ‘best guess’ approach [5] to scoring patients using tools not designed to be used pre-operatively is far from ideal. Thus it is important to establish useful and accurate methods of identifying the high risk patient on admission, if the Royal College guidelines are to be applied.

Table 1

Surgical risk scores classified by outcome measure and need for intra-operative information.

	Scores predicting mortality	Scores predicting morbidity
Scores not requiring operative information	ASA1	ASA
	APACHE-II	APACHE-II
	Donati score	Goldman cardiac risk index
	Hardman index	Veltkamp score
	Glasgow aneurysm score	VA respiratory failure score
	Sickness assessment	VA pneumonia prediction index
	Boey score
	Hacetteppe score
	Physiological POSSUM
Scores requiring operative information	Mannheim peritonitis index	POSSUM
	Reiss index	P-POSSUM
	Fitness score
	POSSUM
	P-POSSUM
	Cleveland colorectal model
	Surgical risk scale

Risk scoring methods analysed in this series

We aimed to analyse which risk scoring methods most reliably predicted observed morbidity and mortality in patients undergoing emergency laparotomy in our institution from January–July 2012. We selected 2 pre-operative risk scores (ASA and Lee Index) and 3 post-operative risk scores (POSSUM, P-POSSUM and CR-POSSUM) for use and comparison. These have not been previously compared in a cohort of emergency patients. The ASA and Lee index were chosen because they are easily calculable at the time of admission and are also suggested as pre-operative tools which might be used to assess risk in the Royal College of Surgeons report [1] and NCEPOD report [4] respectively. The POSSUM scores were selected for assessment of tools that require additional operative information (Tables 4 and 5).

ASA

The ASA (American Society of Anesthesiologists) classification of fitness for surgery [6], although not originally described as a risk prediction score, has a quantitative association with predicted percentage post-operative mortality (Table 2) [7].

Table 2

Predicted risk of mortality after major surgery performed as urgent/emergency (Adapted from Donati et al. [7]).

ASA class	Age <50	Age 50–69	Age ≥70
I	1.6%	2%	0%
II	4.5%	8.2%	12.9%
III	12.4%	21%	30.6%
IV	29.6%	44.3%	56.8%

Lee Index

The Lee Index has been validated for stratifying the risk of major cardiac complications following non-cardiac surgery [8]. The score consists of six independent points (high-risk surgery defined as intraperitoneal, intrathoracic, or suprainguinal vascular procedures, ischaemic heart disease, congestive heart failure, cerebrovascular disease, insulin therapy for diabetes mellitus, pre-operative creatinine level greater than 176 μmol/l) and is calculated using a scoring system outlined in Table 3.

Table 3

Lee class and risk of major cardiac complications.

Points	Class	Risk
0	I	0.4%
1	II	0.9%
2	III	6.6%
3 or more	IV	11%

POSSUM and modified POSSUM

The POSSUM [9] (Physiological and Operative Severity Score for the enUmeration of Mortality and Morbidity) has been previously considered an accurate predictor of post-operative complications [10]. It uses 12 physiological and 6 operative variables to give a calculated risk of morbidity and death (Table 4).

Table 4

Physiological and operative parameters used to calculate POSSUM and P-POSSUM scores.

Physiological	Operative
Age	Operation type (minor – complex major)
Cardiac comorbidity	Number of procedures
Respiratory comorbidity	Operative blood loss
ECG changes	Peritoneal contamination
Systolic BP	Malignancy status
Pulse rate	CEPOD
Haemoglobin
WBC
Urea
Sodium
Potassium
GCS

The Portsmouth POSSUM (P-POSSUM) [11] was developed following studies showing that POSSUM over-predicted mortality. It utilises the same physiological and operative variables as POSSUM with a modified regression equation. The Colorectal POSSUM (CR-POSSUM) [12] is a dedicated risk-adjustment scoring system for mortality in colorectal surgery (Table 5). For research purposes we decided to calculate the CR-POSSUM for all patients including non-colorectal cases to see if this scoring system might correlate with actual post-operative mortality risk even amongst patient without colorectal pathologies.

Table 5

Physiological and operative parameters used to calculate CR-POSSUM scores.

Physiological	Operative
Age	Operation type (minor – complex major)
Cardiac failure	Peritoneal contamination
Systolic BP	Malignancy status
Pulse rate	CEPOD
Haemoglobin
Urea

Methods

Patients

We performed a retrospective analysis of 108 consecutive emergency laparotomies between January 2012 and July 2012. Laparotomies were performed at the University Hospital Birmingham, a tertiary referral hospital which also provides an acute general surgical service to its local population in Birmingham. Inclusion and exclusion criteria are listed in Fig. 1 and were taken from the National Emergency Laparotomy Audit (NELA). Post-operative morbidity was recorded until the time of discharge or death. 22 cases were excluded due to incomplete data, leaving 86 laparotomy cases in the study population.

Fig. 1

Inclusion and exclusion criteria.

Data extraction

The following data were extracted or calculated from information in the medical records: patient demographics, co-existent morbidity, ASA grade, Lee index classification, POSSUM, P-POSSUM, CR-POSSUM grade, indication and type of surgery, post-operative re-intervention, ITU admission, length of stay, postoperative morbidity (as per Clavien–Dindo classification [13]) and 30-day mortality. The number of hours between surgical admission and documented consultant review, the level of training for anaesthetists and surgeons in theatre and the level of post-operative care received was also obtained from the medical casenotes. There was no level 2 care (high dependency unit) available at our institution which offered only either level 1 care which was ward based care or level 3 care which was a place of intensive or critical care.

Risk calculation

ASA and Lee index were calculated using data present at the time of admission. Where patients were scored ASA 5 on the anaesthetic chart, (6 laparotomies) we used the mortality risk prediction for ASA grade 4 since the Donati et al. conversion table to percentage including only ASA grades 1–4 (Table 2). Post-operative POSSUM morbidity and mortality risk scores were calculated using the online calculator http://www.riskprediction.org.uk/.

Statistical analysis

Fisher's exact test was used to compare numbers of cases in categories of low and high risk as defined by specified cut-off points of the POSSUM morbidity prediction percentage.

Audit of seniority of care

In this aspect of the study we took the standard set by the Royal College that patients scoring above 10% mortality risk on admission should be reviewed within 4 h of admission by a consultant surgeon. We audited this standard of practice at our institution in patients who predicted greater than 10% mortality risk according to ASA and Lee Index in 49 new admissions. It was not appropriate to audit the royal college guidelines in the remaining 37 patients who had emergency laparotomies following elective surgery (23 cases), or had been initially admitted under another specialty (8 cases), or had re-laparotomies following emergency laparotomy (6 cases). This is because these 37 patients were already admitted, and had already been reviewed by a consultant prior to their deterioration or transfer to the surgical department from another specialty.

Results

Following exclusion criteria, 86 emergency laparotomies were performed in 43 women and 43 men. The median age was 63 years (range: 19–86).

The findings at the time of operation were gastrointestinal perforation (21), non-malignant intestinal obstruction (16), anastomotic leak/iatrogenic (13) malignancy (8), bleeding (6), bowel ischaemia (5) and other causes (17).

Predicted outcomes

There was wide variability in the risk prediction amongst the five methods (Fig. 2). The average predicted mortality percentage risks and Lee Index of life-threatening morbidity risk across all patients was, ASA – 26.5% (average ASA grade of 3), Lee Index – 2.5% (average Lee Index Class of 2.3), POSSUM – 29.5%, P-POSSUM – 18.5%, CR-POSSUM – 10.5%.

Fig. 2

Predicted percentage risk in emergency laparotomy according to different risk tools.

The average predicted POSSUM morbidity risk score was 69.4% across all patients.

Observed outcomes

There were 9 deaths (10.5%) within 30 days of laparotomy. The median number of days between operation and mortality was 18 days (range 1–29). There were 7 additional deaths in hospital with a median number of days between operation and death of 56 days (range 39–124).

Complications occurred following 67 laparotomies. The majority (51%) of complications were classified as Clavien–Dindo grade 2 or 3 (Table 6). The three most frequent complications following laparotomy were intra-abdominal collections (13.9%), chest infections (12.8%) and wound infections (8.1%) (Fig. 3).

Table 6

Frequency of complication according to Clavien–Dindo classification.

Clavien–Dindo morbidity classification	Frequency of complications (number of laparotomies)
0	19
1	1
2	28
3a	8
3b	15
4a	4
4b	2
5 (Death)	9

Fig. 3

Frequency of complications following laparotomy.

Correlations of predicted and observed outcomes

Mortality

The mortality prediction score that best correlated with the observed 30-day mortality was the CR-POSSUM which was equal with the actual mortality at 10.5%. ASA, POSSUM and P-POSSUM over-predicted and Lee-Index under-predicted risk.

Morbidity

There was a positive correlation between POSSUM morbidity score with the actual CD morbidity (Fig. 4). Patients who were predicted to have a POSSUM morbidity risk of greater than 50% developed significantly more life-threatening complications (CD 4–5) compared with those who predicted less than or equal to 50% morbidity risk (P = 0.01). This difference became more significant when a higher threshold score was chosen, such as 85% (P = 0.0001) (Fig. 5).

Fig. 4

Average predicted POSSUM morbidity in patient who developed complications according to Clavian–Dindo Classification.

Fig. 5

Frequency of patients developing CD 0–3 and CD 4–5 complications in laparotomies where the POSSUM morbidity score was >50% or >85% morbidity risk.

Length of stay

Post-operative length of stay correlated with predicted POSSUM morbidity scores. The median was 23 days in this study. Patients who had a predicted morbidity score of ≤50% had a significantly lesser length of stay (11 days) than patients who predicted >50% morbidity risk (26 days) (P= < 0.05).

Audit of the Royal College guidelines

There were 49 laparotomies following an acute admission. The median time interval between admission to the acute surgical unit and consultant surgeon review was 15 h (range: 1–88 h). 30 and 2 of these acute cases were predicted as being high risk (above 10%) according to ASA and the Lee Index respectively. In 6 of these cases (20%) consultant review within 4 h was achieved (Table 7).

Table 7

Consultant review of patient in 4 h.

	>10% Mortality (ASA)	>10% risk (Lee Index)
Number of patients	30	2
Number seen within 4 Hours	6	0
Percentage	20%	0%

Of the 79 and 18 patients that were predicted as being above 5% risk according to the ASA and Lee Index respectively, the majority (>80%) were operated on with consultant surgeon present in theatre. Consultant anaesthetists were present in theatre in 43 (50%) of these cases (Table 8).

Table 8

Consultant surgeon and anaesthetist present in theatre.

	Total	>5% ASA	>5% Lee Index
Number of laparotomies	86	79	18
Number with consultant surgeons	71 (83%)	65 (82%)	15 (83%)
Number with consultants anaesthetists	43 (50%)	38 (48%)	12 (67%)

Patients received Level 1 (non-intensive care) and Level 3 (intensive care) care immediately post-operatively in 33 and 53 cases respectively. The average predicted risk using all prediction methods was at least twice in the patients who received Level 3 care compared to those who were managed on the ward (Fig. 6).

Fig. 6

The difference in predicted risk of patients who received Level 1 and Level 3 care immediately post-operatively.

Discussion

The guidelines suggest that risk assessment of the emergency surgical patient should be made upon admission to an acute surgical unit. It is often a relatively inexperienced member of the surgical team who initially reviews the patient at this time and therefore risk scoring tools may aid the clerking doctor in highlighting patients who require urgent review by the consultant (<4 hours), consultant supervision in theatre and level 2 or 3 care post-operatively as stated in such guidelines. The Royal College, however, have not provided clear instructions regarding which risk assessment tool should be used. The ASA and Lee Index have been suggested as potentially useful pre-operative tools. We found, however, that both of these tools inaccurately predicted risk. The ASA grade was found to be 3 or 4 in the majority of cases which was an over-prediction of risk. Other larger studies looking at emergency surgery have observed ASA grades of 1 or 2 in the majority of cases [14]. This difference might be because our study population excluded emergency cases with lower risk, such as, appendectomy. We also observed variation between the anaesthetist's assessments when recording ASA. It has been shown that the term “systemic disease” in the ASA grading method introduces subjectivity where some anaesthetists include the acute surgical problem for which patients requires an emergency laparotomy as a ‘systemic disease’ rather than co-existing past medical systemic diseases [15]. This may have occurred in our study where laparotomies tended to be for higher risk conditions. On the whole, we consider the ASA to be a crude measurement and attempts to translate ASA grade into percentage risk have shown themselves to be inherently contradictory. For example Donati el al observed decreasing percentage risks with age in patients with ASA grade 1 (Table 2). Our study confirms that the ASA poorly correlated with actual risk.

The Lee index under-predicted risk in our cohort, and whilst it has less subjectivity than ASA grading, its disadvantage is that it was not strictly designed to predict mortality risk. Rather, it was originally formulated to predict the risk of major cardiac complications, including cardiac arrest. The underestimation of actual mortality and morbidity rates following laparotomy, renders the Lee Index an unhelpful method of risk prediction in practice, based on these data. It is not possible, therefore, to pre-operatively identify the high risk patient using these suggested risk prediction methods. With such vast differences between the numbers of patients defined as high risk by ASA versus Lee Index, it is consequently not possible to fairly audit national standards for the ‘high risk patient’.

The Mersey General Surgery group did attempt such an audit in 494 procedures (including appendectomies and hernia repairs) across 8 acute trusts using the P-POSSUM score upon admission to predict post-operative risk. They audited three of the national guidelines (consultant surgeon and anaesthetist supervision and level 2–3 care for the high risk patient). In only 46% of the 65 “high risk” cases were all three of these standards met [16]. The problem, however, with using P-POSSUM to define the high risk patient is two-fold. Firstly, when using P-POSSUM scores at the time of admission, operative findings have to be guessed. Secondly, we have demonstrated in our population that P-POSSUM scores over-predicted risk.

It was concerning that the minority (20%) of ‘high risk’ patients, were reviewed by a consultant within 4 h of admission. The limitations of these data, however, is that they are based upon documentation evidence and it is possible that patients had been reviewed or at least discussed with a consultant without documentation of such. It is imperative therefore that all discussions with senior clinicians is evidenced by documentation in the patient case notes. The presence of consultant surgeons in theatre during the operation, on the other hand, was more encouraging, whereas consultant anaesthetist were present in only 50% of cases. In the aforementioned study by the Mersey Surgical Group, it was also found that the number of consultant anaesthetists present was lower (66%) in comparison to consultant surgical presence (71%) in high risk patients as defined by the P-POSSUM score. One could argue that these statistics should be in reverse. The presence of a consultant anaesthetist might be as or more important given that it is the physiology of the patient that influences most the patient's risk score and need for critical care peri-operatively.

In contrast to pre-operative risk scoring methods, the post-operative risk scores reviewed in this study may have a role in guiding the appropriate level of care in the emergency surgical patient. The Royal College and NCEPOD have produced guidelines regarding the importance of improving the level of post-operative care for the high risk patient. We found modified POSSUM scoring to be the most accurate, in particular the CR-POSSUM. The CR-POSSUM was found to be both the easiest to calculate (Table 5) and reflected actual risk well even amongst non-colorectal cases.

In the UK, less than 1/3 of non-cardiac emergency surgery patients are admitted to intensive care (ITU) [17]. In addition, delayed admission to ITU has been associated with increased risk of post-surgical death [18]. A higher proportion (62%) of level 3 care was observed post-operatively in our series of patients and this may explain why the mortality rate in our group was lower by comparison to nation-wide data. When correlating the patients who went to ITU against their retrospective risk score calculations the data implies that patients were receiving Level 3 care appropriately (Fig. 6). There was a particularly appropriate correlation when the CR-POSSUM was used as a predictor of risk with patients managed on the ward having a less than 10% risk which is the recommended threshold for ITU admission. Since admission to ITU post-operatively is dependent upon so many factors such as prioritising patient beds and the anaesthetic or intensivist opinion, we suggest an objective means of calculating need for ITU might be found in a universally used accurate scoring system such as the CR-POSSUM.

Although the observed mortality rate was relatively low in this series, the level of any morbidity was 67% in this series, compared to a 50% morbidity rate described in the “high risk surgical patient report 2011”. The single POSSUM scoring method for predicting morbidity (http://www.riskprediction.org.uk/) correlated positively with the actual morbidity. The greater the threshold POSSUM score for predicting ‘life threatening’ morbidity, defined by a Clavian–Dindo classification of 4–5, the more significant (P < 0.0001) the correlation between predicted and actual morbidity. Thus, POSSUM morbidity scoring can sensitively identify patients who will go on to have non-life threatening or life-threatening complications.

The superior accuracy of P-POSSUM over POSSUM scoring has been previously demonstrated [19]. Previous studies, in predominantly elective colorectal procedures [20] have also concluded that the CR-POSSUM is the easiest scoring system to calculate and is superior in its accuracy when compared to other POSSUM scores. The CR-POSSUM, has not, however, prior to this study been shown to be a superior predictor of risk in a purely emergency cohort of patients.

One major limitation of this study is that the risk scores applied to each patient were calculated retrospectively so that operative information was available for all cases. As discussed, in practice, a ‘best guess’ approach has to be used if one is going to use post-operative modified POSSUM scoring as a tool of risk prediction upon admission. A further limitation is the small number of emergency cases. One major reason for this is due to exclusion of laparotomy for appendicitis and cholecystitis in accordance with criteria drawn up by the National Emergency Laparotomy Audit (NELA). We recognise a further limitation in that the guidance from the Royal College insists upon ‘on-admission’ pre-operative risk scoring and yet the most accurate risk prediction tools that we identified require operative information for calculation. Indeed, one criticism of the national guidance might be the failure to provide indications regarding which risk prediction tool should be applied since most validated scoring systems require information unavailable at the time of admission. Furthermore, one of the suggested tools is the Lee Index which whilst it does not require operative information for calculation is a predictor of major cardiovascular mortality rather than all-cause mortality. It may be that the lack of clarity in these guidelines has lead to poor compliance with risk scoring amongst surgical departments. Indeed, there is no consistently used risk scoring methodology in our institution currently. We suspect many emergency surgical units throughout the UK are also not currently able to agree upon a model to calculate mortality risk at the time of admission so as to guide management.

Conclusion

Pre-operative risk scoring methods are variable, inaccurate and therefore unreliable for selection of seniority of care and the level of post-operative care for the emergency surgical patient. The Royal College strongly advises risk assessment upon admission to hospital in all emergency surgical patients, yet offers no guidance about which risk assessment tool is most suitable. Post-operative risk scoring methods in this series were more accurate, with CR-POSSUM predicting closest to the actual mortality rate. We suggest, therefore, that the CR-POSSUM can be a useful tool for guiding post-operative level of care in patients undergoing emergency laparotomy. We recommend evaluation of further risk prediction tools that can be used at the time of admission if the high risk emergency surgical patient is to be matched with the appropriate seniority and level of care.

Ethical approval

Not applicable.

Source of funding

None required.

Author contribution

Stephen Stonelake: Data collection from medical paper and electronic records, statistical analysis.

Peter Thomson: Data collection from medical paper and electronic records.

Nigel Suggett: Supervision, review and editing of paper.

Conflict of interest

None.

Guarantor

Nigel Suggett.