Development of the Combined Assessment of Risk Encountered in Surgery (CARES) surgical risk calculator for prediction of postsurgical mortality and need for intensive care unit admission risk: a single-center retrospective study.

INTRODUCTION: Accurate surgical risk prediction is paramount in clinical shared decision making. Existing risk calculators have limited value in local practice due to lack of validation, complexities and inclusion of non-routine variables.
OBJECTIVE: We aim to develop a simple, locally derived and validated surgical risk calculator predicting 30-day postsurgical mortality and need for intensive care unit (ICU) stay (>24 hours) based on routinely collected preoperative variables. We postulate that accuracy of a clinical history-based scoring tool could be improved by including readily available investigations, such as haemoglobin level and red cell distribution width.
METHODOLOGY: Electronic medical records of 90 785 patients, who underwent non-cardiac and non-neuro surgery between 1 January 2012 and 31 October 2016 in Singapore General Hospital, were retrospectively analysed. Patient demographics, comorbidities, laboratory results, surgical priority and surgical risk were collected. Outcome measures were death within 30 days after surgery and ICU admission. After excluding patients with missing data, the final data set consisted of 79 914 cases, which was divided randomly into derivation (70%) and validation cohort (30%). Multivariable logistic regression analysis was used to construct a single model predicting both outcomes using Odds Ratio (OR) of the risk variables. The ORs were then assigned ranks, which were subsequently used to construct the calculator.
RESULTS: Observed mortality was 0.6%. The Combined Assessment of Risk Encountered in Surgery (CARES) surgical risk calculator, consisting of nine variables, was constructed. The area under the receiver operating curve (AUROC) in the derivation and validation cohorts for mortality were 0.934 (0.917-0.950) and 0.934 (0.912-0.956), respectively, while the AUROC for ICU admission was 0.863 (0.848-0.878) and 0.837 (0.808-0.868), respectively. CARES also performed better than the American Society of Anaesthesiologists-Physical Status classification in terms of AUROC comparison.
CONCLUSION: The development of the CARES surgical risk calculator allows for a simplified yet accurate prediction of both postoperative mortality and need for ICU admission after surgery. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2018. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

The Combined Assessment of Risk Encountered in Surgery (CARES) model predicts risk for both 30-day mortality and need for intensive care unit (ICU) admission postoperatively with good accuracy. Prediction of the risk for postoperative ICU admission is novel and not currently available for most risk stratification tools. This adds to the utility of the model and aids in decision-making process and health resource planning.

The CARES surgical risk calculator comprises of simple and easily accessible variables available from routine preoperative evaluation for most surgeries.

It is the first risk stratification tool to incorporate the use of red cell distribution width, a novel haematological biomarker which has been shown to be of value in predicting mortality risk.

This is a retrospective study design.

CARES has only been validated in a single centre, hence, there is a need for further prospective studies to externally validate the tool in other institutions across the region.

Introduction and background

About 250 million surgeries are performed worldwide each year, and this number is increasing rapidly.1 As access to surgery improves, the number of patients with postoperative complications will also increase.2 3 Previous studies demonstrated that a large proportion of postoperative mortality occurs in a smaller, distinct group of patients with high-risk characteristics, and <15% from this group were admitted to intensive care units (ICU) postoperatively.4 5 In the preoperative assessment of a surgical patient, it is prudent to counsel the patient on the risks of postoperative mortality and need for critical care monitoring after surgery. Therefore, accurate preoperative prediction and stratification of surgical risks is becoming even more important for perioperative shared decision-making process, guiding allocation of resources and improving patient outcomes.

However, predicting postoperative risks and identifying patients at a higher risk of adverse events have traditionally been based on individual surgeon experience and augmented by published rates in the literature, either from single institution studies or clinical trials.6 Unfortunately, these estimates are typically not specific to an individual patient’s risk factors. Moreover, existing risk stratification tools have their own limitations. The currently available risk stratification tools, for example, American Society of Anaesthesiologists-Physical Status (ASA-PS), Physiological and Operative Severity Score for the enUmeration of Mortality and Morbidity (POSSUM), Surgical Outcome Risk Tool (SORT) and American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) were all derived from the Western population and healthcare systems, from which Asians would serve as outliers in view of the their different socioeconomic, cultural, genetic makeup and healthcare systems. There is a paucity of surgical risk stratification models which is derived from or has been validated in the Asian population. This limits the uptake and applicability of these models in the region.

Furthermore, the individual risk stratification tools suffers from wide inter-user variability (ASA-PS),7 need for data which are not available during the preoperative period (POSSUM),8 9 lack of validation outside the derived population’s region (SORT, ACS-NSQIP) and the complexity of the model itself (ACS-NSQIP). Hence, the inertia to use them may be due to concerns over the accuracy, complexity and also the requirement for a large number of variables. To improve the performance of predictive models, there are recent interests in the use biomarkers such as B-type natriuretic peptides10 11 and cardiac troponin12 to predict mortality. However, these markers may not be easily available across all laboratories and most often are not part of the routine preoperative investigations. More recently, readily available haematological biomarkers such as red cell distribution width (RDW) and degree of anaemia (if present) have been shown to be associated with postoperative mortality risk.13–15 Incorporation of these biomarkers with preoperative clinical factors may improve the accuracy of a surgical risk model.

We aim to develop a locally derived, simple and accurate surgical risk calculator that consisted of readily available preoperative clinical and laboratory variables, which can predict both mortality and ICU admission with just a single set of variables. We hypothesise that the development of this risk stratification tool could help us accurately predict the risk of (1) 30-day postsurgical mortality and (2) requiring admission to ICU for >24 hours during the surgical admission, which may serve as a surrogate for major postoperative complications in the Singapore population.

Methodology

Data source and patients

Institutional Review Board approval was obtained (Singhealth CIRB 2014/651/D) prior to the start of the study, which waived the requirement of individual informed consent. Retrospective data were collected and analysed from the electronic medical records of 90 785 patients aged 18 and older who underwent surgery under general or regional anaesthesia between 1 January 2012 and 31 October 2016 in Singapore General Hospital, a 1700-bedded tertiary academic hospital in Singapore. These clinical records were sourced from our institution’s clinical information system (Sunrise Clinical Manager, Allscripts, Illinois, USA) and stored in our enterprise data repository and analytics system (SingHealth-IHiS Electronic Health Intelligence System (eHINTS)). eHINTS collects reliable data on patient demographics, laboratories, comorbidities and 30-day postoperative outcomes for patients undergoing surgeries from all surgical subspecialties. It integrates information from multiple healthcare transactional systems including administration, clinical and ancillary systems. Mortality data on the system were synchronised with the National Electronic Health Records, ensuring a near complete follow-up. We excluded patients who underwent cardiac surgery, neurosurgery, transplant and burns surgery, and evaluated only the outcomes of the index surgery for patients who underwent multiple surgeries during the study period. After excluding patients with missing data, the final data set consisted of 79 914 cases (figure 1).

Figure 1

Flow chart of the patient cohort. In total, 100 873 index cases were identified from operating theatre listing. We excluded patients who underwent cardiac surgery, neurosurgery, transplant and burns surgery, and evaluated only the outcomes of the index surgery for patients who underwent multiple surgeries during the study period. After excluding the above cases, 90 785 cases remained for consideration. Of these, 10 871 cases had missing variables and the final number of cases included in our patient cohort for statistical analysis was 79 914. LA, Local Anaesthesia; NES, Neurosurgery.

Procedures and definitions

Data collected include patient demographics as well as preoperative comorbidities and laboratory data (table 1). These are routine clinical and laboratory data that were electronically collected during the preoperative anaesthesia assessment visit. Priority of surgery (emergency or elective) and surgical risk classification were based on the 2014 European Society of Cardiology (ESC) and the European Society of Anaesthesiology (ESA) guidelines16 17. Emergency cases in our hospital are classified as cases requiring operation within 24 hours, and they are further subcategorised into their degree of urgency. Missing data were excluded and complete case analysis was done.

Table 1

Descriptive data for the study, including 30-day mortality, any intensive care unit (ICU) admission and ICU admission >24 hours

	Total cohort N=90 785		Derivation cohort N=63 715		Validation cohort N=27 070
	N	Valid %	N	Valid %	N	Valid %
Age (years)
18–29	11 052	12.2	7746	12.2	3306	12.2
30–49	27 078	29.8	18 871	29.6	8207	30.3
50–64	28 227	31.1	19 832	31.1	8395	31.0
65–74	15 837	17.4	11 209	17.6	4628	17.1
75–84	7256	8.0	5132	8.1	2124	7.8
≥85	1335	1.5	925	1.5	410	1.5
Gender
Female	48 708	53.7	34 081	53.5	14 627	54.0
Male	42 077	46.3	29 634	46.5	12 443	46.0
Race
Chinese	64 861	71.4	45 545	71.5	19 316	71.4
Malay	8979	9.9	6321	9.9	2658	9.8
Indian	8012	8.8	5580	8.8	2432	9.0
Others	8927	9.8	6264	9.8	2663	9.8
ASA Classification
I	22 047	25.6	15 366	25.5	6681	26.1
II	49 435	57.5	34 844	57.8	14 591	56.9
III	13 405	15.6	9372	15.5	4033	15.7
IV–VI	1079	1.3	740	1.2	339	1.3
Anaemia
None	62 878	72.5	44 316	72.7	18 562	72.0
Mild	13 006	15.0	9089	14.9	3917	15.2
Moderate/severe	10 863	12.5	7555	12.4	3308	12.8
RDW
>15.7	8478	10.0	5855	9.9	2623	10.4
≤15.7	76 069	90.0	53 535	90.1	22 534	89.6
Grade of CKD
G1	47 948	60.0	33 653	59.9	14 295	60.1
G2	23 635	29.6	16 603	29.5	7032	29.6
G3	5114	6.4	3657	6.5	1457	6.1
G4–G5	3258	4.1	2274	4.0	984	4.1
CVA
Present	1543	2.5	1068	2.4	475	2.5
IHD
Present	4245	6.8	2976	6.8	1269	6.8
CHF
Present	787	1.2	544	1.2	243	1.3
DM on insulin
Present	2003	3.1	1411	3.1	592	3.1
Surgical risk
Low	48 049	52.9	33 715	52.9	14 334	53.0
Moderate	39 014	43.0	27 402	43.0	11 612	42.9
High	3722	4.1	2598	4.1	1124	4.2
Priority of surgery
Elective	72 331	79.7	50 791	79.7	21 540	79.6
Emergency	18 454	20.3	12 924	20.3	5530	20.4
30-day mortality	539	0.6	374	0.6	165	0.6
ICU admission	1799	2.0	1232	1.9	567	2.1
ICU admission >24 hours	1145	1.3	770	1.2	375	1.4

Valid % = % of cases without missing data.

ASA, American Society of Anaesthesiologists’ physical status score; CHF, congestive heart failure; CKD, chronic kidney disease; CVA, cerebrovascular accident; DM, diabetes mellitus; IHD, ischaemic heart disease; RDW, red cell distribution width.

Preoperative laboratory results including full blood count (FBC) and renal panel (RP) were taken as the latest blood results within 90 days before the surgery, and up to the day of surgery but before the start time of surgery. These results include preoperative haemoglobin, red cell distribution width levels and serum creatinine levels. The presence of anaemia was defined by WHO’s gender-based classification of anaemia severity.18 Mild anaemia was defined as haemoglobin concentration of 11–12.9 g/dL in males and 11–11.9 g/dL in females; moderate anaemia was defined for both genders to be haemoglobin concentration between 8–10.9 g/dL and severe anaemia defined as haemoglobin concentration <8.0 g/dL. Pre-existing chronic kidney disease, if present, is graded based on the estimated glomerular filtration rate by the Modification of Diet in Renal Disease equation according to 2012 Kidney Disease: Improving Global Outcomes (KDIGO) guidelines.19 Red cell distribution width (RDW) is reported as a coefficient of variation (percentage) of red blood cell volume with the normal reference range for RDW in this hospital laboratory to be 10.9%–15.7%. Levels >15.7% were defined a priori as high RDW, and this corresponded to the 89th centile of RDW values in our study population. A high RDW has been shown to be associated with an increased risk of mortality that is independent of the severity of anaemia.20 The chosen cut-off value of 15.7% was shown to have a sensitivity of 39.5%, specificity of 89.3%, positive predictive value of 5.3% and negative predictive value of 99.0%.13 The individual components of revised cardiac risk index were defined as per the original study by Lee et al,21 and the ASA status follows that of the ASA-PS definitions.22

Statistics

All analyses were performed using IBM SPSS V.21.0. The data set (90 785 subjects) was randomly divided to 70%:30%, with the former used as the derivation cohort and the latter as validation cohort.

The Combined Assessment of Risk Encountered in Surgery (CARES) surgical risk calculator was developed using ORs of the risk variables obtained from the logistic regression for postsurgical 30-day mortality or ICU >24 hours within the admission. The ORs were assigned rank scores. The model was then validated on the 30% cohort. The Hosmer-Lemeshow (H-L) test for calibration was used to show the goodness of fit for the models developed.

In the initial development of the model, we looked at each outcome individually before combining the significant variables to create a combined risk prediction model for both mortality and need for postoperative ICU stay of >24 hours (ICU >24 hours). For the initial analysis, we included potentially significant variables for each outcome and performed stepwise multivariate logistic regression to obtain the minimum number of variables that retained the accuracy of prediction. The accuracy of prediction was estimated using the receiver operating curve (ROC) and the area under the receiver operating curve (AUROC).

The significant variables for each outcome were then combined to construct a single risk prediction model. We tested the accuracy of the model in predicting either mortality or ICU >24 hours using the ROC and AUROC. H-L tests were performed to show goodness of fit for the model. The performance of the CARES surgical risk calculator was then compared with that of the ASA-PS and the ASA-PS with propensity scoring to adjust for possible differences in the development of the models. Statistical significance was set at P<0.05.

Results

From a total of 90 785 cases, 63 715 (70%) patients were randomly selected into the derivation cohort and the remaining 27 070 (30%) into the validation cohort. Observed mortality in the derivation and validation cohorts were similar, at 0.6%. 1.2% (770 patients) in the derivation cohort were admitted to ICU for more than 24 hours, while that in the validation cohort is 1.4% (375) patients. Descriptive data for these cohorts are summarised in table 1.

Model development and derivation

Mortality outcome

For mortality, 12 variables which were found to be significant in univariate analysis (age, surgical risk, race, anaemia, chronic kidney disease, RDW, presence of cerebrovascular accident, ischaemic heart disease, congestive heart failure, insulin-requiring diabetes mellitus, ASA status and gender) were included. The AUROC (the highest ever AUC to be obtained for this set of data) for this 12-variable black-box model was 0.931 (0.916–0.946). Stepwise logistic regression retained only seven significant variables (table 2) which produced the same AUROC of 0.931 (0.915–0.947) of the 12-variable black-box model.

Table 2

Seven significant variables following stepwise logistic regression for mortality outcome

Variable	OR	Rank score
Age (years)
30–49	3.356	3
50–74	10.482	5
75–84	16.365	6
>85	36.712	8
Surgical risk (moderate/severe)	2.204	2
Anaemia status
Mild	1.448	1
Moderate/severe	2.598	3
RDW>15.7	2.374	2
Ischaemic heart disease	2.066	2
ASA Classification
3	4.582	4
4	19.645	7
Emergency surgery	3.068	3

ASA, American Society of Anaesthesiologists; RDW, red cell distribution width.

Both the ORs and rank scores are presented in table 2. While rounded ORs are commonly used for risk prediction models, some of our ORs are too high and may skew the final score. Hence, to handle these ‘extreme scores’, a ranking system was developed for scoring in the calculator. Additionally, this final rank-based model facilitates utilisation of the risk calculator by keeping it simple enough for regular clinical consult use. This mortality model with an AUROC of 0.928 (0.912–0.945), compared with the original AUROC of 0.931 (0.915–0.947) from the OR-based model, does not compromise accuracy of the risk prediction and at the same time offers increased usability (even in health systems without regular electronic medical records use). The AUROC are shown in online supplementary appendix figure 1.

The H-L test for calibration demonstrated good fit for the final model (P=0.79) predicting postoperative mortality for both the derivation and validation cohort (online supplementary appendix figures 2 and 3).

ICU>24-hour outcome

For the ICU >24- hour outcome, 13 variables which were significant in the univariate analysis (age, surgical risk, race, anaemia, RDW, presence of cerebrovascular accident, ischaemic heart disease, congestive heart failure, chronic kidney disease, insulin-requiring diabetes mellitus, ASA status, surgical priority and gender) were included. The AUROC (the highest ever AUC to be obtained for this set of data) for this 13-variable black-box model was 0.876 (0.861–0.890). Stepwise logistic regression retained seven significant variables (age, surgical risk, anaemia, congestive heart failure, ASA status) with AUROC of 0.873 (0.858–0.888). Table 3 shows the seven significant variables and their corresponding OR and rank scores.

Table 3

Seven significant variables following stepwise logistic regression for intensive care unit >24-hour outcome

Variable	OR	Rank score
Age (years)
30–49	1.134	1
50–74	1.731	2
75–84	2.009	2
≥85	1.548	2
Surgical risk (moderate/severe)	5.207	3
Anaemia status
Mild	1.352	1
Moderate/severe	1.588	2
ASA Classification
3	5.199	3
4	29.481	4
Emergency surgery	1.660	2
Male gender	1.322	1
Congestive heart failure	1.465	1

ASA, American Society of Anaesthesiologists.

For the similar reasons as with the mortality model above, a risk score model using the rank of the ORs was developed, with an AUROC of 0.867 (0.852–0.882), retaining accuracy of the risk prediction. The AUROC are shown in the appendix (online supplementary appendix figure 4).

The H-L test for calibration showed good fit for the final model (P=0.81) predicting need for ICU stay >24 hours in both the derivation and validation cohorts (online supplementary appendix figures 5 and 6).

Combined modelling for both mortality and ICU

To further increase the ease of use for CARES surgical risk calculator, we explored a combined model that is accurate in predicting both mortality and morbidity with just a single set of variables. The above results of separate modelling for each outcome predictions demonstrated robust and accurate individual model in predicting respective outcomes. We now combine the predictors to create a single model. This model consists of nine variables: age, surgical risk, anaemia, RDW, ischaemic heart disease, ASA, surgical priority, gender and presence of congestive heart failure.

The combined predictors are shown in table 4.

Table 4

Final Combined Assessment of Risk Encountered in Surgery model, combining predictors for both mortality and intensive care unit (ICU) admission

Variable	Mortality		ICU		Combined
	OR	Rank score	OR	Rank score	Rank score sum
Age (years)
30–49	3.015	3	1.089	1	4
50–74	9.050	5	1.635	2	7
75–84	14.481	6	1.918	2	8
≥85	34.232	8	1.643	2	10
Surgical risk (moderate/severe)	2.159	2	4.788	3	5
Anaemia status
Mild	1.352	1	1.411	1	2
Moderate/severe	2.926	3	1.608	2	5
RDW>15.7	2.160	2	1.248	1	3
Ischaemic heart disease	1.955	2	1.095	1	3
ASA Classification
3	4.463	4	4.786	3	7
4	18.010	7	26.832	4	11
Emergency surgery	2.897	3	1.782	2	5
Male gender	1.198	1	1.335	1	2
Congestive heart failure	1.281	1	1.408	1	2

ASA, American Society of Anaesthesiologists; RDW, red cell distribution width.

These OR-based nine variables yielded an AUROC of 0.936 (0.920–0.953) for mortality and 0.874 (0.859–0.889) for ICU. Using the rank scores, the AUROC are 0.934 (0.917–0.950) and 0.863 (0.848–0.878) for mortality and ICU, respectively, which again show that accuracy was not compromised. The ROCs are shown in figure 2. The corresponding H-L plots are available in the appendix (online supplementary appendix figures 7 and 8).

Figure 2

Receiver operative curves (ROCs) for mortality and intensive care unit (ICU) >24-hour outcomes in the derivation cohort when the combined model was used to predict the above outcomes. These combined OR model yielded an area under the ROC (AUROC) of 0.936 (0.920–0.953) for mortality and 0.874 (0.859–0.889) for ICU. Using the rank scores, the AUROC are 0.934 (0.917–0.950) and 0.863 (0.848–0.878) for mortality and ICU, respectively, which again show that accuracy was not compromised.

We tested the rank score model on the validation cohort. The AUROC was 0.934 (0.912–0.956) for mortality and 0.837 (0.808–0.868) for ICU >24 hours. The ROCs are shown in figure 3. The corresponding H-L plots are also shown in the appendix (online supplementary appendix figures 9 and 10).

Figure 3

Receiver operative curves (ROCs) for mortality and intensive care unit (ICU)>24-hour outcomes in the validation cohort when the combined model was used to predict the above outcomes. The area under the ROC (AUROC) was 0.934 (0.912–0.956) for mortality and 0.837 (0.808–0.868) for ICU>24 hours.

The cumulative rank scores were subsequently categorised into different bands classifying the risk of mortality as low, low–moderate, moderate–high or high, and a corresponding mortality probability was assigned to each band (table 5). Clinical decision-making suggestions are included in the table.

Table 5

Risk categories and clinical decision making

Risk	Cumulative rank score	30-day mortality risk (%)	Risk of postoperative ICU stay>24 hours	Suggestions for clinical decision making
Low	0–10	0	0.1	Proceed with surgery
Low–moderate	11–20	0.2	0.9	Search for modifiable risk factors and optimise if possible
Moderate–high	21–30	1.9	4.9	As above and arrange for appropriate postoperative monitoring/clinical care Plan for possible ICU admission/high dependency care
High	>30	11.5	14.9	As above and consider alternative surgical or non-surgical options if appropriate Strongly consider not proceeding without availability of ICU bed

ICU, intensive care unit.

Applicability of the CARES model for combined mortality and ICU prediction

We provide a hypothetical example of a patient to illustrate the application of the model. A 76-year-old Chinese man with history of hyperlipidaemia, hypertension, previous ischaemic heart disease with stent inserted with no evidence of congestive heart failure is scheduled for an elective right hemicolectomy for colorectal cancer. FBC shows a starting haemoglobin of 12 and an RDW of 12.1.

Using the scoring method, the patient would have a combined rank score of 8+5+2+3+7+2=28, which places him in the moderate–high-risk category predicting a 1.9% risk of 30-day postsurgical mortality and 4.9% risk of need for postoperative ICU stay for >24 hours.

Using these figures, we would be able to counsel the patient appropriately and plan for optimisation of modifiable risk factors preoperatively and appropriate perioperative monitoring and surveillance.

Comparison with ASA-PS

When the performance of combined model CARES was compared with both the ASA-PS and its propensity model, CARES performed better than both with an AUROC of 0.934 (0.917–0.950) for mortality and 0.863 (0.848–0.878) for ICU >24 hours (see table 6).

Table 6

Area under the receiver operating curve (AUROC) comparison between Combined Assessment of Risk Encountered in Surgery (CARES) and American Society of Anaesthesiologists (ASA) for mortality and intensive care unit (ICU) >24 hours

Model	AUROC (95% CI) for mortality	Standard error*	AUROC (95% CI) for ICU>24 hours	Standard error*
CARES	0.934 (0.917 to 0.950)	0.009	0.863 (0.848 to 0.878)	0.008
ASA	0.871 (0.846 to 0.907)	0.013	0.772 (0.754 to 0.791)	0.009
ASA-propensity	0.879 (0.851 to 0.846)	0.014	0.763 (0.744 to 0.783)	0.010

The corresponding ROCs are shown in online supplementary appendix figure 11.

*Under non-parametric assumption.

Discussion

We developed and internally validated a simple, locally derived CARES surgical risk calculator comprising nine preoperative variables to predict both 30-day mortality and need for ICU (ICU admission for >24 hours), respectively, in adults undergoing non-cardiac and non-neurological surgery. The nine variables are age, gender, ASA status, surgical priority, surgical risk, presence of anaemia, RDW, presence of ischaemic heart disease and congestive heart failure. The web-based version of this risk calculator is currently under construction.

Presently, the use of preoperative risk stratification systems is not routinely done due to combination of factors such as the complexity of the system, the need for additional non-routine preoperative tests, use of intraoperative and postoperative variables and the inability to calculate an individual percentage mortality and morbidity risk.23 The CARES surgical risk calculator comprises simple and easily accessible variables available from routine preoperative evaluation for most surgeries, with the exception of healthy patients coming for the lowest risk surgeries, which may not even need an FBC.24 The inclusion of more variables would greatly increase the time taken to collect data and thereby reduce the usability of the tool. Ease of use and face validity are two important factors that may encourage widespread, routine use of risk prediction tools,25 both of which are true for the CARES surgical risk calculator. The CARES surgical risk calculator stood out from most other risk stratification tools not only in that it was derived from the intended Southeast Asian population but it also predicts both postoperative mortality and the need for postoperative ICU care using a single set of easily accessible variables. This provides convenience and improves efficiency to the clinician in a busy outpatient setting.

The prediction of postsurgical 30-day mortality risk is an important clinical outcome that is of interest to both surgeons and patients and therefore aids in shared decision making.26 Thirty-day all-cause postoperative mortality is a widely accepted, valid and relevant outcome measure of surgical care.27 For this study, the postoperative mortality data were synced with the National Registry of Death data, ensuring the integrity and completeness of the data.

The prediction of the risk of ICU admission for >24 hours postoperatively is novel and not available for most current risk stratification tools. The capability to predict ICU admission for >24 hours after surgery would aid physicians to determine the postoperative patient disposition plan before surgery and therefore healthcare resource allocation. This could improve patients’ outcome by reducing failure to rescue events28 and improve the efficiency of the valuable ICU bed allocation. The disposition of a patient immediately after surgery usually depends on both objective and subjective factors. Objective variables include patient comorbidities, preoperative evaluation, surgical risk and intraoperative haemodynamic. Subjective factors are usually operator-dependent and involve the clinical judgement and comfort level of the anaesthetist and surgeon involved.29 30 While an ICU admission by itself would not be a useful measure of morbidity, length of stay in ICU may be seen as an indirect measure of morbidity-related outcome.31 We defined ICU admission for >24 hours as a significant outcome on observation that patients who were discharged from ICU within the first 24 hours may have been safely monitored postoperatively in a lower intensity unit.

CARES reflects the critical significance of age as a predictor of risk. Age is a significant independent predictor, and this should be reflected in the risk counselling. This will help to increase the awareness of the impact of ageing on mortality among both the clinicians and patients and their careers.

Despite being a single-centre, locally derived risk stratification tool, CARES has a number of advantages. It is the largest study to develop and validate a surgical risk stratification tool in the heterogeneous, multiracial cohort of patients undergoing non-cardiac surgery in the Southeast Asia region. The CARES tool is a parsimonious model, consisting of only 9 preoperative variables, of which includes 7 clinical variables and 2 simple preoperative blood tests which are used to calculate both mortality and ICU admission risk, compared with 18 preoperative, intraoperative and postoperative variables for POSSUM and 22 preoperative patient risk factors for the ACS-NSQIP model.8 32 Despite the small number of variables used to compute surgical mortality and morbidity risk, CARES demonstrated high performance in our population cohort for both outcomes. This may be due to the use of biomarkers, which could increase the performance of risk prediction tools. Our CARES model incorporated the use of RDW which is obtained from the almost routine preoperative FBC test. Recent studies have shown the value of RDW in predicting postoperative mortality.13 14 20 33

In the development of our risk prediction tool, we focused on the utility and usability of the tool. We computed the ORs of each variable category and assigned a rank score to each of them. The cumulative rank scores were matched to corresponding risks for the outcomes. This resulted in a more user-friendly calculator, without compromising on the accuracy. This could lead to better physician and patient acceptance in using the tool for shared decision making as well as healthcare resource planning (see tables 2 and 5).

The ASA-PS is one of the most commonly used risk prediction tools currently in Singapore. We compared the AUROC of the CARES surgical risk calculator with that of ASA-PS and found that CARES showed considerable superiority in performance in both the mortality and need for ICU prediction. With easily available preoperative data, good accuracy and the availability of a web-based calculator, it is hoped that adoption of the CARES tool in calculating preoperative surgical risk may exceed that of other models.

The limitations for this study include the inherent nature of retrospective data.

In our data collection, we also did not include the presence of chronic obstructive pulmonary disease (COPD) as a variable for the development of our risk stratification tool. COPD itself has been not been conclusively shown to be strongly associated with increased postoperative mortality in non-cardiac or non-thoracic surgeries.34 35 One major weakness of our study is the lack of proper definition in our classification on the priority of surgery. As this is a retrospective study, we were unable to give an a priori definition of emergency surgery. We classified the priority of surgery according to that recorded in surgical records. However, in our centre, emergency surgery is further subdivided into four categories, with category A being surgeries that require operation within 6 hours of admission and category D being surgeries that can wait up to 24–48 hours. While the Surgical Risk Tool36 uses the National Confidential Enquiry into Perioperative Deaths classification of surgical priority which differentiates between scheduled, elective, urgent and emergency surgeries,6 the lack of discrimination in the priorities of our emergency surgery data contributes to possible classification bias.

Furthermore, while the CARES tool is based on a large data set, it has only been validated in a single centre, hence, there is a need for further prospective studies to validate the tool in other institutions across the region. Despite this limitation, the sample analysed was representative of the local population and the demographics of hospital admissions in Singapore37 38 The results are generalisable owing to the broad representation of the range of surgical specialties. It predicts both postoperative mortality risk and need for ICU in our local population and has the potential to be a specific risk prediction tool in the Southeast Asian population.

The CARES surgical risk calculator has many firsts. Besides being the first locally derived preoperative risk prediction tool and the first calculator to predict both mortality and ICU need after surgery, it is also the first risk stratification tool to incorporate the use of RDW. The generalisability of the model in international cohorts remains unknown and needs to be explored. External validation of the CARES tool is integral to test its validity further, as is the periodic recalibration and re-evaluation of the model to maintain validity with healthcare advancements, and demonstrate both utility and accuracy in predicting mortality and morbidity. The CARES tool has a potential to be expanded to be used for prediction for other postoperative morbidities, and further prospective studies should be focused on this. Likewise, studies evaluating the impact of risk stratification on improving patient outcomes through individual care planning should be a research priority as there is an opportunity to improve outcomes substantially.

Conclusion

The CARES tool provides an accurate prediction for mortality and need for postoperative ICU among surgical patients in Singapore. It is easily accessible and should be used in conjunction with clinical judgement to aid shared decision making and plan for ICU resource allocation. The development of the web-based calculator further facilitates user accessibility and utility of the tool.