A Nomogram for Predicting In-Hospital Major Adverse Cardio- and Cerebro-Vascular Events in Patients Undergoing Major Noncardiac Surgery: A Large-Scale Nested Case-Control Study.

Purpose: Few evidence-based predictive tools are available to evaluate major adverse cardio- and cerebro-vascular events (MACCEs) before major noncardiac surgery. We sought to develop a new simple but effective tool for estimating surgical risk. Patients and
Methods: Using a nested case-control study design, we recruited 105 patients who experienced MACCEs and 481 patients without MACCEs during hospitalization from 10,507 patients undergoing major noncardiac surgery in Beijing Chaoyang hospital. Least absolute shrinkage and selection operator (LASSO) regression and likelihood ratio were applied to screen 401 potential features for logistic regression. A nomogram was constructed using the selected variables.
Results: Chronic heart failure, valvular heart disease, preoperative serum creatinine >2.0 mg/dL, ASA class, neutrophil count and age were most associated with in-hospital MACCEs among all the factors. A new prediction model established based on these showed a good discriminatory ability (AUC, 0.758 [95% confidence interval (CI), 0.708-0.808] and a well-performed calibration curve (Hosmer-Lemeshow χ2 = 7.549, p = 0.479), which upheld in the 10-fold cross-validation (AUC, 0.742 [95% CI, 0.718-0.767]. This model also demonstrated an improved performance in comparison to the modified Revised Cardiac Risk Index (RCRI) score (increase in AUC by 0.119 [95% CI, 0.056-0.180]; NRI, 0.445 [95% CI, 0.237-0.653]; IDI, 0.133 [95% CI, 0.087-0.178]. The decision curve analysis showed a positive net benefit of our new model.
Conclusion: Our nomogram, which relies upon simple clinical characteristics and laboratory tests, is able to predict MACCEs in patients undergoing major noncardiac surgery. This prediction shows better discrimination than the standardized modified RCRI score, laying a promising foundation for further large-scale validation.

Introduction

With the aging of population and the advancement of medical technology, the need for major noncardiac surgery in patients with heart diseases is also increasing in recent years.1 Within the over 200 million patients undergoing major noncardiac surgery in the world each year,2,3 more than 10 million of them suffer from major cardiac complications,4,5 which leads to increased morbidity and mortality.6 Therefore, it is of great importance to evaluate the risk of major adverse cardiovascular events (MACEs) before major noncardiac surgery.

There are three preoperative cardiac risk indexes that are commonly used and endorsed in guidelines: the modified Revised Cardiac Risk Index (RCRI),7 the National Surgical Quality Improvement Program (NSQIP) Myocardial Infarction and Cardiac Arrest (MICA) calculator,8 and the American College of Surgeons NSQIP (ACS NSQIP) Surgical Risk Calculator.9 These indexes are routinely used in clinical practice for predicting the risk of MACEs in patients undergoing noncardiac surgery. However, there are several potential imperfections worth noting as well. First of all, the endpoints evaluated by these indexes do not include cerebrovascular diseases whose incidence has largely increased in recent years.6 Second, previous studies have shown the potential utility of including biomarkers and clinical lab indexes into preoperative risk indices as an approach to identify patients at higher risk of MACEs.10–15 But none of these existing calculators included biomarkers or clinical lab indexes except for serum creatinine. Third, these risk models were developed more than a decade ago, during which the surgical methods as well as medical care has improved by leaps and bounds, along with the characteristics of the disease itself. For example, the modified RCRI score underestimated 50% of the adverse events in recent cohort studies,16,17 which raised concern that these models might no longer be applicable to current patients. Lastly, the ACS NSQIP surgical risk calculator includes 21 patient-specific variables. It is difficult for surgeons to be familiar with each of these variables and to apply them proficiently.

For these reasons mentioned above, there is an increasing need for estimating patient-specific not only MACEs but major adverse cardio- and cerebro-vascular events (MACCEs) in a user-friendly format. In this study, we undertook a retrospective investigation into the characteristics of patients undergoing major noncardiac surgery. We then developed a novel nomogram using selected features and compared this new prediction model to the standardized modified RCRI score to identify patients at risk of MACCEs before major noncardiac surgery.

Materials and Methods

Participants and Variables

We performed a single-center nested case-control study on patients aged ≥ 18 years who underwent major noncardiac procedures at Beijing Chaoyang Hospital from January 1st, 2017 to June 30th, 2021. The study was carried out in compliance with the principles specified for research on patients in the Declaration of Helsinki and the research protocol was approved by the ethics committee of Beijing Chaoyang Hospital (2021-S-476). The need for informed consent was waived due to the retrospective study design and all the data were anonymous.

Exclusion criteria included the followings: 1) patients who underwent emergency surgery, transplant, trauma or low-risk procedures (including breast surgery, dental surgery, endoscopic procedures, ophthalmic surgery, gynecological surgery and plastic surgery based on the American College of Cardiology/American Heart Association guidelines),18 2) patients with American Society of Anesthesiologists classification (ASA class) of V or VI, 3) patients with advanced malignant tumor undergoing palliative surgery, 4) patients with congenital heart disease or cardiomyopathy, 5) patients with a history of other surgeries within 4 months, 6) patients who were submitted to reoperation or reintervention, and 7) patients with incomplete data that were necessary to extrapolate the modified RCRI score.

The endpoint of this study was defined as the occurrence of MACCEs, including all-cause death, acute myocardial infarction, cardiac arrest, heart failure, ventricular fibrillation, complete heart block and ischemic stroke after surgery during hospitalization. The definition of acute myocardial infarction referred to the guideline of the Fourth Universal Definition of Myocardial Infarction.19 Heart failure was defined as acute heart failure or acute exacerbation of chronic heart failure.20 Ischemic stroke was defined as an episode of neurological dysfunction caused by focal cerebral, spinal, or retinal infarction.21 The diagnosis of MACCEs were consulted and confirmed by a cardiologist or neurologist at the time of hospitalization and were further cross-checked by two independent doctors during data examination.

Based on previous studies, potential relevant preoperative and intraoperative variables including demographic information, medical history and laboratory examination were extracted from the database and analyzed.

Statistical Analysis

Continuous variables are presented as median (interquartile range). Categorical variables are expressed as absolute values and percentages. To compare differences between the two groups, we used Mann–Whitney U-test for continuous variables and chi-square test for categorical data.

The adaptive least absolute shrinkage and selection operator (LASSO) was used to identify risk factors of MACCEs in patients undergoing major noncardiac surgery, and the optimal value of λ was determined via 10-fold cross-validation. The area under the receiver operating characteristic (ROC) curve (AUC) was used to assess the performance of the model. Calibration was calculated with the Hosmer–Lemeshow χ2 test and visualized by calibration plot. To favor the clinical implementation and workability of the risk model, we formulated a coefficient-based nomogram. Clinical usefulness and net benefit were estimated by decision curve analysis (DCA). Internal validation was performed using 5 repetitions of 10-fold cross-validation. Using the AUC, the unclassified net reclassification improvement (NRI) and integrated discrimination improvement (IDI), we compared the performance of our model to the modified RCRI score. The difference in the AUC between ROC curves were tested using the DeLong test. Due to lack of ischemic stroke and non-cardiac death in the endpoint events of the modified RCRI score, we also performed a sensitivity analysis after excluding patients who experienced ischemic stroke or non-cardiac death during hospitalization. The p values were 2-sided, and α level of 0.05 was considered statistically significant. All analyses were performed by R software (4.0.2 version).

Results

Patient Characteristics

Of all 23,765 cases with aged ≥ 18 years who underwent major noncardiac surgery from January 1st 2017 to June 30th 2021 in Beijing Chaoyang Hospital Jingxi branch, 13,258 cases fell to the exclusion criteria: 13,215 cases underwent emergency surgery, transplant, trauma or low-risk procedures, 2 cases with congenital heart disease or cardiomyopathy, 18 cases with a history of other surgeries within 4 months, 4 cases were submitted to reoperation or reintervention, and 19 cases with incomplete data that were necessary to extrapolate the modified RCRI score. Of the remaining 10,507 cases, 118 cases experienced in-hospital MACCEs. Thereinto, 13 cases were excluded due to the following reasons: 5 cases with ASA grade V, 7 cases with advanced palliative surgery for malignant tumor, and 1 case died unexpectedly due to irrelevant accident. Thus 105 patients with MACCEs were included in the final analyses. As for the control group, we used stratified randomization to avoid extreme imbalance with the MACCEs group. The remaining 10,389 patients without MACCEs during hospitalization were first divided into five groups (intraperitoneal surgery, thoracic surgery, peripheral arterioplasty, major orthopedic surgery and major urological surgery) according to the operation type of the MACCEs group. Then, at a ratio of 1 to 5, we randomly sampled 525 cases stratified by these five groups as the control group. Among these, 44 cases were excluded due to the following reasons: 27 cases with ASA grade V and 17 cases with advanced palliative surgery for malignant tumors. Finally, 481 patients without MACCEs were included as controls (). Table 1 shows the comparison of characteristics between controls and MACCEs group. Patients with MACCEs were older, had higher ASA class, more ST-T changes on ECG and more medical history of chronic heart failure (CHF), atrial fibrillation(AF), ischemic heart disease, dialysis, hypertension, chronic respiratory disease and valvular heart disease (VHD). Regarding laboratory examination, patients with MACCEs had lower lymphocyte count, higher neutrophil count, N/L% and creatinine. Table 2 shows the end-point events of recruited cases. Among the 105 cases of MACCEs group, AMI occurred in 19 cases (18%), HF in 56 cases (53%), stroke in 18 cases (17%) and 26 cases (25%) died.

Table 1

The Characteristics of Recruited Cases

Characteristics	Controls (n = 481)	MACCEs Group (n = 105)	p value
Sex (Men)	245 (51)	59 (56)	0.385
Age (years)	66 (56–79)	76 (65–84)	< 0.001
HLP	145 (30)	29 (28)	0.692
Diabetes	132 (27)	28 (27)	0.967
Insulin treatment	49 (10)	8 (8)	0.533
HBP	232 (48)	64 (61)	0.024
Stroke	67 (14)	23 (22)	0.057
CHF	6 (1)	16 (15)	< 0.001
AF	23 (5)	23 (22)	< 0.001
Ischemic heart disease	79 (16)	33 (31)	< 0.001
Dialysis	1 (0)	7 (7)	< 0.001
Chronic respiratory disease	29 (6)	13 (12)	0.038
VHD	11 (2)	17 (16)	< 0.001
Disseminated Cancer	22 (5)	7 (7)	0.517
Current smoker within 1 year	112 (23)	26 (25)	0.844
High-risk type of surgery	194 (40)	49 (47)	0.278
General anesthesia	311 (65)	72 (69)	0.515
PLT	210 (168–258)	208 (163–267)	0.629
Neutrophil count	4.17 (3.14–5.9)	4.95 (3.48–6.98)	0.007
Lymphocyte count	1.5 (1.07–1.96)	1.2 (0.86–1.75)	< 0.001
N/L%	2.68 (1.75–4.71)	3.72 (2.58–7.46)	< 0.001
Cr > 2.0 mg/dL	8 (2)	11 (10)	< 0.001
ST-T changes on ECG	109 (23)	40 (38)	< 0.001
Surgery type
Intraperitoneal surgery	227	50
Thoracic surgery	38	8
Peripheral arterioplasty	30	6
major orthopedic surgery (hip joints or spine)	182	37
Major urological surgery	4	4
ASA Class
1	52 (11)	3 (3)	< 0.001
2	330 (69)	48 (46)	< 0.001
3	94 (20)	47 (45)	< 0.001
4	5 (1)	7 (7)	< 0.001
The modified RCRI score
0	183 (38)	19 (18)
1	220 (46)	51 (49)
2	56 (12)	22 (21)
3	19 (4)	9 (9)
4	1 (0)	3 (3)
5	2 (0)	0 (0)
6	0 (0)	1 (1)

Notes: Data present as median (interquartile range) for continuous variables or n (%) for counting data. High-risk type of surgery included intraperitoneal, intrathoracic or suprainguinal vascular procedures according to the modified RCRI score.

Abbreviations: HLP, hyperlipidemia; HBP, high blood pressure; CHF, chronic heart failure; AF, atrial fibrillation; VHD, valvular heart disease; PLT, platelets; N/L%, neutrophil to lymphocyte ratio; Cr > 2.0mg/dL, preoperative serum creatinine > 2.0 mg/dL; ECG, electrocardiogram; RCRI, the modified Revised Cardiac Risk Index.

Table 2

The End-Point Events of Recruited Cases

MACCEs	Controls (n = 481)	MACCEs Group (n = 105)
AMI	0	19(18)
HF	0	56(53)
Death	0	26(25)
Stroke	0	18(17)

Note: Data present as n (%) for counting data.

Abbreviations: MACCEs, major adverse cardiovascular and cerebrovascular events; AMI, acute myocardial infarction; HF, heart failure.

Variable Selection and Nomogram Construction

Using LASSO logistic regression, we screened 20 predictors from 401 potential variables. Then, we calculated the likelihood ratios and selected six strongest predictors: CHF, VHD, Cr > 2.0 mg/dL, ASA class, neutrophil count and age. Among these, CHF referred to patients with insufficient cardiac function at New York Heart Association (NYHA) class III to IV. Valvular heart disease was defined as moderate to severe cardiac valve incompetence or stenosis. The importance of each variable is shown in Figure 1A. Integrating the six variables, we were able to build a nomogram for predicting in-hospital MACCEs in patients underwent major noncardiac surgery (Figure 1B).

Figure 1

Selected variables and the constructed nomogram. (A) Importance of each variable as measured by partial Wald χ2 minus the predictor degrees of freedom. (B) Nomogram for the prediction model. CHF, chronic heart failure; VHD, valvular heart disease; Cr 2.0 > mg/dL, preoperative serum creatinine > 2.0 mg/dL.

Performance of the New Prediction Nomogram

The ROC curve of the new prediction nomogram is shown in Figure 2A with an AUC of 0.758 [95% CI, 0.708–0.808]. The calibration plot of the model showed an overall good agreement between the predicted and observed risks, which was further supported by Hosmer–Lemeshow test (Figure 2B, χ2 = 7.549, p = 0.479). The internal-validated AUC of the model was 0.742 [95% CI, 0.718–0.767] based on 5 repetitions of 10-fold cross-validation.

Figure 2

Comparison between the new prediction nomogram and the modified RCRI score. (A) AUC of the new prediction nomogram (0.758) and the modified RCRI score (0.639). (B) Calibration plot of the new prediction nomogram. (C) Decision curve analysis evaluation of the two models. (D)The ROC curves of the new prediction nomogram in comparison with modified RCRI score in subgroup patients excluded cerebrovascular diseases or non-cardiac death.

In comparison with the modified RCRI score, the prediction model we established demonstrated an increase in AUC of 0.119 [95% CI, 0.056–0.180] (Figure 2A). We also calculated the unclassified NRI and IDI between these two risk models. Compared with the modified RCRI score, the new prediction had significant improvement in reclassification as assessed by the NRI (0.445 [95% CI, 0.237–0.653]) and IDI (0.133 [95% CI, 0.087–0.178]). The decision curve graphically showed a larger net benefit across the range of MACCEs risk of the new prediction nomogram compared with the modified RCRI score (Figure 2C).

Sensitivity Analysis

Since the modified RCRI score did not take ischemic stroke and non-cardiac death into consideration, we performed a sensitivity analysis after excluding 36 patients who experienced ischemic stroke or died due to non-cardiac cause during hospitalization. Compared with the modified RCRI score, our risk model remained a better discrimination with an increase in AUC of 0.159 (Figure 2D, p < 0.0001) and an improvement in reclassification as assessed by the NRI (0.631 [95% CI, 0.385–0.878]) and IDI (0.171 [95% CI, 0.110–0.232]) in these subgroup patients.

Discussion

This study developed a novel nomogram for the prediction of MACCEs in patients undergoing major noncardiac surgery. The nomogram showed good discriminative ability and performed well on calibration plot, with a higher accuracy and better discrimination than the modified RCRI score.

In comparison with previous risk models that predicted adverse events,7,9,22 the new prediction model added the following new factors: history of valvular heart disease and the neutrophil count. Valvular heart disease is common among patients undergoing noncardiac surgery and has been associated with increased perioperative cardiac morbidity and mortality.23 Another reason for including valvular disease in our model is the increasing number of patients with valvular disease brought by aging population and advancing of treatments in recent years. Neutrophils are major and direct effectors of the inflammatory response which contributes to the whole atherosclerotic process, from its initiation, progression, plaque destabilization and ultimately to cardiovascular events.24–26 Therefore, the neutrophil count, as a simple and inexpensive tool, seems to be a factor worth taking into account for prediction of MACCEs in patients undergoing major non-cardiac surgery.

Compared with the modified RCRI score, the final variables of our new prediction nomogram did not include the variable of stroke/TIA, ischemic heart disease (IHD) and insulin therapy for diabetes. However, these factors were already included in the ASA class which was a feature of the new prediction nomogram. High-risk procedures (intraperitoneal, intrathoracic or suprainguinal vascular procedures) in the modified RCRI score were also not a significant influencer in the new prediction nomogram, which might contribute from the significantly reduced perioperative complications of these procedures because of the rapid development of laparoscopic, thoracoscopic and vascular interventional techniques. On the other hand, the inclusion of laboratory tests in the new prediction nomogram attenuated the prognostic importance of many clinical variables.

In contrast to the NSQIP MICA and the ACS NSQIP surgical risk calculator, details of the operation were not part of the new prediction nomogram. From one side, the details and type of operation could change during the operation and sometimes more than two combined operations were performed. On the other side, surgical types and techniques are improving with each passing day. There is no risk estimation tool currently available that covers nearly all operations across multiple subspecialties. Thus, the details of operation may not be reliable representatives of patient characteristics for estimating the risk of MACCEs.

The new prediction model we established was able to evaluate not only MACEs but also cerebrovascular events which were included in any of the previous models. More laboratory tests were incorporated into our model to personalize the evaluation and to meet the standards of precision medicine. In order to derive an index that might be able to facilitate routine practice, the variables we included were easily obtained and well generalized.

There are also limitations to our study. First of all, the prediction model was established based on single-center data. External validation was not performed, which limited the generalization and the evidence level of the results. Secondly, as a retrospective study, we were unable to systematically monitor all biomarkers that might have predictive ability, hence also unable to include them in the predictive model. Thirdly, as a retrospective controlled study, our project did not include all cases without MACCEs, which might increase the risk for selection bias. Therefore, we randomized the enrollment of the cases without MACCEs to minimize this bias as much as possible.

Conclusion

In conclusion, we developed a new and easy-to-implement risk model for the prediction of MACCEs in patients undergoing major noncardiac surgery. This risk model showed better discrimination than the standardized modified RCRI score. It may be a promising decision-aid tool for both surgical patients and surgeons.