Can we predict postoperative complications in elderly Chinese patients with hip fractures using the surgical risk calculator?

PURPOSE: Hip fractures are associated with poor prognosis in elderly patients partly due to the high rate of postoperative complications. This study was aimed to investigate whether the surgical risk calculator is suitable for predicting postoperative complications in elderly Chinese patients with hip fractures.
METHODS: The incidence of postoperative complications among 410 elderly patients with hip fractures was predicted by the surgical risk calculator and then compared with the actual value. The risk calculator model was evaluated using the following three metrics: Hosmer-Lemeshow test for the goodness-of-fit of the model, receiver operating characteristic curve (ROC) (also referred as C-statistic) for the predictive specificity and sensitivity, and the Brier's score test for predictive accuracy.
RESULTS: Preoperative risk factors including gender, age, preoperative functional status, American Society of Anesthesiologists grade, hypertension, dyspnea, dialysis, previous cardio-vascular history, and cerebrovascular disease were positively correlated with the incidence of postoperative complications in elderly patients with hip fractures. The predicted complication incidence rate was well matched with the actual complication rate by Hosmer-Lemeshow test. The model had high sensitivity and specificity for predicting the mortality rate of these patients with a C-statistic index of 0.931 (95% CI [0.883, 0.980]). The surgical calculator model had an accuracy of 90% for predicting the reoperation rate (Brier's score <0.01).
CONCLUSIONS: The surgical risk calculator could be useful for predicting mortality and reoperation in elderly patients with hip fracture. Patients and surgeons may use this simple calculator to better manage the preoperative risks.

Introduction

Approximately 1.6 million hip fractures occur annually around the world, and this number is increasing by 25% every 20 years due to population growth.1 From 1990 to 2006, the number of hip fractures increased by 2.76 times in females and 1.61 times in males in the ≥50-year-old population in China.2 Due to the rapid aging of the population, this number is expected to continue growing.3 The poor prognosis of hip fracture in elderly patients is related to the high incidence of postoperative complications. One-third of elderly patients with hip fractures died within 1 year after injury, resulting in a heavy financial burden for individuals and society.1,4–6 Therefore, a method is urgently needed by which clinicians can accurately and objectively evaluate and predict the incidence of postoperative complications and reduce the perioperative risk in elderly patients with hip fractures.

The American College of Surgeons National Surgical Improvement Program (ACS-NSQIP) Surgical Risk Calculator was developed in 2013.7 Personal data from the preoperative period to 30-day post-operative complications were collected from >1.4 million patients from 393 hospitals in the USA between 2009 and 2012. The calculator was produced using the database generated from these high-quality data.8 Surgeons can calculate the chance of postoperative complications for a patient given the Current Procedural Terminology code (CPT code) and the patient’s preoperative data.

Several orthopedic-related articles have applied the ACS-NSQIP surgical risk calculator to identify preoperative risk factors that increase the risk of postoperative complications in patients.9,10 However, these studies mostly focused on a single risk factor. A recent study evaluated the ability of the surgical calculator to predict complications within 30 days in patients with full-hip arthroplasty.11

In the present study, we identified the preoperative risk factors, compared the predicted and actual postoperative complication rates, and evaluated the goodness-of-fit, sensitivity, specificity, and accuracy of the prediction model in a Chinese population of elderly patients who underwent surgical treatment for hip fracture.

Materials and methods

This study retrospectively analyzed 417 cases of hip fracture in elderly patients who underwent surgical treatment in our hospital from January 2014 to December 2016. Seven cases were lost during the follow-up. The inclusion criteria were age ≥60 years, unilateral femoral neck fracture, and artificial femoral head replacement (CPT code 27,125). The exclusion criterion was a Glasgow Coma scale <3. This study was approved by the ethics committee of Beijing Shijitan Hospital, affiliated to Capital Medical University. All procedures performed in studies involving human participants were in accordance with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. Written informed consent was obtained from patients or their legal guardians.

A total of 21 preoperative entries, including the patient’s basic profile (age, gender, weight, height, functional status, and smoking history), comorbidities (American Society of Anesthesiologists grade, hypertension, diabetes, congestive heart failure, ascites, hormone therapy, COPD, dialysis, renal failure, sepsis, ventilation support, and metastatic cancer), and the nature of the procedure (CPT code, emergency surgery or elective surgery), were entered into the surgical risk factor calculator. Body mass index (BMI, kg/m2) was divided into six categories: underweight (BMI <18.5), normal (18.5≤ BMI ≤25), overweight (25< BMI ≤30), grade 1 obesity (30< BMI ≤35), grade 2 obesity (35< BMI ≤40), and grade 3 obesity (BMI >40).

After entering the patient’s data into the calculator, the predicted incidence of complications within 30 days after surgery was calculated. Postoperative complications defined by this calculator include “major complications” (cardiac arrest, myocardial infarction, pneumonia, progressive renal insufficiency, acute renal failure, pulmonary embolism, deep venous thromboembolism [DVT], reoperation, deep incisional surgical site infection [SSI], organ space SSI, systemic sepsis, unplanned intubation, urinary tract infection [UTI], wound disruption) and “any complication” (superficial incisional SSI, deep incisional SSI, organ space SSI, wound disruption, pneumonia, unplanned intubation, pulmonary embolism, DVT, ventilator use >48 h, progressive renal insufficiency, acute renal failure, UTI, stroke, cardiac arrest, myocardial infarction, reoperation, systemic sepsis). All of the predicted risk ratings in this study were set as “1”. The predicted incidence of postoperative complications was compared with the observed values to determine the accuracy of this calculator in the population of elderly Chinese patients with hip fractures.

Statistical analysis

Statistical analysis was performed using Microsoft Excel (Redmond, WA, USA) and SPSS 22.0 (SAS Institute, Cary, NC, USA). Quantitative data are presented as average ± SD. The predicted values and the actual values of the count data were analyzed using paired t-test. All preoperative risk factors were separately compared for each major complication and any complication incidence rate using bivariate correlation analysis. P<0.05 was considered statistically significant. The Hosmer–Lemeshow test was used to evaluate the goodness-of-fit for the logistic regression model, and P>0.05 was considered good fit. The receiver operating characteristic curve (ROC) (also referred as C-statistic) was used to determine the predictive sensitivity and specificity of the model. When the C-statistic is >0.83, its predictive ability is considered as mid range, indicating that the model has a good ability to predict the incidence of complications, and when the C-statistic is <0.5, the model is considered not specific or sensitive.12 The Brier’s score is defined as the average squared difference between the patient’s predicted probabilities and observed outcomes (1 or 0 depending on event or non-event). If the predicted value is close to the observed value, the Brier’s score approaches 0.0.7,13,14

Results

A total of 410 patients underwent artificial femoral head replacement, including 132 men and 278 women, with an average age of 80.32±7.514 years (range, 60–100 years). The basic characteristics of the patients are summarized in Table 1. Bivariate correlation analysis showed that gender, age, preoperative functional status, ASA grade, hypertension, dyspnea, dialysis, previous cardiovascular history, and cerebrovascular disease were associated with postoperative complications (P<0.05). Although age, preoperative functional status, ASA grade, dyspnea, and cerebrovascular disease were associated with a higher incidence of postoperative complications, none of them were significantly correlated (Figure 1).

Table 1

Patient demographics and risk factors as recorded in the risk calculator and bivariate analysis for complications

Characteristic	Cases, n (%)	Postoperative complications		P-value
		No	Yes
Gender				0.028*
Male	132 (32.2)	68	64
Female	278 (67.8)	175	103
Age (years)				0**
60–64	14 (3.4)	12	2
65–74	65 (15.9)	50	15
75–84	214 (52.2)	130	84
≥85	117 (28.5)	51	66
BMI (kg/m2)				0.134
Underweight	51 (12.4)	30	21
Normal	238 (58.1)	133	105
Overweight	98 (23.9)	64	34
Obese grade 1	23 (5.6)	16	7
Obese grade 2	0	0	0
Obese grade 3	0	0	0
Functionally independent				0**
Yes	218 (53.2)	165	53
No	192 (46.8)	78	114
Emergency surgery				0.898
Yes	136 (33.2)	80	56
No	274 (66.8)	163	111
History of severe COPD				0.508
Yes	12 (2.9)	6	6
No	396 (96.6)	237	161
ASA class				0**
1	1 (0.2)	1	0
2	79 (19.3)	70	9
3	286 (69.8)	159	127
4	44 (10.7)	13	31
Chronic steroid use				0.477
Yes	23 (5.6)	12	11
No	387 (94.4)	231	156
Ventilator dependent				N/A
Yes	0	0	0
No	410 (100)	243	167
Metastatic cancer				0.962
Yes	10 (2.4)	6	4
No	400 (97.6)	237	163
Diabetes				0.353
Yes	139 (33.9)	78	61
No	271 (66.1)	165	106
Hypertension				0.025*
Yes	261 (63.7)	144	117
No	149 (36.3)	99	50
Dyspnea				0**
Yes	64 (15.6)	23	41
No	346 (84.4)	220	126
Acute renal failure				N/A
Yes	0 (0)	0	0
No	410 (100)	243	167
Ascites within 30 days				0.228
Yes	1 (0.2)	0	1
No	409 (99.8)	243	166
Systemic sepsis within 48 h				N/A
Yes	0 (0)	0	0
No	410 (100)	243	167
Dialysis				0.022*
Yes	9 (2.2)	2	7
No	401 (97.8)	241	160
Current smoker within 1 year				0.438
Yes	69 (16.8)	38	31
No	341 (83.2)	205	136
Congestive heart failure within 30 days				0.072
Yes	20 (4.9)	8	12
No	390 (95.1)	235	155
Previous cardiac event				0.024*
Yes	174 (42.4)	92	82
No	236 (57.6)	151	85
Preexisting cerebrovascular disease				0**
Yes	121 (29.5)	54	67
No	289 (70.5)	189	100

Notes:

P<0.05;

P<0.01.

Abbreviations: ASA, American Society of Anesthesiologists; BMI, body mass index; N/A, not applicable.

Figure 1

Correlation between preoperative risk factors and postoperative complication rates.

The actual incidence rate of complications within 30 days of surgery is summarized in Table 2. The incidence rates of major complications and any complications were 32.7% and 40.7%, respectively. Complications with a high incidence rate included pneumonia (23.7%), heart complications (17.1%), and UTIs (10.2%), and others with a lower complication rate were thrombosis (4.6%), renal failure (4.6%), rehospitalization (4.6%), SSI (1.5%), mortality (1.5%), and reoperation (0.5%). The predicted average incidence rates of major complications and any complications were 13.26%±6.05% and 14.06%±6.58%, respectively. There was a significant difference between the actual incidence rates of major complications and any complications (P<0.01). The mean values of hospitalization days and actual hospitalization days were 5.74±1.92 days and 19.65±16.31 days, respectively (P<0.01).

Table 2

Event occurrence rates, Hosmer–Lemeshow test values, and C-statistic analysis results for postoperative complications

Outcomes	Events, n (%)	Hosmer–Lemeshow test P-value	C-statistic (95% CI)
Major complication	134 (32.7)	0.405	0.706 (0.652, 0.759)
Any complication	167 (40.7)	0.186	0.729 (0.680, 0.778)
SSI	6 (1.5)	0.319	0.579 (0.357, 0.800)
Pneumonia	97 (23.7)	0.609	0.746 (0.690, 0.801)
Cardiac complication	70 (17.1)	0.527	0.704 (0.634, 0.774)
UTI	42 (10.2)	0.384	0.631 (0.550, 0.713)
Venous thrombosis	19 (4.6)	0.411	0.641 (0.500, 0.782)
Renal failure	19 (4.6)	0.058	0.791 (0.669, 0.913)
Readmission	19 (4.6)	0.451	0.691 (0.573, 0.810)
Reoperation	2 (0.5)	0.604	0.803 (0.729, 0.878)
Death	6 (1.5)	0.557	0.931 (0.883, 0.980)

Abbreviations: SSI, surgical site infection; UTI, urinary tract infection.

The Hosmer–Lemeshow test was used to evaluate the postoperative complication rate and the actual observed value of the surgical risk calculator. Our results showed that the predictive morbidity rate of the surgical risk calculator was well fitted with the actual complication rate (P>0.05; Table 2).

Based on the ROC analysis (Table 2), the C-statistic for mortality was 0.931 (95% CI [0.883, 0.980]), indicating that the model was sensitive and specific for predicting mortality within 30 days postoperation in elderly patients with hip fracture (Figure 2). The C-statistic predicted values for remaining outcomes were <0.83.

Figure 2

The area under the ROC curve for mortality within 30 days postoperation.

Abbreviation: ROC, receiver operating characteristic.

The Brier’s score for the incidence of reoperation was <0.01, suggesting that the surgical calculator model had an accuracy of 90% for predicting reoperation in elderly patients with hip fractures. However, the Brier’s scores for other complications were >0.01, indicating that the predictive accuracy of this model was low for other postoperative complications (Figure 3).

Figure 3

The Brier’s scores for the predicted and observed values of outcome measurements.

Abbreviation: OR, operating room.

Discussion

Hip fracture is a common type of fracture in elderly people, who are at high risk of postoperative pulmonary infection, DVT, and other complications, resulting in a significant economic burden and high mortality. If the patient’s physical condition allows, surgical treatment should be performed as soon as possible. However, elderly patients often have complications such as a variety of internal diseases, and a surgical risk calculator is a simple and cost-effective assessment method that can be helpful since the perioperative assessment of the disease could be difficult. Thus, in this study, we investigated whether the surgical risk calculator is suitable for predicting the incidence of postoperative complications in elderly Chinese patients with hip fractures. Our results suggested that a variety of preoperative risk factors were positively or negatively correlated with the incidence of postoperative complications. The predicted complication incidence rate by surgical risk calculator was well matched with the actual complication rate. The surgical risk calculator showed high sensitivity and specificity for predicting 30-day mortality in elderly patients with hip fractures, and the accuracy of the surgical risk calculator for predicting reoperation was 90%.

In elderly patients, multiple organ dysfunctions bring a greater surgical risk. Unfortunately, this calculator does not include preoperative risk factors for elderly patients with a history of common cerebrovascular diseases and heart diseases. Among the risk factors for additional assessment in this study, 42.4% of the patients had a history of heart disease and 29.5% had cerebrovascular disease. Our data also confirmed that cardiovascular disease history and cerebrovascular disease history were correlated with postoperative complications. The lack of preassessment of these risk factors may be the cause of significant differences between the predicted and actual observed values of postoperative complications and any complication rates, as well as the correlation analysis of preoperative risk factors and postoperative complication rates. In addition, one of the common complications in surgical treatment, especially orthopedic surgery, is thrombosis,15,16 and preoperative risk factors in the surgical risk calculator do not include thrombosis-related risk factors, such as the international normalized ratio value, which may also reduce the predicted incidence rate of venous thrombosis in elderly patients postsurgery. Arce et al also noted that several previously identified variables were not encompassed in the surgical risk calculator, which could explain the failure of this calculator to predict the risk of 30-day postoperative complications in a cohort of patients who underwent head and neck reconstruction surgery.17

Similar to the previous finding that this universal risk calculator lacks validity in arthroplasty, pulmonary surgery, and orthopedic surgery,11,17–19 our study also suggested that the mean incidence rates of postoperative major complications and any complication were significantly different from their respective actual observed values. However, the Hosmer–Lemeshow test showed that the morbidity rate predicted by the surgical risk calculator fit well to the actual complication rate. Moreover, C-statistic analysis showed that this model was good at predicting mortality, and Brier’s score showed 90% accuracy for predicting reoperation in the elderly patients with hip fractures. These contradictions suggest that the sensitivity, specificity, and accuracy of the surgical risk calculator for predicting the incidence of other postoperative complications of hip fracture are not good enough. The possible explanations are as follows. First, this calculator does not include preoperative risk factors for elderly patients with a history of common cerebrovascular diseases and heart diseases or risk factors for orthopedic surgeries.20,21 Second, there is variation among surgeons for evaluation of the risk factor ratings due to the lack of preoperative risk factors in the calculator. There could be Class 1 or Class 2 standard deviation even if adjustment is applied.8 Future studies with further subdivision of the risk rating would result in a more objective evaluation of the model. Finally, there was a significant difference between the predicted number of days of hospitalization and the actual number of days of hospitalization, which could be related to the differences between countries. In China, patients need to be hospitalized until the removal of stitches, which could contribute to the low hospital readmission rate, because it is easy to notice and immediately treat any postoperative complication during the weeks of hospital stay.

This study has several limitations. First, the elderly patients included in this study were likely critically ill patients, since the authors worked at a large tertiary hospital. This could result in a higher incidence of postoperative complications than the model determination. Second, the sample size in this study was relatively small. Third, there are differences in medical technology level, population characteristics, and patient composition ratio between China and the country in which this surgical risk calculator was developed, and this could result in a contradiction in the evaluation of the predictive ability.

In conclusion, although the predictive accuracy and power of the surgical risk calculator need to be improved, its value as a simple and inexpensive risk assessment tool for patients and doctors in developing countries is undeniable. Future studies with larger sample sizes and more risk stratifications are desired.