A Novel Nomogram for Predicting Morbidity Risk in Patients with Secondary Malignant Neoplasm of Bone and Bone Marrow: An Analysis Based on the Large MIMIC-III Clinical Database.

Objective: Bone and bone marrow are the third most frequent sites of metastases from many cancers and are associated with low survival and high morbidity rates. Currently, there are no effective bedside tools to predict the morbidity risk of these patients in general intensive care units (ICUs). The main objective of this study was to establish and validate a nomogram to predict the morbidity risk of patients with bone and bone marrow metastases.
Methods: Data on patients with bone and bone marrow metastases were extracted from the Medical Information Mart for Intensive Care III (MIMIC-III) database. The patients were divided into training and validation cohorts. The data were analyzed using univariate and multivariate Cox regression methods. Factors significantly and independently prognostic of survival were used to construct a nomogram predicting 30-day morbidity. The nomogram was validated by various methods, including Harrell's concordance index (C-index), area under the receiver operating characteristic curve (AUC), calibration curve, integrated discrimination improvement (IDI), net reclassification index (NRI), and decision curve analysis (DCA).
Results: The study included 610 patients in the training cohort and 262 in the validation cohort. Multivariate Cox regression analysis showed that temperature, SpO2, Sequential Organ Failure Assessment (SOFA) score, Oxford Acute Severity of Illness Score (OASIS), comorbidities with coagulopathy, white blood cell count, heart rate, and respiratory rate were independent predictors of patient survival. The resulting nomogram had good discriminative ability, as shown by high AUCs, and was well calibrated, as demonstrated by calibration curves. Improvements in NRI and IDI values suggested that the nomogram was superior to the SOFA scoring system. DCA curves revealed that the nomogram showed good value in clinical applications.
Conclusion: This prognostic nomogram, based on demographic and laboratory parameters, was predictive of the 30-day morbidity rate in patients with secondary malignant neoplasms of the bone and bone marrow, suggesting its applicability in clinical practice.

Introduction

Bone and bone marrow are the third most frequent sites of tumor metastases, following the lungs and liver.1 Once diagnosed, bone metastases can rarely be cured, indicating a short-term prognosis in cancer patients. Breast and prostate cancers are the most frequent origins of skeletal metastases (over 70%).2 Bone metastases are much more common than primary bone cancers. The median survival times of patients with bone metastases of melanoma, and of lung, bladder, renal, prostate, breast, and thyroid cancers, are low, being 6 months, 6 months, 6–9 months, 12 months, 12–53 months, 19–24 months, and 48 months, respectively.3 Bone and bone marrow metastases are also associated with high morbidity rates, including pathologic fractures, severe pain, impaired mobility, hypercalcemia, and bone marrow aplasia.2

Effective methods for diagnosing bone and bone marrow metastases have not yet been developed, especially for patients in general intensive care units (ICUs). The development of a severity assessment model is essential to stratify patients at risk of mortality.4 Although several critical-care scoring tools are currently available for clinical use, no effective bedside prognostic scoring models can predict mortality in patients with bone and bone marrow metastases.

A nomogram is a commonly used tool based on several key variables and parameters that can mathematically predict patient prognosis, including disease progression and death.5 The ability to calculate the probability of a clinical event based on several critical factors6 can give rise to a powerful and easy-to-use method for predicting outcomes in individual patients.7

The main objective of the present study was to identify factors independently predictive of survival in patients with secondary malignant neoplasms of bone and bone marrow. The patient cohort and the factors were selected from the Medical Information Mart for Intensive Care III (MIMIC-III) database, and a prognostic nomogram for improving prediction of overall survival (OS) of these patients was developed.

Materials and Methods

Data Source

All the data were extracted from the MIMIC-III database (version 1.4), which contains information about over 58,000 patients hospitalized at the Beth Israel Deaconess Medical Center in Boston, MA, including 38,645 adult and 7875 neonatal patients.8 Information on patients in this database was anonymous, thus informed consent was not required. The research personnel participated in courses from the US National Institutes of Health (NIH) and were authorized to access the database (No: 40269496).

Study Population

Data were extracted with the Structured Query Language (SQL) program in PostgreSQL (version 13.5), and the database was screened for patients with secondary malignant neoplasm of bone and bone marrow using the International Classification of Diseases, ninth edition (ICD-9), code 1985. Patients aged <18 years and those who died within 24 h following admission to an ICU were excluded. The primary outcome was all-cause death rates 30 days after hospital admission.

Information was extracted from the MIMIC-III database using hadm_id and icustay_id. Demographic and clinical factors included age, gender, ethnicity, marital status, insurance, comorbidities, vital signs, laboratory parameters, severity scoring system, and survival information. Comorbidities, as assessed using the Elixhauser Comorbidity Index, included hypertension, obesity, diabetes, chronic pulmonary, renal failure, liver disease, heart disease, fluid electrolyte disorders, alcohol abuse, and anemia. Vital signs included mean heart rate, blood pressure (MBP), respiratory rate, temperature, and percutaneous oxygen saturation (SpO2) during the first 24 h of ICU stay. Laboratory parameters included anion gap (AG); hematocrit; bicarbonate, creatinine, chloride, glucose, hemoglobin, potassium, sodium, and blood urea nitrogen (BUN) concentrations; white blood cell (WBC) and platelet counts; international normalized ratio (INR); prothrombin time (PT); and partial prothrombin time (PTT). Severity scoring systems included angus score, Simplified Acute Physiology Score II (SAPSII), Sequential Organ Failure Assessment (SOFA) score, acute physiology score III (APSIII), and Oxford Acute Severity of Illness Score (OASIS).

Statistical Analysis

Parameters with >20% missing values were not included in the study. Values missing for other parameters were filled in using multiple imputation with the “mice” package of R open-source software.

The dataset was randomly divided into a training cohort (70%) and a validation cohort (30%). The training cohort was utilized to construct the nomogram, and the validation cohort was used for external validation. Categorical variables were reported as frequency and percentage and compared using the chi-square or Fisher’s exact test. Continuous variables were reported as mean- and standard-deviation or as median and interquartile-range, with the Shapiro–Wilk test utilized to determine whether the data were normally distributed.

Variables associated with 30-day all-cause mortality rate were initially determined using univariate Cox regression analysis. Factors differing significantly in univariate analyses (P < 0.05) were included in a multivariable Cox regression model with forward stepwise selection, with the results of the multivariate analysis visualized using a nomogram. The predictive accuracy of the nomogram was evaluated using Harrell’s concordance index (C-index) and the area under the receiver operating characteristic curve (AUC).9 The consistency between predicted probabilities and actual outcomes was assessed using a calibration curve.10 Model accuracy was compared by determining net reclassification improvement (NRI), with the effectiveness of improvements determined by calculating integrated discrimination improvement (IDI).11 The clinical value of the predicted models was determined by decision curve analysis (DCA).12

Statistical analyses were performed using R software (version 3.6.1, CRAN) and SPSS software (version 24.0, Chicago, IL), with P < 0.05 considered statistically significant.

Results

Baseline Characteristics of Patients

This study enrolled 872 patients with secondary malignant neoplasm in bone and bone marrow, 610 in the training cohort and 262 in the validation cohort. The training cohort included 262 (42.6%) women and 350 (57.4%) men, of median age 63.8 years (IQR = 55.8–73.4 years), whereas the validation cohort included 120 (45.8%) women and 142 (54.2%) men of median age 63.7 years (IQR = 56.0–73.5 years). Most of the patients in the training and validation cohorts were white (>70%) and married (>59%), and had medicare (>47%) or private (37%) insurance.

The baseline clinicopathological characteristics of the training and validation cohorts did not differ significantly (Table 1). The median length of hospital stay was 8 days (IQR = 5.54–13 days) in the training cohort and 8.5 days (IQR = 4.89–14.4 days) in the validation cohort, whereas the median length of ICU stay was 2 days in both cohorts. The 30-, 60-, and 90-day mortality rates in the training cohort were 37.2% (n = 227), 48.6% (n = 297), and 56.7% (n = 346), respectively. The 30-day period was selected for further analyses.

Table 1

Baseline Demographic and Laboratory Characteristics of Patients with Secondary Malignant Neoplasm of Bone and Bone Marrow in MIMIC-III Database

Variables	Training Cohort (N = 610)	Validation Cohort (N = 262)	P value
Hospital stay time, days	8.01 [4.54, 13.0]	8.53 [4.89, 14.4]	0.343
ICU stay time, days	2.08 [1.19, 3.97]	2.00 [1.17, 3.81]	0.194
30-day mortality, n (%)	222 (39.2%)	110 (35.5%)	0.675
Gender			0.428
Female	260 (42.6%)	120 (45.8%)
Male	350 (57.4%)	142 (54.2%)
Age	63.8 [55.8, 73.4]	63.7 [56.0, 73.5]	0.859
Ethnicity			0.037
White	493 (80.8%)	199 (76.0%)
Black	44 (7.2%)	34 (13.0%)
Asian	20 (3.3%)	9 (3.4%)
Hispanic	14 (2.3%)	6 (2.3%)
Other	39 (6.4%)	14 (5.4%)
Insurance			0.902
Government	12 (2.0%)	5 (1.9%)
Medicaid	51 (8.4%)	22 (8.4%)
Medicare	289 (47.4%)	133 (50.8%)
Private	252 (41.3%)	99 (37.8%)
Self-Pay	6 (1.0%)	3 (1.1%)
Marital Status			0.745
Married	384 (63.0%)	156 (59.5%)
Single	103 (16.9%)	56 (21.4%)
Widowed	73 (12.0%)	29 (11.1%)
Divorced	29 (4.8%)	11 (4.2%)
Separated	8 (1.3%)	5 (1.9%)
Other	13 (2.2%)	5 (1.9%)
Comorbidity [Yes], n (%)
Hypertension	280 (45.9%)	113 (43.1%)	0.211
Diabetes	87 (14.3%)	34 (13.0%)	0.376
Obesity	10 (1.6%)	1 (0.4%)	0.897
Liver disease	38 (6.2%)	17 (6.5%)	0.221
Heart disease	245 (40.2%)	101 (38.5%)	0.593
Alcohol abuse	19 (3.1%)	8 (3.1%)	0.794
Chronic pulmonary	122 (20.0%)	47 (17.9%)	0.670
Renal failure	71 (11.6%)	29 (11.1%)	0.206
Coagulopathy	109 (17.9%)	37 (14.1%)	0.747
Fluid electrolyte disorder	251 (41.1%)	120 (45.8%)	0.651
Anemias	22 (3.6%)	10 (3.8%)	0.257
Severe Score
Angus	0 [0, 1.00]	0 [0, 1.00]	0.167
SAPSII	40.5 [33.0, 52.0]	41.0 [35.0, 50.0]	0.630
SOFA	3.00 [2.00, 6.00]	3.00 [2.00, 5.00]	0.331
OASIS	31.0 [26.0, 38.0]	31.0 [26.0, 37.0]	0.432
APSIII	42.0 [32.0, 56.0]	42.5 [32.0, 56.0]	0.670
Laboratory tests
Anion gap (mmol/L)	14.0 [12.0, 16.5]	14.0 [12.0, 16.5]	0.782
Bicarbonate (mg/dL)	24.0 [21.3, 26.0]	23.5 [20.5, 26.0]	0.402
Creatinine (k/uL)	0.850 [0.600, 1.23]	0.800 [0.600, 1.24]	0.741
Chloride (mEq/L)	104 [100, 107]	104 [98.4, 107]	0.484
Glucose (mg/dL)	128 [109, 157]	124 [104, 145]	0.021
Hematocrit (%)	30.0 [27.3, 33.6]	29.8 [27.1, 33.0]	0.537
Hemoglobin (g/dL)	10.0 [9.05, 11.2]	9.88 [9.03, 11.2]	0.621
Platelet (k/uL)	216 [140, 310]	213 [119, 299]	0.491
Potassium (mEq/L)	4.15 [3.81, 4.56]	4.15 [3.78, 4.57]	0.809
Sodium (mEq/L)	138 [135, 140]	137 [134, 139]	0.085
PTT (s)	29.3 [25.9, 36.8]	30.5 [26.3, 36.8]	0.914
INR	1.29 [1.10, 1.50]	1.25 [1.15, 1.47]	0.337
PT (s)	14.2 [13.1, 16.0]	14.0 [13.2, 15.9]	0.199
BUN (mg/dL)	19.5 [13.4, 28.9]	20.8 [13.0, 30.8]	0.177
WBC (k/uL)	9.93 [6.35, 14.5]	9.39 [6.27, 13.2]	0.172
Vital signs
Heart rate (min−1)	92.4 [79.3, 104]	92.7 [79.2, 105]	0.596
MBP (mmHg)	76.9 [69.7, 84.7]	76.0 [69.9, 83.3]	0.935
Respiratory rate (min−1)	18.4 [16.0, 21.8]	19.0 [16.3, 22.7]	0.090
Temperature (◦C)	36.7 [36.4, 37.1]	36.7 [36.3, 37.0]	0.211
SpO2 (%)	97.3 [95.9, 98.6]	97.1 [95.7, 98.2]	0.424

Nomogram Construction

Univariate analyses showed that significant predictors of 30-day mortality were age, angus score, SAPSII, SOFA, OASIS, APSII, comorbidities with liver disease, heart disease, coagulopathy, fluid electrolyte disorder, AG, bicarbonate, chloride, glucose, hemoglobin, potassium, sodium, PTT, INR, PT, WBC, heart rate, respiratory rate, temperature, and SpO2 (Table 2). All of these factors were included for the multivariate Cox regression analyses, which found that factors predictive of improved 30-day survival included temperature (hazard ratio [HR] = 0.68, P = 0) and SpO2 (HR = 0.94, P = 0.006), whereas risk factors included SOFA (HR = 1.08, P = 0.042), OASIS (HR = 1.03, P = 0.022), comorbidities with coagulopathy (HR = 1.71, P = 0.001), WBC count (HR = 1.01, P = 0.024), heart rate (HR = 1.01, P = 0.013), and respiratory rate (HR = 1.06, P = 0.003) (Table 3). A nomogram was established based on the significant variables identified in multivariate analyses (Figure 1). The nomogram showed that temperature had the greatest impact on prognosis, followed by respiratory rate, WBC, SpO2, OASIS, SOFA, heart rate, SAPSII, APSIII, and coagulopathy.

Table 2

Univariate Cox Regression Analysis Based on First 24 h Data in the Training Set

Variables	OR	95% CI	P value
Gender	0.95	0.73–1.23	0.69
Age	1.01	1.00–1.02	0.011
Angus	2.03	1.57–2.62	0.000
SAPSII	1.05	1.04–1.06	0.000
SOFA	1.22	1.18–1.27	0.000
OASIS	1.08	1.06–1.09	0.000
APSII	1.03	1.03–1.04	0.000
Liver disease	2.1	1.36–3.23	0.001
Coagulopathy	1.83	1.36–2.45	0.000
Fluid electrolyte disorder	1.62	1.25–2.09	0.000
Heart disease	1.41	1.09–1.82	0.008
Anion gap	1.13	1.10–1.16	0.000
Bicarbonate	0.92	0.90–0.95	0.000
Chloride	0.96	0.94–0.98	0.000
Glucose	1	1.00–1.01	0.001
Hemoglobin	0.91	0.85–0.98	0.019
Potassium	1.33	1.08–1.64	0.007
PTT	1.02	1.01–1.03	0.000
INR	1.33	1.16–1.52	0.000
PT	1.03	1.01–1.04	0.002
Sodium	0.96	0.93–0.98	0.001
BUN	1.01	1.01–1.01	0.000
WBC	1.02	1.01–1.03	0.000
Heart rate	1.02	1.01–1.03	0.000
Respiratory rate	1.11	1.08–1.15	0.000
Temperature	0.7	0.56–0.89	0.003
SpO2	0.88	0.86–0.91	0.000

Table 3

Multivariate Cox Regression Analysis Based on First 24 h Data in the Training Set

Variables	OR	95% CI	P value
SOFA	1.08	1.00–1.16	0.042
OASIS	1.03	1.00–1.05	0.022
Coagulopathy	1.71	1.24–2.36	0.001
WBC	1.01	1.00–1.03	0.024
Heart rate	1.01	1.00–1.02	0.013
Respiratory rate	1.06	1.02–1.09	0.003
Temperature	0.68	0.55–0.84	0.000
SpO2	0.94	0.89–0.98	0.006

Figure 1

Nomogram predicting 30-day mortality. The point of each variable was summed to obtain a total score corresponding to the predicted probability of 30-day survival, shown at the bottom of the nomogram.

Nomogram Validation

C-index analysis was performed to confirm the validity nomogram. The C-index values of the training cohort and validation cohorts were 0.82 (95% confidence interval [CI] = 0.85–0.79) and 0.81 (95% CI = 0.87–0.76), respectively, with these high C-index values for 30-day survival indicating that the model had good discriminative ability. A comparison of the predictive abilities of the nomogram and the SOFA scoring system showed that the AUC values of the nomogram in the training and validation cohorts were 0.821 (95% CI = 0.787–0.855) and 0.811 (95% CI = 0.755–0.866), respectively, with both being significantly higher than the AUC values of the SOFA system (Figure 2). The optimal cutoff point in the training cohort was 0.331, with a sensitivity of 0.718 and a specificity of 0.786. In the validation cohort, the optimal cutoff point was 0.390, with a sensitivity of 0.780 and a specificity of 0.714. The calibration curve showed good consistency with the 45-degree ideal line (Figure 3). Compared with the SOFA system, the NRI values of the nomogram in the training and validation cohorts were 0.751 (95% CI = 0.634–0.975) and 0.725 (95% CI = 0.669–1.17), respectively, whereas the corresponding IDI values in the training and validation cohorts were 0.129 (95% CI = 0.101–0.157) and 0.235 (95% CI = 0.179–0.289), respectively, indicating that this nomogram shows better recognition ability than the SOFA system. The DCA curves show that the nomogram has large net benefits in predicting 30-day survival (Figure 4).

Figure 2

ROC curves of the training and validation cohorts, generated to validate the discrimination of the nomogram. The SOFA scoring system was used for comparison.

Figure 3

Calibration plots showing the relationship between the predicted probabilities based on the nomogram and the actual 30-day survival rates of the training and validation cohorts.

Figure 4

DCA curves of the training and validation cohorts. The abscissa is the threshold probability, and the ordinate is the net benefit rate. The horizontal line indicates that all samples were negative and were not treated, with a net benefit of 0. The oblique line indicates that all samples were positive. The red line shows the net benefit of SOFA score, and the blue line shows the net benefit of the nomogram.

Discussion

This study revealed that temperature, SpO2, SOFA, OASIS, comorbidities with coagulopathy, WBC count, heart rate, and respiratory rate were independent risk factors for 30-day survival in patients with secondary malignant neoplasms of bone and bone marrow. These indicators were used to construct a nomogram to estimate the 30-day mortality rate after hospitalization. The nomogram model was validated using multiple methods, including AUC, calibration curve, IDI, NRI, and DCA. The optimal values were evaluated according to the Youden index, and the sensitivity and specificity were determined.

Disease severity and clinical outcome for adult patients are frequently predicted using scoring systems, such as SAPS II, APACHE III (APS III), SOFA, and OASIS, with these systems often used for investigational purpose. APACHE III and SAPS II scores on admission were shown to be related to patient mortality. For example, univariate analyses showed that APACHE III and SAPS II scores were predictors of poor outcomes in lung cancer patients admitted to the medical ICU,13 and that APACHE II score was an independent factor of survival in patients with hematological malignancies.14 In multivariate analyses, however, APACHE III and SAPS II scores were not significantly predictive of survival in patients undergoing hematopoietic cell transplantation.15,16 SOFA score, the most frequently used scoring system in clinical practice, was shown to be effective in assessing the prognosis of cancer patients in the ICU. In contrast to APACHE III and SAPS II scores, higher SOFA scores,17–19 and higher logistic organ dysfunction (LOD) scores20 were associated with an increased risk of mortality in patients with both solid and hematological malignancies. The OASIS scoring system, established using a machine learning algorithm based on the variables of the APACHE II and including 10 easily determined indicators,21 was shown to be predictive of the prognosis of critically ill patients.22 OASIS scores and SOFA scores did not differ significantly in predicting the short-term prognosis of patients in the ICU.23 The present study shows that the SOFA and OASIS scoring systems are significant predictors of 30-day mortality in patients with bone metastases. The developed nomogram includes both indicators, with the nomogram model being more predictive of mortality than the SOFA score.

The present study also found that WBC count was a significant independent risk prognostic indicator for patients with bone metastases. An increase in WBC count is indicative of many types of cancer, especially bone marrow-related cancers, such as acute myeloid leukemia and chronic myeloid leukemia.24 Evaluation of vital signs showed that heart rate, respiratory rate, body temperature, and SpO2 were independent risk factors for patients with secondary malignant neoplasms of bone and bone marrow, with lower body temperature being associated with increased risk. The finding, that coagulopathy was also a risk factor, was unsurprising, as hypothermia can induce coagulopathy due to the reversible impairment of platelet aggregation and the resulting impairment of platelet plug formation.25,26 One meta-analysis showed that even mild hypothermia increased blood loss by approximately 20%.27 Among critically ill adult patients with cancer, mechanical ventilation for respiratory failure was found to be the most consistent predictor of poor prognosis, with mortality rates >40%.17,28,29 SpO2 reflects the supply of oxygen to the body and the degree of hypoxia, serving as a factor associated with critical illness.30 The present nomogram showed that higher respiratory rate and lower SpO2 level correlated with poor prognosis.

Although the model predicting a 30-day mortality rate in patients with bone and bone marrow metastases developed in the current study showed good performance and fitness, this study had several limitations. First, the data were extracted from a public database of a single center in the United States, which could result in population bias and thus limit the applicability of the model. Second, the model included laboratory parameters collected after hospital admission, and patients were not followed up. Third, the database is relatively old, suggesting the need for validation using external data from our own center.

Conclusions

The present study describes the first prognostic nomogram for predicting 30-day mortality rates of patients with secondary malignant neoplasms of bone and bone marrow. This nomogram can be easily applied in clinical practice.