Development of a Nomogram for Predicting the Efficacy of Preoperative Chemotherapy in Osteosarcoma.

BACKGROUND: Due to the obvious heterogeneity of osteosarcoma, many patients are not sensitive to neoadjuvant chemotherapy. In this study, the clinical characteristics and auxiliary examinations of patients with osteosarcoma were used to predict the effect of preoperative chemotherapy, so as to guide the clinical adjustment of the treatment plan to improve the prognosis of patients.
METHODS: In this study, 90 patients with pathologically confirmed osteosarcoma were included, and they were randomly divided into training cohort (n=45) and validation cohort (n=45). A prediction model of preoperative chemotherapy efficacy for osteosarcoma was established by multivariate logistic regression analysis, and a nomogram was used as the visualization of the model. The ROC curve and C-index were used to evaluate the accuracy of the nomogram. Decision curve analysis (DCA) was used to evaluate the net benefit of the nomogram in predicting the efficacy of neoadjuvant chemotherapy under different threshold probabilities.
RESULTS: In the study, the age, gender, location, tumor volume, metastasis at the first visit, MSTS staging, C-reactive protein (CRP), erythrocyte sedimentation rate (ESR), alkaline phosphatase (ALP), and lactate dehydrogenase (LDH) were used in the multivariate logistic regression analysis and the construction of the nomogram. The AUC and C-index of the training cohort were 0.793 (95% CI: 0.632, 0.954) and 0.881 (95% CI: 0.776, 0.986), respectively. The AUC and C-index in the validation cohort were 0.791 (95% CI: 0.644, 0.938) and 0.813 (95% CI: 0.679, 0.947), respectively, which were close to the training cohort. DCA showed that the model had good clinical application value.
CONCLUSION: Based on the clinical characteristics of patients and auxiliary examinations, the nomogram can be good used to predict the efficacy of preoperative chemotherapy for osteosarcoma.

Introduction

Osteosarcoma is the most common type of bone cancer and is commonly found in large bones in the area of bone with the fastest growth rate.1 It is a malignant bone tumor that mainly occurs in children and adolescents,2 and is the third most common tumor in adolescents.3 The advent of neoadjuvant chemotherapy has improved the prognosis of osteosarcoma,4 with the survival rate increasing from less than 20% to 60–70%.3,5,6 However, in recent years, the survival rate of osteosarcoma has not been further improved.7,8 Endogenous or acquired drug resistance is one of the important causes of treatment failure and poor prognosis in patients with osteosarcoma.9 Osteosarcoma is one of the most heterogeneous solid tumors,10 and different patients with osteosarcoma have obvious differences in the efficacy of preoperative chemotherapy. At present, tumor cell necrosis rate of more than 90% is considered as effective chemotherapy, while tumor cell necrosis rate of less than 90% is considered as unsatisfactory chemotherapy efficacy, and endogenous drug resistance may exist.11,12

Neoadjuvant chemotherapy allows patients to achieve a tumor-free margin while preserving the limb.13 It also increases the chance of relapse-free survival in patients with osteosarcoma.14,15 Therefore, the choice of chemotherapy regimen for newly diagnosed patients is extremely important, and reasonable chemotherapy regimens are an important way to prevent the progression and metastasis of osteosarcoma. The NCCN guidelines state that standard first-line chemotherapy consists of a combination of cisplatin, doxorubicin, ifosfamide, and high-dose methotrexate. The preferred regimen is the combination of cisplatin and doxorubicin or the combination of cisplatin and doxorubicin plus high-dose methotrexate.4 Although many patients with osteosarcoma respond to standard chemotherapy regimens, there are still many patients who do not respond well to chemotherapy. We hope to predict the efficacy of chemotherapy based on the clinical characteristics of patients at the time of initial diagnosis and the relevant ancillary examination results. Therefore, chemotherapy regimens can be adjusted to improve the therapeutic effect for patients who may be ineffective. This study retrospectively analyzed the clinical, laboratory, and imaging data of 90 patients with osteosarcoma who underwent preoperative chemotherapy. The primary purpose of this study was to develop a nomogram that predicted the efficacy of preoperative chemotherapy in patients with osteosarcoma and to evaluate the probability that a patient would be effective (necrosis rate greater than 90%).

Materials and Methods

Patients

This study was retrospective and approved by the Ethics Committee of Peking University People’s Hospital. It included 90 patients with osteosarcoma who were hospitalized in the Musculoskeletal Tumor Center, Peking University People’s Hospital. The patients were admitted between March 2015 and December 2018. Inclusion criteria: 1) Patients with the initial diagnosis of osteosarcoma; 2) The patient had single osteosarcoma; 3) The diagnosis was confirmed by biopsy. Exclusion criteria: 1) Patients with other malignant tumors; 2) Patients with infection; 3) Patients with incomplete clinical characteristics or auxiliary examination results; 4) Patients who did not receive neoadjuvant chemotherapy; 5) Patients undergoing radiotherapy for other reasons.

Clinical Characteristics and Auxiliary Examination Items

Clinical characteristics and auxiliary examination items, including age, gender, location, tumor volume, metastasis at the first visit, neoplasm staging, C-reactive protein (CRP), erythrocyte sedimentation rate (ESR), alkaline phosphatase (ALP), and lactate dehydrogenase (LDH) were obtained from the medical records. The tumor sites are the trunk, humerus, femur, tibia, and fibula. Musculoskeletal Tumor Society (MSTS) staging system was used for tumor staging. CRP, ESR, ALP, and LDH were divided into normal group and elevated group according to a normal reference value. The calculation method of tumor volume adopted the previous method (height × length × width × π/6).16–18 These length indicators were derived from CT imaging.

Construction of the Nomogram

The nomogram is widely used to calculate the diagnostic probability of the disease. It can reduce the complicated statistical prediction model to a probability value.19,20 A prediction model of preoperative chemotherapy efficacy for osteosarcoma was established by multivariate logistic regression analysis, and a nomogram was used as the visualization of the model. The “rms” package was used for multivariate logistic regression analysis. The regression coefficients were obtained for each variable by logistic regression analysis. They show how much each indicator contributes to the model. “rms” package was also used for the construction of the nomogram. In the study, the age, gender, location, tumor volume, metastasis at the first visit, MSTS staging, CRP, ESR, ALP, and LDH were used to construct the nomogram. It intuitively predicts the probability that preoperative chemotherapy will be effective in patients with osteosarcoma.

Statistical Analysis

R software (Version 3.6.1) and SPSS 22 were used for statistical analysis. Chi-square test and t-test were used to test the balance of sample distribution between the training cohort and validation cohort. The “rms” package was used for logistic regression analysis, nomogram plotting and C-index calculation. The prediction model of preoperative chemotherapy efficacy for osteosarcoma was established by multivariate logistic regression analysis, and a nomogram was used as the visualization of the model. C-index and ROC curves were used to evaluate the accuracy of the nomogram.21 The “ROCR” package was used to plot the ROC curve and calculate the AUC value. The net benefit is defined as the proportion of true positives minus the proportion of false positives, weighted by the relative harm of false-positive and false-negative results.21,22 The “rmda” package was used to plot the decision curve and evaluate the net benefits under different threshold probabilities. Two-tailed P-values were used, and the statistical significance was set at P < 0.05.

Results

Patient Characteristics

In this study, 90 patients with osteosarcoma were included, and the clinical and auxiliary examination results of each patient were shown in Figure 1. The process of patient inclusion was shown in . The patients were arranged from small to large according to age. In this study, patients were randomly divided into the training cohort (n=45) and validation cohort (n=45), and all clinical characteristics and ancillary examination results of patients were summarized (Table 1). Except for LDH, the sample size of other indicators showed no significant difference between the training cohort and the validation cohort (P > 0.05). This indicates that the samples are evenly distributed in the two queues. There were 27 patients with good chemotherapy efficacy, accounting for 30%. They ranged in age from 3 years to 56 years, with an average age of 17.99±11.55 years. The proportion of male patients was 65.6%, and that of female patients was 34.4%. The locations of the tumors were trunk, humerus, femur, tibia, and fibula, 48.9% of which were located in the femur. The MSTS staging showed that all the 90 patients were at stage II or stage III. Among them, most of the patients in stage II were stage II B and few were in stage II A, so they were uniformly classified as stage II in this study. At least 30% of patients have elevated CRP, ESR, LDH, and ALP. In addition, approximately 34.4% of patients had metastases at the time of their first visit.

Figure 1

Clinical characteristics and ancillary examination results of 90 patients with osteosarcoma. Patients were randomly divided into the training cohort (n=45) and validation cohort (n=45). Each line in the heat map represents a patient. Values for age and tumor volume were standardized.

Table 1

Summary of Clinical Characteristics and Adjuvant Examination Items in Patients with Osteosarcoma

Characteristics	Training Cohort (n=45)	Validation cohort (n=45)	Total (n=90)	t or χ2	P-value
Necrosis rate
< 90%	33 (73.3%)	30 (66.7%)	63 (70.0%)	0.476	0.490
≥ 90%	12 (26.7%)	15 (33.3%)	27 (30.0%)
Age (mean, years)	18.31	17.67	17.99	0.263	0.793
Gender
Female	15 (33.3%)	16 (35.6%)	31 (34.4%)	0.000	1.000
Male	30 (66.7%)	29 (64.4%)	59 (65.6%)
Location
Trunk	7 (15.6%)	10 (22.2%)	17 (18.9%)	1.509	0.680
Humerus	5 (11.1%)	6 (13.3%)	11 (12.2%)
Femur	22 (48.9%)	22 (48.9%)	44 (48.9%)
Tibia and fibula	11 (24.4%)	7 (15.6%)	18 (20.0%)
Tumor volume (mean, cm3)	407	389	398	0.198	0.844
MSTS staging
II	24 (53.3%)	17 (37.8%)	41 (45.6%)	2.195	0.138
III	21 (46.7%)	28 (62.2%)	49 (54.4%)
CRP
Normal	29 (64.4%)	30 (66.7%)	59 (65.6%)	0.049	0.824
Elevated	16 (35.6%)	15 (33.3%)	31 (34.4%)
ESR
Normal	14 (31.1%)	17 (37.8%)	31 (34.4%)	0.443	0.506
Elevated	31 (68.9%)	28 (62.2%)	59 (65.6%)
LDH
Normal	36 (80.0%)	25 (55.6%)	61 (67.8%)	6.156	0.013
Elevated	9 (20.0%)	20 (44.4%)	29 (32.2%)
ALP
Normal	35 (77.8%)	29 (64.4%)	64 (71.1%)	1.947	0.163
Elevated	10 (22.2%)	16 (35.6%)	26 (28.9%)
Metastasis at the first visit
Yes	31 (68.9%)	28 (62.2%)	59 (65.6%)	0.443	0.506
No	14 (31.1%)	17 (37.8%)	31 (34.4%)

Note: Chi-square test and t-test were used to test the balance of sample distribution between training training cohort and validation cohort.

Abbreviations: ALP, alkaline phosphatase; CRP, C-reactive protein; ESR, erythrocyte sedimentation rate; LDH, lactate dehydrogenase.

Construction of a Nomogram to Predict the Efficacy of Chemotherapy for Osteosarcoma

In the multivariate regression analysis, the regression coefficient for intercept was −20.492. The regression coefficients for location (humerus, femur, tibia, and fibula) were 20.866, 21.839, 19.935 respectively. The regression coefficients for age, gender, tumor volume, metastasis, MSTS staging, CRP, ESR, LDH and ALP were −0.064, −3.583, 0.004, −0.125, 2.788, 0.872, −0.955, −2.094, −0.620 respectively. Nomogram includes point, clinical characteristics, auxiliary examinations, total point, linear predicted value, and corresponding probability values (Figure 2). The first line showed the point range (0–100) corresponding to each clinical characteristic and auxiliary examination of the patient. Patients’ clinical characteristics and ancillary examinations have corresponding points in the nomogram. The total points of all clinical characteristics and auxiliary examinations of each patient corresponds to the position on the last line, which is the probability value of effective chemotherapy. Besides, it is 9.27 points per unit of the linear predictor.

Figure 2

The nomogram to predict the efficacy of chemotherapy for osteosarcoma. The first line is the point range for the variables. The second to 11th lines are the ten clinical characteristics and ancillary examination items included in the study. The last one represents the probability value of a high necrosis rate (≥90%).

Evaluation of the Accuracy of Chemotherapy Efficacy Prediction Nomogram

In this study, the ROC curve and C-index were used to evaluate the accuracy of the prediction nomogram. ROC curve showed that the training cohort has a good predictive performance, with an AUC of 0.793 (95% CI: 0.632, 0.954) (Figure 3A). The optimal cut-off value of the total point was 189.839. Grouping with the optimal cut-off value, we found that the sensitivity was 0.583, the specificity was 0.939, the positive predictive value (PPV) was 0.778, the negative predictive value (NPV) was 0.861, and the accuracy was 0.844. The AUC of the validation cohort was 0.791 (95% CI: 0.644, 0.938) (Figure 3B), which was close to that of the training cohort. There was no difference in ROC results by comparing the training cohort and verification cohort (P=0.987) (Figure 3C). Besides, the C-index, Somers’ Dxy rank correlation (Dxy), and R-squared index (R2) in the training cohort were 0.881, 0.763, and 0.536 respectively, and they were 0.813, 0.627, and 0.358 respectively in the validation cohort (Table 2).

Figure 3

Evaluation of the accuracy of the chemotherapy efficacy prediction nomogram. (A) ROC is used to evaluate the predictive power of the chemotherapy efficacy prediction nomogram. The AUC in the training cohort is 0.793 (95% CI: 0.632, 0.954). (B) The AUC in the validation cohort is 0.791 (95% CI: 0.644, 0.938). (C) ROC results of the training cohort and the validation cohort were compared.

Table 2

The Validation of Chemotherapy Efficacy Prediction Nomogram

Parameter	Training Cohort	Validation Cohort
C-index	0.881	0.813
Dxy	0.763	0.627
R2	0.536	0.358

Abbreviations: Dxy, Somers’ Dxy rank correlation; R2, R-squared index.

Clinical Application Value of Chemotherapy Efficacy Prediction Nomogram

DCA was used to evaluate the potential clinical application of this nomogram. The study showed that the threshold probabilities in the training cohort were 1% and 58% (Figure 4A). That is, when the patient’s individual threshold probability is between 1% and 58%, there is a net benefit to using the nomogram to decide whether to receive the chemotherapy. In addition, the threshold probabilities for the validation cohort were 4% and 67% (Figure 4B).

Figure 4

Decision curve analysis for the chemotherapy efficacy prediction nomogram. (A) The red line represents the chemotherapy efficacy prediction nomogram of the training cohort. When the threshold probability is between 1% and 58%, the nomogram has a relatively good net benefit. (B) The blue line represents the chemotherapy efficacy prediction nomogram of the validation cohort. When the threshold probability is between 4% and 67%, the nomogram has a relatively good net benefit.

Discussion

In this study, a nomogram was constructed to predict the efficacy of chemotherapy based on the clinical characteristics and auxiliary examination results before chemotherapy for osteosarcoma. The probability of the effectiveness of chemotherapy can be easily determined by the nomogram, so as to find out the patients who may have poor efficacy of chemotherapy in advance. Modification of treatment regimens in these patients is expected to improve patient outcomes.

Neoadjuvant chemotherapy regimens for osteosarcoma mainly include cisplatin, doxorubicin, ifosfamide, and high-dose methotrexate.23 The combination of neoadjuvant chemotherapy and surgery has greatly improved the prognosis of osteosarcoma. However, due to the presence of drug-resistant tumor cells,23 there are still 20–30% of patients who are ineffective.24 The nomogram can integrate different variables to generate the probability of each clinical event, fulfilling our need for personalized treatment.22 In the study, the age, gender, location, tumor volume, metastasis at the first visit, MSTS staging, CRP, ESR, ALP, and LDH were used to construct the nomogram. These indicators are clinically common and belong to the non-invasive examination. At present, the abnormal elevation of ALP in blood examination may have a certain suggestive effect on osteosarcoma, so it is often included in many studies.25,26 Therefore, ALP was included in our study. In addition, CRP and ESR increases are more obvious in malignant tumors, especially those with rapid growth, they are of significance for the prognosis of osteosarcoma.27 LDH is also associated with poor prognosis in patients with osteosarcoma.28 The NCCN guidelines also indicate that ALP and LDH are associated with the diagnosis and prognosis of osteosarcoma.4 These indicators are readily available at the time of hospitalization, so we included them in our study. The nomogram showed that the site of the tumor contributed more to the total points, and osteosarcoma in the extremities had a greater probability of responding to chemotherapy. In addition, tumor volume also contributed greatly to the total points, and the larger the tumor volume was, the higher the probability of high necrosis rate was. Of course, this may be related to insufficient blood supply caused by larger tumors. There are mainly two aspects of the quality of the general evaluation model. One is the Goodness of Fit of the model. The common evaluation index mainly includes R2. It represents the percentage change in the dependent variable that the model can explain. It is worth noting that R2 is only a reference index, not a decisive one. Another is the prediction accuracy of the model. The C-index is essentially an estimate of the probability that the prediction results will be consistent with the actual observed results. Similarly, Dxy was used to evaluate the model’s discrimination ability. In the study, the ROC curve and C-index were used to verify the accuracy of the nomogram, and both of them reached a high level. The validation cohort also further confirmed the reliability of the nomogram. DCA is used to evaluate the potential clinical impact of biomarkers or models and is an analytical method used to recommend treatment or intervention behavior.29 It is an appropriate method to evaluate alternative diagnostic and prognostic strategies and has advantages over other commonly used measures and techniques.30 The results of DCA in this study indicated that the nomogram for predicting the efficacy of chemotherapy for osteosarcoma had a good clinical application value.

There are some limitations to our research. This study is a retrospective study. Due to incomplete information, we could not include more indicators for statistical analysis, even though some indicators may be meaningful for the diagnosis of osteosarcoma. Second, this was a single-center study, and the patient origin may lead to bias in the results. In addition, the incidence of osteosarcoma is 3–4 per million,31 which is a relatively rare tumor. In the study, the strict screening was carried out according to the above exclusion criteria. Therefore, only 90 patients were included in the study, which is a weakness of this study.

In conclusion, this study presents and validates a nomogram to predict the efficacy of chemotherapy for osteosarcoma. Through the clinical characteristics and auxiliary examination results of the nomogram, we can easily predict whether the patient will have a good chemotherapy efficacy.