Risk factors associated with skeletal-related events following discontinuation of denosumab treatment among patients with bone metastases from solid tumors: A real-world machine learning approach.

Background: Clinical practice guidelines recommend the use of bone-targeting agents for preventing skeletal-related events (SREs) among patients with bone metastases from solid tumors. The anti-RANKL monoclonal antibody denosumab is approved for the prevention of SREs in patients with bone metastases from solid tumors. However, real-world data are lacking on the impact of individual risk factors for SREs, specifically in the context of denosumab discontinuation. Purpose: We aim to identify risk factors associated with SRE incidence following denosumab discontinuation using a machine learning approach to help profile patients at a higher risk of developing SREs following discontinuation of denosumab treatment.
Methods: Using the Optum PanTher Electronic Health Record repository, patients diagnosed with incident bone metastases from primary solid tumors between January 1, 2007, and September 1, 2019, were evaluated for inclusion in the study. Eligible patients received ≥ 2 consecutive 120 mg denosumab doses on a 4-week (± 14 days) schedule with a minimum follow-up of ≥ 1 year after the last denosumab dose, or an SRE occurring between days 84 and 365 after denosumab discontinuation. Extreme gradient boosting was used to develop an SRE risk prediction model evaluated on a test dataset. Multiple variables associated with patient demographics, comorbidities, laboratory values, treatments, and denosumab exposures were examined as potential factors for SRE risk using Shapley Additive Explanations (SHAP). Univariate analyses on risk factors with the highest importance from pooled and tumor-specific models were also conducted.
Results: A total of 1,414 adult cancer patients (breast: 40%, prostate: 30%, lung: 13%, other: 17%) were eligible, of whom 1,133 (80%) were assigned to model training and 281 (20%) to model evaluation. The median age at inclusion was 67 (range, 19-89) years with a median duration of denosumab treatment of 253 (range, 88-2,726) days; 490 (35%) patients experienced ≥ 1 SRE 83 days after denosumab discontinuation. Meaningful model performance was evaluated by an area under the receiver operating curve score of 77% and an F1 score of 62%; model precision was 60%, with 63% sensitivity and 78% specificity. SHAP identified several significant factors for the tumor-agnostic and tumor-specific models that predicted an increased SRE risk following denosumab discontinuation, including prior SREs, shorter denosumab treatment duration, ≥ 4 clinic visits per month with at least one hospitalization (all-cause) event from the baseline period up to discontinuation of denosumab, younger age at bone metastasis, shorter time to denosumab initiation from bone metastasis, and prostate cancer.
Conclusion: This analysis showed a higher cumulative number of SREs, prior SREs relative to denosumab initiation, a higher number of hospital visits, and a shorter denosumab treatment duration as significant factors that are associated with an increased SRE risk after discontinuation of denosumab, in both the tumor-agnostic and tumor-specific models. Our machine learning approach to SRE risk factor identification reinforces treatment guidance on the persistent use of denosumab and has the potential to help clinicians better assess a patient's need to continue denosumab treatment and improve patient outcomes.

Introduction

Bone metastases are common among patients with solid tumors and indicate poor prognosis. Disease progression is often associated with skeletal-related events (SREs), defined as pathologic fractures, spinal cord compression, bone radiation therapy, and bone surgery [1], [2], [3]. Majority of bone metastases across all solid tumors are associated with breast, lung, and prostate, with an estimated 10-year US incidence varying by tumor type, and with prostate cancer patients at highest risk followed by breast and lung cancer patients [1], [2], [4], [5]. Additionally, inpatient costs for solid tumor patients with bone metastases and SREs were significantly higher than those without SREs and increased further with multiple events [6], [7], [8].

The pathophysiology of bone complications is characterized by deranged osteoblast and osteoclast activity at the site of the bone metastasis, potentially creating an environment that supports tumor growth and bone destruction [2]. Though rarely curable, the spread of bone metastasis may be slowed by treatment with bone-targeting agents (BTAs). International treatment guidelines recommend the use of a BTA for SRE prevention among patients with bone metastases from solid tumors for a ≥ 2-year period [3], [9], [10]. The European Society for Medical Oncology (ESMO) recently issued updated treatment recommendations with an emphasis on treatment duration. In their statement, the ESMO suggested assessing the dosage and treatment schedule for individual patients based on the risk for an SRE and the overall status of tumor control before initiating BTA therapy, which should generally continue indefinitely, including into the hospice setting [3], [7].

Previous retrospective studies have identified individual clinical and/or biologic factors, including race, sex, Eastern Cooperative Oncology Group (ECOG) performance status, human epidermal growth factor receptor-2 positivity, number of bone metastases, and elevated serum alkaline phosphatase, as factors contributing to individual patient’s risk of first SRE, both in the presence and absence of BTAs [11], [12], [13], [14], [15], [16], [17]. Denosumab (XGEVA®) is a human monoclonal antibody with neutralizing activity against receptor activator of nuclear factor κβ ligand (RANKL). It is approved for the prevention of SREs in patients with multiple myeloma and with bone metastases from solid tumors when administered at a dose of 120 mg subcutaneously (SC) every 4 weeks (Q4W) [18], [19], [20], [21], [22]. Real-world treatment patterns in the US suggest that the duration of denosumab treatment is often < 1 year, indicating potential underutilization of denosumab [23], [24]. To the best of our knowledge, there are no comprehensive studies examining the impact of individual risk factors relative to real-world treatment patterns of denosumab, specifically in the context of denosumab discontinuation.

An aggregate analysis of multiple SREs over a long follow-up period across various patients and time points requires considerable effort and expertise, while demanding the development of more automated approaches for handling the variety and volume of data. A machine learning approach enables inference of key connections between complex data in a comprehensive manner with greater accuracy than conventional methods [25]. Additionally, prediction of impending SREs, particularly pathologic fractures, could considerably improve the management of metastatic bone disease [26]. To date, no study has made use of a machine learning approach to examine clinical SRE risk factors in the context of early BTA treatment discontinuation. Here, we aim to identify risk factors associated with SRE incidence following denosumab discontinuation using a machine learning approach to help inform optimal clinical SRE-prevention strategies consistent with current treatment guidelines.

Patients and methods

Study design

This was a retrospective observational analysis coupled with an estimation study using a classification model approach based on information collected from patients with incident bone metastases from solid tumors, treated with denosumab (Fig. 1). Patient data from the Optum PanTher Electronic Health Record (EHR) repository were collected from January 1, 2007, to September 1, 2019. Since this study used deidentified data containing no protected health information (PHI), informed consent of participants was not obtained.

Fig. 1

Study design. Baseline, 6 months prior to initial bone metastasis diagnosis; T0, time period from initial solid tumor diagnosis to initial bone metastasis diagnosis; T1, time period from initial bone metastasis diagnosis to denosumab initiation; T2, time period on denosumab treatment (Q4W dosing schedule [28 ± 14 days]). Follow-up, time period within which initial occurrence of SREs following denosumab discontinuation is extracted (84–365 days from the last denosumab dose). aEnd of patient observation will be defined as one of five events following denosumab washout: (i) 365 days after final denosumab claim, (ii) patient death, (iii) patient’s final visit, drug or medical event (up to the end of the study observation period, September 1, 2019), (iv) denosumab restart, (v) switch to another BTA. Patients with end of patient observation occurring before SRE in follow‑up are excluded from this analysis. BTA, bone–targeting agent; Q4W, every 4 weeks; SRE, skeletal-related event.

Eligibility

Adult patients (age ≥ 18 years) with a solid tumor (breast, prostate, lung, colorectal, liver, pancreatic, other gastrointestinal, head and neck, bone and connective tissue, endocrine, malignant melanoma, gynecologic, genitourinary, renal, brain and nervous system, or other solid tumors) diagnosed on or after January 1, 2007, and initial bone metastasis diagnosis on or after the primary solid tumor diagnosis were eligible to be included in the study. Eligible patients received ≥ 2 doses of denosumab on a Q4W (once every 28 ± 14 days for this study) dosing schedule, and had experienced ≥ 1 SRE during follow-up or had a minimum follow-up period of 84 days up to 365 days from the last denosumab dose (regardless of SRE status at follow-up). Patients administered denosumab after November 1, 2010 (US approval date) and on or after the initial bone metastasis diagnosis date, those given the drug before the baseline period (180 days prior to initial bone metastasis diagnosis), and those who received other classes of BTAs prior to both bone metastasis and/or denosumab initiation were eligible. Patients who received denosumab at a dosing schedule other than Q4W (28 ± 14 days) or concomitant BTAs during the initial denosumab treatment and/or washout period were excluded from the study.

Outcomes

The primary objective for this study was to identify risk factors associated with SREs within 84 to 365 days following the last denosumab dose of the first round of continuous Q4W dosing in patients with bone metastases from solid tumors through evaluation of classification model risk predictions. The exploratory objective was identification of risk factors associated with SREs following denosumab discontinuation in individual tumor types (with sufficient sample size). As described earlier by Aly et al. [27], SREs were identified in the EHR data using claims that indicated spinal cord compression, pathologic fracture, bone palliative radiotherapy, or bone surgery. Individual SREs from each event type were identified from a unique set of International Classification of Diseases (ICD)-9, ICD-10, Healthcare Common Procedure Coding System (HCPCS), or Current Procedural Terminology (CPT) codes. Spinal cord compression events were identified using codes indicating “unspecified disease of spinal cord” only. Pathologic fractures were determined using codes indicating “pathologic fractures” or “other fractures,” excluding codes that occurred ≤ 2 weeks after “trauma” codes. Bone palliative radiotherapy events were defined using codes indicating receipt of external beam radiation therapy or radioisotopes. Bone surgery was identified using codes indicating any bone surgical procedure (list of codes used is available in Supplementary File 1).

Model development

A classification model was generated to predict the risk of first symptomatic SRE during 84–365 days from the last denosumab dose. To develop the model, each patient was classified as either an SRE patient (positive class) or a non-SRE patient (negative class) based on the presence or absence of a qualifying SRE in the follow-up period. Subsequently, the model was trained to learn patterns that were indicative of SREs following the last denosumab dose from the 64 patient exposure variables (Supplementary Table 1 in Supplementary File 2) included in this analysis. Extreme gradient boosting (XGB) was leveraged as the model architecture [28]. Model hyperparameters were selected using Bayesian optimization with five-fold cross-validation repeated ten times. If the number of patients without an SRE outweighed the number of patients with an SRE, we checked the model for bias toward the majority class using undersampling, oversampling, and penalization of minority class errors in the loss function. Sequential-based variables were evaluated separately and combined with cross-sectional variables to predict SRE risk.

Model evaluation

The patient cohort was split into training and testing datasets in a 4:1 ratio using stratified random sampling to ensure equal representation of the target variable, age, sex, denosumab treatment duration, and tumor type across subgroups. The model was trained on the training set and evaluated on the testing set using area under the receiver operating curve (AUROC), F1 score (weighted average of precision and recall), sensitivity, specificity, and precision.

Risk factor analysis

The best performing model, selected according to the highest AUROC cross-validation score, was used to extract global risk factors from the full set of 64 variables including demographics, prior SREs, denosumab exposure, other medications, comorbidities, disease progression, and hospital/clinic visits (Supplementary Table 1 in Supplementary File 2). Shapley Additive Explanations (SHAP) was used to quantify the importance of a variable in predicting SRE risk following denosumab discontinuation by calculating the magnitude and direction that a particular risk factor caused patients’ predicted risk to deviate from the average predicted risk, thereby quantifying how important that factor was in elevating or reducing the risk. Risk factor SHAP values with greater absolute magnitudes were considered to be more important in risk calculations. Therefore, of the 64 exposure variables included in this analysis, those that had sufficiently large SHAP values were extracted as primary risk factors from the model. The threshold of sufficiency was determined using a forward variable selection approach in the order of SHAP variable rankings.

Analyses

Models were evaluated using AUROC, F1 metrics, sensitivity, specificity, and precision on the testing dataset. Once validated, this model was leveraged to identify primary SRE risk factors through SHAP analysis in models developed on both tumor-agnostic and tumor-specific cohorts. Shapley values quantified the magnitude and direction that particular risk factors impacted the predicted SRE risk scores. Univariate analyses on risk factors with the highest importance from pooled and tumor-specific models were also conducted.

Results

Of the 104 million patients comprising the full cohort in Optum PanTher EHR, approximately 9,000 patients with bone metastasis having started denosumab treatment after November 1, 2010, were evaluated based on the eligibility criteria (Supplementary Fig. 1 in Supplementary File 2); 7,586 patients without any SRE event between 84 and 365 days and having follow-up of < 365 days from the last denosumab dose were excluded from the study. Overall, 1,414 patients were included in the study, of whom 1,133 (80%) patients were assigned to model training for hyperparameter tuning with K-fold cross-validation and 281 (20%) were assigned to the testing set for model evaluation (Supplementary Fig. 2 in Supplementary File 2). Testing set evaluation was comparable to cross–validation scoring and determined how well the model performed on the data it was not trained on.

Overall, 40% of patients had breast cancer, 30% had prostate cancer, 13% had lung cancer, and 17% had other cancers (Table 1). The median age at inclusion was 67 (range, 19–89) years and the median duration of denosumab treatment was 253 days; 490 (35%) patients experienced ≥ 1 SRE following denosumab discontinuation.

Table 1

Demographic and clinical characteristics of eligible patients at baseline.

Characteristic	N = 1,414
Sex, n (%)
Male	663 (47)
Female	751 (53)
Age, years, median (range)	67 (19–89)
Solid tumor type, n (%)
Breast	563 (40)
Prostate	421 (30)
Lung	180 (13)
Other*	250 (17)
Time since initial bone metastasis diagnosis, days, median (range)	45 (0–2,620)
SRE prior to denosumab initiation, n (%)
Yes	378 (27)
No	1,036 (73)
Denosumab treatment duration at Q4W dosing, days, median (range)	253 (88–2,726)
SRE on denosumab, n (%)
Yes	459 (32)
No	955 (68)
SRE following discontinuation, n (%)
Yes	490 (35)
No	924 (65)

Q4W, every 4 weeks; SRE, skeletal-related event.

Other solid tumor was defined as one of the following: colorectal cancer, liver cancer, pancreatic cancer, head and neck cancer, bone and connective tissue cancer, endocrine cancer, malignant melanoma, gynecologic cancer, genitourinary cancer, renal cancer, other gastrointestinal cancer, brain and nervous system cancer.

Risk factors for SREs identified from the tumor-agnostic model

The tumor-agnostic model was meaningful, as evidenced by an AUROC score of 77% and an F1 score of 62%; model precision was 60% with 63% sensitivity and 78% specificity (Table 2). SHAP identified several significant factors that predicted an increased SRE risk following denosumab discontinuation, including the cumulative number of SREs before initiation and during denosumab treatment, shorter denosumab treatment duration (< 10 months vs > 20 months), ≥ 4 clinic visits per month with at least one hospitalization (all-cause) event from the baseline period up to discontinuation of denosumab, younger age at bone metastasis (≤ 65 years vs > 65 years), and shorter time to denosumab initiation from bone metastasis (≤ 3 months vs > 10 months) (Fig. 2). The risk of SREs was also higher among patients with prostate cancer (34%) vs those with breast cancer (29%).

Table 2

Model performance metrics for best performing model – tumor-agnostic and tumor-specific models.

Patient cohort used for modeling	Performance metrics of model on test dataset
	AUROC	F1 score	Precision	Sensitivity	Specificity
All tumor types	0.774	0.617	0.602	0.633	0.776
Breast	0.726	0.522	0.514	0.530	0.767
Prostate	0.815	0.667	0.621	0.719	0.750
Other solid tumors	0.668	0.567	0.542	0.594	0.704

AUROC, area under the receiver operating curve; F1 score, weighted average of precision and recall.

Fig. 2

Overall variable impact – all patients with solid tumors (declining importance). X-axis represents the risk of SRE post-cessation, with higher risk towards the right and lower risk towards the left. Logarithmic variable values are indicated by the scale. Y-axis represents variables affecting the risk of SRE. More important variables are at the top and less important variables are towards the bottom. Color represents variable value. Red indicates the variable value is higher (e.g., higher number of SREs), while blue indicates the variable value is lower (e.g., lower number of SREs). The first row, cumulative number of SREs, can be interpreted as a higher number of cumulative SREs (red) is associated with a higher risk of SRE (moving towards the right). Conversely, a lower number of SREs (blue) is associated with lower risk of SRE (moving towards the left). The second row, denosumab duration, can be interpreted as a longer denosumab duration (red) is associated with a lower risk of SRE (moving towards the left), while a shorter denosumab duration (blue) is associated with a higher risk of SRE (moving towards the right). BM, bone metastasis; SRE, skeletal-related event.

Measured from a period of 6 months prior to initial bone metastasis diagnosis up to denosumab therapy discontinuation, the cumulative incidence of SREs showed a positive correlation with SRE risk. The number of SREs was directly proportional to an elevated risk of SREs following denosumab discontinuation until a saturation point of four SREs was reached. Consistent with prior SREs associating with an increased SRE risk, the absence of any SRE represented the lowest risk group (Fig. 3). Of patients with 0 (569 [40.2%]), 1 (272 [19.2%]), 2 or 3 (194 [13.7%]), and ≥ 4 (98 [6.9%]) prior SREs from the training set, 19%, 36%, 58%, and 73%, respectively, developed SREs 3 to 12 months after denosumab treatment discontinuation.

Fig. 3

Univariate analysis – correlation between risk factor and SRE risk following denosumab discontinuation from four top risk factors. Y-axis values are in log scale. Y-axis represents the risk of SRE, with higher risk towards the top and lower risk towards the bottom. Values on the left panels of the third and fourth figures represent number of patients with no prior SRE; values on the right panels represent number of patients with at least one SRE before and during denosumab treatment, respectively. SRE, skeletal-related event.

The timing of an SRE relative to denosumab initiation also impacted risk, with the presence of ≥ 1 SRE after denosumab initiation having a higher impact on SRE risk following denosumab discontinuation than the presence of ≥ 1 SRE before denosumab initiation (Fig. 3). Longer denosumab treatment duration was associated with a lower SRE risk 3 to 12 months following discontinuation. Overall, 38% of patients with ≤ 10 months of denosumab treatment (655 [46.3%]), 32% of patients with 11–20 months of denosumab treatment (280 [19.8%]), and 26% of patients with > 20 months of denosumab treatment (198 [14.0%]) developed SREs 3 to 12 months following discontinuation of denosumab treatment (Fig. 3). The risk of SRE reached a low plateau point after 20 months up to the longest duration point of 66 months, indicating that denosumab treatment durations beyond 20 months did not affect SRE risk. However, there were limited data after 33 months.

Risk factors for SREs identified from the tumor-specific model

Results from tumor-specific model analyses showed similar risk trends for patients with breast cancer, prostate cancer, and lung and other solid tumors as that seen for the tumor-agnostic model analyses, with cumulative number of SREs and denosumab duration ranked as common top risk factors in the tumor-agnostic model and all three tumor-specific models (Fig. 4).

Fig. 4

Common and unique risk factors for breast cancer, prostate cancer, and lung cancer and other solid tumors. *Variable also a top predictor in tumor-agnostic model. SRE, skeletal-related event.

The breast cancer prediction model resulted in an AUROC score of 73% (Table 2). SHAP identified several top-ranked factors that predicted an increased SRE risk following denosumab discontinuation in breast cancer patients (Supplementary Fig. 3 in Supplementary File 2), including denosumab duration of ≤ 8 months, time to denosumab initiation from bone metastasis ≤ 2 months, prior SREs, and ≥ 2 clinic visits per month (with at least one hospitalization [all-cause] event and/or at least one emergency room visit from the baseline period up to discontinuation of denosumab).

The prostate cancer model (AUROC of 82% [Table 2]) revealed the top three SRE risk factors that increased SRE risk 3–12 months after denosumab discontinuation (Supplementary Fig. 4 in Supplementary File 2), denosumab duration ≤ 10 months, ≥ 1 cumulative number of SREs, and > 5 unique anticancer drugs prescribed to a patient since initial solid tumor diagnosis.

Duration of denosumab treatment (≤ 10 months or > 10 months) was the top-ranked risk factor for the “other solid tumors” model including lung cancer (AUROC of 67% [Table 2]). Risk of SRE 3–12 months after denosumab discontinuation remained high till ≤ 10 months of denosumab treatment (Supplementary Fig. 5 in Supplementary File 2).

Discussion

This is the first study identifying key risk factors for SREs after denosumab treatment discontinuation using a machine learning model. Our tumor-agnostic classification model predicted that a higher cumulative number of SREs, timing of SRE relative to denosumab initiation, a higher number of hospital visits, and a shorter denosumab treatment duration were significant factors that are associated with an increased SRE risk after discontinuation of denosumab. Patients with ≥ 1 SRE before and/or during denosumab treatment were found to be at a higher risk of SRE occurrence after denosumab discontinuation compared with patients with no SREs.

The receiver operating characteristic (ROC) curve shows the sensitivity of the classifier by plotting the rate of true positives to the rate of false positives. The ROC curve provides nuanced details about the behavior of the model, but it is difficult to compare many ROC curves to each other. For ease of analysis, the area under the curve (AUC) summarizes the ROC curve into a single number. The perfect model would have a true positive rate of 100% corresponding to an AUROC of 1.0. The performance metrics for our model depicted a high AUROC score (0.774) for the tumor–agnostic model, while prostate cancer had the highest AUROC (0.815) among the tumor–specific models. These scores were comparable to those in other classification model studies for disease prediction or quantitative traits [29], [30], [31].

In our study, 36% of patients in the training set across all tumor types who had ≥ 1 SRE before denosumab initiation (168 [27%]) vs 69% of patients who had ≥ 1 SRE during denosumab treatment (183 [32%]) developed subsequent SRE after denosumab discontinuation. Moreover, of the patients who experienced SREs during denosumab treatment, those with an SRE within the last 6 months of therapy (given a treatment duration > 6 months) were at an even greater risk than those with an SRE during the early phases of therapy. Previous studies have also shown that the time to first SRE and SRE frequency, along with the presence of extraskeletal metastases posed significant risk for the development of SREs [11], [14], [15], [32]. A long-term, retrospective study showed that patients with bone-metastatic breast cancer with a history of palliative radiation therapy and elevated serum calcium levels at the time of BTA initiation were at a high risk of developing SREs [32].

The presence of ≤ 2 previous SREs was also a major predictor of SREs 3 to 12 months following denosumab discontinuation for the tumor-specific models of breast cancer, prostate cancer, and combined lung cancer and other solid tumors. The timing of SREs was a top predictor for patients with all tumor-specific models except prostate cancer. Patients with breast cancer who had SREs during denosumab treatment were more susceptible to developing an SRE after denosumab discontinuation. Patients with other solid tumors became susceptible to developing SREs after denosumab discontinuation if they experienced SREs either before and/or during denosumab treatment, although SREs before denosumab initiation held more weight than those during treatment for this patient group. This suggests that if patients developed SREs during denosumab treatment, it was preferable to continue treatment since, alternately, the patient was at a higher risk of developing additional SREs upon discontinuation of denosumab. This principle is in contrary to cancer therapeutics where treatment failure typically suggests a change of therapy.

Our models predicted that a shorter duration of denosumab treatment was associated with a higher risk of SRE occurrence after denosumab discontinuation across all tumor types compared with longer treatment durations. Of the patients in this analysis, those undergoing denosumab treatment for a period of < 10 months (655 [46.3%]) were at an elevated risk of SRE. Incidentally, 29% of patients in the 1–10 months group, 34% of patients in the 11–20 months group, and 40% of patients in the ≥ 20 months group experienced SREs during denosumab treatment. The impact of denosumab duration on risk decreased incrementally with each additional month beyond 10 months until ∼ 20 months of therapy. Notably, among the patients experiencing SREs during denosumab treatment, 67% in the 1–10 months group, 53% in the 11–20 months group, and 41% in the ≥ 20 months group had SREs after denosumab discontinuation, indicating that extended treatment with denosumab reduced risk of SRE incidence even in patients experiencing SREs during denosumab treatment. SREs in the longer denosumab treatment groups were more likely to be refractory disease in the bone. Studies assessing other BTAs have also demonstrated that patients receiving prolonged (≥ 2 years) BTA treatment were less susceptible to developing SREs [33], [34], [35]. The protective effect of prolonged denosumab treatment duration on SRE risk after denosumab discontinuation was not shared by patients with lung cancer nor by patients under the age of 65 (data not shown). Further research is needed to determine how denosumab treatment duration affects the risk of SRE in these subpopulations.

Denosumab treatment duration across all three tumor types was inversely proportional to SRE risk after denosumab discontinuation. Similar effects were shown previously, with longer denosumab treatment durations found to be superior to zoledronic acid in reducing the risk of SREs after discontinuation of treatment [22], [24], [36], [37]. While prostate cancer patients showed only a mild reduction in risk at treatment durations of > 10 months, other solid tumor patients showed a substantial decrease in SRE risk after 10 months of treatment. However, it was difficult to determine how long-term denosumab treatment duration affected SRE risk in patients with other solid tumors as the median denosumab duration in this cohort was 211 days (range, 88–2,276), with < 5% of the patients being treated with denosumab for > 2 years.

Our model predictions showed that longer time to denosumab initiation was associated with lower SRE risk following denosumab discontinuation. The overall negative association appears counterintuitive because early treatment with a BTA should reduce SRE risk. However, potential prior BTA use and physician bias in treatment initiation patterns might confound the benefit of early denosumab initiation. Patients treated with other classes of BTAs, such as zoledronic acid before denosumab initiation, had a low incidence of SREs following denosumab discontinuation. Previous studies have shown that physicians have a propensity to adopt a BTA among patients with more aggressive disease (e.g., several sites of bone metastases) and presence of clinical symptoms (e.g., pain) compared with little or no BTA adoption among patients with metastatic disease or other “low-risk” features in the real world [38].

Our models predicted a higher risk of SREs with a greater number of clinic visits including ≥ 1 inpatient hospital visit. This prediction is supported by previous reports on the risk of developing SREs in relation to inpatient hospitalizations associated with metastatic bone disease [39], [40], [41]. We found a frequency of 4–6+ visits per month, regardless of visit type from 6 months before diagnosis of bone metastasis up to denosumab discontinuation, were associated with an increased SRE risk 3–12 months following discontinuation. In 569 (40.2%) patients from the training set with no prior SRE, 16% of patients with 0–3 average visits per month, 20% with 4–6 visits per month, and 41% with 7+ visits per month had SRE after denosumab discontinuation. In 564 (39.9%) patients with prior SREs, 42% of patients with 0–3 average visits per month, 47% with 4–6 visits per month, and 59% with 7+ visits per month had SRE after denosumab discontinuation. Therefore, while there appeared to be a correlation between the presence of an SRE and the number of visits prior to denosumab discontinuation, the trend of increasing SRE risk after discontinuation with an increased number of visits persisted even in patients not experiencing any SRE before denosumab discontinuation.

Age was identified as a top-ranked risk factor in both the tumor-agnostic model and other solid tumor model, with younger age (< 56 years) being associated with an increased SRE risk after denosumab discontinuation in both models. It is difficult to determine if this effect was truly caused by rapid progression of disease in younger patients or by censoring bias, as older individuals who died or restarted therapy prior to experiencing an SRE in the follow-up period could be excluded from the study, giving more weight to the younger high-risk patients who survived. Conversely, the underlying disease may be more aggressive at younger ages vs older ages, for example, patients with breast cancer tended to be younger and had lytic metastases [42]. With cancer type, patients with prostate cancer were possibly at higher risk than those with breast cancer since prostate cancer is more aggressive than breast cancer [43], and also because patients with breast cancer were more likely to die in the follow-up period before an SRE than patients with prostate cancer giving more weight to prostate cancer. Whereas some studies report a higher incidence of SREs in patients with lung or breast cancer than in those with prostate cancer [44], [45], individual SREs such as pathological fractures [45] and spinal cord compression [44] occurred at a higher incidence in prostate cancer than in breast cancer. This could be due to the osteoblastic bone metastasis observed in patients with prostate cancer characterized by both bone and tumor proliferation. The metastatic tumor also heightened osteoclastic activity [46], [47]. Further, the bone tissue formed during prostate cancer metastasis is spongy and weak due to the poor alignment of osteoblast cells [47]. Together, a weak bone structure and activation of osteoclasts could increase the risk for SREs in patients with prostate cancer.

Among the specific tumor types analyzed, a higher number of different anticancer drugs was an important risk factor in both prostate cancer and other solid tumor groups, but not in patients with breast cancer. Patients with prostate cancer who received ≤ 4 unique anticancer drugs had fewer SREs after denosumab discontinuation than those who received > 5 anticancer drugs. This could be due to a higher disease burden or more aggressive disease that was progressing through multiple antitumor regimens. In the other solid tumors model, patients receiving ≥ 2 unique anticancer drugs had an elevated risk of SRE after denosumab discontinuation. Efficacious treatment for other solid tumors, specifically lung, is more limited than for prostate cancer possibly because prostate is primarily an osteoblastic bone metastasis and may need more progression before an SRE occurs compared with osteolytic and mixed lesions that are generally at a higher risk for SREs [1], [48]. This might explain why a lower number of anticancer drugs in other solid tumors was sufficient to increase SRE risk after denosumab discontinuation compared with an increased number of drugs required in the prostate cancer model [48]. Importantly, the number of anticancer drugs given at any one time or polypharmacy should not be confused with the treatment regimen each of these drugs is used in. While part of this analysis assessed the number of drugs given at any one time in diverse types of cancer, further investigations are required into how drugs given at particular lines or sequences of therapy may impact the risk of SRE after denosumab discontinuation.

Results from our classification model thus address the significant need to generate data that will help provide evidence for the consequences of denosumab discontinuation and support a narrative towards appropriate and optimal treatment duration with denosumab and how it may unduly expose patients to underlying SRE risk. By extension, this work may also help augment understanding of the appropriate use of denosumab with persistence of indefinite duration as an optimal SRE prevention strategy.

Limitations

A limitation of the model design was a possible overestimation or underestimation of SRE risk for some patient types due to exclusion of patients who died or restarted denosumab before experiencing an SRE within 365 days of follow-up (right censoring). Certain variables identified to be significant SRE risk factors in previous studies (e.g., number of bone metastases, pain progression, bone turnover markers, markers for primary disease progression) could not be captured from this dataset. We were also unable to determine the reasons for denosumab discontinuation such as hypercalcemia, perceived risk of serious adverse events like osteonecrosis of the jaw (ONJ) with prolonged exposure, physician perception that therapy is not working, insurance coverage/drug affordability, or remission of the initial tumor. We could not access all this information from the claims data, specifically when the reason was not clinical in nature and was based on physician’s perception of risk or effectiveness. Mostly, patients with ONJ were taken off therapy based on a general clinical perception of risk; however, likely no ONJ event ever occurred, resulting in the absence of any ONJ claim. Since we did not have access to patient dental records in this dataset, which might have had more detailed information on patient dental history and examinations prior to and during denosumab treatment, it was hard to know retrospectively which patients were at high risk for ONJ. Finally, due to the observational design of the study, these results should not necessarily be interpreted as causal. Further work is needed to determine the causal mechanisms behind the correlations.

Conclusion

In conclusion, our classification model identified risk factors that played a crucial role in predicting the incidence of SREs after denosumab treatment discontinuation. Since the increased frequency of SREs results in higher healthcare and hospitalization costs, often affecting treatment efficacy, SRE risk should be considered to optimize patient outcomes when making denosumab treatment decisions. Our machine learning approach to SRE risk factor identification has the potential to help clinicians better assess a patient’s need for denosumab treatment persistence and improve patient outcomes.

CRediT authorship contribution statement

Dionna Jacobson: Conceptualization, Methodology, Supervision, Visualization, Writing – original draft, Project administration, Data curation, Writing – review & editing, Formal analysis, Resources, Investigation, Software, Validation. Benoit Cadieux: Conceptualization, Methodology, Supervision, Visualization, Writing – original draft, Writing – review & editing, Formal analysis, Resources, Investigation, Validation. Celestia S. Higano: Writing – review & editing, Formal analysis. David H. Henry: Writing – review & editing, Formal analysis. Basia A. Bachmann: Project administration, Data curation, Visualization, Writing – review & editing, Formal analysis, Resources, Investigation, Validation. Marko Rehn: Conceptualization, Methodology, Supervision, Visualization, Writing – original draft, Writing – review & editing, Formal analysis, Resources, Investigation, Validation. Alison T. Stopeck: Writing – review & editing, Formal analysis. Hossam Saad: Writing – review & editing, Formal analysis, Resources, Investigation.

Declaration of Competing Interest

Dionna Jacobson, Benoit Cadieux, and Basia A. Bachmann, were employees of Amgen. Marko Rehn and Hossam Saad are employees and stockholders of Amgen.

Celestia S. Higano – consulting fees: Bayer, Ferring, Clovis Oncology, Blue Earth Diagnostics, Janssen, Hinova, Pfizer, AstraZeneca, Carrick Therapeutics, Novartis, Merck Sharp & Dohme, Astellas Pharma, Myovant Sciences, Genentech, Menarini; contracted research support: Aragon Pharmaceuticals, AstraZeneca, Medivation, Emergent BioSolutions, Bayer, Roche, Astellas Pharma, Clovis Oncology, Ferring Pharmaceuticals, eFFECTOR Therapeutics; ownership interest: CTI BioPharma CORP; honoraria: Astellas Pharma.

David H. Henry has nothing to disclose.

Alison T. Stopeck – consulting fees: Amgen, AstraZeneca, Athenex, BeiGene, Exact Sciences; contracted research support: Amgen, Genomic Health, AstraZeneca, Seattle Genetics; speaker’s bureau: Genomic Health; honoraria: Amgen.