Dynamic prediction of overall survival: a retrospective analysis on 979 patients with Ewing sarcoma from the German registry.

OBJECTIVES: This study aimed at developing a dynamic prediction model for patients with Ewing sarcoma (ES) to provide predictions at different follow-up times. During follow-up, disease-related information becomes available, which has an impact on a patient's prognosis. Many prediction models include predictors available at baseline and do not consider the evolution of disease over time.
SETTING: In the analysis, 979 patients with ES from the Gesellschaft für Pädiatrische Onkologie und Hämatologie registry, who underwent surgery and treatment between 1999 and 2009, were included.
DESIGN: A dynamic prediction model was developed to predict updated 5-year survival probabilities from different prediction time points during follow-up. Time-dependent variables, such as local recurrence (LR) and distant metastasis (DM), as well as covariates measured at baseline, were included in the model. The time effects of covariates were investigated by using interaction terms between each variable and time.
RESULTS: Developing LR, DM in the lungs (DMp) or extrapulmonary DM (DMo) has a strong effect on the probability of surviving an additional 5 years with HRs and 95% CIs equal to 20.881 (14.365 to 30.353), 6.759 (4.465 to 10.230) and 17.532 (13.210 to 23.268), respectively. The effects of primary tumour location, postoperative radiotherapy (PORT), histological response and disease extent at diagnosis on survival were found to change over time. The HR of PORT versus no PORT at the time of surgery is equal to 0.774 (0.594 to 1.008). One year after surgery, the HR is equal to 1.091 (0.851 to 1.397).
CONCLUSIONS: The time-varying effects of several baseline variables, as well as the strong impact of time-dependent variables, show the importance of including updated information collected during follow-up in the prediction model to provide accurate predictions of survival. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

A dynamic prediction model predicts survival at different times during follow-up.

Updated patient information is used for predictions as it becomes available.

Local recurrence and distant metastasis are included in the model.

The effect of several risk factors changes over time.

Dynamic prediction is graphically illustrated for selected patients.

Introduction

Ewing sarcoma (ES), first introduced in 1921 by James Ewing, is a type of tumour that forms in bone or soft tissue.1 It usually occurs in adolescents and young adults and accounts for 2% of childhood cancers.2 Several types of treatment are available: chemotherapy, radiotherapy, surgery, etc.3 Five-year overall survival (OS) for localised disease is between 65% and 75%, and patients with metastases have 5-year OS less than 30%.4

Previous studies consider the effect of risk factors from baseline.5 6 Such models can predict OS from the start of the study, however results become inaccurate during follow-up. Over time the baseline covariates do not change, however their association with OS may change. Additionally, events such as local recurrence (LR) or distant metastasis (DM) may occur during the evolution of the disease, which changes a patient’s prognosis. The occurrence of intermediate events is not accounted for in these models. Other studies choose these intermediate events as outcome variables.7 8 However, it remains difficult to evaluate the relationship among them. A dynamic prediction model overcomes these problems, as it assesses the survival probability from different times . Such a model can be developed using a landmarking approach.9

The primary aim of this study was to develop a dynamic prediction model for OS based on a large cohort of patients with ES treated according to the EURO-E.W.I.N.G 99 (EE99) protocol (EUROpean Ewing tumor Working Initiative of National Groups-Ewing Tumor Studies).10 The model predicts the probability for a patient to survive an additional 5 years from a particular prediction time point. Patients with specific characteristics are used to illustrate the results of the prediction model.

Methods

Patients

A retrospective study based on patients from the Gesellschaft für Pädiatrische Onkologie und Hämatologie registry treated in or according to the EE99 protocol was conducted. All patients were treated between 1999 and 2009 and were followed until 2017. They were assigned to six courses of vincristine, ifosfamide, doxorubicin and etoposide chemotherapy as induction treatment followed by a local treatment according to the protocol. Local treatment consisted of surgery with or without radiotherapy or definitive radiotherapy. Surgery was preferred if viable, and was avoided only when an inoperable lesion could not be completely removed or when the tumours were located on critical sites where a complete surgery was dangerous. Implications of applying preoperative radiotherapy included the clinical progression of tumour extension or anticipated marginal or intralesional resectability. Postoperative radiotherapy (PORT) was advised in case of intralesional or marginal surgery and poor histological response (<90% necrosis) regardless of surgical margins.10 Maintenance therapy was applied to patients after local treatment.

The original EE99 data contained 1175 patients, but only patients who underwent surgery (with or without radiotherapy) were used to build the dynamic model. A total of 979 patients, 470 study patients and 509 registry patients (treated complying with the protocol but not randomised), were eligible for the analysis.

Measures

Predictors measured at baseline were8: gender, age (0–10, 11–18, and >18 years old), primary tumour localisation (non-pelvic/pelvic), tumour volume at diagnosis (<200 mL/≥200 mL), histological response (<90%/90%–99%/100% necrosis), surgical margin (wide/marginal/intralesional), disease extent at diagnosis (localised/pulmonary-only metastasis/other or combined metastasis) and adjuvant therapy (no PORT/PORT). These variables were chosen based on previous studies. Time-dependent variables, recorded during follow-up, were LR, DM pulmonary-only (DMp) and DM extrapulmonary with or without pulmonary metastasis (DMo).

Histological response was determined by the percentage of necrosis in the specimen, where the group ‘100% necrosis’ indicated no viable tumour cells. Surgical margins were considered as wide and marginal/intralesional. Disease extent at diagnosis and tumour volume were combined into one categorical covariate: localised disease and tumour volume <200 mL (R1); localised disease and tumour volume ≥200 mL (R2loc); pulmonary-only metastasis and any tumour volume (R2pulm) and extrapulmonary metastasis with or without pulmonary metastasis and any tumour volume (R3). LR was defined as local regional recurrence after initial complete response. DM was defined as new metastatic disease or recurrence of metastatic disease after initial complete response. As a result, seven baseline prognostic factors and three follow-up variables were included in the analysis.

Statistical analysis

Kaplan-Meier’s methodology was applied to estimate OS. To estimate 5-year dynamic OS (DOS), a proportional landmark supermodel was employed.9 11 12 A landmark model uses updated information from patients still in follow-up at a specific landmark time point to estimate survival. A landmark supermodel combines landmark models built for different to make predictions from different prediction times .

To build the landmark supermodel, were set every 3 months between 0 and 5 years after surgery. At each , a corresponding landmark data set was constructed by selecting all patients still in follow-up. A Cox proportional hazards model was estimated based on all covariate history information available at time . The status of LR, DMp and DMo at each landmark time was determined and treated as a fixed factor. These landmark Cox models were then combined into a supermodel.

All main covariate effects, the linear and quadratic terms of time ( and ) and all interactions of covariates with and were included in the model. Interactions allow for the effect of a covariate to change over time. A backward selection was employed to select time-varying effects for each variable. Interactions with and were dropped if they were not significant to obtain the final model. A p value of ≤0.05 was considered significant.

The validity of the prognostic model can be evaluated by determining model calibration. A well-calibrated model ensures small differences between observed and predicted survival probabilities. The model was internally calibrated by the heuristic shrinkage factor which is used in the case that no external data set is available.9 Its value ranges from 0 to 1, where a value approaching 1 suggests a good calibration.

Another measure of model strength is the discriminative ability, which indicates whether the model predicts higher risk for patients with an early death event compared with those with later or no event. To assess the discriminative ability, the concordance index (C-index) was used.9 13 A C-index equal to 0.5 refers to ‘no discriminative ability’, while a C-index equal to 1 indicates perfect discrimination. In the dynamic setting with a prediction window of years, the C-index at time t was obtained by considering event times within the window To account for overfitting, a cross-validated prognostic index was used to compute the C-index.

Among 979 patients, 206 had missing information about some covariates. As most statistical methods are based on complete data sets, multiple imputation was applied by using the R-package Amelia II.11 14 Ten complete data sets were imputed and used for all statistical analyses. All 10 data sets shared the same observed values with the original data, while missing values were imputed using estimated sufficient statistics. The analyses were performed on all imputed data sets and results were combined by using Rubin’s rule.15 All statistical analyses were conducted in the R-software environment.16

Patient and public involvement

Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this study.

Results

This study included 979 patients, with a median follow-up of 7.6 years (95% CI 7.2 to 8.0 years, right panel figure 1) estimated by reverse Kaplan-Meier method.17 The 5-year OS for the whole cohort was 68% (95% CI 65% to 71%, left panel figure 1).

Figure 1

Estimated survival (left panel) and censoring distributions (right panel).

Table 1 shows the characteristics of all patients, 470 study and 509 registry patients. Apart from the number of patients presenting with extrapulmonary metastasis (study: n=0, registry: n=83), there was no difference in the covariate distribution between registry and study patients. At the time of surgery, about 20% of patients with ES had metastatic disease.

Table 1

Patient demographics and treatment characteristics after surgery

Characteristic	N (%)
Total	979
Gender
Male	587 (60.0)
Female	392 (40.0)
Age (years)
0–10	252 (25.7)
11–18	451 (46.1)
>18	276 (28.2)
Primary tumour localisation
Non-pelvic	810 (82.7)
Pelvic	169 (17.3)
Tumour volume at diagnosis (mL)
<200	576 (58.8)
≥200	403 (41.2)
Disease extent at diagnosis
Localised	769 (78.5)
Pulmonary-only metastasis	127 (13.0)
Extrapulmonary metastasis	83 (8.5)
Histological response (%)
100	360 (36.8)
90–99	223 (22.8)
<90	206 (21.0)
Missing	190 (19.4)
Surgical margin
Wide	685 (70.0)
Marginal	150 (15.3)
Intralesional	97 (9.9)
Missing	47 (4.8)
Postoperative radiotherapy
Surgery only	547 (55.9)
Surgery+PORT	432 (44.1)

PORT, postoperative radiotherapy.

Figure 2 provides the number of patients at risk at each landmark time point and the status of patients at that time. In total, 336 (34%) patients died, 53 (5%) developed LR, 76 (8%) DMp and 207 (21%) DMo.

Figure 2

Number of patients at risk at each landmark time point . Left panel: red—patients with LR; blue—patients without LR. Middle panel: red—patients with DMp; blue—patients without DMp. Right panel: red—patients with DMo; blue—patients without DMo. DMo, distant metastasis extrapulmonary; DMp, distant metastasis pulmonary only; LR, local recurrence.

Dynamic prediction model

Table 2 displays the HR along with 95% CI for all risk factors in the dynamic model. The covariates with only time-constant effects are displayed in the upper part of the table. No significant difference in survival was found between males and females, and between wide and marginal/intralesional surgical margins. A significant association between age and survival was found with HR for a patient older than 18 years compared with a patient under 10 years equal to 1.812 (95% CI 1.268 to 2.590). LR, DMp and DMo showed a significant time-constant effect with HR equal to 20.881 (95% CI 14.365 to 30.353), 6.759 (95% CI 4.465 to 10.230) and 17.532 (95% CI 13.210 to 23.268), respectively. A sensitivity analysis is provided in online supplemental file 1, where the dynamic model was fitted on the complete data (without the use of imputation).

Table 2

Estimated HR along with their 95% CI (n=979) for all prognostic factors

	Log(HR)	HR	95% CI	P value
Covariates with time-constant effects
Gender (female vs male)	0.061	1.063	0.830 to 1.362	0.629
Age (years)
0–10
11–18	0.160	1.174	0.830 to 1.660	0.366
>18	0.595	1.812	1.268 to 2.590	0.001
Surgical margin (marginal/Intralesional vs wide)	0.029	1.029	0.776 to 1.365	0.798
Local recurrence (yes vs no)	3.039	20.881	14.365 to 30.353	<0.001
Distant metastasis pulmonary-only (yes vs no)	1.911	6.759	4.465 to 10.230	<0.001
Distant metastasis extrapulmonary (yes vs no)	2.864	17.532	13.210 to 23.268	<0.001
Covariates with time-varying effects
Prediction time (ref: time of surgery, per year)
t_p	−0.451	0.637	0.502 to 0.809	<0.001
tp2	0.045	1.046	0.999 to 1.094	0.059
Primary tumour localisation
Constant
Pelvic versus non-pelvic	0.332	1.394	1.022 to 1.903	0.036
Linear time-varying effect
Pelvic versus non-pelvic	−0.217	0.805	0.678 to 0.955	0.013
PORT
Constant
PORT versus no PORT	−0.256	0.774	0.593 to 1.010	0.061
Linear time-varying effect
PORT versus no PORT	0.408	1.504	1.196 to 1.893	<0.001
Quadratic time-varying effect
PORT versus no PORT	−0.066	0.937	0.892 to 0.983	0.009
Histological response
Constant
100% necrosis
90%–99% necrosis	0.356	1.428	1.043 to 1.954	0.028
<90% necrosis	0.871	2.389	1.725 to 3.307	<0.001
Linear time-varying effect
100% necrosis
90%–99% necrosis	−0.084	0.920	0.711 to 1.191	0.538
<90% necrosis	−0.661	0.516	0.386 to 0.691	<0.001
Quadratic time-varying effect
100% necrosis
90%–99% necrosis	0.005	1.005	0.951 to 1.063	0.774
<90% necrosis	0.119	1.126	1.059 to 1.198	<0.001
Disease extent
Constant
R1
R2loc	0.508	1.662	1.218 to 2.269	0.001
R2pulm	0.977	2.657	1.850 to 3.817	<0.001
R3	1.409	4.091	2.693 to 6.215	<0.001
Linear time-varying effect
R1
R2loc	−0.187	0.830	0.638 to 1.078	0.163
R2pulm	−0.339	0.713	0.531 to 0.957	0.025
R3	−0.452	0.636	0.410 to 0.989	0.045
Quadratic time-varying effect
R1
R2loc	0.039	1.040	0.988 to 1.095	0.139
R2pulm	0.063	1.065	1.001 to 1.132	0.045
R3	0.057	1.058	0.936 to 1.196	0.367

PORT, postoperative radiotherapy.

The other covariates, that is, primary tumour localisation, PORT, histological response and disease extent, were found to have time-varying effects. To illustrate how time affects the HR for the covariate PORT (PORT vs no PORT), in table 3 HRs for 5-year DOS for patients treated with PORT at different prediction time points are shown. The HR at 1 year () after surgery for a patient who was given radiotherapy compared with a patient who did not get radiotherapy is computed as follows:

Table 3

HRs along with their 95% CIs for 5-year dynamic overall survival for a patient treated with PORT after surgery at different prediction time points (reference category no PORT)

t_p	Constant	Linear time-varying effect	Quadratic time-varying effect	HR	95% CI	P value
0	0.774	1.504^0	0.937^0	0.774	0.594 to 1.008	0.058
1	0.774	1.504^1	0.937^1	1.091	0.851 to 1.397	0.492
2	0.774	1.504^2	0.937^4	1.350	0.994 to 1.833	0.055
3	0.774	1.504^3	0.937^9	1.466	1.002 to 2.145	0.049
4	0.774	1.504^4	0.937^16	1.398	0.848 to 2.304	0.189
5	0.774	1.504^5	0.937^25	1.171	0.578 to 2.373	0.662

PORT, postoperative radiotherapy.

Patients treated with PORT have 0.774 times lower risk of dying within 5 years than those without PORT immediately after surgery. After 1 year the HR increases to 1.091 and after 2 years to 1.350 (see table 3). Figure 3 displays the values of HR over time and indicates that PORT has a protective effect immediately after surgery, but the effect declines with time.

Figure 3

Time-varying HR for the PORT. Blue: with PORT; red: no PORT (reference). Dashed line: pointwise CI for HR of PORT. PORT, postoperative radiotherapy.

The time-varying effects of tumour in the pelvis, histological response and disease extent can be computed and interpreted in the same way. Tumour in the pelvis seems to have a strong effect right after surgery, but the impact declines with time. Figure 4 displays the change in HRs for histological response and disease extent at diagnosis as a function of time. The left panel shows that poor histological response has a very strong effect immediately after surgery which declines with time. The right panel shows that the HR of R3 strongly decreases with time. More detailed information about the values of the HRs at different prediction time for histological responses and disease extent status are shown in table 4.

Figure 4

Time-varying HR for the histological response (left panel) and disease extent at the diagnosis (right panel). The red lines represent reference categories.

Table 4

HRs and their 95% CIs for 5-year dynamic overall survival for a patient with different histological responses (reference category 100% necrosis) and different extent disease at diagnosis (reference category R1) at different prediction time points

Histological response (‘100% necrosis’ as the reference)
t_p	90%–99% necrosis		<90% necrosis
	HR	95% CI	HR	95% CI
0	1.428	1.046 to 1.950	2.389	1.037 to 1.966
1	1.320	0.987 to 1.767	1.388	0.988 to 1.764
2	1.233	0.863 to 1.762	1.023	0.862 to 1.763
3	1.163	0.743 to 1.822	0.955	0.752 to 1.799
4	1.108	0.611 to 2.009	1.131	0.632 to 1.943
5	1.066	0.460 to 2.470	1.698	0.482 to 2.360

Dynamic prediction for selected patients

Figure 5 displays the probabilities of dying within 5 years at different time points for patients with specific characteristics and different disease progression status. This figure illustrates how severely LR affects the 5-year DOS probability. In figure 5A, the probability of death within 5 years from various times is shown for two male patients younger than 10 years, with wide margin, tumour in a non-pelvic area, R1 disease extent, 100% necrosis for histological response, treated with radiotherapy and without DM. The probability of dying within 5 years at different for a patient with and without LR is depicted by the red and blue lines, respectively. For example, if a patient is still alive after 1 year from surgery, the probability of dying within the next 5 years from prediction time is 85% and 10% for a patient with and without LR, respectively. A large difference is observed between these two patients; however, the difference becomes smaller over time. Figure 5B, C shows that the probability of dying within 5 years is close to one for the entire period for patients with the same characteristics as those in figure 5A but with DMp and DMo respectively. Figure 5D, E, F displays the probability of dying within 5 years from different times for two patients with other characteristics.

Figure 5

5-year probability of death estimates for patients with different characteristics and at different states of disease progression. (A–C) ‘0–10 years old’, wide margin, tumour in non-pelvic area, R1 disease extent condition at diagnosis, 100% necrosis for histological response and with radiotherapy treatment. (A) Patient without DM at the time of prediction (). (B) Diagnosed with DMp before . (C) Patient diagnosed with DMo before . (D, F) ‘>18 years old’, wide margin, tumour in pelvic area, R3 disease extent condition at diagnosis, <90% necrosis for histological response and without radiotherapy treatment. (D) Patient without DM at . (E) Diagnosed with DMp before . (F) Patient diagnosed with DMo before . Blue line: without LR; red line: with LR. DM, distant metastasis; DMo, DM extrapulmonary; DMp, DM pulmonary-only; LR, local recurrence.

Model calibration and discrimination

A heuristic shrinkage factor of 0.993 indicated a good model calibration. To assess the discriminative ability of the model, dynamic cross-validated C-indices were computed at six time points. Values were equal to 0.67, 0.76, 0.84, 0.85, 0.79 and 0.78 at 0, 1, 2, 3, 4 and 5 years since surgery, respectively. The high values of the C-index indicate good discrimination of the dynamic prediction model.

Discussion

The dynamic model predicts the probability of surviving 5 years from prediction time points () after surgery. It has been applied to ES data to enable predictions based on updated information during follow-up. This model can be used to support personalised treatment decisions for patients with ES at different time points during follow-up. The model is well calibrated, as shown by the high value of the heuristic shrinkage factor, and exhibits good discrimination as indicated by the high C-index values.

LR and DMo present high HR values and time-constant effects. These results suggest that patients who experience one of these adverse events after surgery have about 20 times higher risk of dying within 5 years than those who do not. Several studies have shown the negative effects of experiencing LR and DM. The OS of patients with relapsed ES is poor, with 5-year OS equal to 23%; patients who relapsed within 2 years after diagnosis of primary disease have 5-year OS equal to 7%.18 19

Since the intermediate events (LR, DMp and DMo) have strong effects on patients’ prognoses, it is crucial to update predictions over time in order to get accurate information. This study provides new insight into the change of the effect of risk factors over time and finds some significant time-varying effects. PORT, one of the time-varying effect variables, was protective at the beginning, but no significant association with survival was found as time goes by. However, whether to give PORT treatment is controversial. The results of this research are in line with several studies of ES where the decision to give RT after surgical excision in ES depends on multiple factors.7 20 The prognostic effects of the primary tumour location,21 histological response,22 tumour size and disease extent at diagnosis21 23 24 are consistent with previous studies. The dynamic model takes updated information into account and provides predictions by using a simple model, which can be applied in individualised treatment decisions. Another strong point of this study is the large cohort of patients.

However, some limitations are present. The occurrence of LR is relatively small with only 53/979 (5.41%) and some covariates have unbalanced subgroup sizes (eg, surgical margin). More than half of the patients were treated according to the protocol, and the assignment of radiotherapy did not completely follow recommendations. Therefore, some caution is required in the interpretation of the results.

This study applies dynamic prediction methodology to estimate 5-year DOS probability by using updated patient information. The results of this research are important contributions to the field. To the best of the authors’ knowledge, this is the first study where DOS for patients with ES is addressed.