Development and validation of a prediction index for hand-foot skin reaction in cancer patients receiving sorafenib.

BACKGROUND: This study describes a repeated measures prediction index to identify patients at high risk of ≥grade 2 hand-foot skin reaction (HFSR) before each week of sorafenib therapy.
METHODS: Data from 451 patients who received a sorafenib (400 mg bid) as part of a clinical trial were reviewed (Escudier B, Eisen T, Stadler WM et al. Sorafenib in advanced clear-cell renal-cell carcinoma. N Engl J Med 2007; 356: 125-134). Generalized estimating equations were used to develop the final risk model. A risk-scoring algorithm (range 0-58) was then derived from the final model coefficients. External validation was then carried out on a new sample of 1145 patients who received sorafenib under an expanded access program.
RESULTS: Pretreatment white blood cell count, female gender, good performance status, presence of lung and liver metastases and number of affected organs were predictors for ≥grade 2 HFSR. A nonlinear association between HFSR risk and treatment duration was also identified where risk was maximized at week 5 followed by a gradual decline. Before each week of therapy, patients with risk scores>40 would be considered at high risk for developing ≥grade 2 HFSR.
CONCLUSIONS: The application and planned continued refinement of this prediction tool will be an important source of patient-specific risk information for the development of moderate to severe HFSR.

RCT Entities:   Population Interventions Outcomes

introduction

Sorafenib is an orally administered inhibitor of several tyrosine protein kinases, including VEGFR, PDGFR and Raf [1]. It has been approved by regulatory agencies for use in patients with advanced renal cell (RCC) and hepatocellular carcinoma (HCC) following the results of randomized trials that demonstrated an improvement in progression-free and overall survival [2-4]. As with many other targeted therapies, sorafenib is associated with dose-limiting adverse events (AEs) [5]. Among these, hand-food skin reaction (HFSR) is noteworthy due to a painful erythema, edema and desquamation of the palms and soles, which can lead to decreased quality of life [6, 7]. In the pivotal randomized trial in advanced RCC, the incidence of grade ≥2 HFSR was ∼18% [2]. In a recent meta-analysis of phase II and III studies and expanded access programs of patients with solid tumors, the risk of HFSR in patients receiving sorafenib was increased more than seven times relative to control [relative risk = 7.50, 95% confidence interval (CI) 3.9–14.4] [8]. Prospective evaluations of patients on sorafenib showed that HFSR developed in 60% of patients, with 23% being grade ≥2 in severity [9].

Grade ≥2 HFSR is of clinical concern because it can lead treatment delays, dose reductions and/or premature treatment discontinuation of sorafenib, even in situations where other toxic effects may be manageable and the disease is responding to therapy. Even though risk factors for HFSR such as female gender and patient performance status have been identified, they have not been specific to sorafenib [10, 11]. Therefore, occurrences of HFSR during sorafenib therapy are largely considered to be unpredictable. As a result, oncologists generally take action only after the event occurs (i.e. reactively). This involves trying to ‘rescue’ the patient from a subjectively unpleasant situation and then making adjustments to the regimen such as dose reductions, delays and/or the initiation of supportive care interventions. Clinical care could be substantially improved and dose intensity could be maintained if these episodes of significant HFSR could be accurately predicted, with steps taken in advance to prevent their occurrence in the first place (i.e. proactively). Such steps might include the use of appropriate supportive care medication as well as forewarning the patient and initiating a more intensive early monitoring scheme and action plan for early intervention.

The realities of health care systems around the world preclude such arrangements being put in place, for all patients, throughout all cycles of sorafenib therapy. What may be possible, however, is a highly focused strategy based on the accurate prediction of patients at higher than average risk, applied ‘just-in-time’ to preempt episodes of grade ≥2 HFSR. In other words, it should be possible and economical to intervene preventatively if we knew who was at higher risk and when (i.e. at what point in time) the risk would become unacceptably high. These goals could be achieved through the development and validation of an HFSR prediction tool that could be applied at multiple time points (i.e. repeated measures).

The risk of HFSR during sorafenib therapy may vary over time. This has been the case with other toxic effects such as neutropenia and emesis [12, 13]. Therefore, the advantage of a repeated measures compared with a pretreatment only tool is that the former allows risk to be continually reassessed with each additional week of sorafenib therapy. Repeated measures models have been successfully developed for other toxic effects such as chemotherapy induced emesis and neutropenia [13, 14]. In this study, the development of a repeated measures (i.e. by week) prediction index for grade ≥2 HFSR in patients receiving sorafenib is described.

materials and methods

patients and treatment

Patient data for developing the repeated measures risk index for HFSR were obtained from the study reported by Escudier et al. [2], a large randomized placebo controlled trial evaluating sorafenib in patients with advanced RCC. In this clinical trial, a total of 903 advanced-stage RCC patients who were previously exposed to cytokines were randomized to receive oral sorafenib 400 mg twice daily (n = 451) or to placebo. Baseline data collection included patient demographic information, Eastern Cooperative Oncology Group performance status, biochemistry, number and site of metastases, median duration of disease and previous therapies. Over the first 8 weeks of sorafenib therapy, 43 patients developed ≥ grade 2 HFSR. By the end of the study, the median duration of sorafenib was 23 weeks with 80 of 451 (17.7%) patients developing ≥ grade 2 HFSR as defined by the National Cancer Institute–Common Toxicity Criteria (NCI–CTC). Therefore, the primary risk model end point was grade ≥2 HFSR as defined by the NCIC–CTC over the first 8 weeks of therapy.

predictive factors for HFSR and development of scoring index

There is evidence to suggest that sorafenib induced HFSR is most prevalent within the first 4 weeks of therapy [15]. Consequently, clinical outcomes data for the first 8 weeks of sorafenib therapy were used to build the risk model and associated scoring index. For the first 8 weeks of sorafenib therapy, the database from the Escudier et al., trial provided 3608 weeks of complete outcomes data. Patient demographic and clinical variables were screened for possible inclusion into the risk model. To identify the set of factors with the largest potential contribution to HFSR risk, those with a P value ≤0.25 in a simple logistic regression with the dependent variable of HFSR were retained for further consideration. This is a recommended approach for removing weak prognostic covariates so that a more manageable set of variables can be submitted to multivariate techniques [16].

To determine the final set of risk factors, generalized estimating equations (GEE) were used, which adjusts for the clustering by week of therapy within a patient [17, 18]. A GEE model was chosen since multiple weekly observations for the same patient would be expected to violate the independence assumption of standard logistical regression. The set of initially retained risk factors was analyzed in the GEE model. The likelihood ratio test was then used in a backwards elimination process (P < 0.05 to retain) to select the final set of risk factors for retention into the model. To evaluate the presence of a nonlinear relationship between HFSR risk and week of therapy, the quadratic form of time (i.e. time2) was assessed in the model. The goodness of fit of the final model was then assessed with the Hosmer–Lemeshow test. Model calibration was evaluated by estimating a smooth calibration line between the observed and predicted outcomes [19]. The calibration curve would equal one (optimal) if the observed and predicted probabilities agree perfectly.

Nonparametric bootstrapping was applied to test the internal validity of the final prediction model [20, 21]. Resampled data (1000 iterations) were used to generate bootstrap estimates of the regression coefficients of the multivariable model. The CIs of the regression coefficient estimates from the bootstrap sampling were then compared with the values calculated by the GEE regression analysis.

From the GEE regression model, the contribution of the individual risk factors to HFSR risk was weighted with the model coefficients. To simplify calculations using these weights in the risk algorithm, the coefficients were transformed by multiplying each by a constant (derived by trial and error) and then rounding to the nearest unit value. A summary of HFSR risk score was assigned to each patient by simply adding up transformed coefficient values (points) for each risk factor they possessed.

external validation of HFSR prediction index

The external validation sample consisted of 1145 patients (a mix of advanced-stage RCC and HCC) who received sorafenib as part of a European expanded access program (i.e. EU-ARCCS) [22]. This database provided 9160 weeks of complete outcomes data. However, it is important to note that since this data were not derived from a clinical trial, there were some baseline variables such as hemoglobin (Hb) and white blood cell (WBC) data that were not collected.

The predictive accuracy of the final risk scoring index was then determined by measuring the specificity, sensitivity and area under the receiver operating characteristic (ROC) curve [23, 24]. Discrimination refers to the ability of a diagnostic test or predictive tool to accurately identify patients at low and high risk for the event under investigation and is often presented as the area under the ROC curve. A predictive instrument with an ROC of ≥0.70 is considered to have good discrimination, and an area of 0.5 is equivalent to a ‘coin toss’. All the statistical analyses were carried out using Stata, V11.0 (Stata Corp., College Station, TX).

results

The clinical and disease characteristics of patients in the model derivation and validation samples are presented in Table 1. Patients in both groups were comparable with respect to mean age, gender and presence of lung and liver metastases. However, there were a higher proportion of patients in the derivation dataset with a performance status of zero or one and with multiple organ involvement. Notwithstanding, it is important to recall that this is not a randomized trial but an exercise to develop an HFSR prediction model from unique patient samples. Therefore, imbalance between model derivation and validation samples should be expected and even encouraged to ensure that the prediction model can be applied to a variety of patients being treated with sorafenib.

Table 1.

Characteristics of patients in the derivation and validations samples

Characteristic	Derivation sample (n = 451)a	External validationsample (n = 1145)a
Mean age (range)	59.1 (19–86)	60.9 (18–84)
Female gender	30.2%	25.3%
ECOG performance status (%)
0	48.7	40.1
1	49.8	45.2
2	1.6	14.7
Mean baseline Hb (g/dl) (SD)	12.8 (2.1)	N/A
Mean baseline WBC (×109 cells/l)	7.4 (2.8)	N/A
Median number of organs involved (range)	3 (1–3)	2 (1–3)
Multiple (more than one) organ involvement	85.8%	76.1%
Presence of liver metastases	79.6%	73.0%
Presence of lung metastases	25.7%	27.7%
Development of grade ≥2 HFSR	43 affected patientsover 8 weeks (9.5%)	220 affected patientsover 8 weeks (19.2%)

aThe model derivation sample consisted of RCC patients treated within a clinical trial. In contrast, the external validation sample consisted of both RCC and hepatocellular carcinoma who received sorafenib under an expanded access program.

ECOG, Eastern Cooperative Oncology Group; Hb, hemoglobin; HFSR, hand-foot skin reaction; N/A, data not available; RCC, renal cell carcinoma; SD, standard deviation; WBC, white blood cell.

development of HFSR risk prediction model

After the initial univariate screening of potential predictor factors, the variables associated with HFSR retained for further analysis were gender, age, performance status, baseline Hb and WBC count, presence of liver and lung metastases and number of organs involved. The development of the prediction model was continued with multivariable GEE regression analysis and the backwards elimination process. The final variables retained in the model that were significant predictive factors (P < 0.05) for HFSR were female gender, patient performance status of one or two, presence of liver and lung metastases, two or more organs involved and a normal baseline WBC count (Table 2). A nonlinear association between HFSR risk and duration of therapy was also identified where risk was maximized at week 5 followed by a gradual decline (Figure 1). The CIs of regression coefficient estimates from the bootstrap sampling were comparable with the values calculated by the GEE regression analysis, supporting the internal validity of the model.

Table 2.

Final HFSR prediction model developed from the derivation dataset

Variablea	Odds ratio (95% CI)b	Impact on HFSR relative risk
Female gender	1.68 (1.30–2.18)	Increased by 68%
ECOG PS ≥1	0.50 (0.38–0.65)	Decreased by 50%
Lung metastases at baseline	0.58 (0.43–0.79)	Decreased by 42%
Liver metastases at baseline	1.75 (1.33–2.29)	Increased by 75%
2 or more organs involved	1.71 (1.10–2.66)	Increased by 71%
Baseline WBC >5.5 (×109 cells/l)	1.56 (1.15–2.09)	Increased by 56%
Time (week of therapy)	1.54 (1.18–2.0)	Increased by 54% per week of therapy
Time2	−0.96 (0.93–0.99)	Nonlinear association between risk and time

aThese are the final variables that were retained following the application of the likelihood ratio test (P < 0.05 to retain) in a backwards elimination process.

b95% CI determined by nonparametric bootstrapping.

CI, confidence interval; ECOG PS, Eastern Oncology Cooperative Oncology Group performance status; HFSR, hand-foot skin reaction; WBC, white blood cell.

Figure 1

Probability of developing hand-foot skin reaction for a female renal cell carcinoma patient, ECOG 1, liver metastases only and white blood cell 7.0. ECOG, Eastern Oncology Cooperative Oncology Group.

development of HFSR scoring system

A risk scoring system was then developed from the point estimates of the regression coefficients and the intercept generated from the analysis. Each of the final regression coefficients retained in the model provided a statistical weight for that factor's contribution to the overall risk of HFSR. The scoring system was then adjusted by adding a constant across all scores to ensure that none were <0. The final product was a scoring system between 0 and 58 where higher scores were associated with an increased risk for a HFSR event (Table 3).

Table 3.

Risk scoring system for ≥ grade 2 HFSR in patients receiving sorafenib

Predictive factor	Start sorafenib
Initial starting score	Initial score = 20
Impact of baseline factors
Female gender	Add 6
ECOG PS ≥1	Subtract 7
Lung metastases at baseline	Subtract 7
Liver metastases at baseline	Add 6
2 or more organs involved	Add 9
Baseline WBC >5.5	Add 5
Impact of HFSR risk by week of therapy
Week 1	Add 4
Week 2	Add 7
Week 3	Add 10
Week 4	Add 11
Week 5	Add 12
Week 6	Add 11
Week 7	Add 10
Week 8	Add 8
Total composite risk scorea	?

aThe probability of developing HFSR during that given week while on sorafenib therapy can then be estimated from supplemental Figure S1 or supplemental Table S1 (available at Annals of Oncology online).

ECOG PS, Eastern Oncology Cooperative Oncology Group Performance Status; HFSR, hand-foot skin reaction; WBC, white blood cell.

Factors that elevate the overall score are considered to be positive risk factors for HFSR. For instance, female gender requires the addition of six units and is thus a risk factor for the development of HFSR. In contrast, patients with a performance status of one or two are at a reduced risk for HFSR, so such patients would have seven units subtracted from their cumulative risk score. This risk scoring system can then be applied to an individual patient before the start of sorafenib and then weekly in order to monitor the risk of HFSR throughout the entire course of therapy. For illustration, a 60-year-old RCC patient (base score = 20) who is female (add 6) with a baseline performance status of one (subtract 7), who has liver metastases only (add 6) and a normal WBC (add 3) would have a risk score of 34 before starting sorafenib therapy. This would correspond to a model estimated-HFSR risk of ∼7% during the first week of therapy. As sorafenib is continued, the HFSR risk would increase until week 5 and then begin to decline (Figure 1). Therefore, the data would suggest that the most critical period of HFSR development is during the first 5 weeks of sorafenib.

All patients in the derivation sample were assigned a risk score based on the above system. The risk score for each patient was then compared with the corresponding probability of developing a HFSR, as predicted from the final GEE regression model. The data suggested a nonlinear relationship between risk score and probability of HFSR (supplemental Figure S1, available at Annals of Oncology online). The model development was continued with an ROC curve analysis and a measurement of the area under the ROC curve on both the derivation and external validation datasets. The findings suggested that the area under the ROC curve in both the derivation and external validation samples was acceptable; 0.67 (95% CI 0.64–0.70) versus 0.60 (95% CI 0.58–0.62), supporting the internal and external validity of the scoring system.

The final step in the development of the prediction tool was the identification of a high-risk score threshold or ‘cut-off’, which optimized sensitivity and specificity and was able to minimize the misclassification rate. Five risk score categories were developed (supplemental Table S1, available at Annals of Oncology online). The analysis identified a risk score threshold of >40 as being the range where sensitivity and specificity are optimal and a high proportion (77.9%) of patients were correctly classified (supplemental Table S1, available at Annals of Oncology online). Using a risk score threshold of >40 would capture patients with a HFSR risk of ∼15% before each week of sorafenib therapy (supplemental Figure S1, available at Annals of Oncology online).

discussion

This study describes the development and external validation of a predictive index designed to estimate the absolute risk of HFSR in patients receiving sorafenib therapy. From a list of potential predictor variables, the final risk model contained seven variables that were retained by the statistical elimination process. The finding that females and good performance status were significant independent risk factors for HFSR was consistent with the literature [9, 10]. In addition, the analysis revealed the nonstatic nature of HFSR toxicity whereby the incidence increased by week 5 followed by a gradual decrease (Figure 1). This suggests that risk increases by cumulative drug exposure with the first few weeks being the most critical [15, 25].

Sorafenib is primarily metabolized in the liver, undergoing oxidative metabolism, mediated by CYP3A4 as well as glucuronidation mediated by UGT1A9 [1]. Patients with a normal WBC at baseline would also more likely have a starting dose of 400 mg twice daily as opposed to a lower dose. Therefore, the findings that a patient with liver metastases and a normal WBC count are at a higher risk for HFSR are consistent with the cumulative drug exposure hypothesis. The observation that patients with lung metastases have a lower HFSR risk was unexpected and requires further investigation.

The intent of the prediction exercise was to provide oncologists a tool that would enable them to estimate the percent risk of HFSR prior each week of sorafenib exposure. To facilitate medical decision making, several cut points with risk thresholds were developed (supplemental Table S1, available at Annals of Oncology online). These risk score thresholds are not fixed and can vary based on the patient or oncologist's risk tolerance. Some people may prefer to select a lower risk threshold before the initiation of preventative interventions. A lower score threshold such as >30 would have a greater sensitivity (88.9%), which would minimize the false-negative rate (i.e. more people would receive preventative interventions who actually needed them). In contrast, the impact of using a lower threshold is a higher rate of false positives (i.e. more patients who do not really need the preventative interventions would receive them). Based on our suggested risk threshold, the hypothetical female RCC patient described earlier would be classified as ‘low risk’ with an estimated-HFSR risk of ∼7% before starting sorafenib. However, by the third week of sorafenib exposure, she would be considered at ‘high risk’ for HFSR according to the model because her score would be 41.

Our study has some limitations. Our statistical power to identify important risk factors was limited with only 43 patients developing ≥ grade 2 HFSR in our model derivation sample. The predictive accuracy of our index was adequate with an area under the ROC curve of 0.67 suggesting that it was able to correctly discriminate two of every three exposed patients. However, there is room for improvement. WBC data were not available in the external validation sample. Therefore, the contribution of baseline WBC count to the overall risk calculation could not be evaluated during the external validation exercise. The model only considered data on readily measurable variables. Hence, not all the variability was accounted for in the analysis. Additional factors such as biological and molecular features of the tumors could be involved in the pathogenesis of HFSR. However, this data were not available for all patients. Future HFSR risk models should consider the inclusion of these factors in order to improve overall predictive accuracy.

Despite these limitations, the final model allowed the development of a risk scoring index for identifying high-risk patients before each week of sorafenib exposure. The scoring index is easy to apply, able to reasonably discriminate between high- and low-risk patients and the threshold can be varied depending on a patient's risk tolerance. Therefore, the application and planned continued refinement of this prediction tool will be an important source of risk information for the practicing oncologist and it could enhance patient care by preempting the development of moderate to severe HFSR.

funding

No direct funding was received for this study.

disclosure

The corresponding author had full access to the data in the study and had the final responsibility for the decision to submit the paper. GD, MDV and MEL received consultancy fees from Bayer HealthCare Pharmaceuticals. JY, LH and FF are employees of Bayer HealthCare Pharmaceuticals.