Proportional odds assumption for modeling longitudinal ordinal multiple toxicity outcomes in dose finding studies of targeted agents: A pooled analysis of 54 studies.

BACKGROUND: Data generated by phase I trials is richer than the classical binary DLT measured at the first cycle used as primary endpoints. Several works developed designs for more informative endpoints, e.g. ordinal toxicity grades and/or longitudinal data which relied however on strong assumptions, in particular the proportional odds (PO) assumption.
METHODS: We evaluated this PO assumption for the dose and cycle on a large database of individual patient data from 54 phase I clinical trials of molecularly targeted agents. The PO model is a specific case of the continuation ratio logit model (CRLM) with null parameters. We compared the PO and CRLM models using the widely applicable information criterion (WAIC). We considered a longitudinal multivariate ordinal toxicity outcome (cutaneous, digestive, hematological, general disorders, and other toxicities).
RESULTS: WAIC suggested that the CRLM model (WAIC = 30911.58) outperformed the PO model (WAIC = 31432.10). Deviance from PO assumption for dose was observed for digestive and general disorder toxicities. There was moderate cycle effect with slight deviance from PO assumption for the other type of toxicity.
CONCLUSIONS: Designs based on PO for dose should be a useful tool for drug with low expected digestive or general disorder toxicity dose-related incidence.

Introduction

In oncology, dose finding phase I clinical trials aim at determining the maximum tolerated dose (MTD) as the dose presenting an acceptable rate of severe toxicity during the first cycle of treatment, also called dose limiting toxicity (DLT). Groups of patients are enrolled at increasing dose levels. A given patient is assigned to a dose that is administered in repeated cycles until treatment failure; intra-patient dose escalation is usually not allowed. At each cycle of treatment, adverse events of various types (digestive, hematological, cutaneous, general disorders, etc.) are measured on a graded scale that ranges from 0 (absence of toxicity) to 4 (severe life threatening toxicity). One of the main limitations of dose finding trials is the limited amount of information extracted from the primary outcome [1,2] that results from (i) the composite nature of the outcome ‘worst observed toxicity’, (ii) the dichotomization of this graded outcome in presence or absence of severe toxicity (DLT), and (iii) the use of data collected at cycle 1 only although more than 50% of the first severe toxicity occur after cycle 1 [3,4].

Recently the European Medicine Agency underlined the importance of analyzing adverse events at all cycles of treatment in order to refine the risk of toxicity and to consider not only severe toxicity but also intermediate toxicity [5]. Some authors have proposed designs based on the longitudinal ordinal toxicity measurements [6,7]. Markov chain models have been explored [8], and others authors have included multiple toxicity constraints in the dose finding design [[9], [10], [11], [12]]. Alternatively, cumulative logit models have been suggested to model the ordinal nature of the graded toxicity, and to estimate the dose effect [13], possibly adjusted for the treatment cycle [1,[14], [15], [16]]. One of their advantages is to match the assumption of increasing toxicity with dose, to provide easy to interpret coefficients, either in terms of odds ratio or absolute probabilities. However, sample sizes for dose finding trials being typically small; to account for multiple toxicity grades, a natural simplification of the cumulative logits is the proportional odds (PO) model that assumes that the effect of covariates (here the dose level and the cycle) is similar on the various cumulative logits. This assumption reduces the number of parameters to estimate and allows using intermediate grades of toxicity to refine the estimate of the risk of severe toxicity [17]. Before we implement such an assumption in prospective clinical trials, we explored the PO assumption on real data. The effects of dose and cycle on various types of toxicity were modeled in a large database of individual data of patients treated in 54 phase I clinical trials of molecularly targeted agents provided to the DLT-TARGETT group, a European Organization for Research and Treatment of Cancer (EORTC)-led initiative. We developed a continuation ratio logit model [18] to account for longitudinal multivariate ordinal toxicity outcomes. In order to draw conclusions applicable to future trials, we based our development on predictive modeling strategy, using the horseshoe shrinkage prior [19] to shrink weak coefficients; we compared the models using the widely applicable information criterion [20] (WAIC).

Material and methods

Study design

Full toxicity data of 54 completed phase 1 studies evaluating molecularly targeted agents (MTAs) was provided by four academic institutions (Cancer Research UK (United Kingdom), EORTC, National Cancer Institute-Canada and National Cancer Institute) and three pharmaceutical companies (Pfizer, Roche and Sanofi). The MTAs were administered as single agent to adult patients with solid tumors. All patients who received at least one cycle of treatment were included in the analysis.

The reader may refer to the publication by Postel-Vinay et al. [4] for complete details about the data collection and the study design.

Toxicity data

All grade 1 or above severity adverse events (AEs) reported as at least ‘possibly drug-related’, which were not present at baseline and occurred between cycle 1 and cycle 6 were selected; in fact previous data showed that the majority of AEs occurred during the first 6 cycles of treatment in dose finding trials [3]. To ensure comparability of the AEs over trials that used different grading systems, the grade of all reported toxicities was harmonized to the National Cancer Institute Common Terminology Criteria of Adverse Events, version 3.0 and labeled for description according to Medical Dictionary for Regulatory Activities (MedDRA) 15. If the same AE was reported at different severity grades during a given cycle, only the worst toxicity grade was taken into account in the statistical analysis. The grades were grouped in 4 levels: no toxicity, toxicity grade 1, toxicity grade 2 and toxicity grade (usually considered as dose limiting toxicity (DLT) when they occur during the DLT evaluation period).

In this report, we focused on four different types of AEs which are typical of targeted agents [3]: cutaneous, digestive, general disorder and hematologic toxicities. These categories were defined from the MedDRA classification preferred items in accordance with NCIC experts to better fit the cancer phase 1 trial context and the most frequent toxicity reported in early phase trials of single targeted agents [21]. A fifth (heterogeneous) type was defined that contained all other toxicities and was labeled ‘other type’.

Models

As toxicity data was collected at the end of each cycle of treatment, whose duration may vary across trials, the treatment cycle, as defined per protocol, was used as time unit, irrespective of its duration in days. The dose for the patient i, , was standardized by the MTD, i.e. the ratio between the planned dose and the MTD of this trial, or the maximal allocated dose during this trial when the MTD was not reached (% of trials).

We used mixed effect multivariate continuation ratio logit model [18] with correlated random effects to jointly assess the relationship between the J types of graded toxicity () denoted and the dose D and cycle C considered as continuous variables. The probability that the patient i presents a toxicity of type j () of grade higher than k () at the cycle c (), was:with the intercept for the grade k, and the parameters associated to the dose and the cycle respectively, and a patient specific random effect distributed according to a centered multivariate normal distribution with covariance matrix . expit denotes the inverse of the logit transform function, i.e. . Of note, the random effects have been set at the patient level to account for the possible correlations of the repeated measurements. For model tractability, we did not consider patient-level random effects nested in trial-level random effects. Conditional on , we assumed independence of the longitudinal AEs measures. With this parameterization, labeled full model in the rest of the paper, the PO assumption for the dose effect then corresponds to and , and for the cycle effect to and . The PO model can then be written as:

Model priors

The model parameters were estimated in a bayesian framework. Despite the large number of trials, patients and observations in this joint analysis, the full model contains 45 fixed parameters plus the random effect covariance matrix. To improve the stability of the estimates, we used Horseshoe shrinkage prior [19] for the P parameters of the fixed effects, , with , and . A normal prior distribution for each parameter was elicited, in which the parameters for the variance prior, and τ, followed the standard half-Cauchy distribution :

τ was common to all components of θ, and was specific of . This approach has common features with the Bayesian LASSO that uses Laplacian prior distributions [22], but it belongs to the global-local shrinkage prior family [23]: a global prior parameter τ shrinks all the model parameters towards 0, and local prior parameters modulates this shrinkage at various degrees. A reparametrisation of τ and in a parameter has been proposed to directly quantify the degree of shrinkage [19] by , with the posterior mean of , its maximum likelihood solution, and the shrinkage coefficient for each parameter p. The distribution of has a horseshoe (‘U’) shape, presenting a first point mass at (i.e. no shrinkage, ) and a second at (i.e total shrinkage, ). The horseshoe prior is therefore expected to apply a more severe shrinkage to zero-coefficients, while avoiding over-shrinkage of larger coefficients, favoring both sparsity and robustness to large signals. Several authors have shown that it outperforms the Laplacian prior for prediction, and provides results comparable to the bayesian model averaging (bayesian gold standard for prediction) without the computation burden [19,[23], [24], [25]].

The covariance matrix of the random effects was modeled as parameters. We used the LKJ prior [26] with the shape parameter (uninformative prior) for the Cholesky factor of the random effect correlation matrix. Parameter posteriors were obtained by an Hamiltonian Monte Carlo sampling scheme [27] (4 chains, 5000 iterations including 1000 burning) using the Stan software [28].

Model selection

In addition of the full and the PO models, a reduced model without the parameters whose credibility intervals that included 0, was fit as a sensitivity analysis. To compare these 3 models, we relied on the widely applicable information criterion [20] (WAIC). It can be viewed as an approximation of cross-validation [20,29] and lower value indicates better compromise between information and model dimension.

Model goodness-of-fit was assessed by graphical representation of the observed proportion of toxicity versus the predicted probabilities of toxicity for the full CLRM and the PO model, extracted from 1000 samples generated from the posterior predictive distribution of the models.

Results

Descriptive results

The 2048 patients in the 54 studies received a total of 5592 cycles. During each cycle, toxicities of various types occurred, resulting in 9904 adverse events detailed by grade in Table 1.

Table 1

Number of toxicities by type and grade.

Grade	Cutaneous	Digestive	General disorder	Hematologic	Others	Total
1	549	1754	403	1344	1513	5563
2	207	794	433	748	946	3128
≥ 3	31	190	345	200	447	1213
Total	787	2738	1181	2292	2906	9904

The Venn diagram in Fig. 1 illustrates the co-occurrences of toxicities in the 5592 reported cycles. The most frequent combinations were: digestive/general disorders/other (), digestive/general disorders/hematologic () and digestive/others (). Distributions of proportions of co-occurrences can be found in the supplementary Table A1.

Fig. 1

Co-occurrence of the (9904) toxicity (all grades) for the 5592 reported cycles.

Exploring the PO assumption

Table 2 provides the parameter estimates and their credibility intervals (CI) for the full model. The dose significantly increased the risk of all types of toxicities; dose effect ranged from 2.98 (CI [2.27: 3.96]) for cutaneous toxicities to 2.16 (CI [1.77: 2.65]) for the ‘other’ type of toxicities. The PO assumption seemed to be plausible for the dose as the credibility intervals of the and odds ratios included the null value. Conversely, for general disorders the odds ratio increased by 0.37 (CI [0.10: 0.67]) for the risk of grade , with additional 0.67 (CI [0.05: 1.23]) for the risk of grade violating the PO assumption. Finally, this assumption did not hold for digestive toxicity as the odds ratio increased by 1.37 (CI [0.84: 1.87]) for the risk of grade.

Table 2

Parameter estimates of the full continuation ratio logit model with their credibility interval for each type of toxicity. Bolded figures correspond to parameters with credibility intervals excluding the null value.

Parameter	Cutaneous	Digestive	General disorder	Hematological	Other
Intercept_Grade≥1	−9.51 [-10.81; -8.38]	−1.79 [-2.16; -1.47]	−2.66 [-3.14; -2.31]	−7.33 [-8.27; -6.62]	−1.93 [-2.37; -1.57]
Intercept_Grade≥2	−3.11 [-3.68; -2.45]	−3.38 [-3.73; -3.00]	−3.02 [-3.38; -2.69]	−1.95 [-2.39; -1.58]	−2.70 [-3.03; -2.39]
Intercept_Grade≥3	−4.85 [-6.43; -3.52]	−4.61 [-5.24; -3.93]	−4.17 [-4.87; -3.42]	−2.52 [-3.04; -2.07]	−2.70 [-3.14; -2.32]
Dose_Grade≥1	2.98 [2.27; 3.96]	2.35 [1.96; 2.72]	2.16 [1.77; 2.65]	2.87 [2.17; 3.75]	2.51 [2.12; 2.98]
Dose_Grade≥2	0.06 [-0.41; 0.43]	0.22 [-0.06; 0.51]	0.37 [0.10; 0.67]	0.30 [-0.01; 0.72]	0.08 [-0.19; 0.38]
Dose_Grade≥3	0.33 [-0.56; 1.41]	1.37 [0.84; 1.87]	0.67 [0.05; 1.23]	0.03 [-0.37; 0.42]	0.26 [-0.03; 0.58]
Cycle_Grade≥1	0.37 [0.27; 0.48]	0.04 [-0.01; 0.11]	0.07 [0.01; 0.14]	0.08 [-0.00; 0.17]	0.18 [0.11; 0.24]
Cycle_Grade≥2	−0.16 [-0.33; -0.02]	−0.08 [-0.17; 0.00]	−0.08 [-0.17; -0.00]	−0.02 [-0.12; 0.05]	−0.09 [-0.17; -0.00]
Cycle_Grade≥3	0.05 [-0.20; 0.37]	−0.01 [-0.13; 0.11]	−0.08 [-0.26; 0.06]	0.00 [-0.10; 0.12]	−0.09 [-0.20; 0.01]

Table 3 contains the PO model parameter estimates and their CI. The results were close to that provided by the full model, which is confirmed by the information criteria vs . Under the PO model, dose effects ranged from 2.55 (CI [2.12: 3.00]) for general disorder toxicities to 3.13 (CI [2.31: 4.01]) for the ‘other’ type toxicities. Assuming a proportional odds model would then lead to under-estimate the dose effect on the risk of digestive grade 3 or more toxicity by 35% ( instead of ), resulting in a large underestimation of the odds ratio. The estimated odds ratio of grade 3 or more hematological toxicity in patients treated at the MTD compared to patients treated at half the MTD is 3.63 (assuming PO) instead of 7.15. This misspecification had more limited impact for general disorder. The difference of the same estimated odds ratio for grade 2 or more was small (3.48 instead of 3.44), but it was higher for grade 3 or more (3.58 instead 4.95).

Table 3

Parameter estimates of the proportional odds ratio logit model with their credibility interval for each type of toxicity. Bolded figures correspond to parameters with credibility intervals excluding the null value.

Parameter	Cutaneous	Digestive	General disorder	Hematological	Other
Intercept_Grade≥1	−9.73 [-10.75; -8.63]	−1.90 [-2.25; -1.55]	−2.89 [-3.34; -2.49]	−7.47 [-8.71; -6.51]	−2.01 [-2.37; -1.65]
Intercept_Grade≥2	−3.52 [-3.83; -3.22]	−3.36 [-3.53; -3.19]	−2.94 [-3.11; -2.78]	−1.73 [-1.92; -1.56]	−2.87 [-3.02; -2.72]
Intercept_Grade≥3	−4.40 [-5.15; -3.72]	−3.24 [-3.48; -3.01]	−3.71 [-4.02; -3.42]	−2.49 [-2.75; -2.25]	−2.72 [-2.90; -2.55]
Dose	3.11 [2.35; 3.87]	2.58 [2.19; 2.95]	2.55 [2.12; 3.00]	3.13 [2.31; 4.01]	2.77 [2.36; 3.17]
Cycle	0.33 [0.23; 0.42]	0.01 [-0.04; 0.06]	0.03 [-0.02; 0.09]	0.06 [-0.01; 0.15]	0.12 [0.07; 0.17]

Conversely, additional cycles of treatment did not significantly modify the risk of digestive, general disorder and hematological toxicities, but increased the risk of cutaneous and ‘other’ toxicities. This is in line with the underlying mechanisms of actions inducing cutaneous rash that is directly related to the drug exposure. Based on the 95% credibility intervals of , the PO assumption was plausible for cutaneous and ‘other’ types of toxicities. Nevertheless, in those later cases, the deviation were of borderline significance as the boundary of intervals for parameters associated with grade 2 or more were 0.02 for the former and for the later. Again, the PO assumption would lead to slightly over estimate the cycle effect for grade 2 or higher cutaneous toxicity. One possible explanation for this over-estimation might be the dose reduction which may have been applied to some patients. Indeed, investigators may reduce the dose after several cycles to avoid upcoming toxic side events leading to a systematic bias when we adjusted on the planned dose instead of the actual dose.

The estimated variance-covariance matrix of the random effects of the full CR-model in Table 4 provides some insight on the correlations between the various types of toxicity.

Table 4

Random effects correlation matrix of the full continuation ratio logit model (correlation estimates and their credibility interval).

	Cutaneous	Digestive	General disorder	Hematologic	Others
Cutaneous	1
Digestive	0.22 [0.15; 0.28]	1
General disorder	0.05 [-0.01; 0.12]	0.45 [0.40; 0.51]	1
Hematologic	−0.18 [-0.27; −0.07]	0.28 [0.23; 0.35]	0.26 [0.21; 0.33]	1
Others	0.01 [-0.06; 0.08]	0.42 [0.36; 0.46]	0.41 [0.36; 0.46]	0.37 [0.32; 0.43]	1

The low to moderate correlations between random effects (from −0.15 to 0.46) suggest that each type of toxicity carries different information, which may also be the consequence of different toxicity profiles according to the investigated agent. This is reassuring that correlation between hematologic and cutaneous toxicity was low as they proceed from different mechanisms. Conversely, general disorders (typically fatigue, mood depressions, pain,…) and digestive toxicity that are commonly associated in clinical practice, were correlated in our data. Of note, the correlations estimated under the PO assumptions were quite similar (cf. Supplementary material Table A2).

Goodness of fit

Fig. 2, Fig. 3 show that the observed probabilities of toxicity at each cycle were included in the prediction interval drawn from the predictive posterior distributions for both the full and the PO models. The goodness of fit of the model was satisfactory.

Fig. 2

Observed (empty circle) vs expected conditional probability given the cycle of each type of toxicity at each cycle according to the PO model. The median expected probability (filled circles) and the 95% prediction interval were obtained from 1000 simulations from the posterior predictive distribution of the model.

Fig. 3

Observed (empty circle) vs expected conditional probability given the cycle of each type of toxicity at each cycle according to the full model. The median expected probability (filled circles) and the 95% prediction interval were obtained from 1000 simulations from the posterior predictive distribution of the model.

Discussion

The richness of this large database highlighted some characteristics of the dose-response relationships for different types of toxicities. Our results suggest that the PO assumption may hold in most cases, but this statement cannot be generalized, specifically for digestive and general disorder toxicities. The cycle effect also depends of the type of toxicity as it ranged from no cycle effect for digestive and hematological toxicities, to a moderate effect attenuated for grade 2 or higher of the other types of toxicity.

Those results may have application both at the design and at the analysis level. Assuming proportional odds for the dose effect enables to incorporate the occurrence of intermediate grades of toxicity in the estimate of the risk of toxicity and hence to increase its precision. In particular, the method proposed by Ref. [17] based on PO models appears as a simple and efficient extension of the continual reassessment method (CRM) for ordinal outcomes that may be applied in various situations. Designs based on PO assumption for dose effect may fail for trials applied to a drug which are expected to induce digestive and/or general disorder toxicities, but they would be more informative than binary CRM designs in the other cases. In case the strict PO assumption appears too strong, an informative prior on the dose parameter in the continuation ratio logit model may be an alternative modeling option. Phase I trials enroll increasing numbers of patients (commonly larger than 100) [30]. More advanced analysis may then be performed to help refining the assessment of the toxicity profile according to the dose and over time. We proposed an analytical tool to explore the dose and the cycle effects on each type of toxicity. In addition, the joint modeling provides estimates of the correlations between the various toxicities, an additional information that is useful for the management of patients during the course of the treatment. One of the statistical issue with the analysis of multiple cycles of treatment relates to the patients who get off-study due to early progressive disease. Follow-up strongly varies across patients. Under the assumption that the risk of early progression is largely independent on the risk of toxicity after adjustment for the dose level, our estimates should not be biased by early drop out. When incorporating repeated cycles of treatment in the analysis, the main limitation is the lack of known model for the relationship between the cumulative drug exposure and the risk of toxicity. As patients are treated at the same dose level over all cycles, the delayed toxicity cannot be disentangled from the cumulative dose effect, if any. Is the toxicity at cycle 3 due to the dose administered at the same cycle or at a previous cycle or is it the consequence of some accumulation? It may depend on the half-life of the compounds, but PK models are often unknown at the time the first in man trials are carried out. This is also the reason why accounting for the actual administered dose is not straightforward. In case of adverse events, some dose reduction may be allowed. How to adjust the model on those dose reductions largely depend on the underlying model for the drug exposure. Would this model be known, our approach could be easily adapted. Our analysis shows that the data generated by phase I trial is richer than the classical binary DLT measured at the first cycle. Analysis of all the collected information is feasible. This may help elaborating new designs with reasonable assumptions for our models to select the optimal dose more reliably.

Declaration of competing interest

All authors declare no conflicts of interests.