Point and Interval Estimators of the Target Dose in Clinical Dose-Finding Studies with Active Control.

In a clinical dose finding study with active control a new drug with several dose levels is compared with an active comparator drug. The main focus of such studies often lies on the estimation of a target dose that leads to the same efficacy as the control. This article investigates the finite sample properties of the maximum likelihood estimation of the target dose and compares several approaches for constructing corresponding confidence intervals under the assumption of a linear dose-response curve and normal error terms. Furthermore, the impact of deviations from the model assumptions regarding the error distribution is explored.

INTRODUCTION

Phase II dose-finding studies aim at the characterization of the dose response relationship of an investigational drug for which the principle proof of concept has been shown, as well as the estimation of a target dose that leads to a predefined response. The investigation of dose response relation and the subsequent selection of a target dose is considered to be a pre-requisite for the transition to a confirmatory pivotal study in Phase III. Several approaches have been proposed for the efficacious planning and analysis of such a dose-finding study. Bretz et al. (2005) and Pinheiro et al. (2006a) proposed a methodology that combines formal hypothesis testing for dose response with flexible modeling of the dose response relationship and estimating a target dose, i.e., a minimum effective dose that produces a clinical relevant effect. For a more detailed discussion we refer to Bretz et al. (2008) and Pinheiro et al. (2006b). The estimation of a minimum effective dose based on a given model can be regarded as a calibration problem, a reverse process to regression, i.e., the estimation of a value for an independent variable that yields an expected outcome for the dependent variable equal to a predefined value. There is an extensive literature not only on these calibration problems, e.g., in quality control, but also related to dose estimation, e.g. Hsu and Berger (1999), Tamhane and Logan (2002), Morales et al. (2006), and Budtz-Jørgensen (2007). Although these approaches focus on a target dose that is defined by a specific effect difference to placebo, there is an increasing interest in describing the dose response relative to an active comparator to be included in the dose-finding trial which, however, has found little attention in the literature with the exception of Källén and Larsson (1999) and Dette et al. (2014). In addition, in some situations the use of the placebo may be unethical even in early stages of the development.

If an active comparator study is planned in Phase II, the interest lies in the determination of a dose range that ensures superiority over the competitor. If, on one hand, the probability of an adverse drug reaction is expected to increase with a higher dose, and, on the other, the efficacy is expected to improve with increasing dose, the dose that leads to the same efficacy as the comparator needs to be defined as precise as possible. Only if this information is available, the confirmatory clinical trials in Phase III can properly be designed. Also, manufactural standards should be considered in the precision requirements on the target dose estimation. Whereas this paper refers to a target dose defined by the dose that leads to the same mean efficacy as the active comparator, the application of a more general definition given by a predefined difference to the control is straightforward.

The precision, with which the target dose is estimated, depends on the dose-response shape, especially the steepness of the dose response as well as on the residual variance and the mean response of the control. Focusing on Phase II dose-finding studies with active control and a linear dose-response function the properties of the maximum likelihood estimator (MLE) of the target dose and several methods for confidence intervals of the target dose were investigated in extensive simulation studies. The investigated confidence interval approaches include an approximation of the standard error (SE) by the ∆-method (Ferguson, 1996), an extension of the method by Fisch and Strehlau (1993) to repeated measures, a parametric bootstrap (Efron, 1979), and a profile likelihood method (Pawitan, 2001). The MLE is characterized in terms of bias, variance, and mean squared error (MSE) whereas coverage probabilities and interval lengths are presented for the confidence intervals. Further the impact of the residual error assumptions will be analyzed by sensitivity analysis to investigate a potential robustness of one of the four methods. In particular, the question of which method is the most useful in terms of coverage probability, interval weights, and numerical effort.

To motivate the settings for the simulation studies, two examples of Phase II dose-finding studies with active control are presented briefly in what follows. Krum et al. (1998) report a study investigating the effect of an endothelin-receptor antagonist called bosentan on blood pressure in patients with essential hypertension. In this study 293 patients were randomly assigned to receive placebo or one of four oral doses of bosentan (100, 500, 1000, 2000 mg per day) or the angiotensin-converting-enzyme inhibitor enalapril (20 mg once daily) as active control for 4 weeks. The second example is a Phase II study presented in Chapple et al. (2004), which investigated the effect of solifenacin on patients with symptomatic idiopathic detrusor overactivity. A total of 255 patients were randomized to receive placebo or one of four doses of solifenacin (2.5, 5, 10, or 20 mg per day), or tolterodine 2 mg twice daily as active control. The primary endpoints were voids/24 h and mean volume voided (ml).

The paper is organized as follows. In Section 2 notation and statistical model are introduced, and an overview over the different methods is presented. In Section 3 the properties of the point estimator of the target dose including bias, variance, and MSE are investigated by deriving approximations for the expressions which are subsequently compared to exact values obtained through simulations for a variety of scenarios. In Section 4 the properties of the various confidence intervals of the target dose are investigated. Therefore, the coverage probability as well as the median interval length are simulated. In Section 5 the robustness of the methods for point and interval estimation of the target dose to deviations from the model assumptions such as normality of the error terms are explored. Finally, we close with a brief discussion and some recommendations.

NOTATION, MODEL, AND METHODS

Notation and Statistical Model

In this paper bold small letters such as or define a vector and bold capital letters such as Σ or a matrix. Also is the transpose of and the Kronecker sum of two matrices is defined as . Further ∼ defines “distributed as”, “convergence in distribution” and defines “approximately distributed as.”

With k groups the dose levels , and an active control (ac) the random variable of the jth person in the ith dose level can be written as where denotes the linear mean function, i.e., , and normally distributed error terms . The random variables of the active control can be written as with expected value µ and error terms . Let be the sample size of all dose levels and the total sample size. The linear model can be written as To determine the target dose d* the equation must be solved. In general the parameters of the dose-response curve, i.e., θ 0 and θ 1, as well as the expected value of the active control µ are unknown and must be estimated from the data. In the next section the necessary parameter estimators will be presented.

Point Estimator of the Target Dose d*

Following standard theory for linear models (Mattai and Provost, 1992) we summarize briefly the parameter estimators in the linear model before deriving an estimator of d*. The expected value of the active control can be estimated by using the mean value of the active control The parameter estimators of the linear dose response function can be written as Furthermore, the variance of the error terms is in practice unknown and must be estimated. Therefore, and are defined so that the variance can be estimated by If these estimators are used as plug-in estimators in the maximum likelihood equation of the target dose, the estimator of the target dose can be written as In the next section several methods to compute a confidence interval for the target dose will be presented.

Confidence Intervals for the Target Dose d*

Approximation of the Standard Error by the ∆-method

All parameter estimators which are needed to determine d* are normally distributed and can be written in a joint distribution with expected value and covariance matrix . The target dose d* can then be written as a function , i.e., . Then the gradient of is given by From Cramér’s theorem (Ferguson, 1996) with the use of the first-order Taylor approximation it can be shown that with the weights of the ith dose level, the weight of all dose levels and the weights of the active control (Dette et al., 2008). Furthermore, let be the plug-in estimator of the unknown variance so that a -confidence interval of the target dose can be constructed as with the -quantile of the standard normal distribution.

An Extension of the Method by Fisch and Strehlau (1993)

An alternative way to determine a confidence interval for the target dose d* is an extension of the method by Fisch and Strehlau (1993) to repeated measures which was originally proposed for calibration problems without replications. Therefore, the expected value of the response of the active control is replaced by in (1) which leads to a nonlinear model. Instead of performing a standard nonlinear regression the maximum likelihood equation of the estimator of the target dose can be rewritten in the following form: and the dose d should be considered as fixed, but unknown. All necessary estimators were defined in Section 2.2 and are identical with the maximum likelihood based estimators. The left side of the equation can be defined as a new random variable V (d) with the following exact distribution: because µ can be written as . The variance of the linear combination of normally distributed random variables can be written as with and . Because d was considered as fixed, the only unknown variable in is the error term σ 2 which can be estimated by presented in Section 2.2. Finally the new variable W (d) is defined as The t-distribution of the variable W (d) only depends on the total sample size N. With the knowledge of the distribution of W (d) a confidence interval of the target dose d* can be computed by finding the limits of the confidence interval of W (d) depending on d. This leads to the following (1 – α)-confidence interval with w(d) the realization of W(d): This method does not always lead to finite confidence intervals, but the conditions for the appearance of infinite confidence intervals can be found in Fisch and Strehlau (1993).

Semiparametric Bootstrap

All estimators which are necessary to estimate the target dose d* are known with their explicit distribution functions. Hence, a parametric bootstrap (Efron, 1979) can be used to construct a confidence interval for the target dose d*. Therefore, the bootstrap estimates have to be generated by simulation based on the parameter estimators gained from the data Then the bootstrapped estimator of the target dose can be computed as This simulation step is repeated n times resulting in the vector of all bootstrapped parameter estimators. The confidence interval is then computed from the ordered vector of the bootstrapped target dose estimators by using, for example, the quantile method. The n th and the n th component of the sorted vector are selected to construct the (1–α)-confidence interval

Profile Likelihood

The idea of this approach is to set the target dose d* to a fixed value and to optimize the maximum likelihood equation depending on this fixed target dose (Pawitan, 2001). The likelihood is given by with the ith row of the matrix in (1) and . If the target dose is set to a fixed and known value d the parameter estimates can be computed depending on d so that the profile likelihood is defined as For convenience the linear model with fixed d can be written as With this notation and standard methods for linear models (Mattai and Provost, 1992) the MLE for and σ 2 depending on d can be constructed as with the ith row of the matrix (d). Then the profile likelihood equation of the target dose d can be written as The maximum of the profile likelihood is at so that . The confidence interval for the target dose can be constructed by using the ratio . Then the Wilks likelihood ratio statistic W (see, e.g., Pawitan (2001)) has the following distribution under H 0 : d = d*: and it follows with . This generates a confidence interval for the target dose which is given by For example a 95%-confidence interval contains all d for which the condition holds.

PROPERTIES OF THE POINT ESTIMATOR OF d*

In this section the properties of the point estimator of the target dose will be investigated through second-order Taylor approximation as well as simulation studies. The focus will lie on the assessment of the bias as well as the MSE.

It is not possible to calculate the exact expected value of in (2) because it is a ratio of dependent random variables. Therefore, an approximation with second-order Taylor approximation will be exploited here. As already described in Section 2.3.1, the target dose d* can be written as a function of the parameter vector . For the calculations the gradient Δ and the Hessian matrix of the function g are required, which are given by With these the point estimator of the target dose can be approximated as Hence, the expected value is approximated by . Then an approximation of the bias of is given by If , the approximate bias is equal to 0. Also the approximate bias is increasing with larger error term variance σ 2 and is decreasing for larger total sample sizes N and for larger slopes of the dose-response function θ 1.

The properties of the point estimator of the target dose and its approximation described above are evaluated in a simulation study motivated by the example studies described in Section 1. We consider studies with k = 5 dose levels which are equidistant on [0, 1], and d 1 = 0, and d 5 = 1. The sample sizes n per dose group for , the sample size of the active control , the mean effect of the active control is set to µ = 1, and the intercept of the linear response function is θ 0 = 0. In all simulations the number of replications is n 10,000 and the number of bootstrap simulations per simulation run is n 10,000. Table 1 summarizes the considered simulation scenarios.

Table 1

Simulation scenarios for different variances σ 2, different slope levels θ 1, corresponding target doses d*, and various sample sizes n per group

σ2	θ1	d*	n
1	1.25	0.8	10,12,…, 30, 40,…, 100
	2	0.5	10,12,…, 30, 40,…, 100
	5	0.2	10,12,…, 30, 40,…, 100
2	1.25	0.8	10,12,…, 30, 40,…, 100
	2	0.5	10,12,…, 30, 40,…, 100
	5	0.2	10,12,…, 30, 40,…, 100

In Fig. 1 it can be seen that the derived approximation of the bias based on the real values explains the trend of the simulated bias quite well. However, there are limitations to the use of the approximation. In various situations, especially for small and moderate sample sizes, the estimated bias approximation is way too large compared to the estimator of the target dose. Therefore, we refrain from using the approximation for bias correction. Also, as we will see below, the bias is fairly small in comparison to the variance. The MSE is given by The variance and the bias could not be calculated directly and therefore the same second-order Taylor approximation as in Section 3 was used to approximate the which leads to Hence, the approximate MSE can be rewritten as In the simulation studies the MSE could be estimated by with denoting the simulated target dose estimator in the ith simulation run. However, this estimator is highly sensitive to outliers which occur in several simulation runs with small group sample sizes. Therefore, instead of the median of the simulated squared differences can be used as an approximation of which is more robust to outliers. This is possible because with and g = MSE. So the expected value of U is E(U) = 1 and the Median(U) ≈ 1 · (1 − 2/9)3. Hence, E (gU) = gE(U) ≈ g · Median(U) · (1 − 2/9)−3. With this approach the estimator of the MSE is not influenced by simulation caused outliers and extremes and therefore will be used in the following simulation study.

Figure 1

Approximated and simulated bias of the estimator of the target dose for various variances σ 2, slopes θ 1, and sample sizes n per group.

The MSE of the estimator of the target dose was simulated for the scenarios described in Table 1 and compared to the approximate MSE. In Fig. 2 it can be seen that the MSE, as with the bias above, depends on the error term variance σ 2, the slope θ 1, and the sample size n per dose group. The MSE is decreasing with larger n and larger slopes θ 1 and is increasing with larger error term variances σ These numerical investigations demonstrate that the estimator of the target dose is not unbiased, but that the bias is relatively small for moderate sample sizes and not really a concern for estimating the target dose.

Figure 2

Mean squared error (MSE) of the estimator of the target dose shown by the simulated MSE and the approximate MSE for different slopes θ 1 = 1.25 (- - -), 2 (—), 5 (…), various variances σ, and sample sizes n per group.

PROPERTIES OF THE CONFIDENCE INTERVALS OF THE TARGET DOSE

In this section the performance of the different methods for constructing a confidence interval will be assessed through simulation studies. The setting of the linear model in the simulation studies is the same as in Section 3. To investigate the behavior of the different interval methods the scenarios described in Table 1 will be investigated at a 95%-confidence level. In Fig. 3 it can be seen that the method of Fisch and Stehlau (F&S), the semi-parametric bootstrap (Bootstrap), and the profile likelihood (PL) perform quite well under nearly all scenarios. Furthermore, it can be seen that especially for small sample sizes the ∆-method (Delta method) is relatively conservative compared to the other three methods. To assess the performance of the different methods for the nominal coverage probability of 95% the dotted lines in Figs. 3 and 6 indicate the simulation error and were derived as . This is roughly ±0.005 which has been considered as a practical irrelevant deviation from the nominal level (Friede et al., 2007).

Figure 3

Coverage probability of the ∆-method, the method by Fisch and Strehlau (F&S), the parametric bootstrap, and the profile likelihood for various variances σ 2, slopes θ 1, and group sample sizes n. The dotted lines indicate the simulation error with 99% probability.

Figure 6

Coverage probability and median interval length of the Δ-method, the method by Fisch and Strehlau (F&S), the parametric bootstrap, and the profile likelihood for t-distributed and log-normal distributed error terms with variance σ 2 = 1, slope θ 1 = 2, and various group sample sizes n. The dotted lines indicate the simulation error (99% probability interval).

Now the median length of the different intervals will be investigated. Because a closed expression of the confidence interval only exists for the ∆-method (see Section 2.3.1) the length will be investigated via simulation. Therefore, the scenarios defined in Table 1 will be simulated under the same conditions as described in Section 3. In Fig. 4 it can be seen that the confidence interval length is relatively similar for all four methods. But in all scenarios the median length of the ∆-method is a little bit smaller than the length for the other three methods. This is possible even if the ∆-method is more conservative than the other three methods because the ∆-method generate symmetric confidence intervals and all other methods generates asymmetric confidence intervals. This will be explained in more detail in a simulated example next.

Figure 4

Median interval length of the ∆-method, the method by Fisch and Strehlau (F&S), the parametric bootstrap, and the profile likelihood for various variances σ 2, slopes θ 1, and group sample sizes n.

To explain the behavior of the median length of the confidence intervals the scenario θ 1 = 1.25, σ 1 with group sample size n = 20 will be investigated. Therefore, the simulated lengths were split in upper and lower confidence interval halves. The results of these simulations are presented in Fig. 5. It can be seen that the median length of the lower confidence interval half of the ∆-method is larger than the median length of the other methods, but on the other hand the median length of the upper confidence interval half is larger for the method of Fisch and Strehlau(F&S), the bootstrap, and the profile likelihood. The reason for this is that the ∆-method, in contrast to the other methods, generates symmetric confidence intervals. This leads to a slightly larger lower confidence interval half and a smaller upper confidence interval half.

Figure 5

Length of the lower and upper interval of the ∆-method, the method by Fisch and Strehlau (F&S), the parametric bootstrap and the profile likelihood for variance σ 1, slope θ 1 = 1.25, and sample size n = 20 per group.

ROBUSTNESS AGAINST NON-NORMAL RESIDUALS

In this section the robustness of the methods to deviations from the assumption of normally distributed error terms will be investigated. Therefore, different non-normal error terms will be simulated in the linear model for the scenarios explained in Table 1. The methods described in the sections above were used to estimate the target dose and to construct intervals under the assumption of normally distributed error terms. As non-normal distributions the t-distribution with 4 degrees of freedom as well as a log-normal distribution were used. All error terms were scaled so that the expected value is equal to 0 and the variance is equal to 1. In Fig. 6 it can be seen, that for t and log-normal distributions the methods perform quite well and that there is basically no difference in the coverage probabilities compared to normally distributed error terms. Furthermore, there is a slight decrease in median interval length for all methods except of the ∆-method. Overall, it appears that these methods are quite robust to the deviations from the distributional assumption of the error term. This might be explained by the fact that the used total sample size, with a minimum of N = 70, is large enough for large sample approximations by the central limit theorem.

REAL DATA EXAMPLE

In this section the methods introduced above are illustrated by a dose-finding study including placebo, four dose levels of Solifenacin (0, 2.5, 5, 10, 20 mg) and 2 mg Tolterodine as active control with sample sizes = (n 1,…, n 5)′ = (36, 40, 37, 33, 34) and n 37 (Chapple et al., 2004). The primary endpoint is the reduction in “Voids/24 h” after 6 weeks from baseline. For the different dose levels and the active control only the mean values of the reduction in “Voids/24 h” are reported in Chapple et al. (2004). The standard deviation (SD) of the error terms is not displayed but can be calculated using the reported p-values of the test statistics which lead to σ ∈ [1.9, 2.5]. We used σ = 2 to evaluate this example. For the log(1+dose) the results of the study are summarized in Fig. 7 as mean responses and SEs. Even though the individual patient data are not available it is possible to calculate the estimator of the target dose as well as the confidence intervals of the ∆-method, the method of F&S, and the parametric bootstrap (Bootstrap) which only need the sample sizes, the mean responses, and the SD by using the SAS macro “DF_AC_LIN_MEAN.” The results of the confidence interval methods are shown in Table 2 for the target dose estimator . For some of the confidence intervals the lower interval limit was set to zero to guarantee a positive dose range. Further it is not possible to calculate the profile likelihood based interval on basis of the mean values alone. Therefore, a dataset was generated with identical mean responses and SD. It is somewhat surprising that with a total sample size of N = 217 it is only possible to exclude the maximum dose as potential target dose. The SAS macros for the linear dose-finding problem based on the raw data (DF_AC_LIN) as well as based on the mean values as presented here are available in the supplementary material.

Figure 7

Voids/24 h decrease from baseline of the experimental drug dose levels and active control at the end of the study displayed as mean ± standard error (SE) reported in Chapple et al. (2004).

Table 2

The estimated target dose and the corresponding confidence intervals (CI) of the ∆-method, the method of Fisch and Strehlau (F& S), the parametric bootstrap (Bootstrap), and the profile likelihood approach (PLH) for the data presented in Fig. 7.

	CI-approach	95%-CI	CI-length
3.011	∆-method	[0.236, 12.001]	11.765
	F&S	[0.000, 13.237]	13.237
	Bootstrap	[0.000, 12.671]	12.671
	PLH	[0.000, 13.018]	13.018

CONCLUSIONS

In this paper we presented an MLE of the target dose and several approaches for confidence intervals of the target dose in clinical dose-finding studies with active control. We found that the MLE of the target dose is slightly biased and that the size of the bias depends on the design of the dose-finding study. If the target dose , then the bias is relatively small. Also the total sample size N, the strength of the dose relationship (the slope θ 1), and the variance σ of the error term have an influence on the size of the bias. For larger variances σ the bias is increasing and for larger slopes the bias is decreasing. With larger sample sizes N the bias approaches zero.

Because the bias and the MSE could not be calculated exactly a second-order Taylor approximation was used instead. These approximations perform overall well and were compared with the simulated bias and MSE in numerous scenarios. We showed that the normal approximation based on the ∆-method is the simplest way to construct a confidence interval of the target dose, but the coverage probability is fairly conservative for small sample sizes. The method by F&S requires numerical optimization and is therefore computationally more involved than the ∆-method. In terms of coverage probability however, the method by F&S outperforms the ∆-method. In nearly all scenarios the coverage probability is within the simulation error from the nominal level. One disadvantage of the method by F&S is that it cannot be easily extended to nonlinear models. The semi-parametric bootstrap is slightly more computer intensive than the other methods because the bootstrapped parameter estimates have to be generated n times. In terms of coverage probability performs the bootstrap quite well in most of the scenarios considered. The profile likelihood performs quite similar to the method by F&S, and the semi-parametric bootstrap, but tends to be slightly more liberal compared to the other methods. The median confidence interval length is quite similar for all methods except of the ∆-method, there the median length is in all scenarios a little bit smaller than in the other methods. One reason for that is that the Δ-method always generates symmetric intervals whereas the other methods yield asymmetric intervals. Even if the median confidence interval is shorter for the Δ-method, it does not mean that the Δ-method is superior over the other methods in general. As Lehmann (1959) argues “short intervals are desirable when they cover the true parameter value but not necessarily otherwise”. Therefore, we selected first the methods which hold the coverage probability well and from this subset the method with the shortest interval length would be used.

As illustrated in the supplementary material the results in terms of coverage probability are quite similar for designs with four or even three dose levels instead of five. Especially in the last design with only three dose levels it is quite difficult to use higher or more complex models than the linear model because of the small amount of information on the dose-response curve. To analyze the influence of a non-normal error term in the linear model a t-distribution as well as a log-normal distribution were simulated. We found that the non-normal error term has basically no influence on the coverage probability and on the median interval length. Therefore linear models or transformed models (linearization, log-linear dose response) with non-normal error terms can be evaluated quite well with the methods described in this paper, for moderate group sample sizes n > 10.

To our knowledge up to now a systematic comparison of the various approaches for confidence intervals of the target dose in dose-finding studies with active control is lacking. Here we presented such a comparison for linear models because these models are of interest in some situations and are subject of recent research (Demidenko et al., 2013). They are used in a wide range of applications such as transformation techniques into linear models (see, for example, Neter et al. (1996) and Box and Cox (1964)), generalized linear models via link function (McCullagh and Nelder, 1989), as well as regression situations with low information of the dose relationship. In these situations the MLEs of more complex models might not converge and the model has to be simplified, which is described in more detail by Jones et al. (2011) and Kirby et al. (2011).

We assumed variance homogeneity, i.e., the variance of the error terms to be the same for all dose levels as well as for the active control. One reason for this assumption was that in the case of equal variances and normally distributed error terms the least square estimators of the linear model theory are identical with the MLEs, resulting in a simple presentation of the various methods. For both approaches, least squares and maximum likelihood and all presented confidence interval methods, it is possible to use separate variance estimators for each dose level and/or only for the active control and the dose levels. This can be done straightforward but the MLH estimators would differ from the least squares estimators and have to be computed numerically.

Of course for larger deviations from a linear dose-response curve or without a useful transformation it will be necessary to fit appropriate nonlinear models. The extension of some of these methods to nonlinear mean functions in the setting of active controlled dose-finding studies is subject to ongoing research with a focus on adaptive designs as presented in Jones et al. (2011) and Kirby et al. (2011).

SUPPLEMENTAL MATERIAL

Supplemental data for this article can be accessed on the publisher’s website.