Efficient design of cluster randomized trials with treatment-dependent costs and treatment-dependent unknown variances.

Cluster randomized trials evaluate the effect of a treatment on persons nested within clusters, where treatment is randomly assigned to clusters. Current equations for the optimal sample size at the cluster and person level assume that the outcome variances and/or the study costs are known and homogeneous between treatment arms. This paper presents efficient yet robust designs for cluster randomized trials with treatment-dependent costs and treatment-dependent unknown variances, and compares these with 2 practical designs. First, the maximin design (MMD) is derived, which maximizes the minimum efficiency (minimizes the maximum sampling variance) of the treatment effect estimator over a range of treatment-to-control variance ratios. The MMD is then compared with the optimal design for homogeneous variances and costs (balanced design), and with that for homogeneous variances and treatment-dependent costs (cost-considered design). The results show that the balanced design is the MMD if the treatment-to control cost ratio is the same at both design levels (cluster, person) and within the range for the treatment-to-control variance ratio. It still is highly efficient and better than the cost-considered design if the cost ratio is within the range for the squared variance ratio. Outside that range, the cost-considered design is better and highly efficient, but it is not the MMD. An example shows sample size calculation for the MMD, and the computer code (SPSS and R) is provided as supplementary material. The MMD is recommended for trial planning if the study costs are treatment-dependent and homogeneity of variances cannot be assumed.

RCT Entities:   Population Interventions Outcomes

INTRODUCTION

Randomized experiments are commonly seen as the best method for evaluating the effect of some new treatment in public health and medicine. However, randomized assignment of individuals is neither always feasible nor always desirable. For instance, a healthy food program for restaurants in high schools can only be implemented at the school level, and so we can randomly assign schools, but not pupils, to treatment or control. Further, even if randomization of individuals is possible, this may be undesirable due to a risk of treatment contamination. For instance, it may be very difficult for a life style counselor in family practice to switch between 2 methods of counseling when switching between patients. Treatment contamination reduces the outcome difference between groups and the power to detect a difference. For these reasons, cluster randomized trials,1, 2 also known as group randomized trials,3 are frequently run. In such trials organizational units (eg, schools or family practices), called clusters, are randomly assigned to treatment or control, and all individuals within the same cluster receive the treatment to which their cluster was assigned. Cluster randomized trials are thus a natural alternative to the randomized experiment when individuals are nested within organizations, such as pupils in schools or patients in health centres. Some examples of cluster randomized trials are a smoking prevention trial and a stress management trial, both run in primary schools with schools as unit of assignment,4, 5 and the assessment of COPD burden trial, run in primary and secondary care.6

However, cluster randomization gives a lower power and precision than individual randomization because outcome variation between clusters within the same treatment group inflates the sampling variance of the treatment effect estimator by a factor known as the design effect. This design effect can easily be as large as 2 or 3, depending on the so‐called intraclass correlation (ICC), which is the ratio of outcome variance between clusters to total outcome variance, and on the sample size per cluster.7 It is thus important to design a cluster randomized trial efficiently within constraints like feasibility and the budget for sampling, treating and measuring clusters and persons. In particular, the sample size at each design level (cluster, person) must be cost‐effective, that is, the power to detect a treatment effect of interest must be maximized at minimum costs. Equations for the optimal sample size in a cluster randomized trial, taking into account outcome variance and costs at each level (cluster, person), have been published.8, 9 These equations are based on 3 restrictive assumptions, however: (1) homogeneity of outcome variance and costs between treatment groups, at the cluster level and at the individual level, (2) a known ICC, and (3) an equal sample size per cluster.

The assumption of an equal sample size per cluster was relaxed by showing that the increased sampling variance of the treatment effect arising from cluster size variation, can be approximated by a simple function of the coefficient of variation (CV) of cluster size, and that this increase can often be compensated by sampling 10% to 20% more clusters.10, 11

The assumption of a known ICC was recently dropped by the derivation of maximin designs (MMDs) which are robust against misspecification of the ICC.12 Two other methods to handle the unknown ICC are the Bayesian approach and group sequential trials.13, 14, 15 This leaves the assumption of homogeneous costs and variances to be relaxed, and there are good reasons for relaxing that assumption.

With respect to costs, the growing literature on cost‐effectiveness cluster randomized trials16 testifies to the heterogeneity of costs between treatments. Some of the costs for running a trial may also differ between treatment arms, for instance if one of the treatments is so new that the staff of the participating clusters needs to be trained into the proper use of that treatment. With respect to outcome variance, substantial heterogeneity of variance, with variance ratios as high as 8 or even 12, was found in some studies in clinical psychology.17 For cluster randomized trials in medical and health care, no similar review is known to us. Going through all cluster randomized trials in a published review,18 and restricting ourselves to quantitative outcomes, we found only a handful of trials reporting the outcome variance or SD per arm, and these suggest at least some heterogeneity. For instance, a control‐to‐treatment variance ratio of 3 was reported for the outcome “confidence” in a trial on the effects of a training of general practitioners and nurses on patient‐centered counseling and material use for Type 2 diabetes patients.19 Similarly, a variance ratio of 1.7 was found for systolic blood pressure in a comparison between computer based clinical decision support and usual care for management of hypertension in primary care.20 Both variance ratios are significantly different from one according to the classical F‐test (no adjustment for clustering could be made because both publications only report the total variance and not the ICC). Further, significant heterogeneity of variance, with a variance ratio of 1.35 for the outcome patient satisfaction, was reported in a trial comparing nurse practitioners with general practitioners.21

For binary outcomes and count data, heterogeneity of variance is even bound to occur if treatments differ in mean outcome. The present paper focusses on quantitative outcomes, but its general approach can also be applied to other outcomes. For example, all equations in the next section have an equivalent for binary outcomes.22

Going back to the literature on optimal design and sample sizes, the homogeneity of variance assumption was relaxed for randomized experiments without clustering by several authors,23, 24, 25 and for experiments with clustering in 1 treatment arm by Moerbeek and Wong.26 Heterogeneity of costs per cluster in a cluster randomized trial was considered by Liu,27 assuming homogeneous variances and sample size per cluster. Allowing heterogeneous costs and heterogeneous variances, Candel and Van Breukelen28 derived the optimal number of clusters per treatment arm, given a fixed instead of optimal cluster size, which was allowed to differ between treatment arms, however. Lemme et al29 derived the optimal number of clusters as well as the optimal cluster size per treatment arm, but application of their result requires the outcome variance per treatment arm per design level to be known in the design stage. Finally, Wu, Wong and Crespi30 derived the optimal treatment to control allocation ratio for a cluster randomized trial with a binary outcome and heterogeneous costs and ICCs, given a fixed instead of optimal sample size per cluster, which was restricted to be the same for both treatment arms. Candel and Van Breukelen as well as Wu et al also discussed MMDs to handle dependence of the optimal design on unknown parameters.

This paper extends the literature in 3 ways. First, both the number of clusters and the cluster size per treatment arm of a cluster randomized trial are optimized as a function of treatment‐dependent costs and treatment‐dependent variances at each design level, cluster and individual. In this respect it extends.27, 28, 30 Secondly, the problem that the variances are unknown in the design stage is addressed by MMD, as in Candel and Van Breukelen28 and Wu et al30, which only requires the user to specify a maximum for the ICC and a range for the treatment‐to‐control outcome variance ratio. Within these ranges we derive robust and efficient designs. Third and last, we compare these designs not only with the commonly used balanced design, which is optimal under homogeneity of variances and costs, but also with the design that is optimal for heterogeneous costs but homogeneous variances. The latter design is a practical alternative to the balanced design because costs, unlike variances, can be known in the design stage.

The outline of this paper is as follows. First, the mixed model for analyzing a cluster randomized trial is specified and the optimal sample size per level is given as a function of the ICC and the cluster‐to‐person cost ratio, for the case of homogeneous variances and costs (Section 2) and for the heterogeneous case (Section 3). Because costs can be known in the design stage while variances cannot in general, we then derive the MMD (Section 4). This is the design with maximum efficiency (minimum sampling variance) of the treatment effect in the worst case scenario, that is, in the case where the treatment‐to‐control variance ratio is such that the efficiency is at its minimum (ie, that the sampling variance is at its maximum), hence the name MMD. We compare the minimum efficiency of this design with that of the balanced design, and with that of the optimal design for heterogeneous costs and homogeneous variances. The comparison is made as a function of the range for the treatment‐to‐control variance ratio, and of the treatment‐to‐control cost ratio. Throughout the paper, we assume an equal sample size per cluster within treatment arms, but not between arms. Cluster size variation within arms can be handled as in,10, 11, 31 which often means increasing the number of clusters per treatment arm with 10% or so. Furthermore, it is assumed that the data will be analysed with a statistical model that takes into account the nesting of subjects in clusters and that allows for heterogeneity of variance at each design level. To help the reader keep track of all mathematical symbols in this paper, Appendix A lists all symbols, with their meaning and the section where they are first used.

OPTIMAL DESIGN UNDER HOMOGENEOUS COSTS AND VARIANCES

Let us start with the statistical model for the treatment effect on a quantitative outcome Y in a cluster randomized trial:

where giving:

Here, Y is the outcome for subject i in cluster j, and X is the treatment assigned to cluster j (1 = treatment, 0 = control). Parameter is the mean outcome difference between treated and control, that is, the treatment effect. The intercept is the sum of a fixed mean intercept plus a random cluster effect which is assumed to have a normal distribution with mean zero and variance . Finally, is the sum of subject and measurement error effects, with mean zero and variance within each cluster. The outcomes of any 2 subjects i and i′ in the same cluster j are correlated due to the shared cluster effect , and this intraclass or intracluster correlation (ICC) is: where is the total outcome variance within conditions.

Because the aim of a cluster randomized trial is to estimate the treatment effect as precisely as possible and to have a maximum power for testing this effect, a logical criterion for the design efficiency is the sampling variance of the treatment effect estimator, , which for model (1) and 50:50 treatment allocation is22:

Here, n is the sample size per cluster, K is the total number of clusters sampled, and so nK is the total number of subjects sampled. The factor [ ( is the design effect (DE) and indicates the sample size needed for a cluster randomized trial relative to that for a classical trial with individual randomization. If ρ is zero, there is no outcome variation between clusters, and cluster randomization is as efficient as individual randomization, as reflected by a DE of one. In practice, the ICC is usually between 0.01 and 0.10 or 0.25, depending on the field of application,18, 32, 33 and so the DE is larger than one and increases with the sample size per cluster, n. For large n, approaches , which depends on the variance and sample size at the cluster level only.

We can increase the power of the trial by increasing n or K, and the latter is more effective because increasing n also increases the DE, thus undoing part of the power gain. Unfortunately, increasing the sample size not only increases the study power, but also the total study cost (money, time and effort), and the cost per cluster may be larger than the cost per subject, raising the question of the optimal sample size per level (cluster, person). To answer this question, we need to combine the optimality criterion in Equation (3) with a cost function relating study costs to sample size. Following,8, 9 we assume a fixed cost c per cluster and cost s per subject, where c and s are in the same currency (eg, US$ or Euro). The total budget B needed for K clusters of n subjects each, ignoring all study costs that do not depend on the sample size, then is: where (c + sn) is the total cost per cluster including the cost per subject within the cluster. The optimal design now minimizes in Equation (3) as a function of n and K, given the constraint in Equation (4). Assuming , this gives8, 9: where the superscript * indicates the optimal design. So, the optimal sample size per cluster, , increases with the cluster‐to‐subject cost ratio c/s, and with the subject‐to‐cluster variance ratio . The optimal number of clusters, , then follows from the budget constraint in Equation (4). Note that the total budget B only affects the optimal number of clusters , but not the optimal sample size per cluster . So, if the budget is increased to gain power, then the extra budget must be spent on more clusters rather than on more subjects per cluster. Further, the optimal design in Equation (5) clearly depends on the ICC, which is usually unknown in the design stage. So, the design is locally optimal only, that is, optimal for 1 specific value of the unknown ICC. Combining Equations (3) and (5) gives for the optimal design, which is the smallest possible , yielding the largest possible power and precision, given our budget, costs, and outcome variance and ICC:

So the smallest possible sampling variance of the treatment effect estimator increases with the costs c and s, in a way which depends on the ICC. Further, increases with the ICC to a maximum , attained if . This last case gives a sample size = 1 per cluster by Equation 5, so that the design is no longer nested and no longer depends on ρ, see Equation (3). So, Equations 5 and 6 only apply if   , which is a reasonable assumption given that the ICC is typically between 0.01 and 0.25,18, 32, 33 and that c > s will almost always hold. If in Equation (6) is still too large for sufficient power and precision, the only solution is to increase the budget B, and spend the extra money on sampling more clusters, see Equation 5. A simple equation for computing the number of clusters needed for a given effect size d (defined as , analogous to Cohen's d for non‐nested trials,34) power, type I error rate α, cost ratio c/s, and ICC, is given in Van Breukelen and Candel.7

OPTIMAL DESIGN UNDER HETEROGENEOUS COSTS AND VARIANCES

Equations (1) to (6) assume that the outcome variances and the costs are homogeneous between treatment arms. The assumption of homogeneous costs is unrealistic for expensive or time‐consuming treatments and was relaxed by Liu.27 The assumption of homogeneous variance is not realistic either, because a treatment which affects the outcome mean will also affect its variance, unless the treatment effect is the same for all clusters and all subjects. The optimal design results thus need to be generalized to heterogeneous costs and variances. In terms of Equation (1) this means that and and thus also the total outcome variance and the ICC can differ between treatment arms. In terms of Equation (4), the costs c and s can likewise differ between arms. Now, the treatment effect estimator is the difference between 2 independent sample means: that of the treated clusters, and that of the control clusters.27 So is under heterogeneity of variance equal to: for any design (. Here, is the ICC, is the total outcome variance, is the sample size per cluster, and is the number of clusters, in the treated arm, and likewise , , and in the control arm. The minimum of is found in 2 steps: First, assume a given budget for the treated arm and for the control arm, and minimize in Equation (7a) subject to the same budget constraint as in Equation (4), but now per treatment arm. Secondly, given this minimum as a function of the budgets and , let B denote the total budget for both treatment arms and let f denote the fraction of that budget which is spent on the treated arm, so and . Then, find that f which minimizes as obtained in step 1. Below these 2 steps are elaborated. Step 1 gives as optimal design for the treated arm respectively the control arm:

Here, is the cost per cluster, and is the cost per subject, in the treated arm, and likewise and in the control arm. Note that the optimal sample size per treated cluster, , does not depend on the budget for the treated arm, , and does not depend on . Only the numbers of clusters per arm, and , depend on the budgets and are not optimal yet. Substituting Equations (7b) and (7c) into Equation (7a) gives upon rewriting: for the design as specified by Equations (7b) and (7c), with and .

Note that Equation (7a) reduces to Equation (3) if , , and hold, and Equation (7d) reduces to Equation (6) if additionally and and hold.

In step 2 of the design optimization, substitute in Equation (7d) and , where f is the fraction of the budget B that is spent on the treatment arm, and then minimize Equation (7d) as a function of f, given B. This gives as optimal split of the budget between the treated and control arm:

The optimal number of treated clusters, , then follows from Equation (7b) by letting , and likewise the optimal number of control clusters, follows from Equation (7c) by letting . Combining Equation (7d) with Equation (8) results in the following minimum variance of the treatment effect estimator under heterogeneity: which reduces to Equation (6) if the costs and the variances are homogeneous. In the latter case, the optimal design is balanced, that is, = and .

Another special case is that of homogeneous variances but heterogeneous costs, giving . We refer to this as the cost‐considered (cc) design. The optimal design of Liu27 is a special case of the cc design in that Liu assumes . This is optimal only if , see Equations (7b) and (7c). Given homogeneity of variances and , implying , the optimal budget allocation ratio is , and the optimal cluster allocation ratio is , which depends on the costs per cluster only. However, it can be shown that this cluster allocation ratio is also optimal if , provided the variances are homogeneous and are optimized with Equations (7b) and (7c) instead of letting .

SOLVING THE PROBLEM OF LOCAL OPTIMALITY: MAXIMIN DESIGN

As Equations (7b), (7c), and (8) show, the optimal budget split between the treated and control arms, and the optimal sample size per design level (cluster, subject), depend on the costs and variances per design level, per arm, making the optimal design locally optimal only. But while the costs can be known in the design stage, the variances cannot, and so the optimal design is vulnerable to misspecification of the variances. A solution is the MMD. Maximin design consists of 4 steps, and each is elaborated after this list:

Specify the parameter space, that is, the region containing all plausible values of the unknown parameter vector on which the optimal design depends;

Specify the design space, that is, the set of all candidate designs (;

For each design in the design space find its worst case, here: the maximum of , denoted as , as a function of the unknown parameter vector ;

Finally, select the design with the smallest , denoted as , or equivalently, the design with the maximum minimum efficiency. This is the MMD, denoted as , which is robust against misspecification of the unknown parameters in that it optimizes the worst case.

Concerning step 1, some constraints are needed on the parameter space, because Equation (7a) implies that is an increasing function of the variance in each treatment arm for any given design(, and Equation (9) implies the same for of the optimal design. A finite maximum is therefore imposed on the (unknown) sum of the outcome variances, , to prevent an infinitely large as the worst case in step 3. Further, assume a bounded SD ratio , where u (u ≥ 1) is based upon prior knowledge and determines the possible amount of heterogeneity of variance, with u = 1 giving homogeneous variance, and allowing extremely heterogeneous variance. This section focusses on 2‐sided intervals for the SD ratio, but Section 6 covers 1‐sided intervals. Combining with gives and likewise for . Finally, in step 1 we assume a common maximum possible value for the unknown ICC in each arm, but allow the actual ICC to differ between arms. This may depend on the field of application, for instance 0.10 in primary care,18, 32 or 0.25 in education,33 and is assumed to satisfy for reasons given in Section 2 below Equation (6).

Concerning step 2, restrict the design space to all designs satisfying the following constraints: and , where B is the total study budget for all costs that depend on the sample size. Without the budget constraint, step 4 would give an infinitely large sample size as the best design.

Concerning step 3, note that is an increasing function of the ICC and of the variance in each arm, see Equations (7a) and (7d) and the text below Equation (6). Therefore, given any design (, is maximized (worst case) by letting and in Equation (7a) or (9);

Finally, concerning step 4, note that for any given parameter vector , including those from step 3, is minimized by taking the optimal design. So the MMD can be derived from Equation (7d) instead of Equation (7a). To begin with, this gives as maximin sample size per cluster in the treated arm, and likewise in the control arm, and as worst case the following sampling variance for the treatment effect: where , which is a function of the costs c and s in each treatment arm, and of . As a special case, gives .

It is important to note that, once the costs and and have been specified by the user, in Equation (10) depends on 2 unknowns only: the total outcome variance in the treatment arm, , and the fraction of the budget B spent on the treatment arm, f. So we can first maximize Equation (10) as a function of within its constraint , and then minimize that maximum as a function of the budget split. This gives the maximin budget ratio , and the corresponding shown in Table 1 (for detailed derivations, see Appendix B). The MMD itself is not given in Table 1, but is obtained by filling in Equations (7b) and (7c), using the budget per arm as given in Table 1, and the known costs for that arm, and the maximum ICC, .

Table 1

Maximin budget split and maximum variance of the treatment effect for heterogeneous costs and variances given a fixed maximum total variance and a fixed total budget B, as a function of the relation between u (square root of maximum treated‐to‐control variance ratio) and p (square root of treated‐to‐control cost ratio)

Relation of p to u	Budget Split f^m1−f^m	Maximum Varβ^_1 for the Maximin Design	Equation nr for Referencing
1u≤p≤u	p2	g_cρ_maxV_maxB×1+p^2	(11)
p  > u	pu	g_cρ_maxV_maxB×pu+1^2u^2+1	(12)
p<1u	pu	g_cρ_maxV_maxB×p+u^2u^2+1	(13)

As Table 1 shows, the maximin budget ratio depends on whether p is within the range for the SD ratio or not. If p is within this range, the budget ratio and do not depend on that range. If  , then treatment is more expensive than control (p > 1) and costs are more heterogeneous than variances. Consequently, more budget is allocated to treatment than to control (  ). Likewise, implies that treatment is cheaper than control (p < 1) and that costs are more heterogeneous than variances, resulting in a higher budget for control ( ). Figure 1 plots the maximin budget ratio against u from 1 to 5 (implying a range from 0.04 to 25 for the variance ratio), for p from 2 to 5 (implying a cost ratio from 4 to 25 in the case ). If p < 1 (treatment cheaper than control), then Figure 1 applies in that it then shows the control‐to‐treatment budget ratio.

Figure 1

Maximin budget ratio treatment‐to‐control, based on the minimum efficiency ( ) criterion, as a function of the range for the treatment‐to‐control SD ratio, for p = 2, 3, 4, 5 (p = √ of treatment‐to‐control cost ratio if c/s is homogeneous)

COMPARISON WITH THE BALANCED DESIGN AND THE COST‐CONSIDERED DESIGN

This section compares the MMD with the balanced design (which is optimal for homogeneous costs and homogeneous variances) and the cc design (which is optimal for homogeneous variances but heterogeneous costs). It does so by considering the relative efficiencies of all designs, as defined by the ratios of their ’s.

Relative efficiency of balanced design versus MMD

Consider first the case (homogeneous cluster‐to‐subject cost ratio), so that holds for both designs (see Equations (7b) and (7c) with ), and , the treatment‐to‐control cost ratio. Balanced allocation then implies , that is, the budget allocation ratio equals the cost ratio. This is the MMD if , that is, if costs are more homogeneous than variances, see Equation 11 in Table 1.

If   , the relative efficiency (RE) of the balanced design versus the MMD is the ratio of in Equation 12 to that in Equation 11 in Table 1. If , the RE is the ratio of in Equation 13 to that in Equation 11. In both cases, the RE increases in u, and RE > 0.95 if . Far outside these boundaries, the balanced design can be inefficient, with a minimum of for and (homogeneous variances, expensive treatment), and for and (homogeneous variances, expensive control). To show this, Figure 2 plots the RE of the balanced design versus MMD against u for various p.

Figure 2

Relative efficiency of the balanced design versus the MMD as a function of the range for the unknown SD ratio, for various p (assuming homogeneity of c/s)

For , the RE of the balanced design is obtained as follows: First, compute the design with Equation 5, assuming and and . Then, compute for this design using Equation (3), assuming and letting .

Relative efficiency of cost‐considered design versus MMD

Because the balanced design is inefficient for very heterogeneous costs, and costs can be known in the design stage, an alternative to the balanced design is the cc design, which is optimal for heterogeneous costs (p ≠ 1) and homogeneous variance (u = 1). Following Equation (8), the cc design has the budget allocation ratio (see the definition of p below Equation (10)). This is the limiting case of the MMD for (homogeneous variances). If , we get . However, the RE of the cc design versus the MMD as discussed below is not restricted to this special case.

Using , with , and using Equation (10), we get for the cc design if the variance is actually heterogeneous:

If p = 1, Equation (14) does not depend on and gives the same result as Equation 11 in Table 1. But the design is balanced only if which, for p = 1, implies homogeneity of costs.

If p > 1 (treatment more expensive than control), then Equation (14) is maximized by maximizing within the constraint of Section 4, giving , and the resulting worst case, , is shown in Equation 15 in Table 2.

Table 2

Cost‐considered budget split and maximum variance of the treatment effect for heterogeneous costs and variances given a fixed maximum total variance and fixed total budget B, as a function of the range of p (square root of treated‐to‐control cost ratio)

Range of p (a)	Budget Split f^cc1−f^cc	Maximum Varβ^_1 for the Cost‐Considered Design	Equation nr for Referencing
p  =  1	p	g_cρ_max2V_maxB	‐
p  > 1	p	g_cρ_maxV_max1+pB×pu^2+1u^2+1	(15)
p < 1	p	g_cρ_maxV_max1+pB×p+u^2u^2+1	(16)

The cost‐considered design assumes homogeneous variances and so its budget split does not depend on the SD ratio parameter u (square root of maximum variance ratio). But the maximum (worst case) of the cost‐considered design does depend on the actual heterogeneity of variance as expressed by u, except if p = 1.

If p < 1 (treatment cheaper than control), then Equation (14) is maximized by minimizing within the same constraint, giving and as worst case Equation 16 in Table 2. Note that, under homogeneity of variances (u = 1), Table 1 and 2 give the same results.

The relative efficiency (RE) of the cc design versus the MMD with respect to the optimality criterion now follows from Equations 11 to 16, for instance, as the ratio of in Equation 11 to that in Equation 15 if 1 < p < u. Figure 3 plots the RE of the cc design versus the MMD as a function of u for various p. The results can be summarized thus:

Figure 3

Relative efficiency of the cost‐considered versus the MMD as a function of the range for the unknown SD ratio, for various p (without assuming homogeneity of c/s)

If , the RE of the cc design is a decreasing function of u, with a minimum of 0.83 if and or , and a maximum of 1 if and ( or ), or if and ( or ).

If , the RE of the cc design again decreases in u, but now with a minimum of 0.89 if and , or if and , and a maximum of 1 if either or or . So the RE of the cc design compared with the MMD design, using as criterion max , is always above 0.80.

Relative efficiency of balanced versus cost‐considered design

Finally, if (homogeneous cluster‐to‐person cost ratio), the RE of the balanced versus the cc design follows from Equations 11, 15, and 16 in Tables 1 and 2 and is plotted in Figure 4. If , the balanced design is more efficient. If , the cc design is more efficient. At the interval boundaries, both designs are equally efficient.

Figure 4

Relative efficiency of the balanced versus the cost‐considered design as a function of the range for the unknown SD ratio, for various p (assuming homogeneity of c/s)

Comparing all 3 designs, MMD, balanced and cc, we get for the case :

For balanced is optimal and cc is fairly efficient, with a minimum RE of 0.83;

For and for , balanced is more efficient than the cc design, which in turn has a minimum RE of 0.89 once p is outside the interval ;

For p outside the interval , the cc design is more efficient than the balanced design, and highly efficient due to its minimum RE of 0.89. The balanced design becomes inefficient if while p moves away from 1, for instance, RE < 0.80 if u = 1 and p > 3 or p < 1/3, meaning a homogeneous variance and a cost ratio above 9 or below 1/9.

Relative efficiency of the MMD if the variances turn out to be homogeneous

Up to now, the variance was assumed to be heterogeneous and so the MMD was always at least as efficient as, and usually more efficient than, the balanced and the cc design, which both assume homogeneous variances. But how efficient is a MMD based on heterogeneous variance (u > 1 assumed) if the variance turns out to be homogeneous (u = 1 actually) in the analysis? Consider again the 3 cases in Equations 11 to 13.

If is assumed in the design stage, the MMD is balanced, at least for  (Section 5.1). Its RE compared with the cc design if u = 1 (homogeneous variances) turns out to be the case, then follows from Section 5.3 and Figure 4 with u = 1;

If  is assumed, the maximin budget ratio . Because p > u > 1, this budget split is in‐between that for the cc design, , and that for the balanced design, . The MMD itself is obtained by filling in the budget and costs per treatment arm in Equations (7b) and (7c), which depend on u only through the budget, and so the MMD is in‐between the balanced design and the cc design. It then follows from Figure 4 with u = 1 that, if the variances are actually homogeneous, the MMD is always more efficient than the commonly used balanced design, and less efficient than the cc design. The differences between the 3 designs are small for p up to 2, and still acceptable for p up to 3, implying a treated‐to‐control cost ratio up to 4, respectively up to 9, in the case where . Finally, the case becomes the case  by switching the 2 treatment arms, thus giving the same RE results as the case p > u.

MAXIMIN DESIGN FOR ONE‐SIDED INTERVALS FOR THE VARIANCE RATIO

Sometimes, a 1‐sided interval may perhaps be assumed for the SD ratio, for instance if we know from other studies that treatment always increases outcome variance (or always decreases it). This gives the following result (for details, see appendix B):

If and , or if and , then the budget split for the MMD is , which is the budget split for the cc design. The is then obtained from Table 1 or 2 by filling in u = 1 (see Appendix B, and note that Tables 1 and 2 give the same results if u = 1). In all other cases the MMD and are the same as for the 2‐sided case in Table 1. So if the more expensive arm has the smaller variance, then the MMD for 1‐sided intervals for the SD ratio is the cc design. But if the more expensive arm has the larger variance, then the MMD for 1‐sided intervals for the SD ratio is the same as for 2‐sided intervals. The MMD for the various cases in this and the preceding sections are summarized in Table 3. It follows from this table that the maximin budget split is always between  (the cc budget split) and  (the split for the balanced design if ).

Table 3

Maximin budget split for heterogeneous costs and variances as a function of the relation between u (maximum SD ratio) and p (square root of treated‐to‐control cost ratio), for 2‐ and 1‐sided intervals for the SD ratio (note: Budget split gives the cost‐considered design; budget split gives the balanced design if )

Relation of p to u	p<1u	1u≤p≤1	1 ≤ p ≤ u	u < p
Range for SD ratio
1uu	pu	p2	p2	pu
[ 1 ,  u]	p	p	p2	pu
1u1	pu	p2	p	p

POWER AND SAMPLE SIZE CALCULATION

It has been shown how the MMD depends on 2 factors: , which is a function of the costs in each treatment arm and of the maximum ICC, and , which is the range for the SD ratio . It has also been shown how efficient the balanced and cc design are relative to the MMD with respect to their minimum efficiency (maximum sampling variance). For practical use, these results must be translated into a procedure for computing the sample size needed per treatment arm to have sufficient power to detect a treatment effect of given size. This procedure consists of the following 5 steps for the MMD (the cc design is obtained by choosing , and the balanced design is obtained by additionally letting ):

First, specify the costs  and choose plausible values for , , and the treatment effect . Instead of specifying and , one can also specify a smallest clinically relevant effect size (a generalization of Cohen's d to heterogeneous variances, see the end of Section 2). Because the maximin sample size per cluster depends on neither nor (see step 3 below), and the maximin number of clusters per treatment depends on them only through d (see steps 2 and 4 below), one can also, without loss of generality, specify d;

Second, compute the sampling variance that gives the desired power level (1 − , for a given type I error rate  and true treatment effect , with the following equation: Here, denotes the p‐th percentile of the standard normal distribution. For 1‐tailed testing, replace with . Note that Equation (17) is also used for the classical RCT with individual randomization;

Third, compute the maximin sample size per treated cluster, , with Equation (7b), using as costs  and as ICC , and the maximin sample size per control cluster, , using as costs  and as ICC , and compute the functions and in Equation (7d) and the ratio ;

Fourth, compute the maximin budget ratio  from and , the plausible range for the SD ratio, using Table 1. Then compute the total budget B by equating as given in Table 1, with as computed in step 2 with Equation (17), and compute the budget per treatment arm as and ;

Fifth and last, compute the number of clusters per treatment arm as , and the number of clusters per control arm as , following Equations (7b) and (7c), and round both numbers upward to ensure sufficient power. The budget B will increase a bit by this.

This procedure is easily implemented in standard statistical software, and an implementation in SPSS and in R is available as supplementary material. Two additional remarks are needed, however. First, we do not round off the sample sizes per cluster, and , for 2 reasons: (1) to ensure that all further computations can use the equations in this paper without modification, and (2) because unplanned variation in sample size per cluster always occurs in practice, and so the average sample size per cluster will rarely be an integer anyway. Such unplanned variation leads to some power loss which can be restored by increasing the number of clusters with a percentage that depends on the CV of cluster size, and an increase of the number of clusters with 10% respectively 20% is often a reasonable respectively very safe adjustment.10 This adjustment is not included into the presented syntax, because the user may prefer another percentage, depending on the expected CV. Of course the user can choose to round and upward after step 5 to guarantee sufficient power, and this will lead to some increase of the study budget that is needed.

Secondly, step 2 of the procedure above assumes that the test statistic for the treatment effect, , has a standard normal distribution under the null hypothesis, and that has a standard normal distribution under the alternative hypothesis, where . This ignores the fact that SE is a function of the unknown variances and must be estimated. The correct reference distribution is the Student t‐distribution. This follows from the equivalence of mixed regression analysis of individual data following Equation (1) with the unpaired t‐test of treatment versus control using clusters as units of analysis and cluster means as outcome.35 Under heterogeneity of variance of the cluster means, the degrees of freedom for this t‐test obey an expression due to Welch and Satterthwaite,36, 37 which reaches a minimum df = if the variance ratio goes to zero or infinity, and a maximum df = if the variance ratio goes to one. The df determine the percentiles and , to be used in step 1 instead of and respectively, and thus also the budget and sample size needed, as these are proportional to (. The case gives the standard normal distribution, but the Student t‐distribution is close to that for df = 60 or so. For smaller df, ccalculations in28, 38 show that, at least if the number of clusters per arm is 8 or more, the power loss due to finite df is compensated by having 2 extra clusters per arm if α = 0.05 two‐tailed, or 4 extra clusters per arm if α = 0.01 2‐tailed. With less than 8 clusters in a given arm, add 3 instead of 2 clusters to that arm if α = 0.05, and 4 if α = 0.01. This holds for a planned power of 80% or 90%, and under strong heterogeneity of variance. This adjustment is included into the computer code. Not included into the code for simplicity, but stated here for completeness, is the following exception: Under complete homogeneity of variance and costs the MMD becomes balanced, and if there are at least 8 clusters per arm we then need to add only 1 (if α = 0.05) or 2 (if α = 0.01) clusters per arm.38 This only holds under complete homogeneity of variance, however, not if heterogeneity is allowed but the MMD happens to be balanced as discussed in Section 5.1. Having said this, let us now consider an example of MMD.

Example

Strong heterogeneity of outcome variance has been found in randomized trials in clinical psychology, with variance ratios up to 12 or even 16,17 but results from cluster randomized trials in health care are scarce and suggest mild heterogeneity, such as a variance ratio of 3.19 Further, study costs per cluster and per subject are rarely reported. As an exception, Moerbeek et al reported a cluster‐to‐subject cost ratio c/s = 26 in a smoking prevention trial in primary education, but gave no information on the costs per arm.39

Our example therefore makes the following assumptions in step 1 of the sample size calculation: Let u = 1 or 2 or 3 (homogeneous variances, respectively a treatment‐to‐control SD ratio between 0.50 and 2, or between 0.33 and 3). Further, the maximum ICC is = 0.10 or 0.20.18, 32, 33 Finally, the costs are homogeneous at each level (cluster, subject), or heterogeneous at the cluster level (with ), or at the individual level (with ), or at both levels (with ), and the cluster‐to‐subject cost ratio in case of homogeneity at both levels or of heterogeneity at both levels. These scenarios assume that treatment is at least as expensive as control. The reverse case gives the same designs apart from switching and and, except if , also switching and (see Sections 3 and 4 for technical details).

Combining the input parameters above gives 3*2*4 = 24 different scenarios for the MMD. For each scenario we let = 5 and = 200, so that the standardized treatment effect size d = 0.50,34 where which, under homogeneity of variance reduces to . Remember that and do not affect either the maximin sample sizes per cluster, (see step 3 of the sample size calculation above), or the budget allocation ratio (see Table 1), or the cluster allocation ratio  (steps 4 and 5 above). Instead, and only affect the total budget and the total number of clusters needed. This completes step 1 of the maximin sample size calculation. Next, in step 2, let α = 0.05 two‐tailed and let the desired power be 90%. This gives as maximum allowable sampling variance for the treatment effect, = 2.3815 by Equation (17). Following the steps 3‐4‐5 of the sample size calculation procedure above for each of the 24 scenarios then gives the maximin sample sizes in Table 4, with a separate row per scenario. To better see the relation between the input parameters and the resulting MMD, neither nor have been rounded off. As explained above, need not be rounded off anyway in view of unplanned variation in sample size per cluster in practice, and both need to be rounded upward and then increased by 2. The total budget in the last column of Table 4 is based on the thus adjusted .

Table 4

Maximin design and budget needed to detect a treatment effect of medium size (d = 0.50) with a power of 90% using 2‐tailed testing with α = 0.05, as a function of the range for the SD ratio , the maximum ICC , and the study costs per cluster and per subject in the treated arm , and control arm c , s

Input Parameters				Maximin Budget Split, Maximin Design, Total Budget (a)
1uu	ρmax	ct ,  st	cc ,  sc	p	f^m1−f^m	ntm	ncm	Ktm	Kcm	Bm
[1,1]	0.10	200,10	200,10	1.00	1.00	13.42	13.42	14.04	14.04	11361.58
		360,10	40,10	1.80	1.80	18.00	6.00	9.81	29.42	9680.00
		200,18	200,2	1.46	1.46	10.00	30.00	13.45	13.45	10240.00
		360,18	40,2	3.00	3.00	13.42	13.42	9.36	28.09	9289.76
	0.20	200,10	200,10	1.00	1.00	8.94	8.94	24.33	24.33	15629.91
		360,10	40,10	2.00	2.00	12.00	4.00	16.81	50.44	13360.00
		200,18	200,2	1.33	1.33	6.67	20.00	23.54	23.54	14560.00
		360,18	40,2	3.00	3.00	8.94	8.94	16.22	48.66	12851.26
[0.50,2]	0.10	200,10	200,10	1.00	1.00	13.42	13.42	14.04	14.04	11361.58
		360,10	40,10	1.80	3.24	18.00	6.00	12.61	21.01	10500.00
		200,18	200,2	1.46	2.14	10.00	30.00	15.97	10.93	10220.00
		360,18	40,2	3.00	6.00	13.42	13.42	13.11	19.66	11094.25
	0.20	200,10	200,10	1.00	1.00	8.94	8.94	24.33	24.33	15629.91
		360,10	40,10	2.00	4.00	12.00	4.00	22.42	33.62	14880.00
		200,18	200,2	1.33	1.78	6.67	20.00	26.90	20.17	14800.00
		360,18	40,2	3.00	6.00	8.94	8.94	22.71	34.06	15166.80
[0.33,3]	0.10	200,10	200,10	1.00	1.00	13.42	13.42	14.04	14.04	11361.58
		360,10	40,10	1.80	3.24	18.00	6.00	12.61	21.01	10500.00
		200,18	200,2	1.46	2.14	10.00	30.00	15.97	10.93	10220.00
		360,18	40,2	3.00	9.00	13.42	13.42	14.04	14.04	11361.58
	0.20	200,10	200,10	1.00	1.00	8.94	8.94	24.33	24.33	15629.91
		360,10	40,10	2.00	4.00	12.00	4.00	22.42	33.62	14880.00
		200,18	200,2	1.33	1.78	6.67	20.00	26.90	20.17	14800.00
		360,18	40,2	3.00	9.00	8.94	8.94	24.33	24.33	15629.91

The number of subjects per cluster, and , is not rounded for reasons discussed in the text. The number of clusters per arm, and , has to be rounded upward and increased with 2. The budget is based on the thus adjusted numbers of clusters.

In the first 8 rows, the variances are homogeneous and so the MMD reduces to the cc design, as shown by the fact that the budget ratio (see Section 4 or appendix B for the definition of p). If the cluster‐to‐subject cost ratio is homogeneous, so , then the sample size per cluster is homogeneous, that is, (rows 1, 4, 5 and 8). If the cluster cost is homogeneous, so , then the number of clusters is homogeneous, that is, (rows 1, 3, 5 and 7). This only holds under homogeneity of variance, however (see the end of Section 3). Finally, if the costs are completely homogeneous, so and both hold, then the MMD is balanced at each design level, that is, and (rows 1 and 5), in line with Tables 1 and 2 (noting that p = 1).

Rows 9 to 16 show the same cost scenarios but now for heterogeneity of variance with as range for the SD ratio . Again, homogeneity of the cluster‐to‐subject cost ratio gives homogeneity of the sample size per cluster (rows 9, 12, 13, 16), and complete homogeneity of costs gives a balanced design at each level (rows 9 and 13), in line with the case p = 1 in Table 1. Also in line with Table 1, the budget allocation ratio is as long as , and is else (given that p < 1 does not occur in Table 4).

Finally, rows 17 to 24 show the same cost scenarios for heterogeneity of variance with as range for the SD ratio . Homogeneity of the cluster‐to‐subject cost ratio now gives a balanced design at each level (rows 17, 20, 21, 24) because, in these rows, the budget allocation ratio equals the treatment‐to‐control cost ratio, which is 9 at each design level in rows 20 and 24, and 1 at each level in rows 17 and 21.

DISCUSSION

Optimal sample sizes per level (cluster, subject) of a cluster randomized trial have been published, based on the assumptions that costs and variances at each level are (1) homogeneous between treatments, and (2) known in the design stage.8, 9 Liu27 extended this to the case of treatment‐dependent costs, still assuming known and homogeneous variances. This assumption is problematic for 2 reasons. First, a treatment which affects the mean of an outcome variable can also be expected to affect its variance, making the assumption of homogeneity of variance realistic only under the null hypothesis. Secondly, trials are run to test hypotheses about the unknown mean outcome difference between treatments, making the assumption of known variances untenable. Unfortunately, optimal design strongly depends on the 4 variances involved, and misspecification of these variances can lead to an inefficient design.

Similar to Candel and Van Breukelen28 and Wu et al,30, but extending their results to optimal instead of fixed sample sizes per cluster, this paper addressed these problems by deriving a MMD which allows for heterogeneity of variances, and is robust against misspecification of the variances by maximizing the minimum efficiency over a range of plausible values for the unknown variances. The MMD was compared with the balanced design, which is optimal under homogeneity of costs and variances, and with the cc design, which is optimal under heterogeneity of costs and homogeneity of variances. The results can be summarized as follows: First, if the costs are less heterogeneous than the variances (more precisely, if ), then the MMD allocates budget to each treatment arm proportionally to the costs as expressed by the function for that arm, see Equations (8) and 11. If the cluster‐to‐person cost ratio is furthermore the same in both arms, then the MMD is balanced. Secondly, if the costs are more heterogeneous than the variances, then the budget allocation to each arm increases with the costs and variance in that arm. Compared with the MMD, the balanced design is still highly efficient (RE > 0.95) if (at least if is homogeneous). The balanced design is inefficient outside that range, that is, if costs are very heterogeneous. Compared with the MMD, the cc design is never optimal (unless u = 1, ie, homogeneous variances), but always fairly to highly efficient, with a minimum RE of 0.83. Compared with the balanced design, the cc design is less efficient if , but more efficient else. Further, if it is known that and p < 1, or that and p > 1 (ie, treatment arm ordering by variances is opposite to ordering by costs), then the cc design is the MMD. In view of these results, the practical recommendation is to use the MMD (which is balanced in some cases). As the preceding section shows, sample size calculation is simple for the MMD, and the computer code for such calculations in SPSS respectively R is provided as supplementary material.

No study is without limitations, and here we mention a few of the present study. First of all, although the derivations for the maximin and cc designs and their comparison allowed the cluster‐to‐person cost ratio c/s to be treatment‐dependent, the comparison of both with the balanced design was limited to the case of a homogeneous cost ratio. This was done to reduce the relative efficiencies (REs) to functions of 2 parameters only, u and p, and thus to get a better understanding of how these REs depend on the heterogeneity of variance, u, and the heterogeneity of costs, p. In Section 5, we explained how the RE of the balanced design can be obtained without assuming a homogeneous cost ratio.

Another limitation is that this study assumes a quantitative outcome. Wu et al derived the MMD for binary outcomes, but they assumed a fixed and homogeneous sample size per cluster.30 Extending their work to optimal and heterogeneous sample sizes per cluster is thus a potential topic for future work, as is the extension of MMD to multicentre/multisite trials, for which optimal designs under homogeneity of variance and costs are given by Raudenbusch and Liu and by Moerbeek et al,9, 40 and under heterogeneity of variances and costs by Lemme et al.41

Data S1. Supporting information

Click here for additional data file.

Symbol	Interpretation	Introduced in section nr
β1	The treatment effect of interest	2
σu02	Residual variance at the cluster level	2
σε2	Residual variance at the individual level	2
σY2	Total residual variance	2
ρ	Intraclass correlation	2
K	Total number of clusters sampled	2
n	Number of individuals sampled per cluster	2
Superscript * for any design factor	Optimal design value of that factor	2
c	Cost per cluster	2
s	Cost per subject	2
B	Budget for the study	2
g(ρ)	Shortcut for ρc+1−ρs^2	2
Subscript t for any symbol	In the treated group	3
Subscript c for any symbol	In the control group	3
f	Fraction of the study budget spent on the treated arm	3
f /(1‐f)	Budget allocation ratio	4
ρmax	Maximum plausible intraclass correlation	4
u and 1/u	Maximum and minimum for the SD ratio σ_Ytσ_Yc	4
V	Shortcut for σYt2+σYc2	4
Vmax	Maximum plausible V	4
p	Shortcut for g_tρ_maxg_cρ_max	4
Superscript m, bal, cc for f/(1‐f)	Maximin, balanced, cost‐considered	4,5