Technical note: Optimal allocation of limited proton therapy resources using model-based patient selection.

PURPOSE: We consider the following scenario: A radiotherapy clinic has a limited number of proton therapy slots available each day to treat cancer patients of a given tumor site. The clinic's goal is to minimize the expected number of complications in the cohort of all patients of that tumor site treated at the clinic, and thereby maximize the benefit of its limited proton resources.
METHODS: To address this problem, we extend the normal tissue complication probability (NTCP) model-based approach to proton therapy patient selection to the situation of limited resources at a given institution. We assume that, on each day, a newly diagnosed patient is scheduled for treatment at the clinic with some probability and with some benefit Δ N T C P $\Delta NTCP$ from protons over photons, which is drawn from a probability distribution. When a new patient is scheduled for treatment, a decision for protons or photons must be made, and a patient may wait only for a limited amount of time for a proton slot becoming available. The goal is to determine the Δ N T C P $\Delta NTCP$ thresholds for selecting a patient for proton therapy, which optimally balance the competing goals of making use of all available slots while not blocking slots with patients with low benefit. This problem can be formulated as a Markov decision process (MDP) and the optimal thresholds can be determined via a value-policy iteration method.
RESULTS: The optimal Δ N T C P $\Delta NTCP$ thresholds depend on the number of available proton slots, the average number of patients under treatment, and the distribution of Δ N T C P $\Delta NTCP$ values. In addition, the optimal thresholds depend on the current utilization of the facility. For example, if one proton slot is available and a second frees up shortly, the optimal Δ N T C P $\Delta NTCP$ threshold is lower compared to a situation where all but one slot remain blocked for longer.
CONCLUSIONS: MDP methodology can be used to augment current NTCP model-based patient selection methods to the situation that, on any given day, the number of proton slots is limited. The optimal Δ N T C P $\Delta NTCP$ threshold then depends on the current utilization of the proton facility. Although, the optimal policy yields only a small nominal benefit over a constant threshold, it is more robust against variations in patient load.

INTRODUCTION

Proton therapy is widely considered a superior form of radiotherapy compared to photons due to the favorable depth–dose curve of proton beams. As a rule of thumb, protons allow for reducing the integral dose to normal tissues surrounding the tumor by a factor of 2–3. , Although the number of proton therapy centers is increasing worldwide,1 it remains a limited resource due to cost and size of proton facilities and not all patients who may benefit from proton therapy have access to it. Consequently, concepts are being developed to optimally select patients for proton therapy to maximize its benefit for the health care system as a whole.

One approach for that is model‐based patient selection, which has been promoted in the Netherlands and has been implemented clinically for several disease sites. , Following this approach, comparative treatment planning for protons and photons is performed in combination with normal tissue complication probability (NTCP) modeling. Patients are selected for proton therapy if the NTCP reduction of protons over photons exceeds a threshold. As an example, head and neck squamous cell carcinoma (HNSCC) is a treatment site that lends itself to the NTCP model–based patients selection because (1) the incidence of HNSCC is too high to treat all patients with protons; (2) many patients suffer from xerostomia and dysphagia, , leaving room for clinically significant NTCP reductions through protons; (3) patients benefit to varying degrees from protons due to the relative location of tumor and relevant organs at risk (OAR). , , Below, we use HNSCC as an example, but we would like to stress that none of the methodology presented is specific to HNSCC and would equally apply to other sites.

In this note, we further investigate the problem of proton patient selection in the context of limited availability of proton therapy slots. The Netherlands use a nation‐wide proton patient selection scheme with a fixed NTCP reduction threshold, and it is assumed that every patient above the threshold can receive proton therapy timely. In this note, we instead consider proton patient selection in a single institution, for example, a photon radiotherapy department with an integrated single‐room proton facility. Such an institution may, on any given day, only have a limited number of proton slots available to treat HNSCC patients, which is smaller than the number of HNSCC patients under treatment. We assume that the goal of the institution is to maximize the benefit of its proton therapy resources, that is, to minimize the expected total number of complications in the population of HNSCC patients treated at the department. In this context, proton patient selection faces the following trade‐off: A high threshold may reject too many patients and lead to proton slots being unused. A low threshold may admit patients with mediocre benefit, who block proton slots for patients who would have a larger benefit.

In this note, we present a method to determine the optimal proton patient selection threshold, depending on the number of currently available protons slots and the number of remaining fractions for proton patients currently under treatment.

METHODS

NTCP modeling

The approach discussed in this work is not limited to any particular treatment site or side effect. It only assumes a given probability distribution over for the patient population, that is, the NTCP reduction that protons yield over photons. For our motivating application, HNSCC, treatment planning studies have evaluated the expected dose reduction from proton treatments. Additionally, several dose–response models have been proposed for the two most common side effects, xerostomia and dysphagia, based on clinical data. , Based on these studies, we model the distribution of the NTCP reduction by a Gaussian distribution with a mean NTCP reduction of 10% and a standard deviation of 5%. This is also consistent with the initial experience in the Dutch proton patient selection system. Thereby, the model contains the possibility that the photon plan is superior in rare cases.

Model of proton facility operation

We consider the operation of a department over discrete time steps corresponding to one working day, ignoring days when no treatments are performed. We assume that, on any given day, there is a probability q that a new HNSCC patient is scheduled for treatment at the department and a decision is made whether the patient is assigned to protons or photons. On that day, the patient has completed the standard diagnostic process, and comparative proton–photon treatment planning has been performed. The parameter q represents the patient load. As an example, we consider in this work, which would correspond to a clinic treating approximately 100 HNSCC patients per year, and thus, two new patients starting treatment each week on average. We further assume a 30‐fraction treatment over 6 weeks, meaning that on average 12 patients are under treatment on any day. The facility is assumed to have N proton slots available for HNSCC patients, where we assume N to be smaller than the number of patients under treatment.

In reality, comparative treatment planning for protons and photons would be performed for each new patient to obtain NTCP values and for photon and proton plans, respectively. To simulate this process, we randomly draw a value for the benefit from a Gaussian distribution, as outlined above.

When a patient is assigned to proton therapy, the patient blocks one proton slot for the following 30 workdays. We assume that each new patient has to start treatment shortly after they are assigned to a treatment slot, that is, a patient can wait only for up to T days for a proton slot to become available. The value of T is a clinical decision to be made by the department to limit the adverse effects of treatment delays. For this work we assume , meaning that a patient must be assigned to photons if all proton slots are blocked for the next 2 weeks.

Determining the optimal NTCP thresholds

We now present the methodology to determine the optimal patient selection thresholds that minimize the expected number of complications. To that end, the problem of proton patient selection is formulated as an (infinite horizon) Markov decision process (MDP). MDP is a mathematical framework that enables optimal sequential decision making under uncertainty in situations where the goal is to maximize expected (long‐term average) reward in a system whose state evolves partly at random and partly as a function of the sequential decisions. The optimal policy (course of action) of the system is state dependent. The details of the MDP formulation of the problem of proton patient selection are as follows:

State.

The state of the MDP represents the state of the proton facility, which can be described by one integer per slot indicating for how many additional days the slot is blocked. (The largest number is, thus, the length of the treatment plus the maximum wait time T.) For the example, if , then the state (5, 14, 27) would indicate that currently all 3 slots are occupied, and the proton patients under treatment have to complete another 5, 14, and 27 fractions, respectively.

Actions and policy.

An action amounts to the decision whether or not to assign a new patient to proton therapy. A policy is given by a threshold for every facility state; if a new patient's is above the threshold corresponding to the state of the proton facility at the time the patient presents, then the patient is assigned a proton slot. Only those states have an assigned threshold in which one of the slots are occupied for less than T remaining days.

State transitions.

Every state has up to two possible successor states, depending on whether the state of the proton facility allows a new patient to be assigned a proton slot. For the example state (5, 14, 27) there are two possible state transitions. If on that day no new patient presents or the patient is assigned to photons, the state transitions to (4, 13, 26). If a new patient presents, it is possible to assign the patient to the slot that becomes available in 4 days, and in that case the state transitions to (34, 13, 26). State transitions are stochastic because the sequence of incoming patients is. The probability for the latter transition is given by the probability of seeing a new patient q multiplied with the probability that the patient's exceeds the selection threshold.

Reward.

Our goal is to minimize the expected number of complications over time, or equivalently, to maximize the (long‐term) average realized by the patients. We may consider the situation that all patients are treated with photons as the baseline; if a patient is assigned to protons, an immediate benefit of is realized. The optimal policy is the collection of state‐dependent thresholds that maximizes the average (per patient) realized  benefit.

Computing the optimal policy.

The optimal policy of an MDP is characterized by the Bellman equation. A general algorithm to compute the optimal policy of an MDP is the iterated value and policy iteration, which can be briefly summarized as follows : Start with any policy, for example, with every threshold being 0 (which means assigning every patient to protons if a slot is available), and then iterate the following two steps until no change is observed in the policy: It can be shown that this iterative process converges to the optimal policy. We remark that the optimal policy can also be computed using linear programming, although that approach was not used in our study.

Policy evaluation. Compute the long‐term expected reward for the current policy and the value of each state s, defined as the expected reward given that the system starts at state s. This can be accomplished by solving a system of linear equations, as follows. Suppose that S is the number of states, that is the vector whose ith component is the immediate reward realized by the given policy in state i, and that is the state transition matrix whose th component is the probability that after the action determined by the current policy the system's next state will be j assuming that its current state is i. Finally, let be a positive discount factor. Then the vector of values of each state is the (unique) solution of the linear system In our computation, we used , as the benefit realized by a patient arriving later is considered to be the same as for earlier patients, but is necessary for the convergence of the algorithm.

Value iteration. For each state s, compute the optimal policy assuming that the long‐term rewards starting from each state are the values computed in the previous step. In our problem, this is a straightforward optimization problem with a closed‐form solution: For every state where a decision is to be made (there is room in the proton facility for a new patient), assign the new patient to a proton slot if and only if their is at least as large as the difference between the values of the states and the proton facility after assigning the patient to protons or photons, respectively. In other words, the optimal threshold is the difference between the values of the two possible subsequent states.

Comparison to a constant threshold

The optimal policy is compared to a simpler patient selection scheme using a constant threshold that is independent of current facility utilization but optimal for the assumed values of N, q, and T and the assumed distribution over in the patient population. The (approximately) optimal constant threshold was computed by simulating the operation of the clinic over an extended period of time for constant thresholds between 0% and 30% in 1% increments; see Figure 3.

FIGURE 3

Top panel: observed benefit using a constant threshold independent of the current utilization of the facility, with the number of slots and maximum delay days fixed. Bottom panel: the fraction of patients assigned to proton therapy. A threshold of 0% corresponds to the first‐come‐first‐served policy. The optimal constant threshold is 11%

RESULTS

We computed the optimal policies assuming that a radiotherapy facility has up to three concurrent proton therapy slots for HNSCC patients, who receive 30‐fraction treatments that begin after a delay of no more than 10 days in order to wait for an available proton therapy slot. We also investigated the effect of delayed treatments, and the robustness of policies against uncertain patient arrival rates.

Calibrating the expected benefit.

The overall benefit of the patient population is measured by the long‐term average benefit, that is, the expected value of NTCP reduction, which is equivalent to measuring the average number of prevented side effects. Based on our model, the expected benefit if everyone receives proton treatment is 10%. By definition, the baseline policy of everyone receiving photon treatment corresponds to a benefit of 0. Since, with two new patients presenting each week, we have 12 patients concurrently receiving treatment on average, but only slots, a first‐come‐first‐served assignment of patients (where an available proton slot is assigned to the next arriving patient) can only yield a benefit of around , that is, around 2.5% for . At the other extreme, if we could treat the quarter of the patients with the highest benefit (corresponding to a threshold of 13.4%), the average for the proton patients would be 16.4%, increasing the overall benefit of protons for the population to 4.1%. However, when the number of proton patients is constrained to on any given day, the 4.1% represents an unachievable upper bound and the benefit of the optimal proton patient selection scheme will lie between 2.5% and 4.1%.

The structure of the optimal policy

The optimal threshold depends on the current utilization of the proton facility. For slots, the optimal threshold varied greatly, from 0.074 in the state when all proton slots are immediately available, to 0.16 in the state when one slot will be available in days and the others have just been taken. Figure 1 shows a two‐dimensional cross section of this policy for the states when one proton slot is immediately available. Unsurprisingly, the threshold is monotone in the availability: When a second slot is available sooner, then admitting a patient with less benefit carries a lower risk of blocking all slots from a patient with greater benefit.

FIGURE 1

A cross‐section of the optimal policy for 3 proton slots assuming that one of them is immediately available. The numbers on the two axes represent the number of days until the other two slots become available, the colors indicate the optimal threshold that a patient needs to exceed to be assigned a proton slot

The long‐term benefit of this policy is ; closing 67% of the gap between the first‐come‐first‐served policy and the unattainable outcome of filling all treatment slots with patients of the highest benefit. Simulating the facility operation using this policy provides further insight into the utilization of proton slots. Recall that with treatment slots and 30‐day treatments the best one can hope for is that 25% of the patients may receive proton therapy. The policy in Figure 1 assigns 24% of the patients to proton therapy and thus uses most of the proton slots. Approximately 5% of all proton slots remain unused and approximately 14% of the treatment days leave at least one proton slot empty. The blue histogram in Figure 2 shows the distribution of values for the patients treated with protons.

FIGURE 2

The distribution of the of patients assigned to proton therapy by the optimal policy (blue histogram) and the best constant‐threshold policy, with threshold of 11% (yellow histogram). Also shown: the distribution of the patient population (solid curve). The constant‐threshold policy favors patients with right above the threshold

Comparison to simpler policies with constant threshold

Figure 3 investigates the simplified strategy of using a constant threshold independent of the current utilization of the facility, for the nominal parameter values , , and . A threshold of 0% corresponds to random patient selection, that is, giving an available proton slot to the next patient who benefits from proton treatment. This yields an average NTCP reduction of 2.5%, reflecting that 25% of patients are treated with protons. The optimal constant threshold of 11% yields an average NTCP reduction of 3.4% (compared to 3.57% for the optimal facility state–dependent thresholds). A constant threshold higher than the optimal leads to many unused proton slots (Figure 3, bottom panel), which rapidly reduces the benefit below even that of the first‐come‐first‐served policy. The lower average NTCP reduction of the constant threshold policy can be explained by two factors. At the optimal constant threshold of 11%, 23% of patients are treated with protons (compared to 24% for the facility state–dependent policy). In addition, the facility state–dependent policy results in a distribution of realized reductions with higher mean (Figure 2).

Dependence on parameters

The optimal policy depends on various parameters. Naturally, the thresholds scale with parameters of the probability distribution over . Doubling both mean and standard deviation would simply double the thresholds and the benefit. Furthermore, increasing N or decreasing q would increase the percentage of patients that can be treated with protons, and thus corresponds to lower thresholds and higher average benefit from protons.

The benefit of treatment delays

We also investigated how the utilization of proton slots and the overall benefit is impacted by the maximum allowed delay T. We fixed the number of proton slots at and considered a maximum delay T ranging from 0 to 10 days. Figure 4 shows the dependence of the benefit on the maximum delay T until treatment must begin. In our example it is clear that, as long as the wait does not compromise the treatment outcome, allowing longer delays is beneficial. This is explained by the increasing utilization of the proton facility allowed by the delayed treatment (Figure 5). When no treatment delay is allowed (), approximately 20% of all proton slots remain vacant, which is reduced to about 5% when a delay of 2 weeks () is allowed (blue bars in Figure 5). This is because patients can be assigned a proton slot even if they present when all slots are taken. An interesting feature of the optimal policy is that, even with a relatively long maximum delay of 10 days, most patients have very little wait; see Figure 6.

FIGURE 4

The dependence of the average benefit of optimal proton slot allocation on the maximum allowed delay T until treatment must begin

FIGURE 5

Availability of proton slots as a function the maximum delay, with the number of slots fixed. Yellow bars (left): the fraction of days with an immediately available proton slot. Blue bars (right): the average fraction of proton slots available

FIGURE 6

The frequency of patients who received proton treatment delayed by t days (). Although the maximum allowed delay is 10 days, a large number of patients can be assigned a proton slot immediately

Robustness of patient selection policies

It is also instructive to compare the performances of the optimal state‐dependent threshold policy and the optimal fixed‐threshold policy that are computed assuming a fixed known patient load q under the assumption of different patient loads than the estimated one. This is indicative of the policies' performance under incorrectly specified or time‐varying patient load. Table 1 shows the computed long‐term expected benefit () of both policies assuming different combinations of the assumed patient loads used to compute the optimal policies and the true arrival rate (corresponding to 1, 2, 3 new patients per week).

TABLE 1

Investigating the robustness of the benefit of the state‐dependent slot allocation policy versus the constant‐threshold policy against unknown or varying patient load. Off‐diagonal entries represent scenarios when the assumed patient load q over‐ or underestimates the true patient load . Note that, for constant , higher patient load means that a lower percentage of patients is treated with protons, leading to a reduced average benefit

	State‐dependent threshold			Constant threshold
	q^∗=0.2	q^∗=0.4	q^∗=0.6	q^∗=0.2	q^∗=0.4	q^∗=0.6
q=0.2	5.72%			5.62%
q=0.4	5.56%	3.57%	2.56%	5.07%	3.40%	2.42%
q=0.6	5.19%		2.60%	3.99%		2.49%

The table reveals that the reduction in the expected benefit that results from a misspecified q is considerably smaller for the state‐dependent threshold policy than it is for the fixed‐threshold policy, particularly when the true patient load is overestimated. This is explained by the fact that the state‐dependent threshold adapts to the lack of new patient arrivals by lowering the threshold of admittance to the proton facility, whereas the fixed‐threshold strategy is too rigid and results in a greater underutilization of the proton facility. Figure 3 also explains why overestimating is worse than underestimating it: The benefit rapidly declines to zero if the threshold is higher than necessary.

DISCUSSION

With the limited availability of proton therapy, efforts should be made to optimally select proton patients to maximize the overall benefit of the existing facilities. NTCP model–based patient selection represents one approach to this problem. , Currently, the approach is implemented with a fixed threshold in countries with a nation‐wide referral system, where one may assume that a proton slot can be made available to an eligible patient in a timely manner. In this note, we instead consider the situation that the number of proton slots on any given day is limited. This may, for example, be the case for an individual clinic with a single room proton machine. The optimal threshold then depends on the number of available proton slots and their current utilization.

We compared patient selection for the optimal, facility state–dependent threshold to the optimal constant threshold, that is, the threshold that is optimal for the number of available slots, the assumed probability distribution over in the patient population, and the average number of patients under treatment, but independent of the current slot utilization. It was found that the additional benefit of facility state–dependent thresholds was quite modest if both policies are evaluated for parameters they were optimized for. Nevertheless, there are arguments in favor of a state‐dependent policy. (1) Using a state‐dependent policy does neither cause any technical difficulties for implementation, nor does it increase the workload for the clinic compared to a constant threshold as the optimal policy needs to be computed only once. (2) A state‐dependent policy is in line with the intuition that one should lower the threshold of the proton facility if underutilized and increase the threshold if the patient load is high. (3) The latter aspect makes the patient selection more robust against variations in the patient load.

Calculating the optimal thresholds (both state‐dependent or constant) depends on parameter values and model assumptions that may not fully represent the complexity of clinical reality. In this work, we assumed that a fixed number of proton slots N is available each day for treatments of a given disease site such as HNSCC. In reality, this number would likely fluctuate because the number of patients of other tumor sites routinely referred to proton therapy will fluctuate. This situation could be modeled via extensions of the MDP model. For example, one could assume that patients with certain tumor sites are guaranteed a proton slot, and that each day there is some probability that such a patient presents. Then, the next available proton slot would have to be assigned to this patient, temporarily reducing the number of proton slots available to HNSCC.

In this note, we assumed that the benefit of protons is described by a single value. In reality, different toxicities of varying severity may be relevant. The patient selection method presented in this note may then be applied with representing a weighted sum of values for different side effects. Although the relative importance of distinct side effects may be debatable, this difficulty is not specific to the method presented here but is inherent to any NTCP model–based selection scheme. For example, in the Dutch scheme, grade ≥2 xerostomia and grade ≥2 dysphagia are treated as equally important, corresponding to equal weights for both toxicities. A reduction of 5% in tube feeding dependence is considered as worthwhile as a 10% reduction in grade ≥2 toxicity, which could be represented by double the weight in the sum of  values.

Another approach to make optimal use of limited proton slots is combined proton–photon radiotherapy with optimal slot assignment. In such treatments, for each patient, some fractions are delivered with protons and others with photons. Loizeau et al. suggested reassigning proton therapy slots on a daily basis by calculating the incremental NTCP reduction from one additional proton fraction for all patients currently under treatment. Every day, the proton slots are then assigned to those patients who benefit the most from one additional proton fraction. Thereby, all proton slots can be used in a near‐optimal manner.

CONCLUSION

In the situation that, on any given day, the number of proton slots is limited, the optimal threshold for proton patient selection depends on the current utilization of the proton facility. Such a facility state–dependent policy yields a modest improvement in the average improvement from protons compared to a simplified policy using a constant threshold when evaluated for the expected patient load. However, the facility state–dependent policy is more robust against variations in patient load.

CONFLICTS OF INTEREST

The authors have no conflicts to disclose.