Quantification of the uncertainties within the radiotherapy dosimetry chain and their impact on tumour control.

BACKGROUND AND
PURPOSE: Dose delivered during radiotherapy has uncertainty arising from a number of sources including machine calibration, treatment planning and delivery and can impact outcomes. Any systematic uncertainties will impact all patients and can continue for extended periods. The impact on tumour control probability (TCP) of the uncertainties within the radiotherapy calibration process has been assessed.
MATERIALS AND METHODS: The linear-quadratic model was used to simulate the TCP from two prostate cancer and a head and neck (H&N) clinical trial. The uncertainty was separated into four components; 1) initial calibration, 2) systematic shift due to output drift, 3) drift during treatment and 4) daily fluctuations. Simulations were performed for each clinical case to model the variation in TCP present at the end of treatment arising from the different components.
RESULTS: Overall uncertainty in delivered dose was +/-2.1% (95% confidence interval (CI)), consisting of uncertainty standard deviations of 0.7% in initial calibration, 0.8% due to subsequent calibration shift due to output drift, 0.1% due to drift during treatment, and 0.2% from daily variations. The overall uncertainty of TCP (95% CI) for a population of patients treated on different machines was +/-3%, +/-5%, and +/-3% for simulations based on the two prostate trials and H&N trial respectively.
CONCLUSION: The greatest variation in delivered target volume dose arose from calibration shift due to output drift. Careful monitoring of beam output following initial calibration remains vital and may have a significant impact on clinical outcomes.

Introduction

Successful radiotherapy outcomes rely upon accurate delivery of dose to the tumour whilst sparing the surrounding organs at risk. Furthermore, the delivered dose should be traceable to a primary standards laboratory (PSL). Uncertainties arise from each stage of the process including machine calibration, treatment planning and patient setup for treatment. Previous work by Bentzen et al. [1] modelled the clinical impact of variations in dosimetry based on thermoluminescent dosimeter (TLD) postal audits conducted across Europe. However, this work only looked at the overall uncertainty of the dosimetry measurement and not the individual contributing factors. An assessment of variation in delivered dose due to initial calibration has been previously published [2] based on analysis of 20 years of UK data from dosimetry audits carried out by the UK PSL. These audits involved on-site visits and followed the code of practice as implemented within the host centre using a secondary standard ionisation chamber [3]. The variation in beam output between machines due to this initial calibration was found to be normally distributed with a standard deviation (SD) of 0.7% with a 95% CI of +/− 1.4% and was based on 81 measurements taken during 47 on-site audit visits [2].

The individual stages of dosimetry calibration can be broken down into individual components of uncertainty. These inherent uncertainties in the delivered dose, which arise from each stage of the calibration chain from PSL to clinic, have not been previously quantified. Each transfer of calibration from one instrument to another contributes to the overall uncertainty on delivered dose. In the present work, results from ionisation chamber measurements performed during on-site audits, which have a lower uncertainty [2], [4] and are better able to estimate the uncertainty in the transfer steps of the absolute dose calibration from the PSL to the clinic [5], [6], have been collated for analysis.

The deviation in delivered dose which occurs directly due to variations in beam output within the clinic was subdivided into four components; 1) Uncertainty arising from the initial calibration of the treatment machine, 2) subsequent systematic shift in calibration at the start of a patient’s treatment course due to drift in beam output, 3) the further drift in beam output during a patient’s treatment, and 4) the random day-to-day fluctuations in beam output.

Tumour control probability (TCP) is a metric commonly used to assess treatment outcomes following radiotherapy [7], [8]. Most commonly, a linear quadratic (LQ) model is used to simulate the variation of TCP with delivered dose [9], [10]. The models in this work were developed based on clinical trial data from two different prostate trials which were selected to highlight the range of dose–response within the same treatment site, but with different stages of disease and a head and neck trial. The individual components of uncertainty have then been used to simulate the TCP response for a population of patients form the TCP models derived from the clinical trial results. Separating the individual components of uncertainty and modelling for different clinical scenarios allowed a more detailed investigation into the potential impact of each aspect of the uncertainty. Hence, the study aimed to quantify, for the first time, the variation in delivered dose due to the four individual components of uncertainty within the dosimetry chain as well as the clinical consequences of each by simulation of TCP using the LQ model for each of the three trials.

Materials and methods

Simulations of TCP response to dose using known uncertainties [2] and data from published clinical trial outcomes were performed using the LQ model for the different subcomponents of uncertainty in dose delivery. The data and input parameters used within the TCP model were derived from a range of dose–response values based on different clinical trials; RT01 [11] and Fox Chase [12] trials for different stages of prostate cancer, and the PARSPORT trial [13] to simulate head and neck treatment response to dose. The use of TCP modelling as a tool for assessing clinical impact of dose variations was appropriate for this work as the magnitude of dose variations is of a similar order to previously published work [1]. Ebert [14] also noted that the sensitivity of TCP modelling made it a reliable tool for indicating potentially successful and unsuccessful irradiation strategies, and its sensitivity to its parameters can in some cases be an advantage.

Simulation of treatment outcome for combined uncertainties

The LQ model [7], [8] was used to simulate the change in treatment outcome due to a change in dose, as identified in this work. The TCP measures were implemented using the LQ model as follows [9], [10], [15]where TCP is based on the particular measure of outcome relating to treatment success, N0 is often considered the initial number of clonogens and was held fixed for the different cases explored. The value of di is the dose delivered during fraction i of n.

Outcomes from the RT01 [11], Fox Chase [12], and PARSPORT [13] trials were used for optimisation of the model parameters for each case. Table 1 summarised the values used in the simulations for each clinical case and their origin is described below.

Table 1

Parameters used for modelling prostate and head and neck patients within this work using the LQ TCP model. Based on parameters published within the literature the values of the standard deviation of the α/β value were determined through optimisation using the model. The α/β and N0 parameter values used for each set of simulations are based on the range of values used within the literature [9], [20], [21], [23] and were optimised to fit the clinical data. The TCP measures are those reported from the corresponding clinical trial. *Determined through model calibration to the corresponding trial data. †The dose of interest was the point on the TCP curve at which the variation in TCP has been assessed unless otherwise specified. This dose represents the typical prescribed treatment dose. bPFS is the biological progression free survival, bNED is the biological non-evidence of disease. + The doses given were the median dose for the groups within the trial. The ranges were less than 71.5GY, 71.5–75.75 Gy and <75.75GY for the 70 Gy, 72 Gy and 76 Gy median doses used in this work respectively. #The 2 year survival is taken from the IMRT arm is this is the current standard of care (the 3D conformal arm reported very similar results of 76% and was within the CI of the IRMT arm).

Parameter	Parameter values
	Prostate(RT01)	Prostate(Fox Chase – Med)	Head and Neck(PARSPORT)
α/β	2 Gy	2 Gy	10 Gy
α/β SD (%)*[absolute value of αSD]	100%[0.06]	5%[0.003]	25%[0.08]
β (held fixed)	0.04 Gy−2	0.04 Gy−2	0.02 Gy−2
Dose/fraction (d)(Held fixed for optimisation)	2 Gy	2 Gy	2.17 Gy
TCP results used for optimisation	Ten year bPFS43% @ 64 Gy55% @ 74 Gy	8 year bNED±19% @ 70 Gy31% @ 72 Gy84% @ 76 Gy	2 year survival#78% @ 65 Gy
Dose of interest†	74 Gy	76 Gy	65 Gy
N0	1 × 104	5 × 104	3 × 105

For each case assessed, the specific values of N0 and the α/β variation were optimised using a linear least-squares regression to fit the published clinical outcome measures [16], [17]. A population of patients was modelled as the combined outcome of a simulated patient population with differing values for the α/β ratios which is a common method of simulating a population of patients [10].

An α/β value of 2 Gy was used when simulating prostate cancer [9], [18], [19] and 10 Gy when simulating head and neck cancer [9], [19]. The value of β was held fixed at 0.04 Gy−2 for the prostate cases and 0.02 Gy−2 for the head and neck case [9], [20], [21]. A summary of the model parameters is included in Table 1 for each case.

The models were implemented using the python programming language to allow random and systematic shifts in delivered dose per fraction to be investigated independently on a fraction by fraction basis and is available online [22].

Due to the statistical nature of the simulated results a population of size n = 1000 was used to ensure consistency and reduce stochastic variations to an acceptable level. The values of N0, β and α/β are representative of values previously published by others [17], [23]. Use of the n = 1000 reduced the standard error of the mean (SEOM) of the simulated population TCP to less than 0.1% in all cases investigated within this work, which was deemed acceptable as the changes investigated are an order of magnitude greater. Variation of additional variables such as tumour size, and N0 are not included in this work.

Using the model parameters summarised in Table 1 simulations were performed for n = 1000 patients for each of the three clinical trial cases. The TCP variation at the dose of interest for each clinical trial case was calculated and the standard deviation was reported to quantify the variation. The simulations kept all parameters fixed and varied the daily dose per fraction throughout the entire treatment course. The variation in fractional dose was randomly drawn from a normal distribution with mean and standard deviation of the parameter being assessed.

The simulations were performed to model the following four components of uncertainty; 1) Variation in dose due to initial calibration. 2) Systematic shift due to output drift. 3) Drift during treatment. 4) Daily fluctuations. The combination of these uncertainties was then used to simulate the overall impact of treating a patient on a randomly selected treatment machine.

Fixed systematic offset due to the initial calibration

Previous work quantifying the variation in dose due to initial calibration [2] was used as a measure of the potential fixed systematic offsets due to initial beam calibration, based on 81 measurements taken by NPL during 47 on-site audit visits. This value was normally distributed with a standard deviation (SD) of 0.7% with a 95% CI of +/− 1.4% [2].

Systematic dose offset at start of treatment due to drift in beam output.

Following initial calibration the beam output will also drift over time [24], [25], [26], [27] leading to an additional shift in dose at the start of the treatment course from that at the initial calibration. Data used by Bolt et al. [24] based on 6MV beam data from 204 machines from 52 UK centres between Jan 2015 and June 2015 was used. Measurement results outside of +/−5% were excluded as these were always linked to repeat measurements indicating invalid results. Weekends and any days with less than 50 results were also excluded.

The SD of the measured beam output across all machines on each treatment day was calculated and the mean of these daily SDs was determined. This calculated mean SD was used to model the additional variation in the dose received due to treatment start date arising due to the drift in beam output over time.

The normality of the data was tested using the Shapiro-Wilk test. The uncertainty from initial calibration and subsequent change in calibration due to output drift were combined in quadrature to give the overall variation in delivered dose on the treatment start date.

Variation in delivered dose due to output drift during course of treatment

Machine beam output is continuously changing over time due to the drift and so each delivered fraction will differ from the first (assuming a non-zero drift rate). For this work a typical treatment course was considered to last six weeks. The drift rate varies for each machine and may depend upon age and model.

From the dataset of 204 machines, the output data for each machine was corrected for any step changes due to recalibration by applying the magnitude of the step change to all subsequent data for that beam. A linear least squares regression was then performed and the gradient (change in beam output per day) was extracted for each machine. Taking the mean and SD of all the machines drift rates then determined the distribution of the range of drift rates.

The number of calibrations performed on each machine was determined from the data provided by identification of distinct step changes or through additional information supplied from the original centre.

The impact of this output drift on mean delivered dose for a population of patients was considered over a typical treatment. A patient was randomly assigned to a machine with a rate of drift drawn from a normal distribution with mean and standard deviation as determined from the measurement data. A six-week treatment was then simulated using this data to determine the mean dose delivered over the treatment course. The simulation was performed taking the initial dose to be that prescribed with daily dose incrementing by the daily drift percentage and assuming linear drift over the treatment duration.

Random dose variations due to daily fluctuation

The data from Bolt et al. [24], which included routine beam output data from 204 treatment machines, was analysed to determine the typical daily fluctuation in output measurement results. From the linear regression of each machines corrected output measurements the variation in measurements was quantified by calculating the SD of the residuals. This was used to model the magnitude of daily fluctuations in beam output.

To simulate the variation in TCP induced due to daily fluctuations in dose the mean dose per fraction was fixed at the prescription value and the dose for each fraction extracted randomly from a normal distribution with mean shift of 0% and the SD as determined from the data.

Impact of machine scheduling on treatment outcome

The impact of treatment on a particular machine was determined by taking a single patient with fixed radiobiological parameters (with nominal values as summarised in Table 1) through simulation of treatment 1000 times on different machines. Each simulated treatment had variations in delivered dose modelled to include the four components described, which were combined in quadrature. From this distribution the daily doses were randomly extracted. The initial systematic shift in dose was held constant throughout treatment (i.e. no machine calibration was included during the treatment course) and subsequent treatments included the combined uncertainty of the drift and daily dose variations. The simulated TCP at the dose of interest for each study was determined for each simulated patient and the variation in TCP across this population calculated for each clinical case study.

Results

Impact of systematic dose variation

The mean of all daily SDs was 0.8% and was seen to be normally distributed. The SD of measured beam output for each day is plotted in Fig. 1. Combining this daily uncertainty (which intrinsically includes drift and daily fluctuations) with the uncertainty of 0.7% from initial calibration resulted in a SD of 1.1% and 95% CI of 2.1%. The variation in simulated TCP for a range of systematic dose offsets between −3% to +3% for each clinical case are plotted in Fig. 2. The dose variation 95% CI of 2.1% gave rise to changes in TCP of up to 11%.

Fig. 1

Standard deviation of output measurements on each day on which greater than 50 measurements were obtained. The dotted line shows the mean value of the standard deviations which is 0.8%.

Fig. 2

Variation in simulated TCP measures due to a systematic shift in dose through treatment. The shaded region indicates the 95% CI for the overall uncertainty of 2.1% which includes initial calibration offset, systematic variation due to output drift and daily fluctuations.

Drift in beam output during treatment

The mean drift in beam output was +0.9% per annum with a SD of 2.3% (95% CI −2.2% to +5.1%). The output drifts met the criteria for normality. This is equivalent to a mean drift of 0.1% (95% CI −0.3% to +0.6%) over a 6-week treatment course. Of the 204 machines included in this study only 42 (20%) had at least one beam output adjustment performed during the 6-month data period. Of these, 35 machines had a single calibration and seven had two calibrations. The likelihood of calibration during a treatment course was only 5%. Therefore, a calibration event was not explicitly included within the simulations in this work.

The variation in simulated TCP results is plotted in Fig. 3 noting the asymmetric 95% CI for clinical situations as beam output tended to trend upwards as reported in previous studies [24], [25], [26], [27]. A 5% annual drift, equivalent to 0.6% drift over the 6-week course of a treatment, resulted in a change in TCP of 0.1%, 0.2%, and 0.1% for the RT01, Fox Chase and PARSPORT cases, respectively.

Fig. 3

Plot showing the effect of beam output drift on predicted population TCP. A linear fit is included for each to indicate the general trend. The shaded region indicates the 95% CI of drifts measured in this work.

Daily fluctuations

Measurement uncertainty (after correction for the linear drift) was 0.2% (1 SD), 95% CI ± 0.4% around the mean beam output and was normally distributed. The simulations including only this variation in daily dose introduced a SD of 0.2%, 0.3% and, 0.1% in TCP for RT01, Fox Chase and PARSPORT cases respectively after the entire treatment delivery.

Overall dosimetric uncertainty and impact on machine scheduling

Combining the above uncertainties in quadrature gave an overall uncertainty in the delivered dose introduced within the clinical calibration chain of 1.1% (1 SD), resulting in a 95% CI of ± 2.1%. A graphical representation of the spread of TCP due to the scheduling of a patient onto different machines is given in Fig. 4. The Fox Chase case, which had the steepest dose response curve, had a 95% CI of 71% to 81% (range +/−5%). The RT01 prostate case and PARSPORT head and neck case had a 95% CI of 53% and 59% (range +/−3%), and 75% and 81% (range +/−3%) respectively.

Fig. 4

Modelled TCP values arising for prostate and head and neck cases considered for a patient population. Solid lines indicate the mean value and the shaded region the 95% CI. The nominal dose is the dose delivered if there are no errors in dosimetry through the calibration chain. The uncertainty modelled is due only to the machine to which the patient is assigned with no biological variations between the nominal patient considered. Modelled machines have a mean output of 0% with a SD of 1.1% arising from calibration and subsequent output variations and are normally distributed. The arrows indicate the spread of TCP at the prescribed dose for each case.

Discussion

The overall uncertainty in delivered dose arising from the clinical calibration chain determined in this work was +/−2.1% (95%CI). This may result in a difference in delivered dose of up to 3–4% between patients treated on different machines within the same centre arising from the dosimetric calibration variations alone. This may be considered a maximum difference between machines when all aspects combine and may occur in up to 5% of cases. While the precise numerical predictions given by the model may not be the ‘true’ values, it is expected they are of the same order of magnitude.

A previous study by Bentzen et al. [1] based on European TLD audits reported a combined uncertainty of 2% (1 SD) for the audit measurement. Here the uncertainty arising from the initial calibration was measured as 0.7% (1 SD). This reduction in uncertainty is due to the use of on-site visits using ionisation chamber measurements and is likely a more accurate assessment of achievable accuracy. The trend in measurements over a number of decades indicates the uncertainty may now be as low as 0.4% [2]. However, this potential improvement has not been included in this work as there is not yet enough data across a large number of treatment machines to verify this. If this improvement holds, the overall uncertainty (95% CI) would reduce from 2.1% to 1.8%.

The additional uncertainty following calibration arising from a shift in output due to drift was 0.8% and was the most significant contribution to overall uncertainty. This variation is monitored within the clinic (through routine output measurements) but is not routinely taken into account for individual treatments due to practicalities of adjusting the delivered dose on a daily basis. The mean rate of drift used in this work (of +0.9% with a SD 2.3%) and previously reported [24] is in line with that reported by others; Hossain reported a rate of 2–4% per year [27], Luketina reported 2.5% per year [26], and Grattan reported a variable rate of change dependant on age of ionisation chamber ranging from 1 to 5% per year [25]. It is noted that the calculated linac drifts met the statistical criteria for normality however there were some outliers that showed particularly increased drift rates which skew the 95% CIs reported. These outliers may warrant further investigation to determine the root cause.

The plot in Fig. 2 indicates the potential non-linearity inherent in the LQ model for larger systematic offsets in dose, however within the clinical range it could be considered linear for practical purposes. Based on the simulation results the relationship between drift in beam output over the treatment course and change in TCP can be approximated as a linear relationship as observed in Fig. 3. The figure also indicates that even if large values of output drift are possible over the course of a typical treatment, the change in beam output is relatively small and gives rise in TCP variations of <0.5%.

Variation in delivered dose is known to have potential clinical significance [1], [28], [29] and this work evaluates the impact of the different components of dose uncertainty on TCP. Each of the four assessed components may be broken down further, e.g. the measurement of beam output includes uncertainty due to equipment setup. However, it was not possible to separate the uncertainty into further individual subcomponents from the data available.

Patient shape and positioning variations will undoubtedly have an impact on the delivered target dose, and this was not considered within this work. These inter and intra-fractional variations are patient specific, with large variations in shape change possible, for example due to weight loss in H&N treatments, or change in bladder and bowel filling in prostate treatments. Over a large population these patient variations would be expected to largely “average out”, however individual patients may undergo shape changes which result in changes in delivered dose to the target volume of the same order of magnitude as the dosimetric variations considered in this work. The TCP predictions given here are for a population of patients and so large numbers of patients would need to be studied to reproduce the results experimentally. However, as the models are grounded on the clinical trial outcomes of a population of patients, the steepness of the TCP curves should be largely representative of the patient population and therefore the relationship of dose to TCP reliable.

Daily fluctuations in dose were of much smaller magnitude than the potential systematic offsets, and thus the significance of this variation was small compared with the mean beam output for the duration of the treatment. Measurement tolerances used may vary significantly [30] dependent upon the device and measurement techniques used within each clinic. The results in this work suggest that systematic shifts in dose due to drift in beam output represent the largest uncertainty within the clinical dosimetry chain and the close monitoring of beam output and associated trends remains vital for accurate dose delivery.

Further studies investigating the impact of changes in tolerance levels on the variation in beam output may be warranted for a full cost-benefit analysis of the frequency of machine calibrations. With modern technologies and electronic record keeping, monitoring data for trends and anomalies is relatively simple to implement and may allow prediction of required machine recalibration and potentially reduction in tolerance levels used, thus helping to improve the overall accuracy of dose delivery.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.