Multi-center evaluation of dose conformity in stereotactic body radiotherapy.

BACKGROUND AND
PURPOSE: Stereotactic body radiotherapy (SBRT) is an emerging technique for treating oligometastases, but limited data is available on what plan quality is achievable for a range of modalities and clinical sites.
METHODS: SBRT plans for lung, spine, bone, adrenal, liver and node sites from 17 participating centers were reviewed. Centers used various delivery techniques including static and rotational intensity-modulation and multiple non-coplanar beams. Plans were split into lung and other body sites and evaluated with different plan quality metrics, including two which are independent of target coverage; "prescription dose spillage" (PDS) and "modified gradient index" (MGI). These were compared to constraints from the ROSEL and RTOG 0813 clinical trials.
RESULTS: Planning target volume (PTV) coverage was compromised (PTV V100% < 90%) in 29% of patient plans in order to meet organ-at-risk (OAR) tolerances, supporting the use of plan quality metrics which are independent of target coverage. Both lung (n = 48) and other body (n = 99) site PDS values agreed well with ROSEL constraints on dose spillage, but RTOG 0813 values were too high to detect sub-optimal plans. MGI values for lung plans were mis-matched to both sets of previous constraints, with ROSEL values too high and RTOG 0813 values too low. MGI values were lower for other body plans as expected, though this was only statistically significant for PTV volumes <20 cm3.
CONCLUSIONS: Updated guidance for lung and other body site SBRT plan quality using the PDS and MGI metrics is presented.

Introduction

Stereotactic body radiotherapy (SBRT) has established efficacy in the treatment of stage 1 non-small cell lung cancer (NSCLC) [1], but not for oligometastases, where prospective randomized trial data has not yet been reported [2]. Evidence should be based on high quality, accurate and deliverable treatment plans, tested across multiple centers [3], [4], [5]. In addition to considering target coverage and doses to organs-at-risk (OARs), tight conformity of the planned dose to the target volume is a hallmark of good SBRT planning; excessive coverage of the patient with the prescription dose or sub-optimal dose fall-off around the target will generally increase the integral dose received by normal tissues.

There are many plan quality metrics [6], [7], [8], [9] which aim to quantify the radiotherapy goals of providing adequate target coverage whilst minimizing the normal tissue dose, some combined into a single metric [10], [11]. Published guidance aims to quantify what plan quality should be achieved in lung SBRT; but guidance is currently limited to the ROSEL study [12] and RTOG 0813 [13], which both include the “R100%” and “R50%” metrics (defined in Methods). While the ROSEL tolerances have been widely adopted nationally [14], the data is based on a planning study of only 26 patients using 9 fixed-field intensity modulated radiotherapy (IMRT). Tolerances from the RTOG trial 0813, also used in the RTOG trial 0915 [15] have also been adopted internationally, however these tolerances are based on the use of 3D conformal (3D-CRT) co-planar or non-coplanar SBRT techniques, potentially calculated with Type-A algorithms, which does not reflect the techniques currently routinely used for lung SBRT [16]. The need for updating RTOG guidance has been raised in more recent publications that reviewed dose conformity in co-planar [17], [18], [19], [20], [21] or non-coplanar [22] IMRT or volumetric modulated arc therapy (VMAT) lung SBRT plans. Recommendations for plan quality metrics in other body sites are scarce; the PACE trial protocol referencing the ASTRO report [23], that the volume ratio of the planning target volume (PTV) D95% to the PTV should be less than 1.2.

Other multi-center SBRT planning studies have typically involved the re-planning of a set of case plans for an individual clinical site with different treatment modalities; spine [24], [25], liver [26] and lung [27]. Reporting the variability of plan quality over a small selection of plans is valuable to verify local plan quality, however, most planning studies in the literature do not correlate the variation of results with patient specific variables, such as PTV size, and so are unable to propose any general planning guidance on what may be practically achievable.

Achieving consistently optimal plans should not be confused with published normal tissue constraints [28] which are based on evidence of patient outcomes. However, there is additional value in increasing the conformity of dose, once target coverage and OAR constraints have been met; that normal tissue doses may be reduced further, reducing the risk of normal tissue effects, secondary cancers and increasing scope for re-treatment.

We report two plan quality metrics on data submitted for QA review in the first 18 months of a national programme evaluating the efficacy of SBRT for oligometastases. These metrics quantify the conformity of the prescription isodose to the target volume and the dose fall-off around the target, but unlike other metrics, are independent of target coverage. Such independence is important if a substantial proportion of plans require compromise in order to maintain safe normal tissue doses. The data shows the spread of results and quantifies what is currently being achieved in SBRT planning in terms of dose conformity. The purpose of the presented metrics and data is to guide planners in the “optimality” of planning, beyond achieving recommended target coverage and OAR constraints.

Methods

To evaluate the efficacy, toxicity, cost and quality of SBRT for oligometastases, a three year “Commissioning through Evaluation” (CtE) programme was established by NHS England in June 2015 with 17 centers participating who were all previously experienced in lung SBRT. The QA of the programme was delegated to the National Radiotherapy Trials QA group, who reviewed plans submitted by the centers against the programme “Service Specification” [29].

Centers were sent pre-prepared spine, lung and adrenal metastases planning benchmark cases which consisted of a computed tomography (CT) dataset with pre-delineated target volume and all relevant OARs, with instructions for the prescription dose (see Supplementary Table S1) and OAR dose constraints. Centers applied their own PTV margin to reflect their local planning process, which were typically 3–5 mm except for spine where 2–3 mm was most common. Patient plans (individual case reviews) were subsequently received for these clinical sites, as well as non-spine bone, lymph nodes, and liver. Not all centers submitted plans for all clinical sites; reflecting the breadth of clinical service they intended to offer. Some centers submitted multiple benchmark plans (particularly for lung), as their techniques changed (e.g. 3D-CRT to IMRT, or IMRT to VMAT), new planning systems were introduced, or treatment equipment changed over the 18 months. Multi-target plans (including the lymph node benchmark case) and submissions which were subsequently re-planned due to sub-optimal plan quality or exceeding clinical constraints were excluded from this analysis. VODCA v5.4 (Medical Software Solutions GmBH, Hagendorn, Switzerland) was used for QA review.

The “Service Specification” document referenced the ROSEL study constraints for lung planning, and included national consensus OAR dose constraints [28]. Centers were asked to produce clinically optimal plans, covering more than 95% of the PTV with the prescription dose, compromising only to maintain OAR constraints, without reference to additional planning metrics or suggested tolerances.

Target coverage was evaluated as the percentage volume of PTV that received 100% of the prescription dose:

Throughout this paper, 100% dose refers to the prescription dose (not the maximum dose) i.e. the dose which would ideally cover more than 95% of the PTV.

Several parameters have been reported in the literature for quantifying the excess prescription dose outside the PTV. The ROSEL lung study and RTOG trial 0813 used R100%, the ratio of the volume of the patient that received at least the prescription dose to the PTV volume:

However, this parameter can be insensitive to poor conformity if the PTV coverage were compromised. A modification to this formula that addresses this problem was the “prescription dose spillage” (PDS) conformity metric:

Body V100% is the volume of the patient that received at least the prescription dose and PTV V100% is the volume of PTV that received at least the prescription dose. The PDS metric has an “ideal” and minimum value of unity, and quantifies the excess normal tissue covered with the prescription dose. The PDS metric was not new; but effectively the inverse of the “healthy tissue conformity index” [30], the “selectivity” component of the Paddick Conformity Index (PCI) [11] and the conformity index reported in the Multiplan (Accuray Inc, Sunnyvale, CA) planning system.

It is integral to SBRT planning that a steep dose gradient is achieved around the target volume to protect normal tissue. Two common metrics that quantify this were:

the ratio of the volume of the patient receiving at least 50% of the prescription dose to the volume of the PTV (called R50% in RTOG 0813):

the ratio of the volume of the patient receiving at least 50% of the prescription dose to the volume of the patient receiving at least 100% of the prescription dose (commonly known as Gradient Index (GI) [31], [32]):

For evaluating plans where PTV coverage were uncompromised (PTV V100% ≥ 95%), the R50% parameter was robust, however, if the PTV covered by the prescription dose were limited, the R50% parameter may have failed to detect a sub-optimal plan. Conversely, the GI was insensitive to plans with unnecessarily large prescription dose volume. An alternative metric was defined as: the volume of the patient that receives at least 50% of the prescription dose to the volume of the PTV receiving at least the prescription dose:

Both the PDS and Modified Gradient Index (MGI) metrics were normalized to the volume of target covered by the prescription dose. This approach was robust to compromised PTV coverage and as the PTV V100% is the high dose region expected to yield local control, we suggest it is more clinically relevant.

The PDS, MGI, R100%, R50% and PCI were calculated for all plans submitted for QA review. The data was split into lung and other body site plans to assess the effect of the low density surroundings typically found around lung targets. The data was further split into PTV V100% volume ranges of 0–20 cm3, 20–40 cm3, 40–60 cm3, 60–90 cm3 and >90 cm3, matching the volume ranges used in current national guidance [14] which includes the ROSEL study values. The effect of treatment modality was also considered. Different groups were compared using the 2-sided Student t-test with a significance level of p < 0.05.

Results

In total 147 plans were reviewed in the first 18 months of the CtE programme: spine & bone 46, lung 48, lymph node 24, adrenal 22, liver 7. No single center submitted more than 10 patients (individual case reviews). All plans met the programme OAR dose constraints. A full breakdown of the clinical site, prescription dose and modality is shown in Supplementary Table S1.

The participating centers used a range of treatment modalities: CyberKnife™ (CK) or Tomotherapy® (HT), using the Multiplan® or VOLO™ planning systems (Accuray Inc., Sunnyvale, CA); or co-planar VMAT, 3D-CRT or fixed gantry angle IMRT using either Varian (Varian Medical Systems, Palo Alto, CA) or Elekta (Elekta AB, Stockholm, Sweden) linacs, and Eclipse™ (Varian, Palo Alto, USA), Pinnacle® (Phillips Healthcare, Best, Netherlands), Monaco® or Oncentra® Masterplan (Elekta AB). Most centers calculated dose with a Type B algorithm for plans with proximal low density areas such as lung and other thoracic sites, which model variations in lateral electron transport. These included model-based algorithms such as collapsed cone convolution and Monte Carlo approaches, although some centers (6/17) used the AAA algorithm in Eclipse.

Target coverage

Fig. 1 shows the variation in target coverage amongst the 93 patient plans only. Only 25/93 (27%) plans had coverage >95%. Coverage was more substantially compromised (PTV V100% <90%) in 27/93 (29%) plans, which included 10/24 (42%) node plans, 3/7 (43%) liver plans, 4/12 (33%) adrenal plans, 6/29 (21%) spine and bone plans and 4/21 (19%) of lung plans. 17% of all plans also had CTV V100% <95%.

Fig. 1

The distribution of PTV coverage, by clinical site, for the 93 patient plans submitted by participating centers.

Plans with PTV V100% >90% had R100% values of 1.1 ± 0.1 (mean ± standard deviation, SD) while more compromised plans had significantly lower R100% values of 0.9 ± 0.2 (p < 0.01). A similar trend was seen with R50% values of 4.8 ± 1.4 and 4.2 ± 1.6 respectively, although this did not reach statistical significance (p = 0.08). No compromised plan exceeded any of the Type B algorithm ROSEL study R100% constraints and only two compromised plans exceed the R50% constraints.

Using the proposed PDS and MGI metrics, there was no statistically significant correlation with PTV coverage: PDS values were 1.1 ± 0.1 (mean ± SD) and 1.1 ± 0.1, for plans with more compromised target coverage and V100% >90% respectively (p = 1.00); MGI values were 5.4 ± 1.9 and 5.1 ± 1.4 respectively (p = 0.31).

Dose spillage and gradient outside target

The proposed metrics of PDS and MGI are plotted against the established metrics of R100% (Fig. 2), R50%, GI and PCI (Supplementary Fig. S1), grouped by target coverage. The mean and standard deviation for all metrics are shown in Table 1, grouped by size of PTV V100%, with lung and other body sites reported separately. Median is also reported as some distributions are skewed. Variation of PDS and MGI with size of PTV V100% for the lung and other body site plans are also shown in Figs. 3 and 4. There were insufficient lung plan submissions with PTV V100% of 60 cm3 or larger to provide meaningful statistics.

Fig. 2

Variation in spillage metrics (R100% and PDS) with PTV coverage, for all 147 plans.

Table 1

Conformity metrics for all 147 plans submitted through the CtE programme, split by PTV V100% volume range.

	PTV V100% (cm3)	Mean ± Standard Deviation (Median)
		PDS	R100%	PCI	MGI	R50%	GI	n
Lung data	<20	1.18 ± 0.09 (1.22)	1.10 ± 0.12 (1.12)	0.79 ± 0.06 (0.79)	7.2 ± 2.3 (6.8)	6.7 ± 2.1 (6.7)	6.0 ± 1.6 (5.4)	9
	20–40	1.11 ± 0.07 (1.09)	1.04 ± 0.07 (1.04)	0.85 ± 0.05 (0.86)	5.4 ± 0.9 (5.3)	5.1 ± 0.9 (5.0)	4.9 ± 0.7 (4.6)	32
	40–90	1.11 ± 0.06 (1.12)	1.02 ± 0.05 (1.04)	0.83 ± 0.06 (0.83)	5.0 ± 0.7 (5.2)	4.5 ± 0.6 (4.6)	4.4 ± 0.5 (4.6)	7
Other body site data	<20	1.17 ± 0.10 (1.17)	1.02 ± 0.18 (1.01)	0.76 ± 0.12 (0.80)	6.0 ± 1.8 (5.6)	5.3 ± 1.8 (5.2)	5.2 ± 1.4 (4.9)	15
	20–40	1.13 ± 0.07 (1.14)	1.01 ± 0.14 (1.04)	0.80 ± 0.10 (0.83)	4.7 ± 1.2 (4.6)	4.2 ± 0.9 (4.1)	4.2 ± 1.0 (3.9)	21
	40–60	1.09 ± 0.05 (1.08)	0.97 ± 0.15 (0.99)	0.81 ± 0.11 (0.84)	4.6 ± 1.1 (4.6)	4.0 ± 0.8 (3.9)	4.2 ± 0.9 (4.2)	19
	60–90	1.09 ± 0.07 (1.06)	1.00 ± 0.13 (0.98)	0.83 ± 0.10 (0.87)	4.8 ± 1.3 (4.3)	4.3 ± 1.1 (4.2)	4.4 ± 1.0 (4.2)	19
	>90	1.10 ± 0.06 (1.09)	1.01 ± 0.09 (1.02)	0.83 ± 0.06 (0.82)	4.6 ± 0.9 (4.6)	4.2 ± 0.8 (4.2)	4.2 ± 0.7 (4.1)	25

Fig. 3

A box and whisker plot showing the prescription dose spillage (PDS) for all other body site and lung plans, grouped by size of PTV V100%. Only 7 lung submissions with PTV V100% greater than 40 cm3 were received (average 58 cm3, range 40–77 cm3). The metric results from these plans have been grouped together and displayed in the 40–60 cm3 column.

Fig. 4

A box and whisker plot showing the Modified Gradient Index (MGI) for all other body site and lung plans, grouped by size of PTV V100%. Only 7 lung submissions with PTV V100% greater than 40 cm3 were received, therefore metric results from these plans have been grouped together and displayed in the 40–60 cm3 column.

There was no statistically significant difference between the means of the lung and other body site PDS results (p = 0.10–0.30). In the 20–40 cm3 range, the mean lung MGI was statistically significantly higher than the mean other body site MGI (p = 0.01). In the 0–20 cm3 and 40–60 cm3 volume ranges, the difference between means did not reach statistical significance (p = 0.09 and 0.20 respectively).

Differences between modalities

The data was analyzed for differences between the delivery modalities for the lung and other body site data. In the lung subset, CK plans had significantly higher PDS (1.2 ± 0.1, mean ± SD) than VMAT plans (1.1 ± 0.1) (p < 0.01). In the other body site data, HT plans had significantly higher MGI (6.0 ± 1.8) than CK (4.4 ± 0.9) (p < 0.01) and VMAT (4.8 ± 1.2, p = 0.02). No other statistically significant (p < 0.05) differences were observed between the modalities. There were no significant differences between mean PTV V100% volumes for the different modalities for the lung or other body site plans.

Forming new guidelines

The data results in Table 1 were used to update the existing national guidelines, with the addition of a new “target” level, based on the median/mean results, which should be achievable for approximately half of clinical patients. The “per protocol” (tolerance) level and “acceptable variation” (minor deviation) range are based on the median/mean results plus 1 and 2 standard deviations respectively. The updated guidance is presented in Table 2.

Table 2

Updated national guidance on dose conformity in lung and other body site SBRT.

	PTV V100% (cm3)	Prescription Dose Spillage			Modified Gradient Index
		Target	Per protocol	Acceptable variation	Target	Per protocol	Acceptable variation
Lung	<20	1.20	<1.25	1.25–1.40	7.0	9.0	9.0–11
	20–40	1.10	<1.20	1.20–1.30	5.5	6.5	6.5–7.5
	>40	1.10	<1.15	1.15–1.20	5.0	6.0	6.0–7.0
Other body site	<20	1.20	<1.25	1.25–1.40	5.5	7.5	7.5–9.5
	20–40	1.10	<1.20	1.20–1.30	4.5	6.0	6.0–7.5
	>40	1.10	<1.15	1.15–1.20	4.5	5.5	5.5–6.5

Discussion

Target coverage and OAR tolerance doses are the predominant planning quality metrics, as they are most closely related to clinical outcomes. In situations where proximal OARs necessitated compromising PTV coverage, maximizing target coverage while respecting OAR constraints was the primary requirement of the CtE programme. PTV coverage was <95% in the majority of patient plans, and substantially compromised (<90%) in almost a third. This occurred in other body sites both more frequently and more substantially than lung plans, perhaps because of the PTV margins used and consequent proximity of OARs. These data confirm the need for dose conformity quality metrics which are independent of PTV coverage, so that values are not confounded by under-coverage, which can be reported separately. The proposed metrics of PDS and MGI satisfy this requirement, with minimal change from the established R100%, R50% and GI.

Fig. 2 and Supplementary Fig. S1 show that the usefulness of the R100% and R50% metrics to highlight over-coverage (spillage) is reduced with increasing compromise of PTV coverage. Therefore, meaningful recommendations cannot be made on achievability when the base dataset contains plans with compromised target coverage. Conversely, objectives for these metrics that are based on uncompromised target plans (or a single site such as lung) should not be universally applied to all body sites, or plans with compromised target coverage. The PCI metric is designed to be a composite of coverage and spillage (the inverse being called “selectivity”), so there is little correlation with PDS. However, this metric is usually applied to intra-cranial radiosurgery, where almost all plans have very high coverage [33], so caution should also be applied in using objectives from cranial sites to extra-cranial SBRT plans.

The Gradient Index metric shows no correlation with target coverage and so guidance on its use has also been given (Table 2). The divergence from the line of equality between GI and MGI at higher values indicates that, in this dataset, the lowest dose fall-off was correlated with the poorest prescription dose conformity.

In spite of metric differences, the reviewed plans compared favorably with the ROSEL and RTOG 0813 R100% constraints (listed in Supplementary Table S2). The results suggest that the RTOG 0813 R100% constraint levels are set too high to usefully detect sub-optimal plans, as the “tolerance” constraint was only exceeded by 8 of 147 (5%) plans and no submissions exceeded the “minor deviation” constraint. This is better conformity than was reported in a study of 300 lung SBRT plans from 4 centers [34] which found that only 79% of plans achieved the RTOG 0813 “tolerance” constraint (R100% 1.14 ± 0.21 (mean ± SD)). The ROSEL study R100% constraint levels are supported by these data, as they closely match the CtE data median plus 1 SD and median plus 2 SD, although we have chosen to increase some of the ROSEL values slightly to reflect the results and the fact the PDS denominator is the PTV V100%, rather than the PTV volume (which will be cause the PDS value to rise slightly).

The ROSEL study R50% constraints were easily achieved. No plans with PTV V100% volumes below 60 cm3 exceeded either constraint, suggesting these constraints should be tightened. However, compared against the RTOG 0813 R50% constraints; 29/147 (20%) exceeded “tolerance” and 92/147 (63%) exceeded “minor deviation” constraints. These data suggest that these constraints are not routinely achievable. The recent study of 300 lung SBRT plans supports this, with 43% exceeding RTOG 0813 R50% “tolerance” and 66% exceeding “minor deviation” levels [34]. The RTOG 0813 constraints in general appear to be mis-matched to the gathered data, with the R100% constraints being too lax (particularly for larger PTVs), and the R50% constraints being too tight. This may be because at the time of the trial most centers used a non-coplanar 3D conformal technique and some used Type-A algorithms, which together would potentially reduce prescription dose conformity but improve intermediate dose conformity. In this study, all participants used Type B or intermediate algorithms (e.g. Eclipse AAA [35], Collapsed Cone, Monte Carlo) and most used co-planar IMRT delivery.

The MGI results were lower for other body site plans, which justifies the use of tighter MGI constraints for other body site plans. The higher MGI for lung plans is expected as the penumbral broadening in low density surroundings reduces the dose gradient around the target.

Helical tomotherapy produced statistically significantly poorer plans (as quantified by the MGI metric) than the other modalities, however the total numbers were small, and plans were clinically acceptable, meeting the constraints set out in CtE and justifying their inclusion in the collated data. In terms of plan quality over a large number of patients with different treatment sites, there was very little difference between C-arm linac and CyberKnife plans. The ability of these platforms to account for intrafraction motion and reduce target volume size will likely be of far greater impact on normal tissue doses.

The data used to update the guidance is limited, particularly for lung PTV volumes greater than 40 cm3 (n = 7). This was also an issue for the ROSEL planning study [12] where only five plans had larger volumes, however both datasets show that above 40–50 cm3, a change in target volume has little correlation with the prescription and medium dose conformity.

The results of this work have since been incorporated into the UK SABR Consortium national guidance [14]. All members of the multi-disciplinary radiotherapy team should be aware that this guidance encourages the highest practical level of dose conformity, which increases the risk of a loss of tumour control probability if the contoured target is too small, and if dosimetric or patient positioning accuracy is compromised.

Dose conformity may be reduced for a variety of good reasons. Complex PTV shapes will be more difficult to conform to and very low density normal lung will cause loss of intermediate dose conformity. Furthermore, creating a very steep dose gradient to spare a proximal OAR may compromise dose conformity overall. The presented guidance values should therefore be used in the wider context of general plan evaluation.

In conclusion, SBRT plans from multiple centers, delivery techniques and body sites have been compared using a range of conformity metrics. The high level of target coverage compromise supports the use of metrics which are independent of this parameter, to clearly show over-coverage and sub-optimal plans. Previous recommendations from lung SBRT trials are only partially applicable to this wider dataset, therefore new guidance has been formulated for the proposed MDS and MGI metrics that covers both lung and other body sites.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.