Using prediction models to evaluate magnetic resonance image guided radiation therapy plans.

Comprehensive analysis of daily, online adaptive plan quality and safety in magnetic resonance imaging (MRI) guided radiation therapy is critical to its widespread use. Artificial neural network models developed with offline plans created after simulation were used to analyze and compare online plans that were adapted and reoptimized in real time prior to treatment. Roughly one third of 60Co adapted plans were of inferior quality relative to fully optimized, offline plans, but MRI-linac adapted plans were essentially equivalent to offline plans. The models also enabled clear justification that MRI-linac plans are superior to 60Co in an overwhelming majority of cases.

Introduction

The use of daily, online adaptive magnetic resonance imaging guided radiation therapy (MRgRT) has grown recently across a variety of clinics. As a result, the potential benefits and practical difficulties of online adaptive MRgRT are beginning to be understood [[1], [2], [3], [4], [5], [6], [7]]. Developing and assessing treatment planning processes and workflows for MRgRT remains a challenge. Daily changes in patient anatomy up to 3 cm in magnitude are possible [[8], [9], [10]]. Adapted plans cannot be optimized and scrutinized with the same level of time and effort as plans developed offline because daily adaptive decisions are being made while the patient is on table [1,2]. Furthermore, using plan-specific optimization parameters to create high quality offline plans at patient simulation can lead to subpar adapted plans with substantionally reduced target coverage [7],.

Even with the growth of online adaptive MRgRT, it remains difficult to assess overall online adapted plan quality relative to fully optimized, offline plans [[1], [2]],[11]. Because it is difficult to simulate the inherent complexities and timely decisions associated with MRgRT adaptive workflows [12], actual approved and treated plans offer the best opportunity to assess and improve online adaptive RT. There is very limited previous work where real, clinically treated plans were used to compare online adaptive and offline MRgRT [13], or 60Co and MRI-linac capabilities. The main objective in this study was therefore to use artificial neural network (ANN) models to analyze a wide variety of previously approved and treated MRgRT plans in order to achieve two primary goals: 1) explore online adaptive plan variability and quality relative to high-quality, offline plans; and 2) compare and contrast 60Co- and linac-based MRgRT.

Materials and methods

Patient characteristics

A total of 125 patients with abdominal cancers treated at our institution with high biologically effective dose (BED), online adaptive MRgRT were used for analysis. Online adaptive MRgRT has been used for abdominal, lung, pelvis, and breast cancers [11], but this study focused on abdominal cancers for two primary reasons: 1) natural alignment with the benefits of MRgRT in terms of enhanced soft tissue contrast imaging and daily anatomy changes, and 2) abdominal cancer cases produced the highest percentage of plans requiring daily adaptation at our institution [1,2,11]. Various treatment sites were included such as pancreas, liver, adrenal, bile duct, etc. but most cases (67%) were pancreas cancer. The patients were stratified based on the type of MRgRT: 60Co (n = 70) or MRI-linac (n = 55). All patients were treated with one of two high BED protocols as discussed in detail previously [14],. Overall, 781 of 975 (80%) treated plans were adapted, so a total of 125 offline and 781 online adapted plans were included. Offline plans were created on the patient's simulation image, received standard planning time, optimization, and checks just like traditional IMRT plans, and served as the starting point of plan adaptation for the first fraction.

Treatment techniques

Detailed descriptions of the specific workflows and treatment planning methods for adaptive MRgRT using the MRIdian 60Co and MRI-linac systems (Viewray, Cleveland, OH) have been presented previously [1,2,11],[[14], [15], [16], [17]]. The following characteristics were particularly relevant to this work. Both offline and online adaptive plans were developed with OAR isotoxicity prioritized over target coverage. Essentially, the OAR constraints discussed in detail previously [14], were hard constraints that could not be exceeded, regardless of the effect on target coverage. Treatment plan deviations manifested mainly in changes in target coverage, so the % of the GTV volume receiving >=95% of the prescription dose (V95) was the main plan quality metric. The prescription dose and dose constraints for OARs were used to guide the plan’s optimization. The four critical OARs of stomach, duodenum, small bowel, and large bowel (OARCRIT) were used in nearly all plans, with other OARs (aorta, esophagus, spinal cord, liver, one or both kidneys) also potentially used but with different dose constraints. The target used for optimization was not the PTV (5 mm isotropic expansion of GTV), but rather the PTVOPT (PTV minus OAR5mm). OAR5mm was OARCRIT expanded by a 5–8 mm isotropic margin. The majority of patients (~80% in this study) had a 5 mm OAR structure expansion for producing PTVOPT and it was held constant for all plans for each patient.

ANN prediction models

ANN models to predict voxelized dose inside the GTV were developed using patient anatomy/geometry information only. The model input variables included GTV size/shape, distance relationships between GTV and OARs, and patient size information [[18], [19], [20], [21], [22], [23]]. Additional details of the model development and testing have been published previously [14],. The prediction models were developed using input variables extracted only from offline plans because they received normal planning time, attention, and analysis prior to their approval and use in patients. In contrast, online adaptive plans were not afforded the time to pursue detailed optimization, so their overall quality a priori was not known.

A cross validation process like that described in our previous work [14], was used to test the ANN models and assess their accuracy and precision. For each iteration of the cross validation, V95 values for the test group of plans were determined from the 3D dose predictions and raw V95 prediction errors were calculated: . Then the mean error, 95% prediction intervals (PI, ±1.96σ), and 95% confidence intervals (CI) of the mean error and 95% PI were all determined as outlined in Bland-Altman analysis [24],. Limits of agreement (LoA) for each model were calculated as the mean error ± 95% PI. In order to minimize the effect of potential outlier plans (plans both inferior and superior to the average) on the trained models, a model refinement process was also incorporated [14,19,20]. Any plans with outside of the model LoA were excluded, the models were re-trained, and new prediction errors and model metrics were calculated. Model refinement excluded 10 out of the 70 60Co offline plans and 5 of the 55 linac plans. The refined models were then used for all plan comparison analysis, with inferior, superior, and acceptable plans identified as described in Fig. 2.

Fig. 2

Bland-Altman plots of V95 predictions for online adapted plans from ANN models developed using offline plans: (a) 60Co, (b) linac, (c) 60Co in linac model. The mean bias and LoA are plotted much like in Fig. 1. In each plot, the data points are defined based on their comparison to the model’s LoA: 1) blue circle = acceptable: within LoA, 2) red square = inferior: , 3) green diamond = superior: . The percentages of how many plans fell into the three categories are also indicated. The entire distribution of V95 values as well as boxplots for each distribution are shown in (d): online = clinical 60Co adapted plans, offline = offline 60Co ANN model predictions for 60Co adapted plans, linac = linac ANN model predictions for 60Co adapted plans. The mean values of each distribution are shown with solid dots, the median values are shown as a solid line. The boxes show interquartile ranges of the distributions, while the whiskers include all values up to ± 2.7σ, with any outliers indicated by a + sign.

Two separate models were developed (60Co and linac) and both used the exact same types of patient anatomy and geometry input variables – those optimized in our previous work on ANN dose prediction models [14],. Adapted plan quality relative to offline plans was determined by inputting the parameters extracted from adapted plans into the models trained with offline plans. The adapted plan predictions from the offline model outputs were then compared to the clinical plan metric. 60Co and linac MRgRT were compared by inputting parameters from 60Co plans into the linac ANN model. Effectively, then, the model outputs reflected the predicted 3D dose distribution that would have been achievable had the 60Co plan actually been planned using the linac.

Results

Dose prediction errors for both 60Co and linac models were ~0.2 ± 3.0 Gy when averaged across all plans. Absolute dose errors were ~3.0 ± 2.0 Gy. As shown in Fig. 1, both models produced V95 predictions that strongly correlated with their respective clinical values, maintained minimal bias, and possessed precision within ±6%. In both models ~95% of plans had within the LoA. As seen in Fig. 2(a), nearly one third (157 plans, 30%) of 60Co online adapted plans were deemed inferior, with clinical V95 values outside the lower prediction range of the ANN model. This observation strongly indicates these adapted plans could have achieved improved target coverage if they were developed and optimized offline. Larger deviations were observed as the clinical V95 decreased, showing that more intrinsically difficult cases tended to produce plans that were more inferior. The 60Co plans identified as inferior had statistically significantly lower mean and max OARCRIT doses relative to those established as adequate quality. Fig. 2(b) shows the overwhelmingly majority (91%) of adapted linac plans had clinical V95 values that fit within the prediction range of the offline model, with only 2% and 7% identified as inferior and superior, respectively. These observations demonstrate that target coverage in linac online adapted plans was essentially equivalent to the expectations set by the offline model.

Fig. 1

Results for clinical vs. predicted V95 values for (a) 60Co and (b) linac offline plans. The predicted V95 values come from the respective ANN model 3D dose predictions. The R2 values of the clinical vs. predicted comparisons are included in (a) and (b). Bland-Altman plots of the V95 prediction errors are shown for (c) 60Co and (d) linac models. The values for mean bias and LoA are indicated in (c) and (d). The mean bias, LoA (solid line), and LoA + 95% CI (dotted line) are also plotted in (c) and (d) to show the precision of the model predictions.

Fig. 2(c) shows that a large majority (78%) of 60Co adapted plans were identified as inferior to the expectations from the linac model. Furthermore, Table 1 shows nearly 40% of 60Co adapted plans had clinical V95 values > 10% lower, and roughly 7% > 20% lower, than the linac model predictions. Finally, Fig. 2(d) demonstrates that the median (mean) V95 values of the three groups of plans compared progressed from 77.5 (77.4) to 81.6 (81.4) to 87.9 (86.8). Although not shown explicitly in Fig. 2, offline 60Co plans were also deemed to be inferior to the expectations of the linac model in terms of target coverage, but at a slightly reduced rate of ~60%.

Table 1

Summary of plan comparisons based on V95 predictions for adapted plans from offline plan models.

	60Co Adapted(n = 516)	Linac Adapted(n = 265)	60Co Adapted in Linac(n = 516)
ΔV95 (%):Mean ± σ	−4.0 ± 5.9	0.4 ± 2.4	−9.4 ± 6.5
Acceptable(mean ± σ)	348 (67%)−1.3 ± 2.5	241 (91%)0.2 ± 1.5	109 (21%)−1.6 ± 1.4
Inferior(mean ± σ)	157 (30%)−10.9 ± 4.9	6 (2%)−7.7 ± 3.2	405 (78%)−11.5 ± 5.6
Superior(mean ± σ)	11 (2%)8.0 ± 1.8	18 (7%)5.3 ± 1.9	2 (0%)4.1 ± 0.8
ΔV95 < -10%	80 (16%)	2 (1%)	203 (39%)
ΔV95 < -20%	7 (1%)	0	34 (7%)

Discussion

This study used ANN prediction models, bolstered by patient- and plan-specific parameters, to comprehensively compare offline, online adapted, 60Co, and linac plans in MRgRT. Our results showed that many 60Co online adaptive plans, roughly one third, were not able to maintain the same level of target coverage as offline plans. These observations indicate that for one third of 60Co adapted plans, a tradeoff of reduced target coverage relative to the benchmark established by comparable offline plans was required in order to ensure meeting all OAR constraints. The statistically significantly lower mean and max OARCRIT dose metrics in inferior 60Co (30%) adapted plans suggest the online re-optimization was not able to push OAR doses sufficiently in order to achieve improved target coverage in all 60Co adapted plans. Unlike 60Co, our results showed that MRI-linac adapted plans were able to maintain target coverage expectations that were equivalent to offline plans with comparable intrinsic difficulty. These results establish that linac-based online adaptive MRgRT can maintain important plan quality metrics equivalent to offline plans that received the requisite time and attention to be fully optimized. This is a key observation to boost the clinical confidence in online plan adaptations with linac-MRgRT. Linac plans also outperformed 60Co plans at a rate of nearly 4 out of 5 and the average increase in target coverage (V95) had the plans been developed with the linac was ~10%. This provided further evidence that linac hardware was better able to produce high quality plans.

The details of the ViewRay MRI-linac system hardware have been outlined previously [25],. As discussed in a recent study [12], the distinction between 60Co and linac plans regarding plan quality is mainly due to the higher beam energy (average ~2 MV for linac; 1.25 MV for 60Co) and the improved multi-leaf collimator design in the linac. Online adapted MRI-linac plans were also shown to be roughly comparable in terms of plan quality while offering improved OAR dose metrics relative to original, unadapted plans [13],. Our results were in line with previously established key conclusions about MRgRT but also expanded upon them by analyzing each plan specifically and including intrinsic plan difficulty.

A limitation of this study was that other plan quality metrics such as dose conformality and OAR dose sparing were not easy to compare because our models could only predict GTV dose. Future work will include using more advanced models to expand 3D dose predictions beyond the GTV to explore a more complete picture of plan comparisons. Another limitation was that the models developed and plans analyzed were only from a single institution. MRgRT workflows and online adaptive planning strategies differ across various institutions. We are hopeful that the results presented here are deemed useful for a better understanding of the difficulties and capabilities of online adaptive MRgRT as a rapidly growing application for improved treatment of cancer with RT worldwide.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.