Dosimetric Evaluation of Pinnacle's Automated Treatment Planning Software to Manually Planned Treatments.

INTRODUCTION: With the advent of complex treatment techniques like volumetric modulated arc therapy, there has been increasing interest in treatment planning technologies aimed at reducing planning time. One of these such technologies is auto-planning, which is an automated planning module within Pinnacle3. This study seeks to retrospectively evaluate the dosimetric quality of auto-planning-derived treatment plans as they compare to manual plans for intact prostate, prostate and lymph nodes, and brain treatment sites.
MATERIALS AND METHODS: Previous clinical plans were used to generate site-specific auto-planning templates. These templates were used to compare the 3 evaluated treatment sites. Plans were replanned using auto-planning and compared to the clinically delivered plans. For the planning target volume, the following metrics were evaluated: homogeneity index, conformity index, D2cc, Dmean, D2%, D98%, and multiple dose fall-off parameters. For the organs at risk, D2cc, Dmean, and organ-specific clinical metrics were evaluated. Statistical differences were evaluated using a Wilcoxon paired signed-rank test with a significance level of 0.05. Statistically significant ( P < 0.05) differences were noted in organs at risk sparing.
RESULTS: For the prostate, there was as much as 6.8% reduction in bladder Dmean and 23.5% reduction in penile bulb Dmean. For the prostate + lymph nodes, decreases in Dmean values ranging from 4.1% in the small bowel to 22.3% in the right femoral head were observed. For brain, significant improvements were observed in Dmax and Dmean to most organs at risk.
CONCLUSION: Our study showed improved organs at risk sparing in most organs while maintaining planning target volume coverage. Overall, auto-planning can generate plans that delivered the same target coverage as the clinical plans but offered significant reductions in mean dose to organs at risk.

Introduction

The current inverse planning algorithm in the Pinnacle3 treatment planning system allows a planner to input various optimizer values to reach a clinically acceptable plan. The consistency, speed, and quality of treatment plans originating from this method of manual optimization relies heavily on the experience of the planner.[1-3] For that reason, treatment planning is often referred to as an “art” rather than a science. When manually optimizing, planners must rely on intuition and clinic standards to decide whether they have reached an acceptable sparing level for organs at risk (OARs). Published limits are used as upper limit dose guidelines, but it is essentially impossible for a planner to know whether optimization attempts have minimized the OAR dose as low as achievable—or at least as low as possible within an acceptable time frame. This approach is strongly dependent on the planner to use intuition and experience to conclude that the plan has been minimized, but there is no standardized method in place to alleviate the variability that a manually optimized approach adds to treatment planning.

There are many commercially available planning solutions to help alleviate these issues, including knowledge-based planning and an auto-planning (AP) engine. A recent study by Wu et al showed that these 2 types of planning solutions have comparable plan quality, specifically when planning oropharyngeal cancer treatments.[4] Auto-planning, which is available in Pinnacle3 Version 9.10 (Koninklijke Philips N.V., Amsterdam, Netherlands), is a tool designed to automate and facilitate inverse optimization of treatment plans by utilizing templates to standardize manual repetitive data entry during the initial plan setup. Auto-planning uses templates combined with an AP engine to drive the plan to a more optimized solution. A major advantage of AP is that its optimizer will continue to minimize the OAR dose even once the constraints have been met. Additionally, AP offers a solution that can bridge the gap between planner experience and speed up the process using templates based on years of clinical plans as a starting point for the optimizer.[5,6]

Recent studies have been performed comparing AP plans with previous clinically delivered plans and have shown promising results. Nawa et al retrospectively analyzed 23 prostate cases against a replanned AP and found that the plan quality was equal, and in some cases better than the previously delivered clinical plan.[7] Another retrospective study performed by Wang et al used AP for hippocampal avoidance during whole-brain radiotherapy planning and found that AP was capable of meeting RTOG0933 dose coverage and constraint objectives while reducing the time required for planning.[8] These studies provided useful investigations of AP and its relevance for treatment planning but were limited to one cancer site. Considering these studies, this investigation looks to provide a more comprehensive evaluation of AP. Specifically, this retrospective study aims to include more sites by dosimetrically comparing the radiotherapy plan quality of AP with manually optimized, previously treated clinical radiotherapy plans for intact prostate (prostate and seminal vesicles), prostate and pelvic lymph nodes, and brain irradiation. Specifically, key objectives were used to quantify and compare dosimetric metrics of AP plans with consideration to the target volume and OARs as compared to the clinically delivered plans. The study was approved by our institution’s IRB as part of an ongoing quality improvement investigation. Since this study was done retrospectively and we did not apply any of these virtual treatments to patients, this study did not require written or verbal consent. Additionally, the study is done on anonymized patient data. Our institution has determined this falls under a global Institutional Review Board (IRB) approval to use these data sets for clinical improvements in treatment planning without a specific IRB approval.

Materials and Methods

Site and Clinical Plan Selection

To benchmark AP’s capability, a total of 59 manually optimized, previously treated clinical plans were retrospectively replanned using AP. These 59 cases were spread over three clinical sites that represented varying levels of geometrical uncertainty patient-to-patient. Twenty (n = 20) intact prostate (prostate and seminal vesicles) cases previously treated with a 6 or 10 MV volumetric modulated arc therapy (VMAT) technique to achieve 79.2 Gy were used. Twenty (n = 20) prostate and pelvic lymph node cases previously treated using 6 MV VMAT to achieve 45 Gy were chosen. Nineteen (n = 19) brain cases previously treated using a dual arc, 6 MV VMAT technique to achieve 60 Gy were selected. These cases ranged from similar tumor location between patients (prostate) to dissimilar tumor location among patients (brain). A sample size of 20 patients was the benchmark for the study. Since these plans were being run on weeknights and weekends, this was a reasonable value for the statistics while still maintaining a reasonable amount of planning time. The cases for this study were selected first by narrowing the search to the site, then sorting newest to oldest until 20 plans were selected. This way there was no bias in selecting the plans. For the brain, only 19 full dual arc plans could be found.

Evaluation of Plans and Statistical Analysis

The planning target volume (PTV) coverage across all sites was evaluated using the homogeneity index (HI),[9] conformity number (CN),[10,11] and dose falloff.[12] The OARs and dosimetric indices used to evaluate them were site-specific but were generally evaluated using Vx(Gy), Vx%, Dmean, or Dmax, where Vx(Gy) is the percentage volume receiving x Gy, Vx% is the absolute volume (cubic centimeters) receiving x % of the prescription dose, Dmean is the average organ dose, and Dmax is the dose that 0.03 cc of the volume receives. Table 1 shows the breakdown of selected OARs and dosimetric evaluation indices by site for the OARs and PTV.

Table 1.

Summary of OAR and PTV Metrics.

Site	OARs	Dosimetric Indices
Prostate + seminal vesicles	PTV	D2cc, HI, CN and R70, R50, and R30
	Bladder	D2cc, Dmean, V80Gy, V75Gy, V70Gy, and V65Gy
	Rectum	D2cc, Dmean, V75Gy, V70Gy, V65Gy, and V50Gy
	Sigmoid, penile bulb, and femoral heads	D2cc and Dmean
Prostate + pelvic lymph nodes	PTV	Dmean, D2cc, HI, CN, R50, and total MUs
	Bladder, rectum, sigmoid, small bowel, penile bulb, and femoral heads	D2cc, Dmean, V80%, V60%, V40%, and V20%
Brain	PTV	D2%, D98%, HI, CN, and falloff
	Brain, brain stem, optic chiasm, right/left optic nerves, right/left cochlea, spinal cord, and right/left eye	Dmax (0.03 cc) and Dmean

Abbreviations: CN, conformity number; HI, homogeneity index; OARs, organs at risk; PTV, planning target volume.

For prostate + lymph nodes, Vx% was chosen, as opposed Vx(Gy), because the prescription was limited to 45 Gy. The absolute volumes were recorded at the designated dose levels. For all sites, the dosimetric indices were assessed in both the manual and AP methods and compared using a paired-sample Wilcoxon signed rank test at the 5% significance level.

Auto-Planning Techniques and Templates

An initial step in AP is to define templates for each treatment protocol. This is an important part of AP as template setup quality plays a critical role in the AP outcome. For all template creation in this study, the parameters of the previously manually optimized plan were used by creating a template with the same isocenter, machine, number of arcs, arc length, collimator angle, couch angle, and energy. Template optimization parameters were assigned according to Quantitative Analysis of Normal Tissue Effects in the Clinic (QUANTEC) values for OAR sparing and target coverage with priority values set based on previous planning experience. The QUANTEC values were also used in the optimization of the original clinical plan. The goal was to make 1 template that could plan multiple cases to at least look like the dose–volume histogram (DVH) from the original plan by aiming to first retain the same PTV coverage as before, then pushing the OARs as much as possible before sacrificing coverage. Essentially, it is about matching the same process as would normally be performed. Since the templates were site-specific and 20 patients per site were to be replanned, 6 patients (approximately one-third of the test size) were chosen to test and tweak the initial template. The choice of 6 seemed appropriate to expose any weaknesses with the templates. These 6 patients per site were used to identify gaps in the AP by running a template and comparing the output to the original clinical plans. Based on the output, the template parameters were adjusted to ensure that the AP template PTV coverage matched that of the clinical plans. In addition, planners created manual optimization structures to help improve homogeneity and decrease hot spots that initially appeared in the PTV. This helped to create a stronger template. Doses prescribed were 60 Gy (2 Gy × 30 fractions) for the brain, 79.2 Gy for the intact prostate, and 45 Gy for prostate with lymph nodes. All plan prescriptions for both the clinical and AP were normalized such that 95% of the PTV received 100% of the dose to ensure that any OAR differences and statistical changes were a direct result of the different optimization algorithms and not an artificially created difference. After using those 6 clinical plans to tweak the template, a final template was completed that incorporated all the edits. This was the template used to replan each site. The use of these final templates on each site gave treatment plans that were close to being acceptable after the initial run. At this point, planners would add individual objectives that would be helpful for that specific case to bring the plan to a clinically acceptable plan.

Results

Wilcoxon signed rank tests were performed between the dosimetric parameters of the clinical plans and AP plans. Tables 2, 3, and 4 show the results of the paired Wilcoxon signed rank test for the intact prostate, prostate + lymph nodes, and brain, respectively. Each table depicts only the dosimetric indices that were statistically significant for each OAR.

Table 2.

Statistically Significant Metrics (P < .05) for Prostate and Seminal Vesicles Plans.a

ROI	Dosimetric Index	Median ± σ
		Clinical	AP	% Change
PTV_7920	HI	1.04 ± 0.01	1.05 ± 0.01	1.2
PTV_7920	CN	0.94 ± 0.03	0.93 ± 0.02	−1.1
PTV_7920	R30	12.6 ± 2.0	12.3 ± 1.4	−2.9
Bladder	D2cc	81.0 ± 0.7	82.3 ± 0.5	1.5
Bladder	Dmean	29.2 ± 9.2	27.2 ± 6.6	−6.8
Bladder	V80Gy	6.8 ± 8.2	9.1 ± 8.1	33.6
Rectum	D2cc	80.3 ± 1.8	81.3 ± 2.7	1.2
Rectum	Dmean	36.2 ± 6.8	35.7 ± 4.7	−1.4
Rectum	V65Gy	16.2 ± 6.8	15.9 ± 6.1	−1.8
Rectum	V50Gy	27.3 ± 8.4	24.9 ± 7.0	−9.0
Sigmoid	D2cc	17.1 ± 19.1	11.0 ± 12.7	−36.0
Sigmoid	Dmean	5.0 ± 4.3	4.1 ± 3.0	−16.5
Penile bulb	Dmean	45.1 ± 19.9	34.5 ± 18.4	−23.5
Femur (right)	D2cc	37.6 ± 5.5	30.9 ± 5.6	−17.7
Femur (right)	Dmean	21.5 ± 3.6	17.5 ± 2.6	−18.6
Femur (left)	Dmean	22.6 ± 2.7	19.8 ± 3.1	−12.7

Abbreviation: AP, auto-planning; CN, conformity number; HI, homogeneity index; PTV, planning target volume; ROI, region of interest.

aAll Dx values are given in units of Gy. All Vx(Gy) values are given in units of percentage. The table lists the ROI, significant dosimetric indices, median values, and percent change for the median value of the dosimetric compared between the clinical and respective auto-planning.

Table 3.

Statistically Significant Metrics (P < .05) for Prostate and Pelvic Lymph Nodes Plans.a

ROI	Dosimetric Index	Median ± σ
		Clinical	AP	%Change
PTV_4500	HI	1.08 ± 0.02	1.06 ± 0.01	−1.6
Bladder	Dmean	33.8 ± 5.3	29.8 ± 6.2	−11.8
Bladder	V40%	130.4 ± 190.2	113.4 ± 154.2	−13.1
Bladder	V20%	160.1 ± 233.2	154.4 ± 212.6	−3.5
Rectum	Dmean	28.0 ± 3.6	25.5 ± 3.7	−8.7
Rectum	V60%	30.2 ± 9.6	28.2 ± 9.9	−6.5
Rectum	V40%	44.4 ± 14.8	39.0 ± 13.5	−12.2
Sigmoid	Dmean	36.7 ± 4.0	31.6 ± 2.3	−13.9
Sigmoid	V80%	20.2 ± 17.4	15.6 ± 12.0	−23.2
Sigmoid	V60%	39.4 ± 24.0	29.0 ± 22.0	−26.4
Sigmoid	V40%	44.1 ± 26.0	41.3 ± 23.9	−6.5
Small bowel	Dmean	16.7 ± 5.8	13.0 ± 5.0	−22.3
Small bowel	V40%	560.3 ± 284.1	348.7 ± 238.2	−37.8
Small bowel	V20%	830.2 ± 30.3	742.3 ± 308.8	−10.6
Femur (right)	D2cc	28.1 ± 4.8	25.4 ± 3.5	−9.6
Femur (right)	Dmean	15.3 ± 7.5	14.7 ± 3.0	−4.1
Femur (left)	D2cc	28.1 ± 3.7	25.6 ± 3.9	−8.9
Femur (left)	Dmean	16.2 ± 3.8	14.0 ± 2.7	−13.7

Abbreviation: AP, auto-planning; HI, homogeneity index; PTV, planning target volume; ROI, region of interest.

aAll Dx values are given in units of Gy. All Vx(%) values are given in units of absolute volume—cubic centimeters. The table lists the ROI, significant dosimetric indices, median values, and percent change for the median value of the dosimetric compared between the clinical and respective auto-planning.

Table 4.

Statistically Significant Metrics (P < .05) for Brain Plans.a

ROI	Dosimetric Index	Median ± σ
		Clinical	AP	%Change
PTV_6000	CN	0.86 ± 0.06)	0.92 ± 0.06	7.2
Brain	Dmean	41.8 ± 9.0	38.6 ± 8.6	−7.7
Brain stem	Dmax	59.5 ± 8.9	62.0 ± 12.0	4.2
Brain stem	Dmean	46.1 ± 16.2	40.2 ± 17.3	−12.8
Optic chiasm	Dmax	55.1 ± 12.3	42.0 ± 17.1	−23.7
Optic chiasm	Dmean	50.7 ± 13.2	32.4 ± 15.6	−36.2
Right optic Nerve	Dmax	52.3 ± 18.0	22.4 ± 21.1	−57.2
Right Optic Nerve	Dmean	28.0 ± 15.6	18.6 ± 16.6	−33.6
Left Optic Nerve	Dmax	40.2 ± 14.7	24.4 ± 14.8	−39.3
Left Optic Nerve	Dmean	23.5 ± 13.2	18.4 ± 12.4	−21.5
Right Cochlea	Dmax	31.2 ± 26.3	14.0 ± 22.5	−55.1
Right Cochlea	Dmean	28.0 ± 26.8	31.2 ± 21.2	11.3
Left Cochlea	Dmax	26.2 ± 19.1	13.9 ± 19.2	−46.7
Left Cochlea	Dmean	23.7 ± 18.8	12.0 ± 18.0	−49.5
Spinal Cord	Dmax	3.3 ± 11.7	2.8 ± 9.2	−14.1
Left Eye	Dmax	21.2 ± 11.5	17.0 ± 9.7	−19.8

Abbreviations: AP, auto-planning; CN, conformity number; PTV, planning target volume; ROI, region of interest.

aAll Dx values are given in units of Gy. The table lists the ROI, significant dosimetric indices, median values, and percent change for the median value of the dosimetric compared between the clinical and respective auto-planning.

Figure 1A, B, and C illustrates a representative comparison of isodose lines between the clinical and AP plan for a representative intact prostate, prostate + lymph nodes, and brain patient, respectively. Figure 2A, B, and C shows the DVH for a representative intact prostate, prostate + lymph nodes, and brain patient, respectively.

Figure 1.

Representative isodose comparison between clinical and replanned auto-planning (AP) for (A) prostate and seminal vesicles, (B) prostate and lymph nodes, and (C) brain.

Figure 2.

Representative normalized, cumulative dose–volume histogram (DVH) comparison between clinical and replanned auto-planning (AP) for (A) prostate and seminal vesicles, (B) prostate and lymph nodes, and (C) brain.

For the intact prostate plans, PTV dosimetric index analysis showed significant changes in HI, CN, and R30. As the HI and CN approach 1, the PTV dose homogeneity and conformity are improved. Our results indicated that AP showed a slight, but statistically significant decrease in dose homogeneity (1.2%), while showing a slight but significant increase in conformity (1.1%). In the OAR analysis, statistically significant reductions were seen in all OAR Dmean values, ranging from as much as a 23% reduction (on median) in the penile bulb to 6.8% in the bladder. Figure 1A shows that the 50% isodose for the intact prostate AP is reduced from the clinical plan, which supports the reduction in Dmean values. Figure 2A shows a representative clinical plan in which 95% of the bladder received 20 Gy, while only 55% of the bladder in the AP received 20 Gy. In addition to the reduction in Dmean for all OARs, there were significant decreases in the V65Gy and V50Gy for the rectum and the D2cc for the sigmoid and right femur. There were also significant increases in the D2cc and V80Gy for the bladder and D2cc for the rectum.

For the prostate + lymph nodes plans, PTV dosimetric index analysis showed only a statistically significant increase in homogeneity (1.6%) when using AP. All the OAR Dmean values were significantly improved with the AP plan. The OARs demonstrated a decrease in Dmean values ranging from 4.1% (on median) in the right femoral head to 22.3% in the small bowel. All the significant results for the OAR were reductions. There were significant decreases in the V40% and V20% for the bladder and small bowel; V60% and V40% for the rectum; V80%, V60%, and V40% for the sigmoid; and D2cc for the right and left femoral heads. Figure 1B demonstrates sparing of the rectum as well as improved homogeneity on the PTV as compared to the clinical plan.

For the brain plans, analysis showed a significant improvement in CN (7.6%) when using AP. Figure 1C demonstrates improved conformity in the PTV. The OAR analysis showed improvement in Dmean for all OARs except for the right cochlea. There was improvement in the Dmax of the optic chiasm, right/left optic nerve, right/left cochlea, spinal cord, and left eye, while there was an increase in Dmax for the brain stem. Some of the largest percentage changes in dosimetric indices were realized in the brain plans and Figure 2C shows, in a representative clinical plan, these dramatic decreases.

Discussion

The results of the study show that, for the 3 sites evaluated, AP was capable of obtaining similar, or in some cases, exceeding the, plan quality of the manual plan. Our results offered similar findings to other research using AP. Nelms et al performed a comparison between AP and manually planned prostate cases.[3] The authors retrospectively replanned 23 prostate cancer cases without lymph node irradiation and compared various dosimetric indices with the clinically delivered plan. In their study, there was a significant difference in HI and CN between the 2 planning methods. Auto-planning showed an increase in HI, indicating the AP was worse. In our study, we saw the same trend with our HI increasing using AP. Their results also showed a reduction in CN as did this study. Additionally, they found significant reduction in the V40, V65, and V70 of the bladder and the maximum femur dose. We found significant increase in the bladder V80% and the D2cc, but our study showed significant decreases in the Dmean for the bladder as well as lower femur max and mean doses.

In another AP evaluation study, Gintz et al performed a comparison between AP and manually planned head and neck cases.[13] They concluded that dose homogeneity scores were better for manual plans over the AP plans, while the trend was significantly reversed for OAR sparing.[13] Hansen et al also performed a comparison between AP and manually planned head and neck cases[12] and showed that target dose coverage was consistent between the two, while OAR sparing was significantly improved through the use of AP.[14] Wang et al performed a retrospective study for whole-brain palliative radiotherapy using AP to attempt to avoid the hippocampus and found that AP was capable of producing plans that met or exceeded the plan quality of manual planning while reducing time spent by the planning team.[8] Our brain AP analysis tracked well with these results as we saw significant reduction in OAR doses, while noting a 7.6% increase in PTV conformity. One caveat to the significant reduction in Dmean values for the brain cases in our study is that the clinical plans all met the maximum dose constraints but were not originally pushed to reduce the Dmean values as far as possible. So, although AP most likely was able to push the Dmean values lower to some extent than the clinical plans, the results may be overstated in this study.

Based on the results of this study as well as those of other similar studies, Pinnacle3’s AP appears to improve the mean dose to OARs while maintaining similar target coverage for various treatment sites. Because of this, there is an opportunity to use AP to improve efficiency and standardize the quality of treatment plans without sacrificing treatment plan quality. In most clinics, medical dosimetrists perform many iterations of a treatment plan as they work to meet physician requirements, dose limits, and tumor coverage. Extra iterations can be justified in the case of complex or rare cases; however, for cases where the beam geometry and constraints are similar, many of the initial repetitive aspects of setting up a treatment planning could be replaced by a template in AP. Once OARs and targets are segmented, AP templates can be run to generate treatment plans that require minimal user editing. Since the AP engine automatically creates its own optimization structures and objectives, it removes the need for the manual iterations commonly performed by the dosimetrist. This allows extra time for more time-consuming cases, or for that matter, extra time dedicated to tasks that could significantly improve treatment plans. Specifically, after AP is executed, the dosimetrist can further optimize the plan based on patient-specific criteria to finalize the treatment plan.

A limitation to this study was that the planning times using AP were not recorded, which would have aided in quantifying the time saving opportunity of AP; however, dosimetrist verbal feedback indicated reductions in overall planning time. Time was not tracked because the plans were run outside of working hours and often a person was not present when the plan finished running to record the completion time.

This new notion of initializing the optimization process with AP and further refining the optimization through dosimetrist input in the final stages has many advantages. In particular, by utilizing AP’s template solution to take care of repetitive tasks, it can be argued that one could cut down on human errors.[15] Another advantage is the standardization of plan quality among varying experience dosimetrists. Batumalai et al and Nelms et al showed that planner experience is a driving force in the quality of an intensity-modulated radiation therapy (IMRT) plan.[1,3] For this study, the clinical plans were originally planned by experienced planners (>5 years in the field), while the retrospective APs were completed by first-year dosimetry students. Given that AP can achieve the same or better quality and these were performed by less experienced planners, it would suggest that plan quality did not depend on planner experience when using AP. A high-quality template will ensure that an inexperienced planner does not waste time using constraints or objectives that will provide inferior plans. With AP being used to drive efficiency in the clinic, the time a patient waits from initial computed tomography scan to treatment could be sped up, ultimately improving patient care.

Conclusion

Comparison of AP to manually planned treatment plans for early and advanced stage prostate cancer as well as brain cancer demonstrated significant changes in OAR doses while offering minimal changes in PTV dosimetric indices. Specifically, AP was shown to be able to produce plans that delivered similar high dose conformity, PTV homogeneity, and dose falloff to the target, however offered significant reductions in median dose to OARs independent of treatment site. The results of this study reinforce results of similar AP studies that suggest that AP may be a valuable clinical tool to standardize plan quality and improve clinic efficiency using high-quality templates coupled with the AP engine.