Evaluating the utility of knowledge-based planning for clinical trials using the TROG 08.03 post prostatectomy radiation therapy planning data.

Background and purpose: Poor quality radiotherapy can detrimentally affect outcomes in clinical trials. Our purpose was to explore the potential of knowledge-based planning (KBP) for quality assurance (QA) in clinical trials. Materials and methods: Using 30 in-house post-prostatectomy radiation treatment (PPRT) plans, an iterative KBP model was created according to the multicentre clinical trial protocol, delivering 64 Gy in 32 fractions. KBP was used to replan 137 plans. The KB (knowledge based) plans were evaluated for their ability to fulfil the trial constraints and were compared against their corresponding original treatment plans (OTP). A second analysis between only the 72 inversely planned OTPs (IP-OTPs) and their corresponding KB plans was performed.
Results: All dose constraints were met in 100% of KB plans versus 69% of OTPs. KB plans demonstrated significantly less variation in PTV coverage (Mean dose range: KB plans 64.1 Gy-65.1 Gy vs OTP 63.1 Gy-67.3 Gy, p < 0.01). KBP resulted in significantly lower doses to OARs. Rectal V60Gy and V40Gy were 17.7% vs 27.7% (p < 0.01) and 40.5% vs 53.9% (p < 0.01) for KB plans and OTP respectively. Left femoral head (FH) V45Gy and V35Gy were 0.4% vs 7.4% (p < 0.01) and 7.9% vs 34.9% (p < 0.01) respectively. In the second analysis plan improvements were maintained. Conclusions: KBP created high quality PPRT plans using the data from a multicentre clinical trial in a single optimisation. It is a powerful tool for utilisation in clinical trials for patient specific QA, to reduce dose to surrounding OARs and variations in plan quality which could impact on clinical trial outcomes.

Introduction

During clinical trials, poor quality radiation therapy (RT) may detrimentally affect the anticipated benefits of an intervention, impacting on clinical trial results [1], [2], [3]. Deviations in tumour volume delineation and inadequate planning can compromise local control and overall survival [1]. A second analysis of the RTOG 0126 prostate cancer trial identified a substantial number of patients at risk of rectal toxicity due to suboptimal plans [4]. A critical component of multicentre trials in radiation oncology is quality assurance (QA) to ensure that participating institutions are delivering consistent doses to the target volumes (TV) and adequately sparing organs at risk (OARs). Deviations from trial protocols have been documented to reduce efficacy and increase normal tissue complication rates [2], [3], [5]. However, the QA process can be resource intensive, yet generic and passing trial QA does not necessarily indicate that the plan is the optimal plan for the patient.

Automated planning is a tool which aims to achieve plan consistency and improved plan quality as well as to increase efficiency in a RT department [6]. It has a further role in performing patient specific QA of treatment plans [7], [8], [9], [10]. A Knowledge-based planning (KBP) model is trained using a library of high quality plans. By correlating geometric features of the plans included in the model with the OAR doses achieved, the model is able to rapidly produce estimated dose volume histogram (DVH) curves for the individual patient based on the OARs and PTVs of a delineated CT scan [11], [12]. Treatment plans are created using optimisation objectives obtained from the predicted DVHs. The OAR doses achieved in the KB plans strongly correlate with the OAR doses predicted by the model [7], [12].

There have been an increasing number of publications reviewing the role of KBP to improve the plan consistency, the efficiency of planning and for treatment plan QA [9], [13], [14], [15], [16], [17], [18], [19], [20], [21]. The TROG 08.03 clinical trial dataset provided an opportunity where plans, created in multiple treatment centres and having undergone centralised QA could be replanned using a KBP model and compared to the original treatment plan (OTP). Our hypothesis was that KBP would produce plans, with less variation in target coverage and similar or improved OAR doses than the original treatment plans in a single optimisation. The purpose of this study was to explore the potential of KBP as a powerful and efficient tool in clinical trials for QA and planning purposes.

Materials and methods

Trans-Tasman Radiation Oncology Group (TROG) 08.03 RAVES study is a recently completed multicentre clinical trial comparing the role of adjuvant or early salvage post-prostatectomy radiation therapy (PPRT) [22], [23] This secondary analysis performed using the TROG 08.03 trial dataset was approved by the TROG cancer research scientific committee and ethics. OTPs and subsequent plans created with KBP were compared against the trial protocol for compliance in target coverage, OAR doses and violations.

TROG 08.03 trial protocol

All patients were contoured according to PPRT consensus guidelines as specified by the trial protocol [24]. The prescribed dose was 64 Gy in 32 fractions to the prostate bed. The trial specified centres had to meet minimum TPS requirements. For patients planned with 3D conformal radiation therapy (3DCRT), the dose was prescribed to the ICRU 50 reference point, at the centre of the PTV [25]. For patients planned using an Intensity Modulated Radiation Therapy (IMRT) technique, the dose was prescribed to a volume so that 98% of the PTV received at least 95% of the prescribed dose. The trial protocol only specified constraints for the PTV, rectum and left femur. There were no constraints for Bladder and right femoral head. Table of dose constraints is available in Supplementary Table 1.

Original TROG 08.03 plan QA

Patients included in the trial were treated by 46 clinicians at 32 different hospitals [26]. All clinicians and sites had to submit a credentialing dummy run prior to recruitment as part of QA. For every patient, TV and the treatment plans were reviewed by an independent QA radiation oncologist before the start of treatment. Any prospectively identified major protocol violations or a sum of minor violations required correction and resubmission [26]. During real time review, data, including doses to targets and OARs as well as violations, were stored on the trial evaluation form. If a resubmission was required, QA was again performed and a second trial evaluation form was completed. The OTP cohort consisted of 65 patients planned with 3DCRT, and 72 that had been inversely planned, 67 with IMRT and 5 with volumetric modulated arc therapy (VMAT). In the IMRT plans, the median number of fields was 7, and the range was 5 to 9.

KBP model creation

RapidPlan (RP) (Varian Medical Systems, Palo, Alto, CA) is a KBP tool which was used for this study [6]. RapidPlan software version 15.6 was used to create the model. The TROG 08.03 PPRT KBP model was an ‘iterative’ model, created using 30 in house PPRT plans from the Northern Sydney Cancer Centre database that had been contoured according to the PPRT consensus guidelines [24], [27]. Thirty 7-field IMRT PPRT plans using 6MV energy were generated by the department expert prostate planners and repeatedly optimised to achieve the clinical objectives detailed in the trial protocol. All plans were reviewed by radiation oncologists. These 30 plans were then included in the development of the iterative IMRT PPRT KBP model. The model was re-trained initially using outputted PPRT IMRT plans as input for the next iteration of the model [27], [28]. Included plans were checked for outliers and none were found. A combination of point values, lines, generated and fixed were used, as demonstrated in Supplementary table 2. No trial patients were included in the development of the KBP model. The model was adapted to achieve TROG 08.03 target dose coverage but with the goal of delivering the lowest possible doses to the OARs. Prior to use, the model was validated using 20 independent patients, ensuring that the plans produced met the trial protocol dose constraints.

Replanning using the TROG 08.03 KBP model

PPRT was delivered to 238 of the 333 patients enrolled on the trial [23]. After central de-identification, 169 patient datasets comprising planning CT scans with original contours and 160 trial evaluation forms were received from TROG. The final cohort of 137 patients comprised only those patients with both a CT dataset and a trial evaluation form. Using the KBP model, with no manual intervention, a new plan was created for each of the patients in the cohort. The plans were created using a 7-field IMRT technique and 6MV photons. All plans were normalised according to the original trial protocol. Only a single optimisation was allowed. The treatment planning time was defined as the total time measured from the start of the optimisation to the end of calculation of the treatment plan.

Evaluation of KBP model performance

In this report, plans created using the KBP model are denoted knowledge-based plans (KB plans) and the original treatment plans on which patients were treated in the trial are denoted OTPs. The KB plans were evaluated for their ability to meet the trial protocol objectives and were compared against the OTPs with respect to the target dose coverage and OAR sparing according to the trial dosimetric parameters (Supplementary Table 1). Protocol violations were recorded and compared with OTP violations. The initial analysis included all 137 patients in the cohort, irrespective of the treatment technique (3DCRT or IMRT/VMAT) used. A second analysis was performed between the 72 inversely-planned OTPs (IP-OTPs) and their corresponding KB plans (IP-KBPs).

Statistical analysis

Statistical analysis was performed to compare the different dosimetric parameters of OTP and KB plans. Welch’s t-tests was used to compare means of the 17 parameters between OTP and KB plans. F-tests were used to compare the variances of the two planning approaches. All analyses were repeated for the IP-OTP. No adjustments have been made for multiplicity. All statistical analysis was performed with R version 3.6.3.

Review of dose constraints

Based on the doses to OARs in the KB plans, achievable dose constraints were created. Suggested dose constraints were based on the 90th percentile and the minor violations on the 99th percentile, with major violations being outside of the 99th percentile.

Results

Each KB plan took less than 5 minutes to complete.

Violations and resubmissions

As demonstrated in Table 1, approximately 70% of OTPs met all dose constraints, with 42 plans having either a minor or major violation or both. There were 7 major and 54 minor violations recorded. The most frequent minor violations occurred in the PTV median and mean dose and rectal V40Gy (%). Resubmission due to violations was required in approximately 20% of the cases, with resubmission occurring more frequently in the 3DCRT groups (approximately 32%). The number of recorded violations and resubmissions did not necessarily correspond as some plans requiring resubmission had multiple violations and other plans with only minor violations did not require resubmission. In the KB plans, all dose constraints were met in 100% of plans, with no major or minor violations recorded (Table 1).

Table 1

Table of minor and major violations according to TROG 08.03 protocol, recorded for the full cohort OTP (orange), the OTP separated according to planning (3DCRT vs inverse planning) (green) and the KB plans (blue). Number of resubmissions are also reported. (Abbreviations: OTP: Original treatment plan, 3D CRT: 3D conformal radiation therapy, KB: Knowledge-based).

Comparison of full OTP cohort with KB plans

Mean doses and standard deviations for dosimetric parameters according to the protocol are represented in Table 2. There was significantly less variation in PTV coverage in the KB plans (p < 0.01). This is visually demonstrated in Fig. 1. KBP resulted in significantly lower doses to OARs. Rectal V60Gy and V40Gy were 17.7% vs 27.7% (p < 0.01) and 40.5% vs 53.9% (p < 0.01) for KB plans and OTP respectively (Fig. 2). Mean rectal dose was also significantly lower (38.3 Gy vs 42.4 Gy, p < 0.01). Of note, the rectal max dose was significantly higher in the KB plans (66.5 Gy vs 64.5 Gy, p < 0.01). In dosimetric parameters to the left femoral head, KBP resulted in significantly lower doses as well as significantly reduced variation in dose (Fig. 3). Of note is the reduction in the range of FH volume receiving 35 Gy, from 0 to 91% in OTP plans down to 0–20% in KB plans.

Table 2

Comparison of mean and standard deviations of trial dosimetric parameters between all the 137 OTPs and their corresponding KBPs. The table also includes the secondary comparison of the mean and standard deviations of the 72 IP-OTPs and their corresponding IP-KBPs. (Abbreviations: OTP: Original treatment plan, KBP: Knowledge-based plan, IP-OTP: Inversely planned Original treatment plan, IP-KBP: Inversely planned Knowledge-based plan).

Structure	Parameter	OTPMean (±SD)N = 137	KBPMean (±SD)N = 137	P value (comparing means)	P value (comparing variances)	IP-OTPMean (±SD)N = 72	IP-KBPMean (±SD)N = 72	P value (comparing means)	P value (comparing variances)
PTV	Mean dose (Gy)	64.5 (±0.7)	64.5 (±0.2)	0.66	<0.01	64.7 (±0.8)	64.5 (±0.2)	0.23	<0.01
	Median dose D50 (Gy)	64.7 (±0.8)	64.7 (±0.3)	0.8	<0.01	64.9 (±0.9)	64.7 (±0.2)	0.08	<0.01
	Max dose D2% (Gy)	66.4 (±1)	67.4 (±0.3)	<0.01	<0.01	66.8 (±1)	67.4 (±0.3)	<0.01	<0.01
	Min dose D98% (Gy)	61.3 (±2)	60.8 (±0.1)	0.01	<0.01	61.4 (±1)	60.8 (±0.1)	<0.01	<0.01
	Percentage covered by 60.8 Gy (V95) %	99.2 (±2)	98.1 (±0.1)	<0.01	<0.01	98.6 (±1)	98.1 (±0.1)
CTV	Mean dose (Gy)	64.8 (±2.4)	65.5 (±0.2)	0	<0.01	65.4 (±0.8)	65.5 (±0.2)	0.93	<0.01
	Max dose D2% (Gy)	66.4 (±1)	67.4 (±0.3)	<0.01	<0.01	66.8 (±1.1)	67.5 (±0.3)	<0.01	<0.01
	Min dose D98% (Gy)	63.8 (±1.3)	64.1 (±0.1)	<0.01	<0.01	64.1 (±0.9)	64.1 (±0.1)	0.84	<0.01
Rectum	V60Gy (%)	27.7 (±6.9)	17.7 (±5.6)	<0.01	0.01	24.7 (±6.4)	19.6 (±5.5)	<0.01	0.22
	V40Gy (%)	53.9 (±7.9)	40.5 (±7.7)	<0.01	0.74	51.7 (±9.0)	42.8 (±8.2)	<0.01	0.4
	Mean rectal dose (Gy)	42.4 (±4.3)	38.3 (±3.9)	<0.01	0.27	40.8 (±4.4)	39 (±4.3)	<0.01	0.81
	Max rectal dose D2% (Gy)	64.5 (±3.6)	66.5 (±0.9)	<0.01	<0.01	64.8 (±4)	66.8 (±0.8)	<0.01	<0.01
Left femur	V35Gy (%)	34.9 (±31)	7.9 (±4.8)	<0.01	<0.01	9.3 (±7.5)	8.8 (±5)	0.62	<0.01
	V45Gy (%)	7.4 (±12)	0.4 (±0.8)	<0.01	<0.01	0.8 (±1.5)	0.6 (±0.9)	0.32	<0.01
	V60Gy (%)	0.1 (±0.9)	0.0	0.30	<0.01	0 (0)	0 (0)	0.38	<0.01
	Mean FH dose (Gy)	28.4 (±7.3)	19.9 (±2.5)	<0.01	<0.01	22.9 (±4.4)	20.5 (±2.6)	<0.01	<0.01
	Max FH dose D2% (Gy)	44.1 (±7.5)	39.6 (±3.6)	<0.01	<0.01	39.9 (±6)	40.5 (±3.7)	0.4	<0.01

Fig. 1

Box and whisker plot demonstrating range of doses (max, min, mean and median) to PTV achieved in OTP and KBP plans. (Abbreviations: OTP: Original treatment plan, KBP: Knowledge-based plan).

Fig. 2

Box and whisker plot demonstrating range of doses according to TROG 08.03 protocol achieved to Rectal OAR in OTP, KBP, IP-OTP and IP-KBP plans. (Abbreviations: OAR: Organ at Risk, OTP: Original treatment plan, KBP: Knowledge-based plan, IP-OTP: Inversely planned Original treatment plan, IP-KBP: Inversely planned Knowledge-based plan).

Fig. 3

Box and whisker plot demonstrating range of doses according to TROG 08.03 protocol achieved to femoral head OAR in OTP, KBP, IP-OTP and IP-KBP plans. (Abbreviations: OAR: Organ at Risk, OTP: Original treatment plan, KBP: Knowledge-based plan, IP-OTP: Inversely planned Original treatment plan, IP-KBP: Inversely planned Knowledge-based plan).

Secondary comparison of the IP-OTP cohort and their corresponding KB plans (IP-KBP)

As demonstrated in table 2, plan improvements were maintained in the IP-KBP group, with significantly less variation in PTV target coverage (p < 0.01) and lower doses to OARs. Rectal V60Gy, V40Gy and mean dose were significantly lower (p < 0.01), although again the rectal max dose was significantly higher in the IP-KB plans (66.8 Gy vs 64.8 Gy) (Fig. 2). The volume percent of the FH receiving each dose was similar between the IP-OTP and IP-KBP, however doses in the KB plans were more consistent with less dosimetric outliers (Fig. 3).

Achievable dose constraints

As demonstrated in Table 3, the volume of rectum receiving 60 Gy was <25% in 90% of KB plans and <32% in 99% of KB plans. The volume of rectum receiving 40 Gy was <51% in 90% and <57% in 99% of KB plans. The volumes of left femur that received 35 Gy was <15% and 20% in 90% and 99% of KB plans respectively.

Table 3

Achievable dose constraints based on the 90th and 99th percentile in the KB plans.

Contour		Dose constraints	Minor violation	Major violation
Rectum	V60Gy	<25%	25 – 32%	> 32%
	V40Gy	< 51%	51 – 57%	> 57%
Left femur	V35Gy	< 15%	15 – 20%	> 20%
	V45Gy	< 1.5%	1.5–4%	> 4%

Discussion

The TROG 08.03 clinical trial data was used to investigate the role of KBP in QA and planning for clinical trials to determine if the quality of the radiation treatment plans could be improved. The KB plans, created in less than 5 minutes, achieved significantly less variation in target coverage as well as significantly lower doses to OARs. As expected, improvements were seen when the comparison was made with all OTPs due to the inclusion of plans created with older planning techniques. However significant improvements were retained when the comparison was made exclusively with IP-OTPs.

These finding were in keeping with a previous KBP prostate study which demonstrated homogeneity of target coverage and lower doses to the OARs [19]. Another study on the implementation of KBP in prostate cancer patients reported on a reduction in the average mean rectum dose by 5.6 Gy [21]. Our study used patients receiving PPRT and the reduction in mean rectal dose was 4.1 Gy for the full cohort and 1.2 Gy for the IP cohort. The smaller reduction may be due to the fact that the anterior rectal wall needs to be covered in the PPRT volumes, making dose reduction more challenging. One exception was the rectal max dose (D2%) which was significantly higher in the KB plans compared to both the OTPs and IP-OTPs. Rectal max dose was not a violation in the TROG 08.03 trial and hence the model was not optimised with this constraint. Subsequent KBP models could be further trained to reduce the rectal max dose.

The trial required all major violations to be addressed and plans resubmitted prior to commencement of treatment. The rate of resubmissions decreased as the study progressed, indicating both the presence of an institutional learning curve and the importance of providing timely feedback to clinicians. The most common dosimetric violation was failure to meet rectal DVH constraints followed by PTV DVH violations [26] but it was difficult for the QA team to know if violations were due to poor planning or challenging anatomy (eg small rectum adjacent to CTV). Consequently, the comprehensive QA process was labour intensive and was at times a barrier to trial accrual [26]. Based on our findings, one potential solution to the challenge of plan quality is using KBP in order to provide patient specific plan QA rather than generic QA, to ensure that only the highest quality plans are included in clinical trials. When a plan is submitted for trial QA, a KBP model can be used to predict DVHs for OARs. These patient specific OAR DVHs can be compared with the DVH from the submitted plan. If the predicted DVH is lower than the submitted DVH, the plan could be considered for resubmission, with the OAR DVH predictions being provided as a guide for input optimisation objectives during the replanning process. When KB models are used to predict achievable OAR DVHs, studies have shown strong correlation between predicted and achieved mean doses, indicating that KB can accurately predict achievable mean doses[7]. Tol et al reported that individualised QA could be performed in a few minutes on head and neck plans and frequently improved doses to OARs, even though the plans had met trial generic plan criteria [14]. A recent publication investigated the role of KBP for real time treatment plan review for stereotactic ablative body radiotherapy for kidney cancer. They were able to provide real time feedback for 77% of their cases with replan and improved OAR doses for 2 cases. All centres reported that the QA check for their treatment plan was useful, despite timeline challenges [20]. It must be acknowledged that not all planning systems are the same and therefore the predicted DVHs may not be achievable on a different planning systems, however they would provide a guide for plan improvement. If a plan is submitted which meets trial criteria but KBP indicates that it can be substantially improved, it would require clinical judgement by the investigator or trial guidelines to determine whether a resubmission is required.

Suggested dose constraints used in treatment plans and clinical trials are based on QUANTEC data. In 2010, updated values were made available from accumulation of 3D treatment planning data [29]. It has been reported that the volume of rectum receiving ≥ 60 Gy is consistently associated with the risk of Grade ≥ 2 rectal toxicity or rectal bleeding [30]. A systematic post-QUANTEC review of tolerance doses for late toxicity after prostate radiotherapy was published by Olsson et al, they reported on the importance of keeping doses at the lower boundary of the tolerance curve to reduce toxicity [31]. With technology such as KBP, it has been demonstrated that lower doses to surrounding OARs can be achieved without compromising dose to targets, and therefore OAR constraints should not necessarily be based on QUANTEC but instead on the lowest doses achievable to ensure the best quality plans. The rectal dose constraint on the TROG 08.03 trial was V60Gy < 40%, yet in 90% of KBP cases, the V60Gy to the rectum was<25% and in 99% of cases the dose was<32%. KBP is an effective tool to guide us to achieve lower OAR doses in plans which would otherwise have been considered to be acceptable.

There were several limitations to this study. Data for the full cohort of patients treated on the original trial was not available. While only the patients with the full dataset were used in our study, there is the potential for bias as the entire cohort was not included. All the plans created with KBP were inversely planned while many of the OTPs were created using 3DCRT. As the purpose of this study was to analyse the potential of KBP as a clinical trial QA and planning tool, rather than to directly compare the two planning techniques, the comparison was felt to be acceptable. In addition, a secondary comparison was made between only the IP-OTPs and their corresponding KB plans to allow for a fairer comparison. DVH curves from the original plans were not available, instead only the contours and dose points as recorded on the trial evaluation forms. While the same dose points were used to make a direct comparison, analysing the full DVH curves for both the OTP and KB plans would have further contributed to the comparison of the two treatment plans. The TROG 08.03 trial was established prior to 2008 when 3DCRT planning was used and dose constraints for the trial are reflective of this. The trial evaluation form only recorded data as specified on the protocol, specifically the Lt Femur, Rectum and PTV, as well as doses to the CTV. There was no data recorded on the dose to the Right femoral head or the bladder and therefore while we had this data from the new KB Plans, a comparison was not possible. To more accurately compare the OTP and the KB plans, information on dose to these additional OARs would provide a more comprehensive plan comparison to ensure dose was not being inappropriately delivered to these structures while sparing the specified OARs.

In our study, KBP was able to create treatment plans in less than 5 minutes with improved OAR sparing despite the OTPs meeting trial plan criteria and passing centralised QA. KBP is a powerful tool which should be utilised in clinical trials for patient specific QA, to reduce dose to surrounding OARs and to reduce the variations in plan quality which can have an impact on the outcome of clinical trials.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.