Using multi-centre data to train and validate a knowledge-based model for planning radiotherapy of the head and neck.

Background and purpose: Knowledge-based radiotherapy planning models have been shown to reduce healthy tissue dose and optimisation times, with larger training databases delivering greater robustness. We propose a method of combining knowledge-based models from multiple centres to create a 'super-model' using their collective patient libraries, thereby increasing the breadth of training knowledge. Materials and methods: A head and neck super-model containing 207 patient datasets was created by merging the data libraries of three centres. Validation was performed on 30 independent datasets during which optimiser parameters were tuned to deliver the optimal set of model template objectives. The super-model was tested on a further 40 unseen patients from four radiotherapy centres, including one centre external to the training process. The generated plans were assessed using established plan evaluation criteria.
Results: The super-model generated plans that surpassed the dose objectives for all patients with single optimisations in an average time of 10 min. Healthy tissue sparing was significantly improved over manual planning, with dose reductions to parotid of 4.7 ± 2.1 Gy, spinal cord of 3.3 ± 0.9 Gy and brainstem of 2.9 ± 1.7 Gy. Target coverage met the established constraints but was marginally reduced compared with clinical plans. Conclusions: Three centres successfully merged patient libraries to create a super-model capable of generating plans that met plan evaluation criteria for head and neck patients with improvements in healthy tissue sparing. The findings indicate that the super-model could improve head and neck planning quality, efficiency and consistency across radiotherapy centres.

Introduction

Radiotherapy treatment planning methods are currently based on the skill of the planner and subjective trial and error optimisations. This means that plan quality can be inconsistent and potentially sub-optimal. As the complexity of treatment planning increases with the implementation of intensity-modulated radiation therapy (IMRT) and advances in machine capability, more focus can be placed on sparing organs at risk (OARs) to minimise toxicity, but this creates greater planning inconsistency between centres and planners due to variable protocols and techniques [1], [2]. With increasingly challenging cases, departments are at risk of spending too much time and resource to produce plans that are not necessarily optimal or consistent in quality [3]. Knowledge-based planning (KBP) models have successfully been implemented clinically in many centres for multiple treatment sites and shown to reduce optimisation time and OAR doses [1], [4], [5], [6], [7], [8], [9]. Additionally, consistent radiotherapy has been correlated with improved overall patient survival [10], [11], [12], and KBP has been shown to reduce treatment planning variability [13]. Studies have suggested that increasing the number of patients within the model library improves model performance [1] as it allows the model to be trained to account for a larger variety of patient geometries. Combining KBP models from different centres could therefore have the potential to improve the quality of radiotherapy treatments due to the larger dataset available for model training. Additionally, by combining models and distributing them internationally, a planning standard could be defined helping to improve international radiotherapy treatment consistency and quality.

This study investigates a platform, currently in prototype form, which allows multiple KBP models to be combined between centres to create a super-model using collective patient libraries, thus increasing the breadth of knowledge and statistics available to the model for training and therefore its applicability and potential success clinically. The aim of the merging platform is to make training models across centres faster, easier and more reproducible to enable the creation of super-models. These super-models will represent the knowledge and best practice of a consortium or network of centres and can be regularly updated based on expert plans to reflect current clinical practice. Sharing such models offers further opportunities, including standardisation of clinical trial planning protocols and quality assurance of treatment plans across centres. New, inexperienced centres could benefit from the expertise of others and gain confidence faster when commissioning new treatment sites. Additionally, the ability to combine data libraries could be particularly useful for rare cancers or techniques where individual centres may take a long time to accrue enough patients to train a model. This study aimed to generate and validate a head and neck super-model by combining the model libraries of three UK centres and training a KBP model on the merged dataset. Validation was performed on 10-patient evaluation groups at each of the three model-contributing centres, and model testing performed on 40 unseen independent patients from the same three centres plus an additional centre. Model performance was assessed using established plan-evaluation criteria detailed in Section 2.2. The overall aim was to assess the feasibility of creating a super-model using the merging platform and evaluate its clinical success in terms of standardisation, efficiency and quality in comparison with standard clinical planning techniques.

Materials and methods

Building the model

The super-model was built by merging the KBP models of three UK centres: University College London Hospital (UCLH), Northampton General Hospital (NGH) and Guy’s and St. Thomas’ NHS Foundation Trust (GSTT). These KBP models were created at each centre using RapidPlan, a commercially available knowledge-based planning application developed by Varian Medical Systems of Palo Alto CA. Each centre installed Varian’s Distributed RapidPlan Platform, currently in prototope form, as well as the Varian Learning Connector (VLC), which gives access to each centre’s exported RapidPlan model. These exported files include arc geometry, structure set and plan parameter information but no patient-identifiable information or CT data, so the VLC has access to anonymised data only, ensuring compliance with data protection and patient reporting regulations. The VLC connects to the Distributed RapidPlan Platform which can be accessed by each centre via a web portal. Each evaluating centre uploaded their model to the platform, and a master evaluator was assigned to perform model merges. The model merge is created by combining the data libraries of all three centres and re-training using this super-model database; the contribution of each centre is therefore weighted based on the size of its data library. In order to achieve successful merges, a structure template was created which defined the common planning target volume (PTV) and OARs to be trained by the super-model. The structure template also defined the optimiser objectives and normal tissue objective (NTO) settings for the super-model. Each centre matched its structures to the template; this process standardised the naming conventions between centres with differing structure labels. A triple-centre merge of 207 patients was performed comprising all head and neck patient subgroups: nasopharynx, larynx, oropharynx and hypopharynx. The patients selected for the library were all required to have primary PTV prescriptions of 65 Gy and elective PTV prescriptions of 60 Gy and/or 54 Gy. The merged super-model was then downloaded via the platform by each contributing centre, plus an additional non-contributing centre also with access to the Varian Learning Platform, and evaluated.

Plan evaluation criteria

To enable comparison of treatment plans generated in multiple ways, plan evaluation criteria were established to serve as assessment points for the quality of the dose distribution. The criteria were based on the recommendations of the International Commission on Radiation Units (ICRU 83), which states that the near-minimum (D98%), near-maximum (D2%) and median (D50%) absorbed doses should be reported for each clinical plan [14]. The mean and maximum (0.1 cc) dose to individual OARs were collected. The OARs selected for comparison were the spinal cord, cord planning organ at risk volume (PRV), brainstem, brainstem PRV and the parotids, as these organs featured in the vast majority of clinical plans from all contributing centres. The final parameter chosen for comparison was the number of Monitor Units (MU) required to deliver each plan, as this provided a measure of plan complexity.

Model validation

The model was validated on a total of 30 patients; 10 from each of the centres contributing to the model library. Multiple optimiser objective combinations were trialled, alongside different parotid structure definitions, various NTO settings and MU objectives before the final super-model structure template was defined. Multiple iterations of model objectives were tested, starting with fully automated PTV and OAR objectives to minimise the user input, and ending with fixed objectives for the PTVs to assess the impact on plan quality. Separating the parotids into left and right, ipsilateral and contralateral and combining them into a single structure were all tested as combinations in the model template to assess which contouring technique provided the best OAR-sparing effect when running the model. Additionally, various NTO and MU objectives were tested with the aim of optimising plan quality and complexity. Each validation iteration involved testing the model on 10 patients, comparing with the previous model validation stage, and consulting with consortium members to evaluate the best model to proceed with for testing and eventually sharing with other centres.

Model testing

The merged super-model was tested on a total of 40 patients; 10 each from four different centres. Three of the testing centres had contributed patients to training the model, although all patients used for testing were independent of the model library and selected randomly. The super-model was run with a single optimisation and no manual input and the plans were normalised to 100% in the high dose target mean. The time required to generate the dose-volume histogram (DVH) estimates and optimise each plan was noted and the established plan evaluation criteria were extracted and compared with the clinical plans. Paired sample t-tests were used to assess whether mean OAR differences between super-model plans and clinical plans were statistically significant (p-value0.05). Normality of the data was tested prior to the analysis with the Anderson–Darling test to ensure statistical validity. Plans were also visually assessed to compare plan quality.

Results

The optimal objectives for the optimiser were found to be fully automated for OARs but fixed for PTVs, as detailed in Table 1. The original model produced plans with high PTV54 D50% doses and therefore an additional V50% objective was added to keep the values within tolerance. The model with combined parotids was also found to produce the lowest parotid doses and therefore the parotid geometries were combined for the final template. The addition of an MU objective was found to be unnecessary so no MU objective was fixed in the optimiser template.

Table 1

A table outlining the optimiser objective template used in the merged Rapidplan super-model. Generated (Gen) objectives are fully automated line objectives created by the RapidPlan model.

Structure	Type	Vol (%)	Dose (Gy)	Priority
	Upper	0	67.0	100
PTV 65	Lower	100	65.0	100
	Lower	97	65.5	100
	Upper	0	62.0	100
PTV 60	Lower	100	60.5	100
	Lower	97	61.0	100
	Upper	0	57.0	90
PTV 54	Upper	50	54.0	80
	Lower	100	54	100
	Lower	97	55	100
Cord	Line	Gen	Gen	Gen
Cord PRV	Line	Gen	Gen	Gen
Brainstem	Line	Gen	Gen	Gen
Brainstem PRV	Line	Gen	Gen	Gen
Parotids	Line	Gen	Gen	Gen

A comparison of plan quality for the clinical plans and super-model RapidPlans at each centre, measured using established plan evaluation criteria, is shown in Table 2.

Table 2

A table outlining the mean (SD) clinical and super-model RapidPlan evaluation results for the 40 patients included in the testing group (10 each from four centres). Super-model results marked with a * are statistically significantly different from their clinical counterpart (p0.05).

Parameters		Clinical	Super-model
	D98%	62.5 ± 0.8	61.9 ± 0.7*
PTV65	D50%	65.1 ± 0.1	65.2 ± 0.6
	D2%	66.6 ± 0.4	66.8 ± 0.5*
	D98%	58.1 ± 0.6	57.6 ± 1.3
PTV60	D50%	60.3 ± 0.2	60.6 ± 0.5
	D2%	62.9 ± 0.9	62.4 ± 0.6
	D98%	52.0 ± 0.7	51.7 ± 0.7*
PTV54	D50%	54.3 ± 0.2	54.8 ± 0.3*
	D2%	61.9 ± 1.4	62.0 ± 1.3
Parotids	Mean	30.9 ± 9.6	26.2 ± 9.0*
Cord	0.1 cc	38.9 ± 4.3	37.8 ± 2.2*
	Mean	23.6 ± 4.7	20.3 ± 3.1*
Cord PRV	0.1 cc	42.7 ± 4.3	42.4 ± 3.0
	Mean	23.6 ± 4.6	20.4 ± 3.1*
Brainstem	0.1 cc	33.2 ± 14.1	31.7 ± 13.3*
	Mean	11.1 ± 9.5	8.2 ± 5.5*
Brainstem PRV	0.1 cc	36.3 ± 15.0	36.2 ± 13.5
	Mean	10.1 ± 6.4	9.1 ± 5.6*
MUs	Total	473 ± 70	520 ± 54*

The coverage of the PTVs was slightly better in the clinical plans, although the RapidPlans were still clinically acceptable in this respect and met the established dose constraint of 98% of the volume receiving 95% of the prescribed dose. The maximum (0.1 cc) values for the cord and brainstem were significantly lower on average across the evaluation group for the RapidPlans. The mean dose for every OAR fell significantly, most notably for the parotids with a mean dose-sparing of 4.7 Gy. The mean values of the cord, cord PRV, brainstem and brainstem PRV dose also reduced significantly across the evaluation groups. This dose-sparing effect is shown graphically in Fig. 1, where the mean OAR values are shown for each individual centre alongside the mean PTV differences. The average optimisation and calculation time for the evaluation group RapidPlans was 10.1 min. The monitor units increased significantly for the Rapidplans suggesting increased plan complexity.

Fig. 1

Bar charts detailing the mean dose values achieved for the PTV65 D98%, PTV60 D98%, PTV54 D98%, parotids, cord and brainstem for clinical plans and super-model RapidPlans for the 10 independent patients tested in each centre.

Fig. 2 shows the mean DVH plots for the primary PTV and two of the modelled OARs at one of the centres to highlight the dose differences across the entire volumes. The plots were created by calculating the average dose (Gy) at each relative volume point for the OARs for each of the 10 evaluated plans. The DVH graphs display a statistically significant reduction in dose across the cord and parotid volumes for the super-model RapidPlans (displayed in red) compared with the clinical plans (displayed in blue). They also show minimal changes between the primary PTV dose recorded for the RapidPlans and clinical plans, with the RapidPlan distributions being marginally less homogeneous.

Fig. 2

Mean DVH plots for the primary PTV, cord and parotids for clinical and super-model RapidPlans generated for 10 patients at UCLH.

Individual plans were compared visually to assess the differences between the clinical plans and super-model RapidPlans; an example is shown below. Fig. 3 shows a comparison of a clinical plan and generated merged RapidPlan for an oropharynx patient. It highlights one of the key differences between the clinical and RapidPlan generated plans that was a significant finding across all of the tested patients; better parotid sparing in the RapidPlan. The dose gradient is much steeper at the parotid-PTV interface which has reduced the parotid mean dose significantly from 34.3 Gy to 22.0 Gy. However, this has caused the PTV65Gy D98% coverage to reduce by 0.2 Gy in the RapidPlan, although it still meets the established objective.

Fig. 3

Left: A transversal slice of a clinical plan for an oropharynx patient. Right: The same slice for the merged RapidPlan generated for the same patient. The isodose colourwash is set to 95% of the PTV54Gy (i.e. 51.3 Gy) to show the difference in dose drop-off at the PTV-parotid interface. The primary PTV (65 Gy), elective PTV (54 Gy), cord, cord PRV, left and right parotid are contoured in red, light green, yellow, pink, light blue and green respectively.

Discussion

A head and neck super-model created by merging the RapidPlan data libraries of three centres was able to produce clinically acceptable plans in terms of established evaluation criteria for a total of 40 patients from four different centres, each contributing 10-patient evaluation groups. One of the testing centres did not contribute to the model building and training process, demonstrating the success of the model in an independent setting. The PTV coverage in the super-model plans was slightly lower than the clinical plans, although still met the clinical objectives, and the mean OAR doses were statistically significantly lower. It is a patient-specific decision by the clinician as to whether more PTV coverage is desirable or whether OAR sparing is more important in particular regions of the patient’s geometry. It is therefore difficult to state which is a better quality plan for each patient as they are both clinically acceptable in terms of statistics but one may be preferable to another for different clinicians. Determination of which plan is clinically better would be by clinicians; this will be the subject of further clinical trials.

With NHS England introducing a new 17-day treatment pathway for category one radiotherapy patients [15], it is becoming increasingly important to reduce the planning time for head and neck treatments. The super-model plans were generated in single optimisations with an average optimisation and planning time of 10 min. No clinical timing data was available, but a study which aimed to evaluate the machine and operator time required to treat head and neck patients identified the average treatment planning time to be 3 h and 8 min [16]. This figure includes the pre-optimisation time required for PRV and optimisation contouring, beam placement and isocentre placement and so cannot be directly compared with the super-model optimisation time. However, even with a generous estimate of a pre-optimisation time of 1 h, using the super-model with a single optimisation still reduces the planning time by roughly two hours per patient. This is very significant for departments with busy workloads and addresses one of the challenges in meeting new more stringent NHS treatment pathways whilst simultaneously improving the consistency of plan quality.

Methods to boost the PTV coverage whilst maintaining good OAR sparing have begun, with testing including varying the NTO settings in the optimiser, and introducing a PTV boost volume and re-optimising the plan with a boost priority value. Both of these methods have been shown to improve PTV coverage, although not without consequence. Relaxing the NTO objective improves coverage but increases the normal tissue dose and boosting the PTV volume increases the MUs and the overall planning time. The super-model already produces plans with higher MUs, most likely due to the higher complexity needed to achieve significantly lower OAR doses, so any further increase would need to be considered with care. The model could be manipulated individually by each centre to achieve the clinical goals of each clinician, whilst still providing a standardised and good quality plan template with the initial optimisation. An MU objective could also be introduced on a centre-by-centre basis depending on specific QA requirements and pass criteria.

Concerning the patient data input into the model, 207 patients were used with a variety of plan sites including: oropharynx, hypopharynx, larynx and nasopharynx. Due to the rare nature of nasopharyngeal cancers, only 23 nasopharynx plans were available for use. Therefore the model data was limited for this type of tumour, reducing the accuracy of the model’s DVH prediction ability. Furthermore, due to the varying protocols and techniques used by different centres, not all of the OARs were present in each plan for all centres and consequently the model was unable to train a number of additional reported head and neck OARs. Only the OARs which were in use at all centres for all patients were included in the model and so, to achieve local objectives/tolerances, it may be necessary for individual centres to add objectives to the optimisation template to drive optimisation of dose to any additional structures of interest. With a larger data set of nasopharynx patient plans and more standardised contouring protocols, additional OARs could be trained, widening the model’s clinical use and reducing the need for manual adjustments at each centre. However, it could be beneficial for the model to remain in its primary format with fewer OARs since planning and reporting techniques vary so significantly between centres. This allows it to be a useful standard and potential building block for a larger group of patients.

To the best of the author’s knowledge, the prototype merging platform used in this study is the first of its kind in that it allows Rapidplan models to be combined without transferring and sending patient data between centres; it simply requires each centre to connect and upload its model to a portal. This ensures data protection laws are complied with and approvals are not necessary each time a new model is compiled. Another successful method to create a multi-centre model was investigated by Panettieri et al. [17] for prostate IMRT, but this required sending anonymised patient data between centres and then re-building and training the model from scratch, which lengthens the merging process and may be subject to a wider range of legal or regulatory restrictions. The accessibility of the merging platform opens the opportunities of the super-model to a wide set of users and allows centres with minimal experience or lack of data to use high-quality and well-tested super-models. Multi-centre validation of models has been common practice for a while [18], [19], [20], and has shown that combining experience between centres can improve model performance and standardisation of radiotherapy [10], [11], [12]. Therefore, the ease with which multiple centres can contribute to and influence models plays an important role in improving radiotherapy treatment.

This study investigates the head and neck region but this method of merging models could be applied to multiple treatment sites, as demonstrated by Panettieri et al. [17] for the prostate, and prescriptions and help to standardise a wide range of radiotherapy treatments. One of the advantages of the platform is that it is very quick and simple to re-merge and therefore update super-models, meaning they can be continuously improved and re-distributed in line with changes in clinical practice and changing prescriptions. Rapidplan has been shown to adapt well to ranging prescriptions [21], when given an appropriate data library, which is more feasible with larger numbers of centres able to contribute data via the merging portal.

Additionally, merging models could be useful tool for clinical trials where standardisation of care is pivotal, and also for quality assurance (QA) purposes where the model could be used to ensure a predetermined planning quality is being maintained. Previous studies have used single institution KBP models as QA tools for multi-institutional clinical trials and found plan quality to improve with the process [22], [23]. Super-models and distributing platforms could enable QA to happen on a larger scale.

The merging platform allowed three centres to successfully merge their patient libraries to create a super-model capable of generating plans that met the plan evaluation criteria for head and neck patients with improvements in OAR sparing. The super-model reduced planning time and provided a solution for improving the consistency of the head and neck planning process. The primary indications are that the super-model will help to improve head and neck planning efficiency and consistency across centres.

Declaration of Competing Interest

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Two of the authors are employees of Varian Medical Systems – a Healthineers company.