The ROCOCO performance scoring system translates dosimetric differences into clinically relevant endpoints: Comparing IMPT to VMAT in an example pilocytic astrocytoma dataset.

BACKGROUND AND
PURPOSE: Proton therapy is expected to outperform photon-based treatment regarding organs at risk (OAR) sparing but to date there is no method to practically measure clinical benefit. Here, we introduce the novel ROCOCO Performance Scoring System (RPSS) translating dose differences into clinically relevant endpoints and apply this to a treatment plan comparison of volumetric modulated arc therapy (VMAT) and intensity modulated proton therapy (IMPT) in 20 pilocytic astrocytoma patients.
MATERIAL AND METHODS: The RPSS was developed on the basis of expert-based weighting factors and toxicity scores per OAR. The imaging datasets of 20 pilocytic astrocytoma patients having undergone radiotherapy were included in this in silico dosimetric comparison trial as proof of principle. For each of these patients, treatment plans to a total dose of 54 Gy (RBE) were generated for VMAT and IMPT and these were compared regarding radiation dose to the clinical target volume (CTV) and OARs. The RPSS was calculated for each treatment plan comparing VMAT and IMPT.
RESULTS: In 40 analysed treatment plans, the average and low dose volumes to various OARs were significantly reduced when using IMPT compared to VMAT (p < 0.05). Using the RPSS, a significant difference between both treatment modalities was found, with 85% of the patients having a lower RPSS in favour of the IMPT plan.
CONCLUSION: There are dosimetric differences between IMPT and VMAT in pilocytic astrocytoma patients. In absence of clinically validated NTCP models we introduce the RPSS model in order to objectively compare treatment modalities by translating dosimetric differences in potential clinical differences.

Introduction

Proton therapy as radiation treatment modality has been introduced in the last decade and is currently available in many countries. It may have an advantage over photon therapy due to its physical properties which allows sparing of organs at risk (OARs) situated behind the tumour [1]. It is known that radiation dose to the brain in long term brain tumour survivors can lead to irreversible cognitive decline, directly affecting the patient’s quality of life [2]. Multiple studies have already shown a dosimetric advantage of proton therapy in this patient group [3], [4], [5], [6], [7]. These studies report significantly reduced low dose volumes using intensity modulated proton therapy (IMPT) compared to volumetric arc photon therapy (VMAT). However, the clinical benefit of these reduced low dose volumes remains unclear. To our knowledge, there is no system available to translate these dosimetric differences in clinical relevant endpoints.

Due to the recent availability of proton therapy in the Netherlands with a maximum of 2200 patients per year, the model based approach has been adopted to decide whether a patient is a candidate for proton therapy depending on the clinical relevance of the dose reduction to the OARs in a photon versus proton treatment plan comparison. A 10% reduction in grade 2 toxicity is required using a validated photon based normal tissue complication probability (NTCP) model in order to become eligible for proton therapy [8]. Unfortunately, there are currently no externally validated NTCP models available for central nervous system (CNS) OARs including those related to cognition. Therefore, in The Netherlands, a ≥ 5% dose benefit in the mean dose (Dmean) hippocampi and/or supratentorial Brain minus the clinical target volume (CTV) is currently adopted as indication for the use of proton therapy, resulting in reimbursement by the health insurance. Until now, no scoring system exists which incorporates all available OARs, resulting in a uniform selection tool until CNS NTCP models become available.

In this paper we introduce the Radiation Oncology Collaborative Comparison Group (ROCOCO) Performance Scoring System (RPSS), which is an assessment of a radiation treatment plan using weighted scores of differences in dose to OARS (Δdose). The purpose of the RPSS is to translate dose differences into a score which reflects expected overall toxicity burden. This score can help clinicians in deciding whether patients will benefit from particle therapy. In order to demonstrate the power of the RPSS, we assessed dosimetric differences and the associated RPSS scores of VMAT and IMPT plans in an example dataset consisting of 20 pilocytic astrocytoma patients.

Materials and methods

ROCOCO performance scoring system

In order to compare treatment plans on an individual patient level, we introduced the ROCOCO Performance Scoring system based on two components. First, a simplified NTCP table was defined, based upon the published European Particle Therapy Network dose constraints in neuro-oncology (Fig. 1) [9], [10]. A score was determined in consensus (LV, DE), ranging from 1 to 5, where 1 indicates that no toxicity is expected in this specific OAR in a dose range, while a score of 5 expresses that severe toxicity is to be expected. When different dose levels have the same number, no clinically relevant difference in toxicity is to be expected between these dose levels.

Fig. 1

Simplified normal tissue complication probability (NTCP) table based upon the published European Particle Therapy Network dose constraints in Neuro oncology, data in adults [9], [10].

Secondly, in order to translate dose to OARs into their expected clinical relevance we introduced a weighting factor for each OAR, based upon expert opinion (LV, DE, IC, ET), ranging from 0 to 1 (Fig. 1). A high weighting factor (0.7–1) indicates a highly clinically relevant OAR for expected late toxicities, while a low weighting factor (0–0.3) indicates a less clinically relevant OAR. Some OARs were assigned a weighting factor of 0. In these OARs dose limits are never exceeded thus not expected to induce toxicity.

Each OAR was then scored between 1 and 5 depending on the dose received in the VMAT and IMPT treatment plan. Next, the RPSS values for the IMPT and the VMAT treatment plans were determined by multiplying the NTCP scores, based upon the doses to the OARs, and the weighting factors for each OAR and summarising these per patient:with p = patient, N = number of OAR, WF = weight factor per OAR, NTCP = normal tissue complication per OAR

Study population

The population, on which the RPSS was assessed, consisted of 20 consecutive pilocytic astrocytoma patients treated with radiotherapy at one of three Dutch university hospitals, the Radboud University Medical Centre (Radboudumc), Nijmegen, the University Medical Center of Amsterdam (AMC), Amsterdam or the Department of Radiation Oncology of Maastricht University Medical Center (MAASTRO), Maastricht. Radiation therapy was either indicated as primary treatment, after resection or after chemotherapy. This ROCOCO in silico trial was approved by the MAASTRO clinic institutional review board.

Target volume and OAR definition

All patients had a planning CT prior to radiotherapy. For stabilisation, each patient received an individual head support and thermoplastic mask. The radiation oncologist of each centre delineated the target volumes using the planning CT after co-registration with the MRI (T1-weighted with contrast agent and T2-weighted/FLAIR images). The individual patient’s gross tumour volume (GTV) and clinical target volume (CTV) were identical for both treatment options (VMAT, IMPT). The margin for CTV was set at 5 mm for all patients independent of the original treatment protocol [11]. For VMAT planning, the CTV was expanded with a 2 mm margin for the planning target volume (PTV), accounting for linear accelerator-based setup errors chosen uniformly for all centres.

All OARs were outlined by one dosimetrist and supervised by a radiation oncologist in training and a radiation oncologist (LV and DE). The following OARs were identified and delineated: brain, brainstem, optic chiasm, retina, cochlea, optic nerves, spinal cord, hippocampi, cornea, lacrimal glands, pituitary gland, lens, supratentorial brain, cerebellum, and anterior and posterior cerebellum [1], [12]. A bilateral OAR (e.g. hippocampus, cochlea, cornea) was termed ‘contralateral’ when located outside the CTV and at the contralateral hemisphere. If the tumour was located centrally or both left and right OARs were situated within the CTV, the OAR was named ‘bilateral’. Severe dental scatter artefacts or metal dental implants were delineated and the density was overridden to that of teeth or tissue. Dental fillings within the treatment beam were an exclusion for IMPT.

Dose prescription and constraints

The prescribed dose to the target volume was 54 Gy (RBE) taking into account the factor 1,1 for relative biological effectiveness (RBE) in proton plans. The dose limits of the OARs were defined in the study protocol and are listed in Supplementary Table 1. A prioritisation was made as a guideline and defined in the study protocol. It was used for planning purposes and not for the weighting factors used to calculate the RPSS.

Table 1

Dose and coverage parameters per organ at risk or target volume for VMAT and IMPT.

Organs at risk	VMAT D2%	IMPT D2%
Brain	55.1 (4.6)	54.9 (1.3)
Brainstem	51.3 (16.3)	50.9 (17.0)
Optic chiasm	31.8 (23.1)	26.1 (27.4)
Cornea ipsi and bilateral	10.1 (8.0)	4.4 (7.1)*
Cornea contralateral	4.3 (3.8)	<0.01 (<0.01)*
Lens ipsi and bilateral	3.5 (2.9)	0.96 (1.9)*
Lens contralateral	1.9 (2.4)	<0.01 (<0.01)*
Optic nerve ipsi and bilateral	30.8 (23.9)	28.3 (26.6)
Optic nerve contralateral	8.2 (6.8)	0.065 (0.16)*
Retina ipsi and bilateral	12.2 (10.0)	7.5 (11.5)*
Retina contralateral	4.6 (4.2)	<0.01 (<0.01)*
Spinal cord	7.7 (17.0)	10.5 (18.5)
Organ at risk	VMAT D40%	IMPT D40%
Hippocampus ipsi and bilateral	17.6 (14.7)	11.4 (16.0)*
Hippocampus contralateral	5.6 (3.6)	1.0 (1.7)*
Organs at risk/target volume	VMAT Dmean	IMPT Dmean
Brain	11,2 (3.9)	7.9 (3.0)*
Cerebellum anterior – CTV	26.5 (11.2)	15.2 (15.0)*
Cerebellum posterior – CTV	17.6 (10.9)	10.7 (12.9)
Cochlea ipsi and bilateral	14.3 (9.6)	<0.01*
Cochlea contralateral	6.0 (6.0)	0.071 (0.065)*
Hippocampus ipsi and bilateral	17.2 (12.8)	11.1 (12.8)*
Hippocampus contralateral	5.7 (3.9)	1.7 (3.2)*
Lacrimal gland ipsi and bilateral	8.8 (7.5)	3.3 (5.8)*
Lacrimal gland contralateral	3.7 (3.7)	<0.01 (<0.01)*
Supratentorial brain – CTV	9.3 (4.3)	6.5 (3.8)*
Pituitary gland	26.5 (24.2)	23.4 (26.4)
Integral dose to the body	6.5 (2.5)	4.5 (2.1)*
CTV54Gy	57.2 (2.4)	55.6 (2.0)*
GTV	57.0 (2.5)	55.4 (1.9)*
CN	VMAT D95%	IMPT D95%
CTV54Gy	0,62 (0.068)	0,57 (0.11)*

D2%: dose to 2% of the volume reported in Gy(RBE); D40%: dose to 40% of the volume reported in Gy(RBE); Dmean: mean dose to organ at risk or target volume in Gy(RBE); D95%: dose to 95% of the volume reported in Gy(RBE) CTV54Gy: Volume of the CTV receiving 54 Gy; CN: conformity number. Numbers between parentheses are standard deviations. * numbers with asterisk are statistically significantly different (p < 0.05).

Photon and proton planning

The photon VMAT plans were made according to current clinical practice at Maastro clinic, using EclipseTM (v11.0 Varian Medical Systems, Palo Alto, CA). 99% of the PTV received at least 95% of the prescribed dose. Cold spots (<95%) and hot spots (>107%) were avoided and no hot spots were allowed outside the PTV.

The proton treatment plans were calculated at OncoRay (Dresden, Germany) using RayStation (v4.65.99, RaySearch Laboratories AB, Stockholm, Sweden). IMPT was used for beam delivery utilising Pencil Beam Scanning (PBS). No PTV concept was used in these plans. Instead, the plans were robust for target coverage of the CTV, considering 21 scenarios for the optimisation process, taking a setup uncertainty of 2 mm and range uncertainty of 3,5% into account. In order to spare the organs at risk and avoid passing through air cavities, beam directions (mostly two beams, sometimes three beams) were chosen individually for each single patient. The beam arrangements did not affect the margin.

Data evaluation and statistical analysis

For OARs and CTV in each of the VMAT and IMPT plans the mean dose (Dmean), maximum dose (Dmax) and near-maximum dose (D2% or D0.1cc, the highest dose to 2% or to 0.1 cc of the volume of interest, respectively) were calculated [13]. For the hippocampus the D40% was calculated (according to Gondi et al. [14]) For statistical analysis, the previously described framework was utilised to centrally extract dose-volume-histograms (DVH) metrics from the 3D dose distributions of each single radiation treatment plan using in-house developed software in Matlab (version 2017a, The MathWorks, Natick, MA) [15]. The VMAT plan was considered the gold standard. In order to adequately compare both treatment modalities, the doses to the CTV were evaluated as no PTV was used for the IMPT treatment plans. Doses were scaled such that 99% of the CTV received exactly 100% of the prescribed dose [54 Gy (RBE)]. The DVH metrics, doses to OARs, and the RPSS data were compared using a two-tailed Wilcoxon ranked sum test to determine the significance of differences between IMPT and VMAT. We consider a p-value of < 0.05 as statistically significant. In order to evaluate the conformity of the CTV coverage in combination with dose to the normal tissue we used the Van ‘t Riet Conformity Number with 1 indicating a perfect conformity [16].

Results

Plan comparison

In total, 40 treatment plans (20 VMAT, 20 IMPT) of 20 consecutive pilocytic astrocytoma patients treated with radiotherapy in the age of 2–21 years were calculated and analysed. Overall, the coverage of the CTV was excellent for both modalities, with a volume receiving 95% (V95%) for the CTV54Gy of 97.7% for IMPT and 98, 5% for VMAT. The dose coverage of the CTV54Gy was significantly higher for the VMAT plans compared to the IMPT plans. Table 1 shows the average scaled Dmean, D40% and D2% for all of the OARs in the IMPT treatment plans compared to the VMAT treatment plans.

Overall, the D2% was significantly lower in 7/12 (58%) OARs for the IMPT plans compared to the VMAT plans. For the Dmean, 9/11 (82%) OARs received significantly less radiation dose when planned with IMPT. The D40% of the hippocampus was significantly lower in the IMPT treatment plans, for both the ipsi- and bilateral as contralateral hippocampus. Fig. 2 shows a comparison of the percentages of volume of the brain minus the CTV, hippocampus ipsi- and bilateral, hippocampus contralateral, posterior cerebellum and supratentorial brain minus the CTV receiving a dose up to 50 Gy (V10Gy – V50Gy). It can be seen that for all these OARs the low dose volumes were statistically significantly reduced using IMPT whereas the high dose volumes were comparable or increased when IMPT was compared with VMAT. The mean integral dose to the body was statistically significantly decreased for IMPT (4,5 Gy) compared to VMAT (6,5 Gy), with p < 0.05.

Fig. 2

The percentage of organs at risk volume receiving a radiation dose between 10 Gy and 50 Gy (V10Gy to V50Gy, respectively); A: contralateral hippocampus, B: ipsilateral hippocampus, C: posterior cerebellum excluding the CTV, D: supratentorial brain excluding the CTV, E: brain.

RPSS

The summed weighted RPSS toxicity score for each modality per individual patient is presented in Fig. 3. In 17/20 (85%) patients, IMPT received a superior (lower) score compared to VMAT. In 2/20 (10%) patients, the score was identical and in 1/20 (5%) patients, VMAT received a superior score of 0.228 compared to 0.235 for IMPT, where all expected toxicities were equal except for the posterior cerebellum minus the CTV which received a higher dose. Overall, the RPSS data for IMPT and VMAT proved statistically significantly different in favour of the IMPT group with a p-value of 0.00023. Fig. 4 shows the delivered doses for specific OARs in patients 9 and 12. Patient 9 had an RPSS in favour of IMPT, which is showed by the plot. In this case, the pituitary gland received an equally high dose with both modalities, all other organs received a lower dose with IMPT, thus resulting in a better RPSS. Patient 12 had an equal RPSS for IMPT and VMAT. The lacrimal gland and pituitary gland received less dose in the IMPT plan, however, the supratentorial brain and hippocampi received equal dose and due to the weighting factors the RPSS score was consequently equal.

Fig. 3

Differences in RPSS scores for VMAT (blue) and IMPT (red) plans for each patient. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 4

Logarithmic plots of differences in Dmean, D0.03cc and D40% per OAR (x weighting factor) for IMPT and VMAT plans for patient 9 (A) and patient 12. (B) The white border of the plot represents the dose constraint.

Discussion

In our study, we found that IMPT was superior in sparing most OARs compared to VMAT. In an attempt to translate the dosimetric differences found in this study into clinically relevant endpoints we compared IMPT and VMAT treatment plans by introducing the RPSS based on expected toxicity for organs at risk. There was a significantly reduced RPSS for IMPT plans in 17/20 patients, meaning less toxicity can be expected in these plans.

The RPSS is a novel scoring system and an example of how a practical scoring can help a clinician in deciding which treatment plan is most beneficial for the patient regarding possible long-term side effects. The dataset used in this study consisted of 20 pilocytic astrocytoma patients. Pilocytic astrocytoma is a rare diagnosis predominantly seen in children [17]. Only a very small subset of children with this diagnosis undergo radiation therapy. It is important to notice that this pilocytic astrocytoma dataset was only used as an example to explain the mechanism and power of the RPSS. The toxicity data used for the RPSS was based on treatment in adults, so the RPSS as shown in this study is only applicable in adults receiving cranial radiation for brain tumors. Although the RPSS is presented in brain tumor patients in this article, the same principle is applicable to all tumor types and comparison of all types of treatment modalities.

There are a few important limitations of the RPSS. The weights of the scoring factors have a big impact on the total score of the RPSS. No data exist yet on how these weight factors should optimally be determined. The Neuro OAR weighting table (Fig. 1) that was used for the RPSS was based upon the European Particle Therapy Network (EPTN) OAR CNS consensus tolerance table which is extracted from an extensive literature search and is not yet externally validated requiring continuous optimization [9]. The weight factors used in this study were decided upon after consensus between only 4 CNS radiation oncologists. And the score from 1 to 5 are big steps and maybe not detailed enough, which may have influenced the final outcomes. The same applies for the weighting factors. If other choices were to be made for the weighting factors, outcomes could be different. This does however reflect current practice internationally because there is no objective scoring system available. Another limitation of the RPSS is that not all potential relevant factors are incorporated into the RPSS system, such as age and location of the tumor. The RPSS is an overall indication whether less or more toxicities can be expected from an overall treatment plan. It is for the clinician to decide which toxicity is important in an individual patient and take all OARs into account before deciding which treatment modality is most suitable for that patient. There is no differentiation between treatable toxicities and non-treatable toxicities in the current version of the RPSS. It was decided to score untreatable and treatable toxicities. One could argue to score only the untreatable toxicity for example resulting in cognitive decline. Radiation therapy can cause late cognitive decline caused by damaging the brain by several mechanisms such as inducing vascular damage, demyelination and white matter changes to certain parts of the brain [18], [19], [20], [21]. Although recently the OARs identified as possibly being relevant to cognition are expanded, future research needs to show if other areas in the brain should be taken into account and delineated when treating a CNS tumour [20]. For example, in the current version of the RPSS the choice was made to give several OARs a weighting factor of 0 based upon the fact that dose limits in these important OARs such as the brain are never exceeded thus not expected to induce toxicity. However, a recent randomized phase II trial shows that the Brain V20Gy was a strong predictor for severe lymphopenia in patients treated with radiation and temozolomide for glioblastoma [22]. In our study a RBE of 1.1 for proton therapy was used. However, there is an increasing uncertainty at the end of the Bragg peak which may lead to a higher value at the distal end [23]. This may affect the calculated RPSS score and needs further exploration in the future.

All of the above limitations show that the RPSS is a proof of principle for now. Future research and data will make optimization of the RPSS possible. In order to implement the RPSS clinically, a broader international concensus is necessary, preferably within the European Particle Therapy Network (EPTN). When implemented, we still expect alterations to the RPSS as new data become available, making it a dynamic process.

The fact that IMPT was superior in sparing most OARs in this dataset is in line with other studies. Eekers et al. [3] compared four treatment modalities (IMRT, VMAT, TOMO and IMPT) in low grade glioma and found a significantly reduced low-dose volume when using IMPT. Toussaint et al. also found that the distribution of doses to brain substructures associated with cognition are consistently lower with proton therapy [24]. Only a few studies reported on neurocognitive outcomes of proton therapy versus photon therapy in patients receiving cranial radiation. Gross et al. [25] reported on 125 paediatric brain tumour survivors who received cranial radiation with either photon or proton therapy. The proton therapy group had higher IQ scores and better scores on processing speed compared to the photon group. This is in line with the findings of Kahalley et al. [26] who examined changes in IQ scores over time in a group of patients (n = 150) who received cranial radiation comparing photon therapy with proton therapy. While the proton group did not show IQ changes over time, the IQ declined with 1.1 points per year in the photon group. Recently, Dutz et al. [27] reported on 62 brain tumour patients treated with proton therapy who’s neurocognition was assessed three-monthly using the Montreal cognitive assessment (MoCa). Overall, the MoCa score and self-reported cognitive function remained stable over time. Patients with declined scores experienced more physical and cognitive deficits compared to patients without decline in scores. In this study slight deterioration of MoCa score was associated with tumours located in the left hemisphere and increase of volume of the anterior cerebellum that received 30–40 Gy (RBE).

It is known that radiation dose to the brain can lead to mostly irreversible cognitive decline, directly affecting the patient’s daily functioning [2]. This cognitive decline develops over many years and the severity can be influenced by multiple factors such as age, use of chemotherapy, shunting procedures, surgeries and the tumour itself [19], [28]. So, individual patient characteristics could also be assumed relevant and were not yet taken into account in the RPSS. At which doses levels the damage mechanisms occur remains unclear, so whether avoiding a low dose bath to the organs at risk will eventually translate into a clinically relevant benefit also remains subject of future research. As mentioned earlier, we found a statistically significant decrease of the integral dose to the body in favour of IMPT. A major concern of increase in low-dose volumes, especially in paediatric patients, is induction of secondary malignancies [29], [30], [31]. Theoretically, IMPT should lead to a decreased risk of induction for secondary tumours. However, the dose–response relationship for induction of secondary cancers has been much debated and there are no validated models to predict the absolute reduction of risk for protons versus photons.

Conclusion

The novel ROCOCO Performance Score (RPSS) was developed and introduced to translate dosimetric differences in clinically relevant benefits. In this study, the RPSS showed a benefit of IMPT over VMAT in the majority of patients in our pilocytic astrocytoma dataset. The RPSS is a proof of principle and can be used in all types of treatment modalities in different types of tumors using the appropriate toxicity scores and weighting factors. It is a first step towards a uniform practical scoring system which is necessary for clinical decision making in proton therapy. In order to optimize the RPSS, registration of late toxicity is obligatory for example in the national Dutch database Proton therapy research infrastructure (PROTRAIT) in order to correlate dose to CNS toxicity. Future studies and international consensus are essential to evaluate, improve and correlate, the RPSS to patient treatment outcome, overall toxicity and the potential benefit of proton treatment.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.