Dummy Run of Quality Assurance Program before Prospective Study of Hippocampus-Sparing Whole-Brain Radiotherapy and Simultaneous Integrated Boost for Multiple Brain Metastases from Non-small Cell Lung Cancer: Korean Radiation Oncology Group (KROG) 17-06 Study.

PURPOSE: Lung Cancer Subcommittee of Korean Radiation Oncology Group (KROG) has recently launched a prospective clinical trial (KROG 17-06) of hippocampus-sparing whole brain radiotherapy (HS-WBRT) with simultaneous integrated boost (SIB) in treating multiple brain metastases from non-small cell lung cancer. In order to improve trial quality, dummy run studies among the participating institutions were designed. This work reported the results of two-step dummy run procedures of the KROG 17-06 study.
MATERIALS AND METHODS: Two steps tested hippocampus contouring variability and radiation therapy planning compliance. In the first step, the variation of the hippocampus delineation was investigated for two representative cases using the Dice similarity coefficients. In the second step, the participating institutions were requested to generate a HS-WBRT with SIB treatment plan for another representative case. The compliance of the treatment plans to the planning protocol was evaluated.
RESULTS: In the first step, the median Dice similarity coefficients of the hippocampus contours for two other dummy run cases changed from 0.669 (range, 0.073 to 0.712) to 0.690 (range, 0.522 to 0.750) and from 0.291 (range, 0.219 to 0.522) to 0.412 (range, 0.264 to 0.598) after providing the hippocampus contouring feedback. In the second step, with providing additional plan priority and extended dose constraints to the target volumes and normal structures, we observed the improved compliance of the treatment plans to the planning protocol.
CONCLUSION: The dummy run studies demonstrated the notable inter-institutional variability in delineating the hippocampus and treatment plan generation, which could be decreased through feedback from the trial center.

Introduction

Whole-brain radiation therapy (WBRT) has long been one of the standard treatment options in treating the patients with brain metastasis from various malignancies [1,2]. Stereotactic radiosurgery (SRS) is an aggressive local treatment option and has been recommended, either alone or in conjunction with WBRT, to the selected patients. The indications for SRS usually include the limited number of metastatic lesions (≤ 3 to 4 lesions), relatively small metastatic tumor size (≤ 4 cm), well-controlled extracranial disease, and good performance status. Compared with SRS alone, WBRT in addition to SRS, though not associated with survival improvement, has the advantage of less frequent new metastatic lesions in the brain in addition to improved local control [3-6]. Neurocognitive dysfunction attributable relation to WBRT, meanwhile, has remained one of the major concerns. To ameliorate WBRT-related neurocognitive dysfunction, hippocampus-sparing WBRT (HS-WBRT), which can achieve conformal dose reduction to the hippocampus, has been suggested as a solution [7,8] and demonstrated improved memory preservation thorough the Radiation Therapy Oncology Group (RTOG) 0933 prospective trial [9].

Simultaneous integrated boost (SIB) during HS-WBRT, which can deliver higher dose to the grossly metastatic lesions while keeping the hippocampus from high radiation dose, has recently been tried. A few planning studies tested the technical feasibility of volumetric-modulated arc therapy (VMAT) in HS-WBRT with SIB [10,11]. Moreover, a few clinical studies reported the efficacy and safety of this approach in treating the patients with multiple brain metastases, and reported no grade ≥ 3 toxicities [12,13]. These studies, unlike the RTOG 0933 trial, have the weakness of lacking routine neurocognitive function evaluation, which can reflect the effect of HS-WBRT. Based on these backgrounds, the Korean Radiation Oncology Group (KROG) has recently launched a prospective phase II multi-institutional trial of HS-WBRT with SIB in treating the patients with multiple brain metastases from non-small cell lung cancer, which incorporates the Seoul-Verbal Learning Test as the neurocognitive function evaluation (KROG 17-06, NCT03366376).

As Gondi et al. [14] previously described, the pre-treatment centralized review in the RTOG 0933 trial was able to prevent the unacceptable protocol deviations significantly. The necessity of the pre-treatment review process was agreed on among the members of the KROG’s lung cancer research section, and we conducted a series of dummy run studies before launching the main trial. Authors would report the results of the variations and variabilities in the target and hippocampus delineation, radiation dose distribution, and treatment technique.

Materials and Methods

1. Treatment planning protocol

The treatment planning protocol is summarized in Table 1, which was primarily based on the RTOG 0933 protocol [9] and has been modified through a preliminary planning exercise by the participating institutions. The planning gross tumor volume (P-GTV) was defined by 2 mm expansion of the gross tumor volume (GTV), each of which has a dimension greater than 3 mm. The planning clinical target volume (P-CTV) was defined as the whole brain parenchyma down to the bottom of C1 spine body excluding the hippocampus region, which was the 5 mm expansion of the actual hippocampus 3-dimensionally. Using the current protocol, a preliminary planning exercise was performed by the participating institutions to assess the variability in the treatment planning and protocol compliance. Following this preliminary exercise, we recognized fairly large inter-institutional diversity in the planning techniques, even though all were to be based on the proposed protocol. To improve the compliance to the protocol, we designed two-step dummy run procedures and analyzed (1) hippocampus contouring variability and (2) radiation therapy planning compliance and variability with contour delineation among the participating institutions.

Table 1.

The treatment planning protocol of KROG 17-06 study

Target/Normal structure	Constraint
	Per protocol	Acceptable
P-GTV[a)]	V95% ≥ 40 Gy	V93% ≥ 40 Gy
P-CTV[b)]	V95% ≥ 25 Gy	V90% ≥ 25 Gy
Hippocampus	Maximum ≤ 20 Gy	Maximum ≤ 22 Gy
	Maximum to 0.1 cm3 ≤ 16 Gy	Maximum to 0.1 cm3 ≤ 18 Gy
	Mean ≤ 12 Gy	Mean ≤ 14 Gy
Optic chiasm	Maximum ≤ 30 Gy	N/A
Optic nerve, both	Maximum ≤ 30 Gy	N/A
P-Lens[c)], both	Maximum ≤ 5 Gy	Maximum ≤ 6 Gy
Eyeball, both	Maximum ≤ 10 Gy	Maximum ≤ 12 Gy

P-GTV: planning gross tumor volume=gross target volume (GTV)+2 mm margin,

P-CTV: planning clinical target volume=whole brain–hippocampal avoiding region (hippocampus+5 mm margin),

P-Lens: planning volume for lens=lens+5 mm margin.

2. Step 1: variation of hippocampus contouring

Computed tomography (CT) and magnetic resonance (MR) images taken on two anonymized patients (patients A and B), who actually had received HS-WBRT without SIB, were provided to all participants. The CT images were obtained using 120 kVp, 400 mA, 1.25 mm slice thickness. The axial T1-weighted multi-planar reconstructed MR images with a 1.0 mm slice thickness were acquired using 3T magnetic flux. Both CT and MR images were obtained with contrast enhancement. For each case, the participants were asked to delineate the hippocampus, P-CTV and other surrounding normal structures on the CT image and to generate the treatment plans according to the protocol (Table 1). Fusion of the MR images to the CT images was mandatorily requested in contouring the hippocampus. The CT-MR image fusion was performed semi-empirically using non-deformable image registration method. The treatment planning priority was proposed by the following order: (1) optic chiasm, (2) left and right optic nerve, (3) hippocampus, (4) left and right lens, and (5) P-CTV. The structure set and radiation dose of each plan were saved as Digital Imaging and Communications in Medicine RT format and then imported into the MIM work-station (MIM Software Inc., Cleveland, OH). When transporting the plan data between different treatment planning systems (TPS) and MIM workstation, we did not observe any significant image or contour distortion which may impact to calculate the dose distribution. The inter-institutional variation of the hippocampus delineation was evaluated using the Dice similarity coefficients (DSC) [15,16] 3-dimensionally. The DSC ranges from 0 to 1, where 0 means that two contours are not similar at all and 1 means that two contours are perfectly matched. The DSC value between 0 and 1 indicates that two contours are partially overlapped. The dose distribution to the hippocampus contours was compared with the reference radiation dose file. The grid size of dose calculation was 2×2×2 mm3. The difference in the hippocampus delineation before and after providing the hippocampus contouring feedback was also analyzed.

3. Step 2: treatment plan compliance

CT and corresponding MR images of another anonymized patient (patient C), who actually had received HS-WBRT with SIB, was distributed to the participating institutions. The participants were requested to delineate the GTV and the normal structures including the hippocampus, and to generate the treatment plans according to the planning protocol as summarized in Table 1. These plans were uploaded in the same method as in step 1 and the inter-institutional variation of the GTV and hippocampus delineations as well as the radiation dose distributions to the target and normal structures were evaluated.

4. Ethical statement

This protocol was approved by KROG (KROG 17-06) and the Institutional Review Board of Samsung Medical Center (2017-08-070) and performed in accordance with the principles of the Declaration of Helsinki. The informed consent was waived.

Results

1. Step 1: variation of hippocampus contouring

During the step 1, the contoured structure sets and radiation dose files were submitted by ten and nine institutions before and after the provision of feedback on hippocampus contouring. Figs. 1 and 2 summarize the inter-institutional variation of the hippocampus contouring among the participants on patients A and B, respectively. The structure set from the trial center was selected as the reference in calculating the DSC of the hippocampus contouring. The radiation dose file from the same institution was also selected as the reference in evaluating the radiation dose distribution. For patient A, before providing the hippocampus contouring feedback, the median volume and DSC of the hippocampus were 4.21 cm3 (range, 2.23 to 8.41 cm3) and 0.669 (range, 0.073 to 0.712), respectively. After providing the feedback, these values were 5.04 cm3 (range, 2.95 to 6.03 cm3) and 0.690 (range, 0.522 to 0.750), respectively. For patient B, before providing the hippocampus contouring feedback, the median volume and DSC of hippocampus were 5.10 cm3 (range, 2.43 to 10.19 cm3) and 0.291 (range, 0.219 to 0.522), respectively. After providing the feedback, these values were 5.00 cm3 (range, 2.52 to 9.72 cm3) and 0.412 (range, 0.264 to 0.598), respectively. After providing the hippocampus contouring feedback, the median DSC were greater in both cases, which meant that the conformity of hippocampus delineation among the participants was improved. For patient A, the inter-institutional variation of hippocampus dose distribution was notably decreased after providing the contouring feedback as shown in Fig. 1E and F. However, for patient B, the dose distribution variation for modified hippocampus contours among the participating institutions was not reduced significantly as shown in Fig. 2E and F. As summarized in Fig. 2C, the scatter degree of volume to reference volume ratio of the hippocampus contours for patient B was similar before and after providing the feedback, while it was reduced greater for patient A as plotted in Fig. 1C. We speculate that the greater variation of hippocampus volume delineation for patient B might induce the smaller change in the inter-institutional variation of the hippocampus dose distribution before and after providing the contouring feedback.

Fig. 1.

Hippocampus delineation in a computed tomography image for patient A before (A) and after providing the hippocampus contouring feedback (B). (C) Change of hippocampus volume-to-reference volume ratio delineated by each participating institution. (D) Dice similarity coefficients before and after providing the feedback in the scatter plots with median lines. (E, F) The inter-institutional variation of the dose distributions to the delineated hippocampus contours from reference dose distribution before and after providing the feedback, respectively.

Fig. 2.

Hippocampus delineation in a computed tomography image for patient B before (A) and after providing the hippocampus contouring feedback (B). (C) Change of hippocampus volume-to-reference volume ratio delineated by each participating institution. (D) Dice similarity coefficients before and after providing the feedback in the scatter plots with median lines. (E, F) The inter-institutional variation of the dose distributions to the delineated hippocampus contours from an identical radiation dose distribution before and after providing the feedback, respectively.

2. Step 2: treatment plan compliance

A total of 12 institutions participated in the step 2. Table 2 summarizes the TPS and techniques; ten institutions used Eclipse (Varian Medical Systems, Palo Alto, CA) as their TPS; one did Pinnacle3 (Philips Health Care, Andover, MA); and one did TomoTherapy (Accuray Inc., Sunnyvale, CA), respectively. Various techniques, including VMAT and step-and-shoot or dynamic intensity-modulated radiotherapy (IMRT) with or without couch rotation (non-coplanar or coplanar), were used. These TPS were used for generating actual patient treatment plans in the participating institutions. Therefore, the accuracy of TPS commissioning has been already established before participating in this dummy run study.

Table 2.

Summary of the treatment planning systems and techniques for the dummy run study

	No.
Treatment planning system
Eclipse	10
Pinnacle3	1
TomoTherapy	1
Technique
VMAT	6
Coplanar	3
Non-coplanar	3
IMRT, step and shoot	2
Coplanar	1
Non-coplanar	1
IMRT, dynamic	3
Coplanar	1
Non-coplanar	2
Helical tomotherapy	1

VMAT, volumetric-modulated arc therapy; IMRT, intensity-modulated radiotherapy.

Similar to the step 1, one structure set from the trial center, was chosen as the reference structure set and the plan from the same institution was selected as the reference dose distribution. Fig. 3A-E shows the GTV and hippocampus delineations on the step 2 (patient C), the contoured volume-toreference volume ratio, and the corresponding DSC for the GTV, P-GTV and hippocampus, and the inter-institutional variations of the radiation dose distribution to the P-GTV and hippocampus when compared to the reference dose distribution. The median volume and DSC of the GTV were 8.49 cm3 (range, 6.02 to 12.1 cm3) and 0.653 (range, 0.567 to 0.752), respectively. For the P-GTV, the median volume and DSC were 22.2 cm3 (range, 12.8 to 32.2 cm3) and 0.708 (range, 0.533 to 0.801), respectively. For the hippocampus, the median volume and DSC were 4.46 cm3 (range, 2.56 to 8.68 cm3) and 0.692 (range, 0.538 to 0.777), respectively. Even though the hippocampus volume had wider range, the range of the DSC was similar to those of the GTV and P-GTV. When using the reference dose distribution, it was shown that the dose distribution to the P-GTV defined by one institution (institution K) was much lower than the others (Fig. 3E). This was because institution K did not follow the planning protocol and delineated all 12 brain metastases including three metastases smaller than 3 mm as the GTV.

Fig. 3.

Gross tumor volume (GTV) (A) and hippocampus delineation (B) in a computed tomography image for patient C. (C) Contoured volume-to-reference volume ratio delineated by each participating institution. (D) Dice similarity coefficients of GTV, planning gross tumor volume (P-GTV) and hippocampus in the scatter plots with median lines. (E) Inter-institutional variation of the dose distribution to the P-GTV and hippocampus contours from a reference radiation dose distribution provided from the trial center. (F) Dose-volume histograms for P-GTV, planning clinical target volume (P-CTV) and hippocampus contours from the 12 dummy run treatment plans for patient C. Each treatment plan was generated based on the two target volumes (P-GTV and P-CTV), hippocampus and other normal structures which were delineated by each participating institution.

Fig. 3F and Table 3 summarize the radiation dose distribution to the P-GTV, P-CTV and hippocampus and the number of the treatment plans that satisfied the dose constraints to the P-GTV, P-CTV, hippocampus and surrounding normal structures for patient C. Each dummy run treatment plan was generated using the contours delineated by each participant. Seven out of 12 institutions satisfied the dose constraint to P-GTV when expanding the dose constraint to the variation acceptable limit. One institution (institution I), which did not meet the dose constraint to the P-GTV, generated the plan by optimizing the dose delivery to 30 Gy instead of 40 Gy for one of the 12 brain metastases. This was because this contour partially overlapped with the hippocampal avoiding region, which was 5 mm extended from the hippocampus. Most of the dummy run treatment plans (11 out of 12) satisfied the dose constraint to the P-CTV with the acceptable variation limit. The variation of the dose distribution to the hippocampus was greater than those to P-GTV and P-CTV due to the relatively small size of the hippocampus. For the hippocampus, nine out of 12 treatment plans satisfied the suggested planning protocol with the acceptable variation limit. For the optic chiasm and right and left optic nerves, most of the plans satisfied the dose constraints. For the planning volume for lens and eyeballs, except for one institution for each structure, all institutions generated the treatment plans satisfying the dose constraints in the acceptable variation limit. Compared to the preliminary planning experiment (data not shown), we found that the compliance of the treatment plans to the planning protocol were impro-ved by applying the plan priority and extended dose constraints to the treatment planning.

Table 3.

Summary of the results of 12 dummy run treatment plans for patient C

Target/Normal structure	No. of treatment plans satisfying the dose constraints
	Per protocol	Acceptable
P-GTV	3	7
P-CTV	6	11
Hippocampus, maximum	6	9
Hippocampus, maximum to 0.1 cm3	5	10
Hippocampus, mean	6	10
Optic chiasm, maximum	11	-
Optic nerve (right, left), maximum	12, 12	-
P-Lens (right, left), maximum	8, 8	12, 11
Eyeball (right, left), maximum	9, 9	11, 11

P-GTV, planning gross tumor volume; P-CTV, planning clinical target volume; P-Lens, planning volume for lens.

Discussion

Unacceptable deviations from the radiation therapy protocol have been demonstrated to have a negative impact on the clinical outcomes [17-19]. Since HS-WBRT with SIB involves a complex technique of IMRT requiring steep dose gradient around the gross metastatic lesions and hippocampus, the protocol deviation could affect the doses to the target volumes and hippocampus and clinical outcomes subsequently. The primary endpoint of the KROG 17-06 study is the intracranial progression-free survival and the secondary endpoints are the verbal neurocognitive function, overall survival, toxicity, and quality of life. As these endpoints could be influenced by protocol deviations, the comprehensive pre-treatment dummy run as a quality assurance program seems quite essential in conducting this kind of prospective study.

In these dummy run procedures, we found several variations in terms of delineating the target volumes, hippocampus and the radiation therapy planning. First of all, the delineation of the hippocampus was quite diverse, although the participants were advised to refer to the contouring atlas for hippocampal delineation on the RTOG website [20]. For re-orientation of the participants, an example case of hippocampus contouring on CT/MR fused images was provided from the trial center as a feedback, which helped decreasing the degree of hippocampus delineation diversity. In addition to sparing the hippocampus, our protocol would involve SIB and IMRT technique to improve the local control of the gross metastatic lesions. The eligibility criteria for the KROG 17-06 study include the patients with (1) three or more brain metastases, (2) with ≥ 3 mm in size, and (3) lesions located outside the 5-mm margin around the hippocampi on both sides. Therefore, the great variation in GTV delineation was induced by one institute’s delineating the small enhancing lesion smaller than 3 mm. Furthermore, the closest distance between the GTV and hippocampus was almost 5 mm in patient C, which made it difficult to fulfill the planning protocol. Authors, however, would expect that this rather tough dummy run study could have enhanced the quality of planning procedures by the participants.

In the pilot experiment, we found the considerable variations in the radiation dose to the target volume and hippocampus and the real-time transport protocol at the participating institutions seemed more or less preliminary and lacked detail. Through these pilot studies, we could develop more detailed planning protocol and improve the compliance to the treatment plan protocol, though tough and difficult cases were selected (Fig. 3). Application of the plan priority and extended dose constraints were quite important in these processes.

Our dummy run studies have a few potential limitations. There were still variations in delineating the target volumes and hippocampus and patient C might not have been an ideal case for this type of dummy run practice. It might be helpful to repeat a third dummy run procedure in order to further reduce the variations. As shown in Table 2, the majority of the planning techniques were linear accelerator (LINAC)-based IMRT including VMAT, while helical tomotherapy technique was used in only one institution. Therefore, we could not compare LINAC-based IMRT and helical tomotherapy as Gondi et al. [7] did.

In conclusion, there were notable inter-institutional differences in planning HS-WBRT with SIB, including hippocampus contouring and target volume delineation. Feedback and re-orientation could reduce the variations, and detailed dose specification with plan priority is quite essential before launching multi-institutional study involving complex IMRT planning.