Variability of Gross Tumor Volume Delineation for Stereotactic Body Radiotherapy of the Lung With Tri-60Co Magnetic Resonance Image-Guided Radiotherapy System (ViewRay): A Comparative Study With Magnetic Resonance- and Computed Tomography-Based Target Delineation.

INTRODUCTION: To evaluate the intra-/interobserver variability of gross target volumes between delineation based on magnetic resonance imaging and computed tomography in patients simulated for stereotactic body radiotherapy for primary lung cancer and lung metastasis.
MATERIALS AND METHODS: Twenty-five patients (27 lesions) who underwent computed tomography and magnetic resonance simulation with the MR-60Co system (ViewRay) were included in the study. Gross target volumes were delineated on the magnetic resonance imaging (GTVMR) and computed tomography (GTVCT) images by 2 radiation oncologists (RO1 and RO2). Volumes of all contours were measured. Levels of intraobserver (GTVMR_RO vs GTVCT_RO) and interobserver (GTVMR_RO1 vs GTVMR_RO2; GTVCT_RO1 vs GTVCT_RO2) agreement were evaluated using the generalized κ statistics and the paired t test.
RESULTS: No significant volumetric difference was observed between all 4 comparisons (GTVMR_RO1 vs GTVCT_RO1, GTVMR_RO2 vs GTVCT_RO2, GTVMR_RO1 vs GTVMR_RO2, and GTVCT_RO1 vs GTVCT_RO2; P > .05), with mean volumes of GTVs ranging 5 to 6 cm3. The levels of agreement between those 4 comparisons were all substantial with mean κ values of 0.64, 0.66, 0.74, and 0.63, respectively. However, the interobserver agreement level was significantly higher for GTVCT compared to GTVMR ( P <.001). The mean κ values significantly increased in all 4 comparisons for tumors >5 cm3 compared to tumors ≤5 cm3 (all P < .05).
CONCLUSION: No significant differences in volumes between magnetic resonance- and computed tomograpghy-based Gross target volumes were found among 2 ROs. Magnetic resonance-based GTV delineation for lung stereotactic body radiotherapy also demonstrated acceptable interobserver agreement. Tumors >5 cm3 show higher intra-/interobserver agreement compared to tumors <5 cm3. More experience should be accumulated to reduce variability in magnetic resonance-based Gross target volumes delineation in lung stereotactic body radiotherapy.

Introduction

Stereotactic body radiotherapy (SBRT) is considered as the standard care for medically inoperable early-stage non-small cell lung cancer (NSCLC) with durable local control.[1-3] It is also a curative strategy in patients with early-stage NSCLC who decline surgical resection or harbor high perioperative risk, as an alternative to surgery. Furthermore, despite lack of high-level evidence of survival benefit and precisely designed criteria for patient selection, SBRT for lung metastases from various primaries is increasingly performed in the clinic nowadays.[4]

The fundamentals of SBRT are the abilities to precisely deliver a high biologically effective dose per fraction to the tumor as well as minimizing the dose to surrounding normal tissues with steep dose gradients. Those can only be performed under the premises of accurate targeting of tumor by image guidance techniques and inverse treatment plan optimizations. Radiotherapy treatment planning begins with a critical step of delineating the gross target volume (GTV). Since only additional 5- to 10-mm margins are added from the GTV for SBRT planning of the lung and no additional margins for encompassing the subclinical disease are taken into consideration, accurate GTV delineation is even more crucial in lung SBRT.[5,6] However, delineating the GTV of the lung is known to largely vary among physicians using computed tomography (CT)-based contouring,[7-9] although the variance is somewhat reduced in the setting of SBRT.[10,11] Moreover, the volume of GTVs may change throughout the treatment course,[12] and GTVs can be significantly affected by artifacts when using the 4D-CT technique.[13] Recently, it has been reported that the interobserver variability is generally larger for magnetic resonance (MR)-based GTV delineation compared to that based on positron emission tomography computed tomography (PET-CT).[14]

The ViewRay (ViewRay Inc, Cleveland, Ohio) is the first commercially available MR-guided radiotherapy system with an inbuilt 0.35-T MR and tri-60Co source used in less than 10 institutions worldwide.[15,16] Static intensity-modulated radiotherapy can be performed by this tri-60Co system and online intrafractional near-real-time cine sagittal magnetic resonance images (MRIs), which allow automated respiratory gating of the tumor, can be acquired during treatment. The size of the planning target volume can potentially be reduced by this automated respiratory gating and with limited margins around the GTV compared to internal target volume-based strategies.[17] Moreover, since MRIs are known to have superior soft tissue resolutions compared to CT images, contouring of organs at risk such as the esophagus, heart, and spinal cord may benefit by MR-based delineation. However, the evaluation of the feasibility of GTV contouring based on the simulation MRIs obtained by the ViewRay and comparison with CT-based contouring have not been performed to date.

Therefore, in the current study, we evaluated the intra- and interobserver variability of MR-based GTV contouring using the ViewRay system. Moreover, we compared the variabilities with those of CT-based GTV contouring and investigated to identify factors affecting the variabilities.

Materials and Methods

This study was approved by the institutional review board of Seoul National University Hospital (IRB No: H-1712-112-907). Between October 2015 and March 2017, a total of 25 patients with 27 lesions underwent MR simulation with ViewRay and CT simulation (Brilliance CT big bore; Philips, Cleveland, Ohio) for lung SBRT. Eighteen lesions were early-stage primary lung cancers, whereas 9 were metastatic lesions to the lung. Of the 27 lesions, 2 lesions were eventually not treated. One patient was lost from follow-up, and the other demonstrated rapid progression of disease in bilateral lungs, during the referral to simulation interval. Of the remaining 25 lesions, 18 (72.0%) were treated with ViewRay and 7 (28.0%) were treated using the TrueBeam STx (Varian Medical Systems, Palo Alto, California) per physician’s preference after SBRT planning. The median prescribed total dose was 60 Gy (range, 48-60 Gy), and the number of fractions was 4 in all lesions. The characteristics of treated lesions are listed in Table 1.

Table 1.

Characteristics of Simulated Lesions.

Variable	n (%)
Site of lesion
Upper/middle lobe	12 (44.4)
Lower lobe	15 (55.6)
Primary disease
Lung cancer	18 (66.7)
NSCLC	16 (59.3)
N/A	2 (7.4)
Others	9 (33.3)
GI tract	4 (14.8)
Head and neck	2 (7.4)
Liver	2 (7.4)
Prostate	1 (3.7)
Treatment by ViewRay
Yes	18 (66.7)
No	9 (33.3)
Dose fractionation	median, 60 Gy in 4 fractions (range, 48-60 Gy)

Abbreviations: GI, gastrointestinal. N/A, not available; NSCLC, non-small cell lung cancer.

Magnetic resonance and CT simulations were both done with mild expiration breath-hold technique in a single scan. Both simulation images were obtained in 2-mm thickness. The near-real-time true fast imaging with steady state precession (FISP) pulse sequence was used for MRI acquisition with the ViewRay system.[18] Computed tomography images were directly imported into the Eclipse system (Varian Medical Systems) after simulation, and simulation MRIs were first extracted from the MRIdian planning system (ViewRay Inc) and then imported to the Eclipse system. Both images were rigidly fused according to the anatomy near the target for every single lesion. Two radiation oncologists (ROs), RO1 (H.G.W.) and RO2 (C.W.W.), independently contoured the GTVs on the MRIs (GTVMR) and CT images (GTVCT) for all 27 lesions (Figure 1). When contouring the GTVMR, the fused CT images can be used for reference and vice versa. Positron emission tomography computed tomography images at the time of diagnosis were available in 17 (63.0%) lesions, and those were also permitted to be used for reference although not fused in the Eclipse system. Since no consensus exist on which window level and width the GTVMR of lung should be delineated using the true FISP sequence, window level and width were selected per physicians’ preference. For GTVCT, contouring on the pulmonary window using −600/1600 HU for optimal visualization was recommended.[19] Eventually, the 4 following GTVs were contoured for each lesion: GTVMR_RO1, GTVCT_RO1, GTVMR_RO2, and GTVCT_RO2.

Figure 1.

Examples of GTVMR and GTVCT contoured by RO1 (red line) and RO2 (orange line). A, A 71-year-old female with a third primary non-small cell lung cancer presenting as clinical stage T1N0. B, A 68-year-old female with lung metastasis of hepatocellular carcinoma origin. Image on the left and right for each patient correspond to the simulation CT and MR images, respectively.

For statistical analysis of contours, they were extracted from the Eclipse system and transferred into the Computational Environment for Radiotherapy Research, version 5.2 (Mathworks, Natick, Massachusetts). Volumes of the 4 GTVs for each lesion were measured. To evaluate the intra- and interobserver agreement levels between contours, the apparent and κ-corrected agreement was utilized.[20] The κ-statistics is an interobserver metric of agreement that can be obtained by chance. According to Landis and Koch, the level agreement is regarded as poor, slight, fair, moderate, substantial, and near perfect when the κ values range <0.00, 0.00 to 0.20, 0.21 to 0.40, 0.41 to 0.60, 0.61 to 0.80, and 0.81 to 1.00, respectively.[20] Agreement levels between GTVMR_RO1 versus GTVCT_RO1, GTVMR_RO2 versus GTVCT_RO2, GTVMR_RO1 versus GTVMR_RO2, and GTVCT_RO1 versus GTVCT_RO2 were calculated for analysis.

All analysis was done using the Statistical Package for Social Sciences, version 22.0 (IBM SPSS, Armonk, New York). To evaluate the statistical difference among volumes and agreement levels between GTVs, a 2-tailed paired t test was used. The level of significance in all exams was set at a cutoff P value of < .05.

Results

The volumes of GTVs were directly measured in the Eclipse system. The mean volumes of GTVMR_RO1, GTVCT_RO1, GTVMR_RO2, and GTVCT_RO2 were 5.76 ± 7.53 cm3, 5.33 ± 8.52 cm3, 5.22 ± 7.27 cm3, and 5.36 ± 7.33 cm3, respectively. Using the paired t test, there was no significant difference in volumes between GTVMR_RO1 versus GTVCT_RO1 (P = .125), GTVMR_RO2 versus GTVCT_RO2 (P = .618), GTVMR_RO1 versus GTVMR_RO2 (P = .182), and GTVCT_RO1 versus GTVCT_RO2 (P = .577). Furthermore, for investigation of whether a certain factor affects a physician to delineate the GTVMR or GTVCT larger than the other, we measured the ratio of the volumes of GTVMR to GTVCT in RO1 and RO2. However, none of tumor size, primary tumor, histology, subpleural location, lower lobe location, emphysematous lung, and addition of PET-CT was shown to affect intraobserver variance between GTVMR and GTVCT in terms of the size in both ROs (Table 2).

Table 2.

The Ratio of Volumes of GTVMR to GTVCT in 2 Radiation Oncologists.

Variables	N	GTVMR_RO1/GTVCT_RO1 Ratio		GTVMR_RO2/GTVCT_RO2 Ratio
		Ratio (Mean [SD])	P a	Ratio (Mean [SD])	P a
Tumor size			.656		.644
>5 cm3	9	1.02 (0.18)		1.05 (0.19)
<5 cm3	18	0.98 (0.29)		1.10 (0.27)
Primary lung cancer			.114		.174
Yes	18	1.05 (0.22)		1.12 (0.22)
No	9	0.88 (0.29)		0.99 (0.27)
Histology
Adenocarcinoma			.975		.278
Yes	15	0.99 (0.22)		1.13 (0.26)
No/unknown	12	1.00 (0.30)		1.03 (0.21)
SqCC			.387		.689
Yes	7	1.07 (0.33)		1.05 (0.16)
No/unknown	21	0.97 (0.23)		1.10 (0.27)
Location
Subpleural			.119		.326
Yes	16	0.93 (0.21)		1.04 (0.27)
No	11	1.09 (0.30)		1.14 (0.18)
Lower lobe			.644		.787
Yes	15	0.97 (0.26)		1.07 (0.27)
No	12	1.02 (0.26)		1.10 (0.20)
Emphysematous lung			.650		.886
Yes	6	1.04 (0.20)		1.10 (0.14)
No	21	0.98 (0.27)		1.08 (0.27)
PET-CT			.510		.498
Yes	17	1.02 (0.24)		1.05 (0.18)
No	10	0.95 (0.28)		1.13 (0.32)

Abbreviations: CT, computed tomography; GTVCT, gross target volumes were delineated on CT; GTVMR, gross target volumes were delineated on MR; PET, positron emission tomography; RO, radiation oncologist; SD, standard deviation; SqCC, squamous cell carcinoma.

a Two-tailed independent t test.

All intra- and interobserver comparisons of GTVs demonstrated substantial agreement with mean κ values of 0.64 ± 0.11 (range, 0.37-0.82), 0.66 ± 0.10 (range, 0.52-0.82), 0.63-0.16 (range, 0.28-0.86), and 0.74 ± 0.09 (range, 0.56-0.91) for GTVMR_RO1 versus GTVCT_RO1, GTVMR_RO2 versus GTVCT_RO2, GTVMR_RO1 versus GTVMR_RO2, and GTVCT_RO1 versus GTVCT_RO2, respectively. When the levels of agreements were compared by paired t test, the mean κ value was significantly higher in the CT-based GTV delineation (0.74 ± 0.09) compared to MR-based GTV delineation (0.63 ± 0.16; P < .001).

Tumor size, primary tumor, histology, subpleural location, lower lobe location, emphysematous lung, and addition of PET-CT were assessed for their effects to agreements between GTVMR_RO1 versus GTVCT_RO1, GTVMR_RO2 versus GTVCT_RO2, GTVMR_RO1 versus GTVMR_RO2, and GTVCT_RO1 versus GTVCT_RO2 (Table 3). Only size of tumor larger than 5 cm3 was proven to significantly increase the level of GTV agreements in all 4 comparisons compared to that of tumors smaller than 5 cm3. Squamous cell carcinoma, compared to other histology, increased the level of agreement only between GTVCT_RO1 and GTVCT_RO2. Other factors did not significantly affect intra- and interobserver agreement of GTV delineation.

Table 3.

Tumor Variables and Agreement Levels of GTV Delineation.

Variables	N	GTVMR_RO1 Versus GTVCT_RO1		GTVMR_RO2 Versus GTVCT_RO2		GTVMR_RO1 Versus GTVMR_RO2		GTVCT_RO1 Versus GTVCT_RO2
		κ (Mean [SD])	P a	κ (Mean [SD])	P a	κ (Mean [SD])	P a	κ (Mean [SD])	P a
Overall	27	0.64 (0.11)		0.66 (0.10)		0.63 (0.16)		0.74 (0.09)
Tumor size			.009		.025		.026		.002
>5 cm3	9	0.72 (0.10)		0.71 (0.08)		0.72 (0.19)		0.81 (0.09)
<5 cm3	18	0.60 (0.10)		0.63 (0.09)		0.58 (0.13)		0.70 (0.07)
Primary lung cancer			.441		.290		.267		.491
Yes	18	0.65 (0.11)		0.67 (0.09)		0.65 (0.16)		0.75 (0.10)
No	9	0.62 (0.11)		0.63 (0.11)		0.58 (0.17)		0.72 (0.09)
Histology
Adenocarcinoma			.232		.734		.254		.280
Yes	15	0.62 (0.11)		0.65 (0.09)		0.60 (0.14)		0.72 (0.08)
No/unknown	12	0.67 (0.11)		0.66 (0.11)		0.66 (0.18)		0.76 (0.11)
SqCC			.332		.418		.167		.023
Yes	7	0.68 (0.11)		0.68 (0.10)		0.70 (0.20)		0.81 (0.08)
No/unknown	21	0.63 (0.11)		0.65 (0.09)		0.59 (0.15)		0.70 (0.12)
Location
Subpleural			.086		.806		.691		.847
Yes	16	0.67 (0.10)		0.66 (0.10)		0.64 (0.16)		0.74 (0.10)
No	11	0.60 (0.12)		0.65 (0.10)		0.61 (0.17)		0.75 (0.09)
Lower lobe			.926		.787		.578		.954
Yes	15	0.64 (0.12)		0.65 (0.10)		0.61 (0.19)		0.74 (0.09)
No	12	0.64 (0.10)		0.66 (0.09)		0.65 (0.13)		0.74 (0.11)
Emphysematous lung			.968		.665		.872		.556
Yes	6	0.64 (0.15)		0.67 (0.09)		0.64 (0.23)		0.72 (0.12)
No	21	0.64 (0.10)		0.65 (0.10)		0.62 (0.14)		0.75 (0.09)
PET-CT			.799		.293		.870		.968
Yes	17	0.64 (0.11)		0.67 (0.08)		0.62 (0.18)		0.74 (0.09)
No	10	0.63 (0.12)		0.63 (0.12)		0.63 (0.14)		0.74 (0.10)

Abbreviations: CT, computed tomography; GTVCT, gross target volumes were delineated on CT; GTVMR, gross target volumes were delineated on MR; PET, positron emission tomography; RO, radiation oncologist; SD, standard deviation; SqCC, squamous cell carcinoma.

a Two-tailed paired t test.

Discussion

The utilization of SBRT for early-stage lung cancer and oligometastases to the lung from various primaries is very common nowadays demonstrating durable local control by delivering biologically ablative doses.[1-3] The ViewRay system, by MR-based automated real-time gating system, enables to overcome one of the obstacles for precise targeting of tumor in lung SBRT, control of respiratory motion. However, unlike tumors of other sites such as the head and neck, central nervous system, prostate, gastrointestinal tract, and so on, where MRI is well-known for its value in radiotherapy target delineation,[21-24] the value of thoracic MRI for target delineation of the lung has been very limited throughout the years due to poor signal to noise ratio as well as artifacts from respiratory and cardiac motion.[25,26] Furthermore, delineation of GTVs, particularly using the true FISP sequence from ViewRay, has never been evaluated to date, and the experience is very immature.[14] Therefore, we evaluated the intraobserver variability between GTVMR and GTVCT as well as the interobserver variability of GTVMR and GTVCT between 2 ROs in 27 lung tumors simulated for SBRT.

In our study, the measured volumes of GTVs were similar in both imaging modalities and physicians with mean volumes ranging 5 to 6 cm3 for GTVMR_RO1, GTVMR_RO2, GTVCT_RO1, and GTVCT_RO2. Furthermore, no statistically significant difference was found according to imaging modality or the contouring RO using the paired t test. In a recent report by Karki et al, GTVMR of the primary tumor located in the lung was shown to be smaller when using the postgadolinium T1-weighted ultrafast gradient echo volume interpolated breath-hold examination and diffusion-weighted MRI compared to GTVCT (mean relative volume compared to PET-CT-based GTV [GTVPET-CT], 1.38 ± 0.44 vs 1.62 ± 0.76).[14] To the authors’ knowledge, Karki and colleagues were the only group to directly compare the volumes of GTVMR and GTVCT to date. Fleckenstein et al compared the GTVPET-CT and GTVMR using the half-Fourier acquisition single-shot turbo spin echo and diffusion-weighted sequence for MRI acquisition.[27] GTVMR was also smaller than GTVPET-CT, as Karki et al have reported. Although we did not fuse the PET-CT images to simulation CT images for GTV delineation and did not compare the sizes of GTVMR and GTVPET-CT, 17 (62.0%) patients had available PET-CT images allowed to be used for contouring references. There was no significant difference in the GTVMR/GTVCT ratio between lesions with and without available PET-CT in both ROs (Table 2). According to our finding, PET-CT as well as other factors did not cause any tendency of volumetric difference between GTVMR and GTVCT for lung SBRT.

GTVCT for lung SBRT, compared to conventional radiotherapy,[7-9] is known to have smaller variability.[10,11] Persson et al had measured the mean standard deviations of distances to a reference contour in axial and craniocaudal directions.[10] They reported a small interobserver variability in 7 independent physicians with mean standard deviations of 0.15 ± 0.08 cm and 0.26 ± 0.15 cm for axial and craniocaudal directions, respectively. Peulen et al also reported a small variability in GTVCT for lung SBRT.[11] They have computed a median surface among GTVCT from 11 ROs and quantified the variability by root mean square of the local standard deviations, which was the variation of perpendicular distances from all points to the median GTVCT. The overall target variability was 2.1 mm by root mean square, and only small uncertainty was observed with standard deviations of 1.2 to 1.8 mm. In our study, we adopted a different method to assess intra-/interobserver variability of contours, the generalized κ statistics.[19,20] This methodology has been used to assess contour variabilities in various cancer types.[28-31] Substantial level of agreement was observed in both intraobserver (GTVMR_RO1 versus GTVCT_RO1, GTVMR_RO2 versus GTVCT_RO2) and interobserver comparisons (GTVMR_RO1 versus GTVMR_RO2 and GTVCT_RO1 versus GTVCT_RO2) with mean κ values ranging 0.63 to 0.74. However, the interobserver agreement was significantly higher between GTVCT_RO1 and GTVCT_RO2 compared to GTVMR_RO1 and GTVMR_RO2. This is mainly thought to be due to shortage of experience since physician training is known to be associated with improved consistency of target contouring in lung cancer.[32,33] Lack of consensus for appropriate window level and width for true FISP MR-based GTV contouring in lung SBRT might also have attributed to the lower interobserver agreement between GTVMR compared to GTVCT.

We have also investigated for factors that might affect agreements between GTVs. Tumors larger than 5 cm3 tended to have significantly higher levels of intra- and interobserver agreement. Particularly, the mean κ value between GTVCT_RO1 and GTVCT_RO2 was 0.81 for tumors larger than 5 cm3, which corresponds to near-complete agreement. For tumors smaller than 5 cm3, small discrepancies among GTVs might have exaggerated the disagreement. Regarding tumor location, Persson et al have demonstrated a pronounced interobserver variance of GTVs in tumors abutting the pleura.[13] However, 16 lesions with subpleural location in our study did not demonstrate significantly higher discrepancy in both GTVMR_RO1 versus GTVMR_RO2 and GTVCT_RO1 versus GTVCT_RO2, compared to lesions surrounded by lung tissue. Magnetic resonance imaging offers superior soft tissue contrast, hence the authors hypothesized an increased interobserver agreement between GTVMR compared to those of GTVCT, which was not confirmed in our results. Since Karki et al have demonstrated that GTVPET-CT of the primary tumor in lung shows the lowest interobserver uncertainty at the tumor–chest wall interface among GTVCT, GTVPET-CT, and GTVMR,[14] adding PET-CT might improve interobserver agreement in GTVMR delineation for subpleural lesions simulated for SBRT. Despite several limitations of PET-CT such as poor spatial resolution, the use of PET-CT is well known to reduce contour variabilities in lung cancer.[34] The benefit is most pronounced for distinguishing the tumor and associated atelectasis.[14,35] However, there was no lesion with associated atelectasis in our study and consequently resulted in showing no difference in contour agreements (GTVMR_RO1 vs GTVCT_RO1, GTVMR_RO2 vs GTVCT_RO2, GTVMR_RO1 vs GTVMR_RO2, and GTVCT_RO1 vs GTVCT_RO2) between patients with and without available PET-CT. The benefit with PET-CT of reducing contour variabilities might be minimal for lesions with somewhat small sizes eligible for SBRT and without associated atelectasis.

One limitation of this study was that the interobserver variability was evaluated between only 2 ROs. Other studies using the same methodology involved over 10 ROs and 2 to 10 cases.[28-31] However, those previous studies were to develop a consensus target volume, whereas the aim of this study was to evaluate the feasibility and variability of target contouring for lung SBRT using an MR-guided radiotherapy system, the ViewRay. Moreover, a large number of the 27 lesions make this analysis quite reliable. Spatial distortions that may have occurred during the fusion of MRIs and CT images would have affected the intraobserver agreement between GTVMR and GTVCT, although it could not be quantified.

In summary, interobserver agreement in true-FISP MR-based GTV delineation for lung SBRT was acceptable at a substantial level. However, CT-based GTV delineation demonstrated significant higher interobserver agreement compared to MR-based GTV delineation. Experience and training for MR-based GTV delineation should be further accumulated. Tumors larger than 5 cm3 showed significantly higher intra- and interobserver agreement levels between GTVs.