Impact of the Intra- and Inter-observer Variability in the Delineation of Parotid Glands on the Dose Calculation During Head and Neck Helical Tomotherapy.

The intra- and inter-observer variability in delineation of the parotids on the kilo-voltage computed tomography (kVCT) and mega-voltage computed tomography (MVCT) were examined to establish their impact on the dose calculation during adaptive head and neck helical tomotherapy (HT). Three observers delineated left and right parotids for ten randomly selected patients with oropharynx cancer treated on HT. The pre-treatment kVCT and the MVCT from the first fraction of irradiation were selected to delineation. The delineation procedure was repeated three times by each observer. The parotids were delineated according to the institutional protocol. The analyses included intra-observer reproducibility and inter-structure, -observer and -modality variability of the volume and dose. The differences between the left and right parotid outlines were not statistically significant (p > 0.3). The reproducibility of the delineation was confirmed for each observer on the kVCT (p > 0.2) and on the MVCT (p > 0.1). The inter-observer variability of the outlines was significant (p < 0.001) as well as the inter-modality variability (p < 0.006). The parotids delineated on the MVCT were 10% smaller than on the kVCT. The inter-observer variability of the parotids delineation did not affect the average dose (p = 0.096 on the kVCT and p = 0.176 on the MVCT). The dose calculated on the MVCT was higher by 3.3% than dose from the kVCT (p = 0.009). Usage of the institutional protocols for the parotids delineation reduces intra-observer variability and increases reproducibility of the outlines. These protocols do not eliminate delineation differences between the observers, but these differences are not clinically significant and do not affect average doses in the parotids. The volumes of the parotids delineated on the MVCT are smaller than on the kVCT, which affects the differences in the calculated doses.

Introduction

Adaptive strategies play an important role in the current management of radiation therapy (RT) (1). The adaptations are generally stratified for two groups, when the potential disagreement between the planned and delivered dose is caused by the intra- or inter-fractions movement and/or deformations of the irradiated structures (2 –4). In both situations, realization of the adaptation requires accurate information of the anatomical changes and the delivered doses during RT. First of them is realised by the procedures of image guidance which are based on the comparison of the actual anatomy from the images gathered during the RT, such as mega-voltage (MV) computed tomography (CT) or cone beam (CB) CT with the corresponding anatomy, acquired before RT on the kilo-voltage (kV) CT (5 –7). The second is the procedure of dose guidance that allows checking and reporting the accuracy between the planned and delivered doses (8 –10).

The inter-fraction changes of the anatomy such as weight loss, tumour and/or organs at risk (OAR) shrinkage and tissue edema are the most common situations causing adaptive actions for patients with head and neck cancers. There are several publications investigating a proper usage of the adaptive procedures in this group of patients. Castadot et al. (11) analysed the anatomic changes occurred during RT. They suggested that adaptive strategies, where patients are re-imaged and possibly re-planned during treatment, are worth evaluating. Piotrowski et al. (12) describes the anatomical parameters that determine the situation when the initial plan should be adapted to the actual anatomy by the creation of a new treatment plan. They noted that adaptation was performed usually after the first two weeks and/or after the first five weeks of the treatment. Schwartz and Dong (13) show that adaptive actions performed twice, after 11th fraction and after 22nd fraction, increase correctness of the doses delivered to the tumour while the doses cumulated in the parotid glands and the integral body dose are effectively reduced. Wu et al. (14) show that the benefit of re-planning is improved sparing of the parotid, where the dose can be reduced by 30% compared to the dose delivered by nonadaptive schemes.

Nevertheless, the information describing the status of dose accumulation during RT, which is used in decision about adaptive actions, is burdened by errors arising from the uncertainties related to the technology (e.g. registration and dose mapping algorithms) and human subjectivity (e.g. manual segmentation of the tumour and OARs) (15 –17).

The helical tomotherapy (HT) allows management of patients that are treated according to the adaptive schemes (18). The current version of the HT system (dated on the end of 2013) does not include models of a deformable image registration and dose mapping algorithms. Therefore, the agreement between the planned dose calculated on the kVCT and the delivered dose calculated on the MVCT is examined in the light of the single fraction of RT (19, 20). Moreover, the HT system has not got auto segmentation tools that routinely help in finding the shapes of the tumour and OARs. As a result, before the comparison of the planned and delivered doses, the shape of the irradiated structures must be manually delineated on the MVCT scans. Therefore, the statistical parameters of the dose distribution calculated for the tumour and OARs are burdened by the error caused by human subjectivity. The differences between the parameters determining the quality of the MVCT scans and kVCT scans such as the noise, contrast and spatial resolution can increase the delineation error, especially for the soft tissue structures that are surrounded by blocks of tissues with similar density (e.g. parotid glands) (21).

The aim of this study was to examine the intra- and interobserver variability in the delineation of parotid glands performed on the kVCT and MVCT scans. The impact of this variability on dose calculation during an adaptive head and neck HT was analysed.

Material and Methods

The study includes the data of ten patients that were pulled at random from the group of 25 patients diagnosed with oropharynx cancer treated with radical concomitant chemoradiation (CRT) in 2010 at the Second Radiotherapy Department of the Greater Poland Cancer Centre (GPCC), Poznan, Poland (12). The patients were in general good condition WHO 0-2, with normal renal, liver, cardiac and bone marrow function. Exclusion criteria were tumours of salivary glands, nasopharynx, sinonasal region, lips and skin, planned palliative treatment and distant metastases.

The study was a part of KBN N N402 352138 grant provided by the State Committee for Scientific Research and was approved by Bioethics Committee at Poznan University of Medical Sciences. All patients signed an informed consent to participate in this study.

Before RT, the standard immobilisation method was used, based on the head and shoulder thermoplastic mask with five fixation points, affixed to the carbon fibrebase plate (CIVCO Medical Solutions, Kalona, IA, USA), followed by kVCT scans performed from the top of the head to manubrium sterni with a 2 mm inter-slice distance in a treatment position (Siemens Sensation Open, Siemens AG, Erlangen, Germany). The parotid glands, larynx, thyroid, oral cavity, mandible, inner ear, spinal cord and brain stem were contoured on the kVCT as OARs. Treatment intent was to deliver 70 Gy to gross tumour volume (GTV), 60 Gy to high-risk volumes and 50 Gy for elective volumes. Preparation of the treatment plan was described in detail in previous publication as well as its realisation on the HT unit (Accuray Inc., Sunnyvale, CA, USA) (12). Each daily fraction of the dose delivery was preceded by MVCT scanning used to the set-up of the actual anatomy according to the anatomy used during plan preparation (kVCT). The region included on the MVCT scans corresponded to the region acquired on the kVCT scans. The inter-slice distance for the MVCT scans was the same as for the kVCT scans (2 mm).

To analyse intra- and inter-observer variability in the delineation of parotid glands the pre-treatment kVCT scans and MVCT scans performed during the first fraction of irradiation were selected. Three observers were asked to delineate the parotid glands for each patient. All of them are well-trained radiation therapists with more than eight years of experience. The delineation of the parotid based on the institutional protocol whom criteria were adopted from the work published by van de Water et al. (22). According to the definition presented by van de Water et al. (22), the parotid gland is enclosed by the parotid fascia derived from the superficial layer of the deep cervical fascia. This gland consists of a deep and superficial lobe that are separated by the extracranial facial nerve passing through the gland. The anatomic boundaries of the parotid gland respected in our study were respectively: (i) in cranial direction – external auditory canal and mastoid process; (ii) in caudal direction – posterior part of submandibular space; (iii) in anterior direction – masseter muscle, posterior border of mandibular bone, medial and lateral pterygoid muscle; (iv) in posterior direction – anterior belly of sternocleidomastoid muscle, lateral side of postosterior belly of the digastric muscle; (v) in lateral direction – subcutaneous fat, platysma and (vi) in medial direction – posterior belly of the digastric muscle, styloid process and parapharyngeal space. All original figures presenting the graphical visualization of the anatomical boundaries of the parotid gland were included in the paper published by van de Water et al. (22).

The observers repeated the procedure of the delineation three times. Time between subsequent delineations was one month. For both modalities (kVCT and MVCT), parotid glands were typically delineated from fixed display windowing images (center = 0 Hounsfield Units (HU), width = 300 HU). The patients were anonymised through replacing characteristic patient’s data by the six-digital token, which has been changed before every delineation. Only main investigator, which supervised delineations performed by three observers (without active participation in delineation process), controlled the homogeneity of the data and adequate identification of tokens and patients. After delineations conducted in the Eclipse software v.10.0 (Varian Medical Systems, Palo Alto, CA, USA), the outlines of the parotid glands were transferred to the in-house software to perform detailed analysis (23).

To determine the intra-observer variability and their dependence on the modality of the images (kVCT or MVCT), three sets of delineations were compared for each observer separately. Three specific examinations were performed: (i) the similarity between the right and left parotid was analysed by the comparison of the volumes of the delineated structures (Wilcoxon test); (ii) the reproducibility of the parotids delineation was checked by the comparison of the volumes determined three times for each patient (Friedman ANOVA) and (iii) the similarity between outlines of the parotids determined on the two different methods of acquisition (kVCT and MVCT) was analysed by comparison of the delineated volumes (Wilcoxon test).

Inter-observer analysis was based on the comparison of the outlined volumes of the parotid glands by each observer on the kVCT scans as well as on the MVCT scans (Friedman ANOVA). Moreover, the analysis of the directionality of the potential shifts between the outlines delineated on the kVCT and MVCT by the observers was performed. The shifts were analysed for the left and right parotid glands separately, in each direction and were defined as the absolute difference between centroids of mass (COM) calculated on the basis of the points from the outlines delineated on the related kVCT and MVCT scans. The shifts obtained by each observer were compared to check inter-observer differences (Friedman ANOVA).

To analyse the impact of the intra- and inter-observer variability for the parotid glands on the dose calculation, the related dose distribution calculated in the Hi-ART planning system v.4.2 (Accuray Inc., Sunnyvale, CA, USA) were transferred to the Eclipse software where the delineations were performed. After adequate connection of the doses and scans, the dose distribution for each outline was analysed. The analysis of (i) the inter-structure variability of the dose in the left and right parotid outlined on the kVCT and MVCT (Wilcoxon test); (ii) the inter-observer variability of the dose on the kVCT and MVCT (Friedman ANOVA) and (iii) the inter-modality variability between the dose obtained for outlines from kVCT and MVCT (Wilcoxon test) were performed. The Wilcoxon test as well as the Friedman ANOVA is designed to analyse paired samples. All tests were performed at α = 0.05. Figure 1 shows the steps of the analysis performed in the study.

Figure 1:

Scheme of the study.

Results

The analysis of the inter-structure variability shows no statistically significant difference (p > 0.3) between the volumes of the left and right parotid, for delineation performed by each observer and for both modalities of imaging (Table I). The ranges of the average volume of the left and right parotid were, respectively, from 23.2 cm3 to 28.5 cm3 (left parotid) and from 23.4 cm3 to 28.3 cm3 (right parotid) on the kVCT and from 20.4 cm3 to 26.1 cm3 (left parotid) and 21.0 cm3 to 25.7 cm3 (right parotid) on the MVCT. These results allowed the inclusion of both (left and right) parotid glands to one group. Therefore, the analyses of the reproducibility of the observer delineation, inter-observer and inter-modality variability were performed for the parotid glands without specification of the sides.

Table I

Inter-structure variability between the volumes of the left and right parotid glands, for delineation performed by each observer and for both modalities of imaging (kVCT and MVCT). p-value resulted from Wilcoxon test performed at α = 0.05.

	kVCT			MVCT
	Volume ± SD (cm3)			Volume ± SD (cm3)
	Left parotid	Right parotid	Inter-structure variability	Left parotid	Right parotid	Inter-structure variability
Observer 1	23.2 ± 2.2	23.4 ± 1.9	p = 0.326	20.4 ± 2.7	21.0 ± 3.2	p = 0.759
Observer 2	28.5 ± 2.9	28.3 ± 2.9	p = 0.987	26.1 ± 3.5	25.7 ± 3.2	p = 0.308
Observer 3	24.5 ± 2.8	25.9 ± 2.7	p = 0.838	23.1 ± 3.4	22.8 ± 3.3	p = 0.610

Abbreviations: SD: Standard deviation; kVCT: Kilo-voltage CT; MVCT: Mega-voltage CT.

Table II shows the intra-observer variability (reproducibility) of the parotid glands delineations performed three times by each observer on the kVCT and MVCT. The reproducibility was confirmed for each observer, for outlines performed on the kVCT (p > 0.2) as well as on the MVCT (p > 0.1). The inter-observer variability of the outlines was statistically significant (p < 0.001) as well as the inter-modality variability of the outlines (p < 0.006) (Table II). The ranges of the average volume of the parotids delineated by the observers were from 23.3 cm3 to 28.4 cm3 on the kVCT and from 20.7 cm3 to 25.9 cm3 on the MVCT. The volumes delineated on the MVCT were smaller than corresponding volumes on the kVCT. For example, average volumes of the parotid glands delineated by third observer were 25.2 cm3 on the kVCT and 22.9 cm3 on the MVCT.

Table II

Intra-observer (reproducibility), inter-observer and inter-modality variability for the volumes of the parotid glands. p-value resulted from Friedman ANOVA for intra- and inter-observer variability and from Wilcoxon test for inter-modality variability. All test performed at α = 0.05.

	kVCT		MVCT
	Volume ± SD (cm3)	Reproducibility	Volume ± SD (cm3)	Reproducibility	Inter-modality variability
Observer 1	23.3 ± 2.1	p = 0.325	20.7 ± 2.9	p = 0.488	p = 0.006
Observer 2	28.4 ± 2.9	p = 0.487	25.9 ± 3.4	p = 0.193	p = 0.002
Observer 3	25.2 ± 2.8	p = 0.231	22.9 ± 3.3	p = 0.301	p < 0.001
Inter-observer variability	p < 0.001		p < 0.001

Abbreviations: SD: Standard deviation; kVCT: Kilo-voltage CT; MVCT: Mega-voltage CT.

The shifts between the parotids delineated on the kVCT and MVCT were presented on Table III. The inter-observer variability of the shifts was non-statistically significant (p > 0.2). For both parotids, the outlines on the MVCT to the outlines on kVCT were shifted inwards the body more than 1 mm in the medio-lateral direction. Shifts in the other directions were insignificant, around 0.5 mm in the anterioposterior direction and close to zero in the cranio-caudal direction.

Table III

The shifts of the outlines of the parotids on the MVCT to the outlines of the parotids on kVCT in the medio-lateral, cranio-caudal and anterio-posterior direction.

	Mean shift ± SD (mm)
	Left parotid			Right parotid
	ML	CC	AP	ML	CC	AP
Observer 1	-1.4 ± 0.7	0.3 ± 0.8	0.4 ± 0.3	1.3 ± 0.7	0.2 ± 0.8	0.5 ± 0.4
Observer 2	-1.1 ± 0.7	0.1 ± 1.0	0.6 ± 0.2	1.5 ± 0.6	0.0 ± 1.0	0.6 ± 0.3
Observer 3	-1.1 ± 0.8	-0.5 ± 1.0	0.5 ± 0.2	1.0 ± 0.7	-0.4 ± 0.8	0.5 ± 0.3
Inter-observer variability	p = 0.905	p = 0.273	p = 0.993	p = 0.407	p = 0.908	p = 0.827
Average observer	-1.2 ± 0.2	-0.1 ± 0.4	0.5 ± 0.1	1.3 ± 0.2	0.0 ± 0.3	0.5 ± 0.1

Abbreviations: SD: Standard deviation; ML: Medio-lateral direction; CC: Cranio-caudal direction; AP: Anterio-posterior direction.

After analysis of the delineation of the parotid glands, the impact of the outlines variability on the dose distribution was examined. As in the outlines examination, the inter-structure variability of the average dose calculated in the left and right parotid was analysed as first. The ranges of the average dose were, respectively, from 23.1 Gy to 24.3 Gy (left parotid) and from 23.0 Gy to 24.3 Gy (right parotid) on the kVCT and from 24.1 Gy to 25.1 Gy (left parotid) and 23.8 Gy to 25.0 Gy (right parotid) on the MVCT (Table IV). There are no statistically significant differences for doses calculated in the left and right parotids on the kVCT (p > 0.4) as well as for doses in parotids on the MVCT (p > 0.5) (Table IV). Moreover, there are no statistically significant differences between doses calculated for the outlines performed by different observers (p = 0.096 on the kVCT and p = 0.176 on the MVCT) (Table IV). Nevertheless, significant differences were found between the doses calculated on the kVCT and on the MVCT, as shown in Table V.

Table IV

Inter-structure and inter-observer variability of the doses calculated in left and right parotid glands. p-value resulted from Friedman ANOVA for inter-observer variability and from Wilcoxon test for inter-structure variability. All test performed at α = 0.05.

kVCT				MVCT
	Dose ± SD (Gy)		Inter-structure variability	Dose ± SD (Gy)		Inter-structure variability
	Left parotid	Right parotid		Left parotid	Right parotid
Observer 1	24.3 ± 1.6;	24.3 ± 1.9	p = 0.919	25.1 ± 1.7	25.0 ± 2.0	p = 0.760
Observer 2	23.1 ± 1.8	23.0 ± 2.4	p = 0.838	24.1 ± 2.1	23.8 ± 2.5	p = 0.683
Observer 3	23.7 ± 1.6	23.4 ± 1.9	p = 0.476	24.5 ± 1.9	24.2 ± 1.9	p = 0.531
Inter-observer variability	p = 0.096			p = 0.176

Abbreviations: SD: Standard deviation; kVCT: Kilo-voltage CT; MVCT: Mega-voltage CT.

Table V

Inter-modality variability of the dose calculated in the parotid glands. p-value resulted from Wilcoxon test (α = 0.05).

	Dose (Gy)		Inter-modality variability
	kVCT	MVCT
Average dose ± SD	23.6 ± 1.8	24.4 ± 1.9	p = 0.009
Minimum dose	3.2	5.7	p < 0.001
Maximum dose	69.1	68.7	p = 0.221

Abbreviations: SD: Standard deviation; kVCT: Kilo-voltage CT; MVCT: Mega-voltage CT.

Discussion

The inter-observer variability is a well known in the literature (17, 24 –26). Usually, important differences among different observers may exist. Wong et al. (27) showed that delineation guidelines might help improve uniformity among radiation oncologists. Guidelines to delineate clinical target volumes in head and neck cancer already exist (28, 29). Additionally, they exist for the OARs involved in xerostomia (22).

In our study, despite delineation of the parotid based on institutional protocol, statistically significant difference between the outlines delineated by different observers was detected (Table II). On the other hand, the usage of the protocols enables to obtain small intra-observer variability and high reproducibility of the delineation (Table II). Moreover, adequate usage of the protocols enables to obtain small variability between the volumes of the left and right parotids delineated by specified observer (Table I). These results suggest that the delineation based on institutional guidelines should be still burdened by the observer’s subjectivity. To reduce inter-observer variability, continuous training of the persons involved in the delineation process is needed.

Analysis of the inter-modality variability shows statistically significant differences between the volumes of the parotids delineated on the kVCT and MVCT (Table II). The volumes of parotids from the MVCT scans were usually smaller than corresponding volumes of the parotids delineated on the kVCT scans. Moreover, systematic shifts inwards the body of the parotids on the MVCT to the parotids on the kVCT were observed in medio-lateral direction (Table III). The systematic tendency of results presented on Tables II and III suggests that these differences were caused by a different quality of the kVCT and MVCT and does not depended on the inter-observer variability.

The inter-observer variability of the parotids delineation has not got any influence on the doses calculated in them (Table IV). Therefore, although the inter-observer variability is statistically significant, it is relatively small and in result, clinically non-significant. Nevertheless it should be noted that the results are burdened by relatively small group of the analysed patients and the results of this study has been obtained by the one institution experience in delineation of parotid glands.

Statistically important differences of the inter-modality variability (kVCT vs. MVCT) compounded by the shifts inwards the body (to the region of high dose where tumour is located) caused the difference between dose calculated on the kVCT and MVCT. The average dose in parotids was higher for the MVCT than for the kVCT (Table V). It is caused by volume of the parotids, which was smaller for delineation on the MVCT than for delineation on the kVCT (Table II). Additionally, the highest increase of the dose was detected for minimum dose (Table V). It is caused by the shifts inwards the body. For the parts of the parotids located externally to the body (region of the low doses, in the part of the parotid glands bordering with subcutaneous fat and/or platysma), the dose increased by the shift inwards the body, while for the parts located interiorly to the body (region of the high doses, in the part of the parotid glands bordering with posterior belly of the digastric muscle, styloid process and parapharyngeal space) the dose did not increase and was as high as before the shift. Figure 2 shows the relation between the volume of the parotids delineated on the kVCT and MVCT and the calculated doses.

Figure 2:

The relation between the volume of the parotids delineated on the kVCT and MVCT and the calculated doses, respectively for (A) minimum dose, (B) average dose and (C) maximum dose.

Conclusion

Usage of the institutional protocols for the parotids delineation enables to obtain small intra-observer variability and high reproducibility of the outlines. These protocols do not eliminate delineation differences between the observers; however, these differences are not clinically significant and do not affect the average doses in the parotids.

The volumes of the parotids delineated on the MVCT are smaller than on the kVCT, which affects the differences in the calculated doses. The doses calculated on the MVCT are 3% higher than corresponding doses on the kVCT. These differences should be taken into account during dose evaluation during adaptive head and neck helical tomotherapy.