Variability of clinical target volume delineation for rectal cancer patients planned for neoadjuvant radiotherapy with the aid of the platform Anatom-e.

OBJECTIVE: Delineation of treatment volumes is a major source of uncertainties in radiotherapy (RT). This is also true for rectal cancer patients undergoing neoadjuvant RT, with a potential impact on treatment quality. We investigated the role of the digital platform Anatom-e (Anatom-e Information Sytems Ltd., Houston, Texas) in increasing the compliance to follow a specific treatment protocol in a multicentric setting.
MATERIALS AND METHODS: Two clinical cases of locally advanced rectal cancer were chosen. Participants were instructed to follow the 2009 Radiation Therapy Oncology Group consensus atlas and asked to manually segment clinical target volumes (CTVs), for both patient 1 and 2, on day 1 with and without the use of Anatom-e. After one week (day 2), the same radiation oncologist contoured again, with and without Anatom-e, the same CT series. Intraobserver (Intra-OV) and interobserver (Inter-OV) variability were evaluated with the Dice similarity coefficient (DSC), the Hausdorff distance (HD) and mean distance to agreement (MDA).
RESULTS: For clinical case 1, no significant difference was found for Intra-OV and Inter-OV. For clinical case 2, no significant difference was found for Intra-OV but a statistically significant difference was found for Inter-OV in DSC when using or not the platform. Mean DCS was 0.65 (SD: ±0.64; range: 0.58-0.79) for day 1 vs reference volume without Anatom-e and 0.72 (SD: ±0.39; range: 0.67-0.77) (p = 0.03) with it. Mean MDA was lower with Anatom-e (3.61; SD: ±1.33; range: 2.85-4.78) than without (4.14; SD: ±2.97; range: 2.18-5.21), with no statistical significance (p = 0.21) The use of Anatom-e decreased the SD from 2.97 to 1.33. Mean HD was lower with Anatom-e (26.06; SD: ±2.05; range: 24.08-32.62), with no statistical significance (p = 0.14) compared to that without (31.39; SD: ±1.31; range: 26.14-48.72).
CONCLUSIONS: The use of Anatom-e decreased the Inter-OV in the CTV delineation process for locally advanced rectal cancer with complex disease presentation planned for neoadjuvant RT. This system may be potentially helpful in increasing the compliance to follow shared guidelines and protocols.

Introduction

A multidisciplinary approach, including total mesorectal excision, radiotherapy (RT) and chemotherapy (CHT) is presently considered the standard of care for the treatment of locally advanced rectal cancer [1], [2]. Pre-operative RT is a well-established option to provide tumor downsizing and downstaging and to increase loco-regional control in this setting [3]. Selection and delineation of clinical target volume (CTV) and organs at risk (OARs) are crucial steps to deliver precise and tailored radiation [4], [5], [6]. While the sites and subsites of irradiation are, to some extent, agreed by physicians, the boundaries of CTVs still remain controversial, leading to inhomogeneous contours and systematic errors with standard deviations as high as 1 cm, as reported in different studies [7], [8]. Generally speaking, most of the heterogeneity is due to the difference in contouring protocols used by the treating physician, but the magnitude of this uncertainty is also related to the imaging modalities and technical approaches used in the delineation process [9]. In the context of pelvic malignancies, one of the major sources of uncertainty is the lack of clearly-defined anatomical boundaries in this region, which may lead to detectable contouring differences among radiation oncologists [8]. From an anatomical point of view, the site of major disagreement is to be located in the upper anterior and inferior aspect of the mesorectum, which is a critical structure for tumor control given the likelihood of microscopic involvement, particularly for locally advanced cases [7], [10]. Several strategies can be implemented to reduce this source of error, including periodic training for radiation oncologists, the use of shared delineation guidelines, and quality assurance processes with online platform for centralized revision [11]. Anatom-e (Anatom-e Information Sytems Ltd., Houston, Texas) is an electronic platform functioning as an image-based delineation system, an image fusion software and a treatment planning tool. It includes a digital atlas combining protocols and guidelines, classified according to tumor site and a vast library of normal tissue structures. All protocols and information can be continuously updated and customised. Interestingly, Anatom-e contains a library of CTVs, including lymphatics at risk, prophylactic volumes and OARs for most of the tumor sites and oncological scenarios, on both intact and post-operative anatomy. The present study was set in order to test the potential role of the platform Anatom-e in reducing the intra- (Intra-OV) and inter-observer variability (Inter-OV), within a multicentric context, in the delineation process of prophylactic volumes in locally advanced rectal cancer patients undergoing neoadjuvant RT. In particular, we tested the efficacy of the platform in homogenizing the compliance of different radiation oncologists (ROs) in following a pre-defined delineation protocol.

Material and methods

This was a multicentric study implemented within the oncological radiotherapy network of the Piedmont region in Italy and endorsed by ‘Rete Oncologica del Piemonte e della Valle d’Aosta’. The study was proposed to 14 centres belonging to the network and 10 of them agreed to participate and were consequently included. For each participating centre, a RO with a ≥5-year experience in the treatment of rectal cancer was selected to participate, on a voluntary basis. Participating centers were provided with the credentials to access the online version of Anatom-e, which has most of the characteristics and tools of the physical platforms (www.anatom.e.com). Two different clinical cases were chosen to be delineated. Both of them were locally advanced rectal cancer classified according to 2010 American Joint Committee on Cancer/Union Internationale Contre le Cancer (AJCC/UICC) staging system, undergoing neo-adjuvant long-course radiotherapy. Patient 1 was a 59 years old male with a Stage IIIB (cT3N2aM0) low lying rectal cancer with a mesorectal node in close proximity to the mesorectal fascia. Patient 2 was a 49 years old female with a Stage IIIC (cT4bN1bM0) rectal cancer located to the low rectum with an anterior spread and infiltration of the posterior wall of the vagina. Detailed characteristics of the 2 clinical cases can be found in Fig. 1.

Fig. 1

Description of the 2 clinical cases.

Target volume selection and delineation

For both cases, planning computed tomography (CT) images were acquired from the second lumbar vertebra down to below the lesser trochanters. All simulation images were acquired with no contrast enhancement and had a 3 mm-slice thickness. Participating ROs were instructed to follow the atlas and the specific instruction found in the Radiation Therapy Oncology Group (RTOG) consensus published in 2009, during the delineation process [12]. Two consensus meetings comprising contouring laboratory were organized before the start of the study to agree on the whole contouring workflow. In general, our study aimed at investigating whether the compliance in following the indications of the RTOG 2009 consensus could be enhanced by the use of an advanced digital platform such as Anatom-e, compared to direct consultation of the paper version of the aforementioned consensus. Participants were asked to manually segment clinical target volumes (CTVs), for both patient 1 and 2, on day 1 with and without the use of the Anatom-e platform. After one week (day 2), the same ROs were asked to contour again, with and without Anatom-e, the same CT scans of the 2 patients. The treatment volumes to be contoured and the corresponding nomenclature and description can be found in Fig. 2. An example of the Anatom-e interface with the specific ontology can be seen in Fig. 3.

Fig. 2

Anatom-e nomenclature of target volumes.

Fig. 3

Visual example of the Anatom-e platform.

Contour analysis

All volumes were imported in the Velocity platform (Varian Medical Systems, Palo Alto, CA). To determine Intra-OV, we analysed and compared contours performed by the same RO on day 1 and day 2. To determine Inter-OV, we compared the contours drawn by all different participants with a ‘ground truth’ contour performed by an experienced RO dedicated to rectal cancer treatment. The ‘ground truth’ contour was driven by the use of Anatom-e and considered as the ‘gold standard’ for comparison. An outline of all different contours obtained with or without the platform can be found in Fig. 4. We decided to analyse overlap between different contours using the Dice similarity coefficient (DSC) which represents the ratio between the overlapping volume and the encompassing volume, with the numerator multiplied by 2. By definition, DCS varies from 0 (no overlap) to 1 (complete overlap) [13]. To explore the distance between contours, we employed the Hausdorff distance (HD), which is the maximum distance of each voxel of the reference set to the nearest point in the comparison set [14]. We also calculated the mean distance to agreement (MDA), which is the average distance that all outlying points in the considered volume must be moved to achieve perfect conformity-overlap with the reference volume [15]. For both HD and MDA, lower values (in mm) correspond to a higher correspondence between the compared volumes.

Fig. 4

Delineated volumes for clinical case 2.

We investigated the overlap between contours performed on day 1 and day 2 by the same operator for Intra-OV and between all contours drawn by the ROs of all the 10 centres participating in the present study and the ‘ground truth’ contour drawn in the reference centre, for Inter-OV.

Statistical analysis

All the results were reported as the sample mean and standard deviation (SD). Comparisons between groups were performed using univariate t-Student test. Multiple subsets of data were analysed on a 8 × 8 grouping categorization. The difference between multiple subsets of data was considered statistically significant if t-Student test gives a significance level P (P value) less than 0.05. The STATA software package (Stata Statistical Software: Release 13.1. Stata Corporation, College Station, TX, 2013) was used for all statistical analysis.

Results

Detailed results can be found in Table 1. For the clinical case 1, no significant difference was found in terms of Intra-OV (same RO; day 1 vs day 2) for DSC, HD and MDA according to the use or not of Anatom-e. In particular, the mean DSC was 0.95 (SD: ±0.43; range: 0.88–0.99) for day 1 vs day 2 without Anatom-e and 0.95 (SD: ±0.32; range: 0.91–0.10) (p = 0.71) with the use of the platform. No difference was also found in terms of Inter-OV (different ROs; day 1 vs ground-truth) for DSC, HD and MDA when using or not Anatom-e. The mean DSC was 0.80 (SD: ±0.04; range: 0.76–0.86) for day 1 vs ground-truth with no Anatom-e and 0.80 (SD: ±0.30; range: 0.75–0.84) (p = 0.68) with the use of the system. For the clinical case 2, no significant difference was found in terms of Intra-OV for DSC, HD and MDA according to the use or not of Anatom-e. In particular, the mean DSC was 0.89 (SD: ±0.11; range: 0.62–0.99) for day 1 vs day 2 without Anatom-e and 0.91 (SD: ±0.77; range: 0.76–0.99) (p = 0.64) with the use of the platform. A statistically significant difference was found in terms of Inter-OV for DSC when using or not the platform. Mean DCS was 0.65 (SD: ±0.64; range: 0.58–0.79) for day 1 vs ground truth without Anatom-e and 0.72 (SD: ±0.39; range: 0.67–0.77) (p = 0.03) with the use of Anatom-e (Fig. 5a). With the use of Anatom-e, mean MDA was lower with Anatom-e (3.61; SD: ±1.33; range: 2.85–4.78) than without (4.14; SD: ±2.97; range: 2.18–5.21), with no statistical significance (p = 0.21) (Fig. 5c). The use of Anatom-e decreased the SD from 2.97 to 1.33 (Fig. 5b). Mean HD was lower (26.06; SD: ±2.05; range: 24.08–32.62) but without statistical significance (p = 0.14) compared to the one obtained without Anatom-e (31.39; SD: ±1.31; range: 26.14–48.72).

Table 1

Measures of intra- and inter-observer variability for both clinical cases.

Clinical case 1
Without Anatom-e				With Anatom-e
	Mean	SD	Range	Mean	SD	Range	p-value
Intraobserver variability (day 1 vs day 2)
DSC	0.95	0.43	0.88–0.99	0.95	0.32	0.91–0.10	0.71
MDA (mm)	0.90	0.76	0.14–2.07	0.77	0.62	0.04–1.71	0.70
HD (mm)	10.22	4.68	4.30–18.81	10.44	4.60	3.40–18.45	0.83
Interobserver variability (day 1 vs ground truth)
DSC	0.80	0.04	0.76–0.86	0.80	0.30	0.75–0-84	0.68
MDA (mm)	3.84	0.91	2.70–5.00	3.82	0.83	2.75–5.16	0.97
HD (mm)	28.91	3.18	23.18–33.49	30.67	3.53	26.11–35-71	0.31
Clinical case 2
Intraobserver variability (day 1 vs day 2)
DSC	0.89	0.11	0.62–0.99	0.91	0.77	0.76–0.99	0.64
MDA (mm)	1.26	1.14	0.49–3.70	1.22	1.15	0.94–3.58	0.94
HD (mm)	13.04	7.69	4.19–29.45	12.70	7.14	6.44–25.56	0.93
Interobserver variability (day 1 vs ground truth)
DSC	0.65	0.64	0.58–0.79	0.72	0.39	0.67–0.77	0.03
MDA (mm)	4.14	2.97	2.18–5.21	3.61	1.33	2.85–4.78	0.21
HD (mm)	31.39	1.31	26.14–48.72	26.06	2.05	24.08–32-62	0.14

Bold values represents results of particular interest.

SD; standard deviation; DSC: Dice similarity coefficient; MDA: mean distance to agreement; HD: Hausdorff distance; mm: millimeters.

Fig. 5

Dice similarity coefficient, Hausdorff distance and mean distance to agreement with and without Anatom-e for clinical case 2.

Discussion

The current management of rectal cancer employs a multidisciplinary strategy which involves different professionals. Hence, it is of crucial importance to verify the quality of the different treatment strategies comprised within the combined modality approach. For example, training, centre’s experience and the quality of total mesorectal excision have be shown to be prognostic factors in rectal cancer patients [16]. Moreover, a central review of pathology report and an efficient feedback to surgeons has been proven to improve the quality of the surgical procedure [17]. Also RT, as a mainstay treatment option in the multimodality management of cancer, needs quality assurance (QA) protocols to constantly check for the quality of treatments (target volume delineation, treatment plan optimization, dosimetric results and delivery reliability) [11]. The contouring process of the target volume is a major source of uncertainty and error in RT and, since, most of the times, this potential error remains constant during the whole RT treatment, it may have a detectable impact on the dose received by the tumor, especially for highly conformal techniques, such as volumetric modulated arc therapy (VMAT) and whenever image-guidance (with consequent CTV to PTV margin reduction) is employed [18], [19], [20]. The factors that most consistently influence target delineation variability include gross disease visibility, disagreement on target definition, extension and interpretation or lack of dedicated contouring protocols [18], [21]. Inter-OV during the delineation process is strongly affected by the imaging modality and technique employed and by the specificity of the observer (specialty, training and personal bias) [18]. This inter-observer variation is detectable even during the delineation of visible and well-circumscribed targets such as in prostate cancer or brain tumors with variation having an average factor of 1.3–2 [18], [21]. It is, of course, much higher in body regions where anatomical boundaries are not necessarily well-defined. The pelvis is paradigmatic in this sense and rectal cancer RT volumes are a good example [4]. Pelvic subsites such as the presacral space, the mesorectum and the lateral lymphnodes are not trivial to be correctly defined on non-contrast-enhanced computed tomography images [4].An even higher variation can be supposed in the evaluation of the extent of the microscopic involvement in the delineation of the CTV. It has been shown that ROs tend to delineate larger volumes compared to physician with different specialties [21]. Hence, standardization of the delineation process is of paramount importance for all tumor sites, including rectal cancer. As an example, in Belgium, the PROject on Cancer of the Rectum (PROCARE) initiative was set to increase the use of guidelines and quality indicators throughout the country, with decentralized implementation of treatment protocols, prospective data registration and consequent feedback supply and benchmarking to improve the homogeneity of CTV delineation in daily clinical practice [11]. Tools allowing for direct visualization of a specific contouring protocol are very useful to increase the consistency of the delineation process. In this sense, the study performed by the Radiation Committee of the Southwest Oncology Group (SWOG) provided an important evidence on the effect of a consensus guideline-based visual atlas with respect to target volume delineation variability in rectal cancer [7]. Authors asked 13 physicians and 1 reference expert to contour both gross tumor volume (GTV) and CTV in a case of cT3N0M0 rectal cancer. The access to the delineation atlas was provided or not on a random basis and observer variations were analysed on a volume basis with the conformation number [7]. The use of the aforementioned atlas resulted in a significantly higher inter-observer agreement between physicians, particularly for pelvic nodal regions [7]. Anatom-e (Anatom-e Information Systems Ltd., Houston, Texas) is an electronic system working as a platform able to drive delineation based on multi-modality imaging, with an advanced image fusion software and treatment planning characteristics. It includes a digital atlas built on the combination of more than 150 contouring atlases using 3 mm axial computed tomography (CT) and magnetic resonance (MR) images acquired in treatment position, with more than 50.000 normal tissue structures available. It contains several treatment protocols and guidelines, classified according to tumor site, and allows for the personalization of institutional protocols. It employs an evidence-based approach with a continuous update of scientific information and literature, being connected online to a central data server. Anatom-e contains a library of CTVs, including lymphatics at risk, prophylactic volumes and OARs for most of the tumor sites and oncological scenarios, on intact and post-operative anatomy. The present study was aimed at investigating whether the platform Anatom-e may increase the adherence to a specific protocol among different centres, with respect to CTV delineation in rectal cancer patients planned to receive pre-operative long-course RT. We chose to follow the RTOG 2009 consensus guidelines since they were constitutively included in the platform and most of the participating centres were familiar with their indications. Patient 1 was a locally advanced low-lying rectal cancer whose level of criticality lied on the proximity of a mesorectal node to the mesorectal fascia. Patient 2 was again a locally advanced low lying rectal cancer extending to the posterior wall of the vagina. For patient 1, no differences in terms of DSC, HD and MDA were found, when using or not the platform, on both Intra-OV (same RO; day 1 vs day 2) and Inter-OV (different ROs; day 1 vs ground-truth). The participants had a high degree of self-consistency since the mean DSC was 0.95 for Intra-OV regardless of the use of the platform and the mean MDA was below 1 mm even with no Anatom-e used. For Inter-OV, mean DSC was 0.80 and mean MDA around 3.8 mm, independently of the platform. The high consistency within and among ROs can be explained by the low number in delineation variables in case 1, with very standard prophylactic volumes to be included in the CTV (mesorectum, presacral space, bilateral internal iliac and obturator nodes) and a very visible mesorectal node close to the mesorectal fascia to drive the volume selection and definition. The other explanation might be that most of the participants were trained in the same centre during the residency program and hence they shared a common background knowledge and contouring approach. In clinical case 2, self-consistency was again very high with a DSC around 0.9 regardless of Anatom-e. Conversely, the inter-OV was significantly decreased by the use of the Anatom-e platform. The DCS for day 1 vs ground truth was significantly influenced by the use of Anatom-e (0.72 vs 0.65 without platform; p = 0.03), as an effect of a higher overlap of all contours with the reference CTV. The mean MDA was lower with the use of the platform (3.61 mm vs 4.14 mm; p = 0.21), but with no statistical significance; the use of Anatom-e decreased the SD from 2.97 to 1.33. That means that the mean distance between the tested contours and the reference volumes were on average lowered by the use of Anatom-e. At the same time the dispersion of the values around the mean values was lowered, as an effect of the increase in homogeneity of the delineation process. Clinical case 2 had a higher number of delineation variables compared to case 1, with also bilateral external iliac and inguinal nodes to be included in the CTV. Moreover, the involvement of the vagina increased the complexity of the selection and delineation of treatment volumes introducing a region of uncertainty represented by the anterior aspect of the CTV. The anterior aspect of the nodal regions within the pelvis is, in general, a source of potential disagreement, because anatomical boundaries are less clear. Moreover, the infiltration of the posterior wall of the vagina pushed ROs to extend anteriorly the CTV to cover the area of tumor spread, but to a different extent depending on the contouring RO. The visual evaluation of delineation variation according to pelvic sub-regions, confirmed the variability for the anterior border of the CTV (cranially to the bladder) and for the anterior border of the lateral lymphnodes, which is a critical boundary since no easily recognizable landmarks are present at that level. This was one of the reason for the increased Inter-OV, which was shown by the lower DSC for case 2 compared to case 1 without the platform (0.65 vs 0.80). This variability was mitigated by the use of the Anatom-e platform which increased the DSC up to 0.72, lowered the mean HD and decreased the SD for HD. This was evident also by visual inspection of the anterior aspects of all contours obtained, which had a higher overlap with the use of the system. This is also in line with the data of Nijkamp et al., where the benefit of the implementation of delineation guidelines based on adigital atlas in rectal cancer patients was particularly observed in the reduction of the disagreement among operators in the anterior region of treatment volumes [22]. The increase of homogeneity in the contouring process has been shown to have a dosimetric impact on target coverage in rectal cancer patients undergoing pre-operative RT [23]. This may have an influence on the quality of the whole RT process and finally on patient’s outcome [24]. Improvement in interactive teaching for treatment volume delineation is also a major need for education and training of professionals in radiation oncology (especially trainees and young specialists) [25]. International professional societies such as ESTRO, the European Society for Radiation Oncology, developed an educational project denominated as FALCON (Fellowship in Anatomic deLineation and CONtouring) to increase the homogeneity in the delineation process, comparing individual contours with endorsed guidelines or expert opinions [26]. This initiative, based on short and interactive workshops, was shown to be effective in reducing Inter-OV in specific clinical contexts [27].

Conclusion

The use of a digital platform such as Anatom-e decreased the inter-observer variability among operators in the delineation process of CTV for locally advanced rectal cancer patients with a complex disease presentation planned to receive neoadjuvant RT. This system may be helpful in increasing the compliance to follow shared guidelines and protocols, potentially reducing discrepancies and discordances in delineated treatment volumes.

Declarations

Ethics approval and consent to participate

Approval for the present study was given by the Review Board of the Department of Oncology of the University of Turin. Written informed consent was acquired from all patients with respect to RT treatment and clinical data management for research purposes.

Consent for publication

Not applicable.

Availability of data and material

Materials in the manuscript are available by contacting the corresponding author at pierfrancesco.franco@unito.it.

Competing interest

We declare that we do not have any competing interest.

Funding

No specific funding was received for the present manuscript.

Authors’ contribution

PF, FA, CF, EG: study conception and design; ET, SM, GCI: data collection; CP, FM, MGR, VT, CB, GP, RV: delineation of clinical target volumes; RR: data analysis; PF; manuscript draft; OB, UR, OB: final revision and approval. All authors read and approved the final manuscript.

Financial support

None to declare.

Conflict of interest

I declare that there is no conflict of interest for the present manuscript.