Does magnetic resonance imaging improve soft tissue sarcoma contouring for radiotherapy?

OBJECTIVE: Soft tissue sarcomas (STS) are a rare, heterogeneous tumour group. Radiotherapy improves local control. CT is used to plan radiotherapy, but has poor soft tissue definition. MRI has superior soft tissue definition. Contour variation amongst oncologists is an important factor in treatment failure. This study is the first to directly compare STS tumour contouring using CT vs MRI.
METHODS: Planning CT and T 2 weighted MR images of eight patients with STS were distributed to four oncologists. Gross tumour volume was contoured on both imaging modalities using in-house software. Images were recontoured 6 weeks later. The mean distance to agreement (DTA), standard deviation of the DTA, dice similarity coefficient (DSC) and contour volume were calculated for each oncologist and compared to a median contour volume. Results for CT and MRI were compared using a pairwise Student's t-test.
RESULTS: When comparing MRI to CT, tumour volumes were significantly smaller, with a difference of 21.4 cm3 across all patients (p = 0.008). There was not a statistically significant difference in the mean distance to agreement or dice similarity coefficient, but the standard deviation of the DTA showed a statistically significant improvement ( p = 0.04). For intraobserver variation, there was no statistically significant improvement using MRI vs CT.
CONCLUSION: Oncologists contour smaller tumour volumes using MRI, with reduced interobserver variation. Improving the reliability and consistency of contouring is needed for improved quality assurance. ADVANCES IN KNOWLEDGE: With further experience, the use of MRI in STS radiotherapy planning may reduce variation between oncologists and contribute to improved local control and reduced treatment toxicities.

Introduction

Soft tissue sarcomas (STS) are a rare, heterogeneous tumour group with an incidence of 4–5/100,000 people in Europe.[1] Radical treatment involves a combination of radiotherapy and surgery for intermediate or high risk disease.[2-4] Combined treatment has been shown to minimize local recurrence whilst maintaining function and moderating long term toxicity.[5-7]

Tumour volume contouring is undertaken on contrast-enhanced CT. However, CT has poor soft tissue contrast making contouring of STS difficult, as the extent of disease is poorly visualized. Indeed, a study by Wang et al demonstrated interobserver variation amongst radiation oncologists when contouring the clinical target volume of STS on pre-operative CT scans alone.[8] Difficulty in contouring, including deviations from protocol and variation in contouring tumour volumes, is an important factor in treatment failure.[9,10] Consensus amongst oncologists when contouring is important for quality assurance in radiotherapy treatment planning, with variation amongst oncologists often cited as the weakest link.

MRI promises improved soft tissue contrast, and offers both anatomical and functional imaging.[11] There is a growing interest in incorporating MRI into the radiotherapy planning pathway for a number of treatment sites, including STS.[12] The use of MRI has grown in STS, e.g. in diagnosis, staging and follow-up.[13] It is recommended that CT and MR images are co-registered for radiotherapy planning,[14,15] although this is not feasible in all centres. Furthermore, the use of MRI in STS radiotherapy planning has not been extensively studied. The major benefit of MRI for STS is that improved visualization of the disease will improve confidence in tumour volume contouring, reducing variation between radiation oncologists. Additionally, better visualization should result in smaller tumour volumes because the full extent of the patient’s disease can be identified. Ultimately, improved accuracy in tumour contouring alongside advanced radiotherapy techniques may allow for dose escalation and improved local disease control.[16,17] Improved accuracy may also minimize normal tissue toxicity to nearby critical structures, such as the brachial plexus in an upper limb STS. This may reduce acute and chronic radiotherapy related toxicities, preserving long-term function for patients.[13]

No published studies have directly compared CT only contouring to MRI only contouring for STS in the same patients. Therefore, the direct benefit of MRI in contouring STS has yet to be fully demonstrated. This work aims to directly compare, for the first time, inter- and intraobserver variation between oncologists contouring tumour volumes for STS on CT and MRI.

methods AND MATERIALS

Eight patients diagnosed with STS and treated with neoadjuvant radiotherapy prior to definitive surgery were selected from a previous research study. This previous study had received approval from the local ethics committee, and its primary objective was to correlate the histological response of soft tissue sarcoma after pre-operative radiotherapy with pre-treatment MRI parameters of DCE (dynamic contrast enhanced) MRI and DWI (diffusion-weighted imaging) MRI. Patient demographics are included in Table 1. All patients underwent pre-treatment radiotherapy planning contrast-enhanced CT and MRI in the supine position, at less than 1 week apart. CT and MRI protocols were the same for all patients. CT slices were 3 mm thick. For MRI, axial T 2 weighted turbo spin echo images were acquired and used for contouring in this study. The imaging parameters were: repetition time 3790 ms; echo time 99 ms; Matrix 448 × 448×29; Slice thickness 10 mm; ETL 13; field of view 400 × 400 mm.

Table 1.

Summary of patient demographics

Patient number	Gender	Age at diagnosis	Tumour location	Histology	FNCLCC grade	Maximum diameter (cm)
1	Male	72	Medial aspect left upper arm	Myxofibrosarcoma	2	9.0
2	Male	56	Left lateral thigh	Myxoid liposarcoma	2	6.0
3	Male	27	Medial aspect right knee	Extraskeletal myxoid chondrosarcoma	3	7.3
4	Male	29	Anterior aspect right lower leg	Spindle cell carcinoma	3	5.1
5	Male	41	Anterior aspect left lower leg	Myxoinflammatory fibroblastic sarcoma	1	2.7
6	Male	62	Medial aspect right forearm	Myxofibrosarcoma	3	5.1
7	Male	24	Lateral aspect left knee	Synovial sarcoma NOS	3	8.4
8	Male	73	Lateral aspect right anterior chest wall	Undifferentiated spindle cell sarcoma	3	5.2

FNCLCC, Fédération nationale des centres de lutte contre le cancer; NOS, Not otherwise specified.

For interobserver analysis, four radiation oncologists trained in contouring STS independently contoured the gross tumour volume (GTV) for each patient on both imaging modalities using in-house contouring software.[18] Contouring was performed in the axial plane using a soft-tissue level and window, each oncologist optimizing this to visualize the disease. CT and MRI scans were fully anonymized and oncologists were blinded to which CT and MRI belonged to the same patient.

Interpatient analysis was performed by first creating a median contour volume from the four oncologist contours. This was done as a whole volume and acted as the truth contour. All individual oncologist contours were compared against this median volume, calculating a signed mean distance to agreement (mDTA).[19] This approach looks for the closest distance between each point on the contour and the truth contour. The mDTA provides a good comparison of each oncologists contour against the truth volume across the whole volume. In addition to the mDTA, the standard deviation (SD) of the DTA between each oncologist and the truth contour was calculated. This indicates the level of agreement around the whole contour, with lower values showing better all-round agreement. Finally, the dice similarity coefficient (DSC) was calculated. This is the ratio of twice the volume of the union of the truth and test contour and the total volume of the truth and test volumes. DSC is summarized by the equation: DSC(A,B)=2(A∩B)/(A + B) where A and B are the truth and test contour volumes respectively, and ∩ is the volume of the intersection of the two contours. A value of 1 indicates perfect overlap and 0 no overlap.[19-21] The volume of each contour, on CT and MRI, was also collected.

Results were combined across all oncologists for each patient on CT and on MRI for comparison. A pairwise Student's t-test was used to test for statistical significance between CT and MRI based outcomes.

Additionally, intraobserver analysis was performed. Three patients (3, 7 and 8) with STS in the lower limbs and on the chest wall were randomly selected for recontouring. These patients were recontoured by three of the oncologists after a minimum delay of 6 weeks from initial contouring. Oncologists recontoured both the CT and MRI of each patient and were blinded to their first attempt, allowing a direct comparison of intraobserver variation on each imaging modality. mDTA, the SD of the DTA and the DSC were calculated by comparing the oncologist’s first contour against their second. Volumes of each contour were also collected to check for consistency. A pairwise Student's t-test was performed to test for statistical significance between each oncologist’s contours on CT and MRI.

Results

Figure 1 illustrates contouring for two representative patients, highlighting the improved soft tissue contrast of the STS on MRI compared with CT. The contours of the four oncologists are shown on both CT images (Figure 1A and B) and MRI images (Figure 1C and D), illustrating the benefit seen with MRI in reducing interobserver variation. All GTV delineations were returned except for one oncologist who did not contour the CT or MRI for Patient 3.

Figure 1.

Representative patient images CT (Figure 1a and b) and MRI (Figure 1c and d) images with contours for two representative patients

The results for interobserver variation showed significant improvement in using MRI for delineation. Figure 2 shows box and whisker plots for mDTA, SD of DTA, DSC and volumes. Table 2 summarizes these results, presenting the mean values across all oncologists and the results of the pairwise Student's t-test. The mDTA between CT and MRI based contours did not show a statistically significant improvement (p = 0.3) However, the SD of the DTA did show a statistically significant improvement on using MRI (p = 0.04). Figure 2b illustrates an improvement for six of the eight patients with the maximum SD on CT of 5.5 vs 4.6 mm on MRI. This indicates that although overall mDTA is not improved on MRI, the oncologists are contouring more consistently. Similarly to the mDTA, the DSC did now show a statistically significant difference in moving to MRI based delineation.

Figure 2.

Summary of results for interobserver variation. Box and whisker plots for mDTA (a), SD of the DTA (b), DSC (c) and volumes (d). For each patient, the box shows the 25th and 75th percentiles, the line shows the median, the dot shows the mean and the whiskers show the range. DTA, distance to agreement; DSC, dice similarity coefficient; SD DTA, standard deviation of the distance to agreement.

Table 2.

Summary of results for interobserver variation

	CT	MR	p
DTA (mm)	0.7	0.6	0.3
SD DTA (mm)	2.0	1.6	0.04
Volume (cm3)	88.7	67.3	0.008
DSC	0.84	0.87	0.1

DSC, Dice similarity coefficient; DTA, Distance to agreement; SD DTA, Standard deviation of the distance to agreement.

Finally, the mean volume contoured on CT was 88.7 cm3 compared to 67.3 cm3 on MRI (p = 0.008). Contouring on MRI resulted in a lower volume for every patient, with Patient 1 and Patient 7 showing a marked reduction in volume on MRI.

Intraobserver results are summarized in Table 3 and show no statistically significant improvement in using MRI. The SD of DTA and DSC showed a modest improvement but these were not significant. The volumes delineated by each oncologist showed slightly better consistency on MRI than CT. The ratio of volumes (median across all oncologists) delineated at each time point was 0.97 for CT and 0.99 for MRI. However, this was not a statistically significant improvement (p = 0.7).

Table 3.

Summary of results for intraobserver variation

	CT	MR	p
DTA (mm)	0.3	0.3	0.6
SD DTA (mm)	2.3	2.1	0.7
DSC	0.88	0.90	0.7

DSC, dice similarity coefficient; DTA, distance to agreement; SD DTA, standard deviation of the distance to agreement.

Mean values across all oncologists are shown, as well as the results of the pairwise Student's t-test.

Discussion

To our knowledge, this is the first study to directly compare inter- and intraobserver variation when contouring tumour volumes for STS on CT only and MRI only. Each patient was imaged with CT for radiotherapy planning and with anatomical T 2 weighted MRI acquired as part of a research study. Our results showed that oncologists uniformly contoured smaller tumour volumes using MRI vs CT, by 21.4 cm3 across all patients (p = 0.008). The improved soft tissue contrast provided by MRI allows the disease to be better visualized. This allows greater confidence in contouring with oncologists better able to minimize the volume they wish to treat. This reduction in volume has the potential to deliver reduced normal tissue toxicities for this group of patients, potentially leading to improved function.

Other groups have previously aimed to quantify inter- and intraobserver variation in STS contouring using CT or MRI.[8,22-24] Wang et al demonstrated interobserver variation amongst 10 radiation oncologists when contouring an upper limb sarcoma on CT. Briefly, oncologist agreement was assessed by calculating apparent volume overlap, and then correcting for agreement by chance using generalized κ statistics. Oncologists had MRI images available, but co-registration with CT images was not possible for all patients as MRI scans were not performed in the treatment position. A result of κ = 0.77 for upper limb clinical target volume contouring indicated less than perfect agreement according to Londis and Koch criteria. However, they did not comment on intraobserver variation.[8] Baldini et al similarly used κ statistics to demonstrate a high agreement between 11 radiation oncologists when contouring the GTV for two retroperitoneal STS patients using CT images.[23] The Roberge study used Boolean analysis to assess the degree of volume overlap as a measure of variation amongst five clinicians when contouring STS on MRI images, with a median tumour volume overlap of 79 and 93% for inter- and intraobserver contours respectively.[22] Sargos et al, distributed co-registered CT and MRI images to six radiation oncologists, demonstrating substantial agreement by κ statistics in a single pre-operative patient.[24] However, the above studies did not directly compare inter- and intraobserver variation between CT and MRI.

Other groups have compared inter- and intraobserver variation amongst CT and MRI for other tumour sites, with mixed results. Rasch et al observed reduced interobserver variation with MRI when contouring head and neck tumours.[25] Al-Hammadi et al reported improved generalized conformity index and accuracy index with MRI v s CT when contouring the post-operative breast cancer lumpectomy cavity for radiotherapy planning, indicating reduced interobserver variation. However, this study had low numbers of patients and oncologists.[26] Karki et al found no difference in interobserver variation between MRI and PET-CT for non-small-cell lung carcinoma radiotherapy planning, when using a bidirectional local distance measure to compare individual clinicians’ contours with a median contour.[27] Barkati et al observed a higher DSC using CT v s MRI for prostate radiotherapy planning, indicating increased interobserver variation with MRI.[28]

Several other groups have looked at the value of adding MRI images to CT images in reducing interobserver variation v s CT alone. Villeirs reported that the addition of MRI to CT reduced both the contoured volume and the interobserver variation for prostate radiotherapy planning v s CT alone.[29] The addition of MRI to CT has been observed to increase the concordance index when contouring high grade gliomas, indicating reduced interobserver variation.[30] CT and MRI image registration has been similarly shown to reduce interobserver variation in other central nervous system tumours.[31]

Interestingly, the mean DTA was not significantly different between MRI and CT (p = 0.3) in this study. However, the SD of the DTA was significantly smaller for MRI (p = 0.04). This suggests more consistent contouring of the tumour around the whole contour between the clinicians. Visual inspection highlighted better agreement in the superior–inferior extent of the disease. The extent of disease along the muscle is difficult to interpret on CT and this may be the great benefit of MRI. The better agreement on MRI in these areas may be the reason for the improvement in the SD of the DTA between oncologists.

No difference in intraobserver variation was found between MRI and CT. This result was surprising given the potential advantages in improved soft tissue delineation that MRI offers, and could partially be due to a small study sample and learned experience from the earlier contouring. Alternatively, the oncologists involved in the study are highly experienced in contouring with CT images, but less so with MRI. Further training and experience using MRI may allow the full potential of this modality to be realized. It may prove advantageous to repeat this study in the future once a higher level of experience has been reached.

There were several limitations to this study. It was conducted at a single centre, and due to the rare nature of STS only four trained oncologists were available for contouring, with only three performing the intraobserver delineation exercise. In the Wang et al and Baldini et al studies, 10 and 11 clinicians respectively returned contours.[8,23] Similarly, we only had 8 patients, limited by the availability of suitable patients with both CT and MRI available, whereas the Roberge et al study had 15.[22] A limitation for all studies looking at inter- and intraobserver contouring variation is that the results of individual clinicians are compared against other clinicians and the mean or median contour. There is a lack of a “Gold-Standard” for comparison of results. T weighted images allow visualization of peritumoral oedema, and the necessary inclusion of this oedema may have led to oncologists contouring larger tumour volumes. Indeed, work by White et al demonstrated that peritumoural signal changes seen on T weighted MRI scans ranged from 0 to 5.3 cm (mean 1.1 cm).[32] This could also account for increased interobserver variation amongst oncologists with MRI due to different interpretations of tumour tissue versus oedema. MRI has the capability to reflect a range of tissue properties. For example, gradient echo imaging may be used for detection of haemorrhage, and T 1 weighted fat suppressed sequences may differentiate haemorrhage from fat. Biopsiable solid tumour can be differentiated from necrotic tissue with gadolinium-enhanced sequences. Gadolinium may also allow for visualization of cystic components of tumours. Vascular structures are more easily visualized with MRI v s CT[33]⁠.

There are also limitations to the analyses. DSC is driven by volume, and large volume contours can have a higher DSC but display poor agreement. Conversely, smaller volume contours may have good agreement but poor DSC. This is illustrated in Figure 2 where patient five has the smallest volume (2d) and the worst DSC (2c) despite showing a reasonable mDTA (2a). DTA may be a better representation of consistency and agreement amongst oncologists. However, by generalizing values to a mean value across the entire contour, small local benefits may be hidden.

There are several challenges to an MRI-based radiotherapy planning system. Many radiotherapy departments lack a dedicated MRI machine and so would have to compete for timeslots with other departments and services for MRI planning scans. Thus, departments may not have access to MRI for all patients, and there are inherent challenges with co-registration of CT and MRI images.[34] MRI scanning is more costly than CT and scanning times are longer. This may be particularly problematic for comorbid patients who struggle to lie flat for extended periods. However, immobilization devices are being made MRI safe. It may be difficult to scan patients in a treatment position in the confines of the MRI bore whilst maintaining image quality, although larger 70 cm bore systems are becoming available at 1.5 and 3.0 T.[34] Additional receiver coil configurations have been developed, as well as devices which raise existing receiver coils above the patient to prevent coil weight disturbing patient anatomy. Such bridge coils are already becoming common in radiotherapy scanning 34⁠. There is potential for large errors to be introduced into the planning process due to geometric distortion in MRI images. The degree of distortion depends on many factors including scanner manufacturer, field strength, main magnetic field inhomogeneity, gradient non-linearity, sequence type, choice of sequence parameters such as bandwidth, and the distance of the region of interest from isocentre[35]⁠. For an individual scanner, rigorous quality assurance should be undertaken to ensure that geometric distortion is minimized. Manufacturer-supplied distortion correction algorithms are commonly applied to MR images, and these have been shown to reduce distortion to less than 2 mm.[36,37] This highlights the importance of rigorous QA and strict adherence to standard protocols when acquiring MR images for RT planning. Additionally, dose calculations may be difficult in the absence of Hounsfield units and density overrides or corrections applied. However, progress is being made in this field, and Philips have a commercial product for planning prostate radiotherapy using MRI.[38] The factors above will need to be considered once clinical benefit has been demonstrated.

Other groups have looked at ways of reducing contouring variation amongst oncologists. Bowden et al showed that providing oncologists with a clear protocol results in a reduction in interobserver variation in lung tumour contouring.[39] However, other work has shown that this effect may be reduced with more experienced oncologists.[40] Other suggestions include involving two oncologists in the contouring of a tumour, and Hollingdale et al recently completed a prospective study highlighting the feasibility of oncologists working with radiologists to increase accuracy and confidence in tumour contours.[41] The above work was carried out for CT imaging but similar principles could be applied to MRI. It was beyond the scope of our study to comment on the effect of the level of training. However, it is likely that comprehensive training would be needed if MRI were to be introduced as standard for radiotherapy planning in STS. Further work could involve a greater number of oncologists and patients, perhaps across several centres. It would be interesting to clarify the reason for increased variation with some images, the effects of training grade, or whether histological subtype is an important factor, and we could also look at the effect of different MRI sequences. For example, the addition of DWI could allow better visualization of tumour vs healthy tissue, and lead to smaller tumour contours.

In conclusion, we aimed to compare inter- and intraobserver variation amongst radiation oncologists when contouring tumour volumes for STS on CT only vs MRI only for the first time. We did not observe a significant difference in intraobserver variation between CT and MRI. However, we showed reduced interobserver variation using MRI, with oncologists delineating smaller volumes and more consistent contours. In radiotherapy treatment planning, accurate margins are important in allowing maximum dose to be given to the area of disease and to minimise dose given to normal tissue. As imaging technology advances, accurate contouring continues to be limited by oncologist interpretation of images. Improving the reliability and consistency of tumour contouring is needed for improved quality assurance. With further experience and training the use of MRI in STS may reduce variation between oncologists and contribute to improved local control and reduced treatment toxicities.