Evaluating the repeatability and set-up sensitivity of a large field of view distortion phantom and software for magnetic resonance-only radiotherapy.

BACKGROUND AND
PURPOSE: Magnetic Resonance (MR)-only radiotherapy requires geometrically accurate MR images over the full scanner Field of View (FoV). This study aimed to investigate the repeatability of distortion measurements made using a commercial large FoV phantom and analysis software and the sensitivity of these measurements to small set-up errors.
MATERIALS AND METHODS: Geometric distortion was measured using a commercial phantom and software with 2D and 3D acquisition sequences on three different MR scanners. Two sets of repeatability measurements were made: three scans acquired without moving the phantom between scans (single set-up) and five scans acquired with the phantom re-set up in between each scan (repeated set-up). The set-up sensitivity was assessed by scanning the phantom with an intentional 1 mm lateral offset and independently an intentional 1° rotation.
RESULTS: The mean standard deviation of distortion for all phantom markers for the repeated set-up scans was < 0.4 mm for all scanners and sequences. For the 1 mm lateral offset scan 90 % of the markers agreed within two standard deviations of the mean of the repeated set-up scan (median of all scanners and sequences, range 78%-93%). For the 1° rotation scan, 80% of markers agreed within two standard deviations of the mean (range 69%-93%).
CONCLUSIONS: Geometric distortion measurements using a commercial phantom and associated software appear repeatable, although with some sensitivity to set-up errors. This suggests the phantom and software are appropriate for commissioning a MR-only radiotherapy workflow.

Introduction

Magnetic Resonance (MR) imaging is increasingly being used within the radiotherapy planning pathway to delineate tumours and organs at risk. Delineation uncertainties are currently some of the largest uncertainties within the radiotherapy planning and delivery pathway [1], with the poor soft tissue contrast of Computed Tomography (CT) images being a significant reason [2]. MR has superior soft tissue contrast which reduces inter-observer variability in delineation and so reduces that uncertainty [3]. In addition functional MR scans such as diffusion weighted and dynamic contrast enhanced have shown potential for demonstrating active tumour sub-volumes which could receive dose boosts [4], [5], [6]. All these advantages have motivated incorporating MR imaging into the radiotherapy planning pathway.

In current practice the radiotherapy planning MR is typically registered with the planning CT, with the MR delineated contours being associated with the CT via this registration. However the MR-CT registration has its own uncertainty, which will contribute to the overall delineation uncertainty [7]. This has driven investigations into an MR-only radiotherapy planning pathway which aims to use the good soft-tissue contrast of the MR without the uncertainty of a MR-CT registration [8], [9].

It is essential that images used for radiotherapy planning are geometrically accurate and MR images can suffer from significant geometric distortions [10]. MR image reconstruction assumes that the combination of the static magnetic field () and the gradient magnetic fields will vary linearly with position and uses this variation to encode spatial information [11]. Therefore inhomogeneities in the static magnetic field, gradient non-linearities and patient magnetic susceptibility effects will all cause geometric distortion by causing the magnetic field to vary non-linearly with position [12]. These first two causes are often called system distortions because they depend on the scanner (and acquisition sequence) but not on the patient [13]. This enables them to be characterised for a particular scanner and acquisition sequence. The system geometric distortions tend to be largest towards the edges of the MR scanner field of view [14]. For most diagnostic imaging and for radiotherapy planning utilising MR-CT registration this is not a significant issue because the anatomy of interest will typically be positioned in the centre of the field of view, where the distortion is least. However geometric distortion at the periphery of the field of view is a serious concern for MR-only radiotherapy planning since distortions in the patient external contour could significantly affect the calculated dose distribution and introduce errors in patient set-up [15], [16], [17]. This makes it vital to be able to measure the geometric distortion over the entire scanner field of view.

MR geometric distortion in general has been well-studied [18] and there have been several studies investigating geometric distortion throughout the entire scanner field of view using in-house phantoms and analysis software. These phantoms have consisted either of orthogonal grids [19], [20] or 3D arrays of points [21], [22], [23]. These studies have demonstrated that the geometric distortion can be measured throughout the entire scanner field of view and that gradient distortion correction software can significantly reduce the measured distortions. Several commerical large field of view distortion phantoms have also been investigated [24], [25]. Tadic et al. demonstrated that measurements of the geometric distortion at a limited number of points, combined with a spherical harmonic analysis method, enables the geometric distortion across the full field of the scanner to be characterised [24]. Antolak et al. demonstrated that the vendor supplied gradient distortion correction software reduced the measured distortions on both MR scanners and combined MR-radiotherapy treatment machines [25].

A large field of view distortion phantom is important both in optimising MR sequences for radiotherapy planning purposes and in carrying out regular quality assurance testing of the MR scanner. In order to use this phantom it is important to ensure that the measurements made are repeatable and to characterise the sensitivity of the measurements to small set-up errors. Wang et al. investigated repeat geometric distortion measurements without moving the phantom between acquisitions and demonstrated very small differences [19]. However this does not take into account any set-up differences, which are necessary to produce a repeatability measurement which applies to the clinical use of the phantom. Price et al. investigated the set-up repeatability of their phantom using repeat CT acquisitions with independent phantom set-up for each acquisition [23]. This does not assess the repeatability of distortion measurements made by the phantom however, just the repeatability of the phantom set-up. To the best of our knowledge the repeatability of geometric distortion measurements using a large field of view phantom and software have not been reported in the literature.

The purpose of this study was to fill this gap by evaluating the repeatability and set-up sensitivity of a commercial large field of view phantom and associated software for distortion measurements on three different MR scanners in three different centres. The focus of this study was evaluating the phantom for a MR-only radiotherapy workflow, however the results are relevant to the wider MR community.

Materials and methods

Distortion phantom and software

The phantoms used in this study were two GRADE phantoms (Spectronic Medical AB, Helsingborg, Sweden), consisting of approximately 1,200 small spherical markers at known positions embedded in expanded foam. This enables the phantom to be large enough to assess the full scanner field of view but still weigh less than . The markers are made of polyethylene glycol and have a diameter of . Spectronic Medical AB customise the external dimensions of the GRADE phantoms to a particular MR scanner. This customisation permits the full scanner field of view to be sampled whilst ensuring the phantom is small emough to fit in the scanner bore. Two different GRADE phantoms were used in this study, one customised for a 3 T Magnetom Prisma (Siemens, Erglangen, Germany) and one customised for a 1.5 T Magnetom Espree (Siemens). The two phantoms were of similar design but slightly different sizes to fit the different scanner bores. The second phantom was also used for measurements on a 3 T Signa PET-MR (GE Healthcare, Wisconsin, USA). An MR image of the second phantom is shown in Fig. 1. The markers are in a grid pattern with a spacing of approximately . The spacing between markers near the edges of the phantom is smaller (), to give a more precise measure of the distortion in the periphery of the scanner field of view.

Fig. 1

Axial (left) and sagittal (right) planes from MR images of the Spectronic Medical GRADE distortion phantom. The small central markers used to identify the phantom and determine its orientation and position can be seen in the middle of the axial view (red circle). (For interpretation of the references to colour in this figure caption, the reader is referred to the web version of this article.)

All distortion analyses were carried out using the associated Spectronic Medical AB GRADE evaluation software. This is an entirely automatic process with a number of steps. First a unique pattern of markers in the centre of the phantom (see Fig. 1) are analysed to identify the phantom and determine its orientation. Secondly the software carries out a deformable registration between the acquired MR image and the reference image of the phantom contained within the software. The deformation field is used to determine the geometric distortion in the MR image. Distortion is given on a marker by marker basis and for the nth marker is given bywhere is the x coordinate of nth marker in the image and is the reference x coordinate of the same marker. , , and are the y and z coordinates for the nth marker in the image and reference respectively. Each marker also has a unique reference number.

Scanning protocols

Three different scanners were used in this study: a 1.5 T Siemens Magnetom Espree (designated 1.5 T MR), a 3 T GE Signa PET-MR (3 T PET-MR) and a 3 T Siemens Prisma (3 T MR). Two sequences were used, a 2D fast spin echo and a 3D gradient echo, as recommended by Spectronic Medical AB. The scanning protocols used for each scanner and sequence are recorded in Table 1. The sequences used were based on recommended parameters provided by Spectronic Medical AB. The sequences used on the 1.5 T MR were modified to ensure sufficient contrast to noise ratio for the automatic analysis software to be able to identify the markers accurately. The modifications consisted of reducing the echo time to the minimum possible and using two 6 channel flexible receive coils (Siemens Body Matrix) and the 24 channel spine receive coil contained in the couch (Siemens Spine Matrix). In this paper a scanning session was defined as acquiring both sequences consecutively. The vendor supplied 3D distortion correction was applied to all scans.

Table 1

Protocols for the sequences used on the three scanners.

Parameter	1.5 T MR		3 T PET-MR		3 T MR
Sequence	2D TSE	3D VIBE	2D FSE	3D GRE	2D TSE	3D GRE
Field of view/mm	450		500		500
Acquisition matrix	448×448		512×512		512×512
Slice thickness/mm	2.5	1.0	2.5	1.0	2.5	1.95
Number of slices	128	256	200	508	128	256
Slice gap/mm	0.0	0.2	0.0	0.0	0.0	0.0
Repetition time/ms	11560	4.81	15000	3.8	15000	4.4
Echo time/ms	17	2.39	96	1.3	94	1.58
Refocusing flip angle/°	130	20	130	20	130	20
Bandwidth/HzPixel^-1	385	510	250	250	390	490
Receive Coil	2 flex coils + spine coil		Body coil		Body coil
Acquisition time/s	1077	553	1666	1244	960	540

Repeatability

Two repeatability investigations were carried out. The first consisted of three sequential scanning sessions acquired without moving the phantom and using the same automatic shim (referred to as the single set-up scans). The second consisted of five scanning sessions carried out using a new phantom set-up and automatic shim between each session (the repeated set-up scans). In between each scanning session for the repeated set-up scans the phantom was removed from the couch and then re set-up. Each phantom set-up was carried out as accurately as possible, using the scanner internal lasers and the cross-hair markings on the phantom. This placed the centre of the phantom as close to the scanner isocentre as possible. The five scanning sessions were carried out on the same day for the 1.5 T MR and 3 T PET-MR scanners and on two consecutive days for the 3 T MR scanner.

The distortion of each marker in the phantom (Eq. (1)) was measured using the automatic analysis software. For each scanner and sequence the mean and standard deviation of the distortion for each marker was calculated for the three single set-up scans ( for marker n) and the five repeated set-up scans (). We have assumed that the distortion for each marker is normally distributed. Occasionally the automated software would not detect some markers in the periphery of the images due to lack of signal. The number of markers undetected was small (maximum of the markers in the images for the 2D sequence on the 3 T PET-MR scanner). Only markers that were common to all images for a given set-up were used in the calculation. The mean standard deviation of all the markers for a given set-up was also calculated, using was the standard deviation of the marker using set-up and N was the number of markers common to all images in set-up . The range in distortion for each marker in the single set-up () and repeated set-up () scans was also calculated. For each scanner and sequence the mean range over all markers for a given set-up was determined ( and ).

Set-up sensitivity

The set-up sensitivity was investigated by acquiring a scan with the phantom set-up with an intentional lateral 1 mm offset. Secondly, and independently, the phantom was scanned with an intentional 1 degree rotation around the centre of the phantom cross-hair. These were considered the maximum set-up error that could occur in clinical use. Each scan included both the 2D and 3D sequences.

The distortion for each marker was compared to the repeated set-up mean distortion for that marker ( for marker n) and the mean standard deviation of distortion for that scanner and sequence (). The percentage of markers with measured distortions within two mean standard deviations of the mean distortion for that marker was calculated (i.e. within for marker n). The percentage of markers with distortions within of the mean distortion for that marker was also calculated (within for marker n). was chosen because the in-plane voxel size for all the acquired scans was .

Displacing or rotating the phantom will physically relocate the markers, meaning they will sample the distortion at different points to those in the repeated set-up measurements. In peripheral regions where the distortion changes rapidly this could impact the set-up sensitivity analysis. To assess the impact of the displacement, the marker positions after the 1 mm lateral offset and the 1 degree rotation were modelled and the distortion at these shifted positions interpolated from the repeated set-up mean distortion. The distortion was not extrapolated beyond what was measured in the repeated set-up experiment because the distortion gradient outside the phantom was unknown. Therefore markers with modelled positions outside the distortion field in the repeated set-up measurements were excluded (mean percentage of excluded markers for all scanners and sequences was for the 1 mm lateral offset images and for the 1 degree rotation images). The percentage of included markers in the set-up sensitivity images that agreed within two mean standard deviations of the interpolated mean distortion for that marker in the modelled position was calculated. A similar percentage within of the interpolated mean distortion was also calculated. The mean repeated set-up distortion for each marker was also compared to the distortion of that same marker at the modelled positions (i.e. laterally offset or rotated). The difference in distortion due to the displacement of the phantom for each marker was calculated. The percentage of markers with differences were determined.

Results

The geometric distortion of the MR images increased with distance from the isocentre (see Fig. 2). Only the distortion measurements for the repeated set-up are shown as the equivalent graphs for the single set-up are very similar. The distortion measurements of the phantom appear to be repeatable, with the repeated set-up mean standard deviation of distortion (defined in Eq. (2)) being and the mean range for all scanners and sequences (see Table 2). The maximum range of measured distortions for a given marker was , and for the 1.5 T MR, 3 T MR and 3 T PET-MR respectively. For distances from the isocentre the 3 T PET-MR maximum range was . This was for both sequences with the repeated set-up.

Fig. 2

The mean distortion for each marker as a function of distance from the isocentre for the 2D sequences (top) and 3D sequences (bottom). The results shown are for the repeated set-up.

Table 2

Table showing the mean standard deviation of distortion (see Eq. (2)) and mean range of distortion for each set-up.

Scanner	Sequence	Mean Standard Deviation/mm		Mean Range/mm
		Single	Repeated	Single	Repeated
1.5 T MR	2D	0.30	0.34	0.70	0.92
1.5 T MR	3D	0.26	0.32	0.61	0.86
3 T PET-MR	2D	0.32	0.32	0.74	0.88
3 T PET-MR	3D	0.12	0.21	0.27	0.58
3 T MR	2D	0.11	0.21	0.25	0.57
3 T MR	3D	0.11	0.20	0.24	0.55

The distribution of the standard deviations for each marker are shown in Fig. 3, Fig. 4 for the 2D and 3D sequences respectively. The single set-up mean standard deviation is less than or equal to the repeated set-up distribution for all scanners and sequences. The standard deviations of distortion for all markers in both sequences and set-ups were for both the 3 T MR and 1.5 T MR scanners. The 3 T PET-MR images had some larger standard deviations (up to for the 2D sequence).

Fig. 3

The standard deviation of distortion for each marker for the single and repeated set-ups for the 2D sequence. Four outliers for the 3 T PET-MR have been excluded in order to show the rest of the results in more detail. The excluded points were in the range to . The dotted line marks a standard deviation of distortion of , which was equal to the in-plane voxel size for the 2D sequence.

Fig. 4

The standard deviation of distortion for each marker for the single and repeated set-ups for the 3D sequence. One outlier () for the 3 T PET-MR has been excluded in order to show the rest of the results in more detail. The dotted line marks a standard deviation of distortion of , equal to the image voxel size.

Analysis of the lateral offset scans found (median of all sequences and scanners) of markers within two mean standard deviations (, given in Table 2) of the mean for that marker (range: 78%–93%) and (median) within of the mean (range: 94%–100%). 91%, 88% and of markers were within two standard deviations of the mean for the 1.5 T MR, 3 T MR and 3 T PET-MR respectively (median of both sequences). The equivalent percentages for of the mean (the in-plane voxel size) were 99%, 100% and .

Using the interpolated mean distortion at the modelled positions of the markers as the comparator found very similar results. The median percentage of markers within two mean standard deviations was (range: 80%–93%) and within was (range: 95%–100%) for all sequences and scanners. The difference in distortion between the modelled position and the repeated set-up mean distortion was small, with only of markers having a difference greater than (median of all scanners and sequences, range: 0.0%–0.6%).

Analysis of the rotation scans found (median of all sequences and scanners) of markers within two mean standard deviations of the mean for that marker (range: 69%–93%) and (median) within of the mean (range: 86%–99%). The proportion of markers within two standard deviations of the mean were 90%, 79% and for the 1.5 T MR, 3 T MR and 3 T PET-MR respectively (median of both sequences). Within of the mean these proportions were 98%, 97% and respectively.

Comparing the rotation images to the interpolated mean distortion at the modelled marker positions appeared to improve the levels of agreement, with the median percentage within two mean standard deviations being (range: 74%–94%) and within was (range: 91%–100%) for all sequences and scanners. The difference in distortion between the modelled and measured positions of the markers in the repeated set-up mean distortion was larger than for the lateral offset, with of markers having a difference greater than (median of all scanners and sequences, range: 0.0%–0.7%).

Discussion

MR-only radiotherapy requires geometrically accurate images across the whole scanner field of view, necessitating appropriate distortion phantoms for commissioning and quality assurance testing. This study has investigated the repeatability and set-up sensitivity of a commercial large field of view geometric distortion phantom and associated analysis software on three different MR scanners to determine if it could be used for these purposes. The distortion measurements of the phantom appeared to be repeatable, with the repeated set-up mean standard deviation of distortion being for all scanners and sequences. The phantom did show some sensitivity to the relatively large intentional set-up errors.

The measured geometric distortion in the MR images increased with distance from the scanner isocentre for all sequences and scanners. This is in agreement with previous studies of geometric distortion with large field of view phantoms [21], [22], [23]. For both sequences the 3 T PET-MR had the largest geometric distortions. However these large distortions () were all at from the scanner isocentre and were likely due to the 3 T PET-MR scans having the longest scan length and the lowest acquisition bandwidth. Truncating the 3 T PET-MR images to give the same scan length as the 1.5 T MR and 3 T MR gave the mean distortions for all markers except one being , which was comparable to the other scanners. This suggests the longer scan length of the 3 T PET-MR images was the primary reason for the large distortion measured. The 3 T PET-MR images were acquired with a longer scan length because both Siemens scanners limited the number of slices that could be acquired, thus limiting the physical scan length for a given slice thickness. The acquisition bandwidth on the 3 T PET-MR was the largest that could be set for these sequences and acquisition parameters. At distances closer to the isocentre the 1.5 T MR scanner had the largest calculated distortions. This was likely due to the higher specified static magnetic field inhomogeneity of the 1.5 T MR scanner, which was in a volume [26]. This was a higher field inhomogeneity than either the 3 T MR ( in a 50 cm Diameter Spherical Volume (DSV),  www.healthcare.siemens.co.uk) or the 3 T PET-MR ( in a 45 cm DSV) [27]. For all scanners the 2D sequences appeared to show larger amounts of geometric distortion than the 3D sequences, which may be due to the 2D sequence being acquired with a larger slice thickness (). The distortions observed on all scanners and sequences were larger than would be clinically acceptable for MR-only radiotherapy if the distortions occurred in the treated region (i.e. for clinical indications requiring images with this scan length). However the bandwidths used in all the sequences are significantly lower than would be used in clinical radiotherapy MR sequences [14]. This study was not investigating the geometric distortion of particular MR sequences and scanners but the repeatability of distortion measurements made using a commercial phantom and software.

The phantom and software appeared to acquire repeatable distortion measurements for all scanners and sequences. The repeated set-up mean standard deviation and mean range were less than the smallest voxel size acquired, suggesting the distortion measurements are precise and repeatable. The standard deviations of distortion for all markers in both sequences and set-ups are for both the 3 T MR and 1.5 T MR scanners. The 3 T PET-MR images have some larger standard deviations (up to ) but these are all at from the scanner isocentre, where there were very large distortions measured (). If the geometric distortion in the image is very large and similar to the peripheral marker separation distance () the deformable registration in the automatic software may not function correctly and deform the acquired image to match the incorrect marker in the reference image. The software would then report an incorrect value for the distortion of this marker. This may be the reason for the large standard deviations observed at the edges of the scan volume for the 3 T PET-MR images. Therefore the phantom may not be making accurate measurements in areas with very large amounts of distortion present. This should not have an impact for clinical scans where the maximum distortion values should be much lower [28].

The repeatability of distortion measurements made using large field of view phantoms has not been previously studied in the literature. Wang et al. found the mean differences in the position of grid points between repeated scans without moving the phantom (corresponding to the single set-up used in this study) to be [19]. Price et al. assessed the phantom set-up repeatability using CT acquisitions with independent set-ups and found the set-up repeatability was very high, with mean rigid registrations between images giving translations in each axis [23]. Both of these results are consistent with the high repeatability of distortion measurements reported in this study.

The mean standard deviation of distortion for the single set-up was less than or equal to the repeated set-up for all scanners and sequences (see Table 2). The mean range was also observed to be larger for the repeated set-up than the single set-up. For the 3 T MR and the 3D sequence on the 3 T PET-MR the single set-up mean standard deviation and mean range were approximately half of the repeated set-up. For the other scanners and sequences the single set-up was closer to the repeated set-up. This same pattern can be seen in Fig. 3, Fig. 4, where the distribution of the standard deviations between the single and repeated set-ups appear similar for the 1.5 T MR and the 2D sequence on the 3 T PET-MR, and more different for 3 T MR and the 3D sequence on the 3 T PET-MR. This suggests that the additional uncertainty caused by small differences in phantom set-up and shim (the cause of the differences between the single and repeated set-up) are not the only significant cause of the uncertainty in the distortion measurement. Reshimming lead to the static field inhomogeneity being similar but non-identical between imaging sessions, with the similarity depending on the positioning of the phantom and the repeatability of the automatic shim. This will result in a non-identical distortion field between imaging sessions [29]. Reshimming between each repeated set-up was essential in order for the repeatability measurements to be relevant to the clinical use of the phantom, since an automatic shim will be applied prior to phantom measurements during different imaging sessions. Our results suggest that small differences in phantom set-up and shim do not substantially impact on the repeatability of the measurements.

However, the set-up sensitivity measurements do appear to show that the phantom has some variation when intentionally set-up with an error. Some of this variation is due to the phantom sampling the geometric distortion at different physical points, which can be seen by the improvement in agreement when that shift is modelled. The improvement is very small for the lateral offset measurements but more substantial for the rotation measurements, where the median agreement within two mean standard deviations goes from to . This is likely due to the markers at inferior and superior ends of the phantom being displaced by several mm by the rotation, and the distortion field changing more rapidly in those peripheral regions. This implies that the phantom needs to be set-up carefully in order for the measurements to be repeatable.

However even the worst set-up sensitivity result had of markers within of the mean of that marker for both set-up sensitivity scans, which is within the in-plane resolution of the scans used. This suggests that the phantom measurements are robust to within one voxel, even in the presence of the set-up errors tested. The magnitude of set-up error experienced in practice is likely to be less than the intentional errors assessed in this study. Therefore the measured set-up sensitivity should not impact on the repeatability of the measurements as long as the set-up is performed accurately. To the best of the author’s knowledge the sensitivity of distortion phantoms to set-up errors has not been assessed in the literature previously. Characterising this set-up sensitivity is an important result for ensuring accurate clinical use of the phantom. Future studies could explore the sensitivity of the distortion measurements to small changes in sequence parameters to further characterise the behaviour of the phantom.

In conclusion, the Spectronic GRADE large field of view MR distortion phantom and software appears to make repeatable measurements. The phantom does appear to show some sensitivity to large set-up errors (up to and ), implying care needs to be taken when setting the phantom up. This study suggests the phantom and software could be suitable for commissioning a MR-only radiotherapy workflow as well as routine MR quality assurance testing for radiotherapy planning.

Conflict of interest

The authors have no conflicts of interest to declare.