MRI interobserver reliability in rectal tumor angulation.

BACKGROUND: When rectal tumors are examined using magnetic resonance imaging (MRI) the perpendicular angulation of the axial T2-weighted image to the tumor axis is essential for a correct measure of the shortest distance between tumor and mesorectal facia.
PURPOSE: The purpose of this study was to determine the interobserver variability in rectal tumor angulation between a radiologist and a radiographer.
MATERIAL AND METHODS: Two observers performed the angulation independently. All MRI examinations were performed using an MRI 1.5 Tesla unit. A Bland-Altman plot was used to assess the interobserver variance and Intraclass correlation coefficient (ICC) statistic was used to assess the interobserver reliability.
RESULTS: MRI was performed in 55 patients with rectal cancer during a one-year period (25 (45.5%) women and 30 (54.5%) men). The median age was 71 years (range 46-87 years). The rectal tumor mean length was 3.9 cm. The interobserver reliability was good (ICC = 0.83, 95% confidence interval 0.72-0.90).
CONCLUSION: Radiographers receiving training will be able to perform MRI rectal tumor angulation.

Introduction

In 2012, the first consensus guideline on Magnetic Resonance Imaging (MRI) in rectal cancer diagnostic was published by the European Society of Gastrointestinal and Abdominal Radiology (ESGAR). The guideline states the importance of having trained radiographers to perform rectal tumor MRI angulation. Depending on the tumor size and placement, the axial and coronal T2-weighted imaging should be angulated perpendicular and parallel to the tumor axis from the upper and middle part of the rectum. Angulation of low rectal tumors may be performed perpendicular and parallel to the tumor axis or the anal canal. Measurement from the tumor wall to the mesorectal fascia is determinant of the tumor staging and preoperative treatment. The ESGAR guideline describes state-of-the art MRI rectal protocol using a surface coil on a 1.5 or 3.0 Tesla MRI system. The MRI rectal protocol as a minimum should include T2-weighted imaging in three planes and a diffusion weighted imaging (DWI) with minimum one high b value (≥800). A slice thickness of ≤3 mm is recommended for axial and coronal image sequences.[1,2]

The demand for imaging worldwide is increasing, and in some regions and countries it may not be possible to hire enough radiologists to meet the growing demand. Consequently, the radiographer needs to perform tumor angulation without assistance by a radiologist. Today, MRI is a standard examination in the staging of rectal cancer,[2,4,5] and in some hospitals, it is normal procedure for abdominal radiologists to perform the rectal tumor angulation when planning MRI images sequences. The objective of the study was to investigate if radiographers can perform this task safely and adequately. The aim of this study was to investigate MRI interobserver reliability and agreement between a radiologist and a radiographer performing MRI rectal tumor angulation. To the best of our knowledge there is no literature on the subject.

Material and methods

Design

This was a retrospective interobserver study performed at the Department of Radiology, Vejle Hospital, University Hospital of Southern Denmark. The hospital has status as Center of Clinical Excellence in the treatment of colorectal cancer.

Patients

From January 1 to December 31, 2019, 117 potentially eligible patients were scheduled for MRI investigation of the rectum. The inclusion criteria were age ≥18 years, rectal cancer diagnosis, and no treatment or surgery initiated. Sixty-two patients were excluded (Figure 1) due to follow-up scans and other diagnoses such as anal cancer and rectal polyps leaving 55 rectal cancer patients for the study. The diagnosis was subsequently confirmed by histopathology reports (47 patients underwent rectal surgery and eight rectal biopsies).

Figure 1.

Patient flow diagram.

Magnetic resonance imaging

All MRI examinations were performed by a 1.5 Tesla unit (Phillips Healthcare, Ingenia, unit release 4.1.33, Best, Netherlands) using an anterior coil. The MRI included T2-weighted sequences in the coronal, sagittal, and axial plans and a diffusion-weighted sequence including five b-values. The patient was placed in the prone position. The MRI scan parameters are shown in Table 1.

Table 1.

Magnetic resonance imaging scan protocol.

Sequences	T2-weighted axial	T2-weighted axial	T2-weighted coronal	T2-weighted sagittal	DWI
Coil	Anterior coil	Anterior coil	Anterior coil	Anterior coil	Anterior coil
TR (µs)	4232	4232	2500	2500	2500
TE (µs)	100	100	90	90	90
Slice thickness (mm)	5	3	4	3	5
Gap (mm)	1.5	0.5	1.5	0.5	0.5
FOV (mm2)	270 × 270	270 × 270	270 × 270	270 × 270	160 × 140
Matrix	336	480	480	480	128
Flip angle (°)	90	90	90	90	90
b-values	—	—	—	—	0, 300, 500, 800, 1000

DWI: Diffusion weighted imaging; FOV: Field of view; TR: time to repeat; TE: time of echo.

The MRI rectal tumor angulations were performed separately by two observers, that is, a radiologist with more than 20 years of experience in colorectal diagnostics and a radiographer with 5 years of experience in MRI including MRI of rectal cancer. Angulations were performed by the two observers using the same image diagnostic screen (21.3″ Monitor CCL358i2 from: Totoku, JVCENWOOD Corporation, Kanagawa, Japan) within 3 days of each other. Prior to the observer study the radiographer received a 30-min case-based introduction from a trained abdominal radiologist on how to perform and measure rectal tumor angulation. The observers had no access to patient information such as previously imaging and reports and did not have the opportunity to discuss the cases with colleagues. The two observers were blinded to each other’s MRI tumor angulation.

Individually, the two observers first measured the specific angulation in each rectal tumor and then made an indication of the tumor angle direction, that is, whether the angle was positive or negative.

The standard position or y-axis was set as the cranio-caudal direction of the patient in supine position. The x-axis is anterior-posterior plane perpendicular to the supine position. Angles above the x-axis were defined as positive and angles below as negative angles. Examples of how to perform the tumor angulation are provided in Figures 2 and 3.

Figure 2.

A 64-year-old female with a 3.7 cm rectal tumor 14 cm above the anal verge seen on the sagittal image. The 3 mm axial images were obtained with an angulation of 100° using the sagittal as planning and showed a T3 tumor from 2 o’clock to 4 o’clock in the left side of the rectal wall, with an outgrowth of 2 mm. The distance of more than 5 mm from the mesorectal fascia was confirmed at histopathology.

Figure 3.

A 70-year-old male with a 5.2 cm tumor 10 cm above the anal verge. With an angulation of 60°, the axial images showed a T3 tumor from 9 o’clock to 6 o’clock with an outgrowth of 6 mm. The distance to the mesorectal fascia was only 2 mm.

The distance to the anal verge as well as tumor height and width was obtained from the radiology report archived in the hospital’s Picture Archive Communication System. The tumor stage of patients having undergone surgery was obtained from the histopathology report.

Ethics

Due to the retrospective design of the study, approval by the regional Ethics Committee was not required. The study was approved by the institutional Review Board (May 2020) of the University Hospital of Southern Denmark and the local Danish Data Protection Agency.

Statistical analysis

STATA statistical software (version 17.0, STATA, Corporation, College Station, TX, USA) was used for analyzing all data including standard deviation (SD) and 95% confidence interval (CI). Intraclass correlation coefficient (ICC) was used to assess the interobserver reliability in the tumor angle (including positive and negative degree). We used a two-way random effects model to estimate the interobserver ICC with 95% confidence intervals (CI). The ICC used the following cut-offs: less than 0.50 = poor, 0.50–0.75 = fair, >0.76–0.90 = good and above 0.90 = excellent. Bland–Altman plot was used to visualize the interobserver agreement. The Bland–Altman plot was calculated with limits of agreement (LoA) assuming a normal distribution of differences.

Results

The study population (N = 55) consisted of (25 (45.5%) women and 30 (54.5%) men). The overall median age was 71 years (range 46–87 years) and in women and men it was 73 and 71 years, respectively.

The mean length and width of rectal tumors was 3.9 cm (SD 0.21, 95% CI 3.45–4.29) and 3.0 cm (SD 0.15, 95% CI 2.71–3.29), respectively. The mean distance from the rectal tumor to the anal verge was 9.9 cm (SD 0.49, 95% CI 8.90–10.88).

The mean tumor angulation was 44.3° (SD 2.79, 95% CI 38.8–49.8).

The mean difference between the radiologist and the radiographer in performing MRI rectal tumor angulation was 5.1° (Figure 4).

Figure 4.

Bland–Altman plot of tumor average (x-axis) and tumor difference (y-axis) in 55 patients. The horizontal solid line represents the mean difference (5.1°), and the dashed horizontal lines correspond to the 95% limits of agreements.

Histopathological tumor stage was not available for the eight patients who did not undergo surgery. Four patients had a tumor stage T1, 14 had T2, 26 had T3 (T3 = 15, T3a = 4, T3b = 3, T3c = 3 and T3d = 1), and three patients had a tumor stage T4. Table 2 shows the distribution of tumor stage, size and distance from the lower tumor edge to the anal verge. Measuring the rectal tumor angle (positive or negative direction) the ICC was good (ICC = 0.83, 95% CI 0.72–0.90).

Table 2.

Tumor length, height, and distance from tumor to the anal verge in T1–T4 tumors.

Variable (mean)	T1 (n = 4)	T2 (n = 14)	T3 (n = 26)	T4 (n = 3)
Length (cm)	2.9	3.5	4.4	3.9
Height (cm)	2.2	2.4	3.4	3.3
Anal verge (cm)	8.5	9.1	11.1	14.3

Discussion

This study focuses on the interobserver reliability of MRI tumor angulation in patients with rectal cancer. We found a good interobserver reliability between the experienced radiologist and the MRI radiographer, indicating that experience may not necessarily be essential in this clinical aspect. The average discrepancy between the two observers was only 5.1° which is not clinically important and would not require a re-scan to ensure correct measurement of the distance to the mesorectal fascia.

The literature holds papers on interobserver reliability between radiographers and radiologists in many areas[7-11] but to our knowledge the present study is the first to report interobserver reliability within MRI angulation of rectal tumors.

It is important to provide insight in the differences or similarities in image perception between radiographers and radiologists. Kjelle et al. found that 45% of radiographers reject images mainly based on suboptimal positioning, whereas 55% of the radiologist reported this as the main reason for image rejection. The study also found radiographers and radiologists to highly agree on image acceptability. Torres-Mejia et al. found it possible for radiographers to interpret screening mammography investigations. In general, radiographers perform well, when given the opportunity and training. We advocate for radiologists to use dedicated trained radiographers for MRI tumor angulation.

In the present study the radiographer had none and the radiologist more than 20 years of experience in tumor angulation. With time and more training, it is likely that the radiographer will perform even better than the present results. Also, radiographers performing MRI rectal tumor angulation will save valuable time and lower the number of interruptions for the radiologist. In the event of any interruption, there is a risk of error.

We found a systematic variation of 5.1°, which in daily clinical would have no diagnostic effect on the distance measurement of distance between and mesorectal fascia. Radiographers should be thoroughly trained before starting to perform rectal tumor angulation. The use of a 3D T2-weighted sequence could theoretically avoid these difficulties, but available evidence is lacking and the ESGAR expert panel did not recommend this approach as a replacement for multiplanar 2D T2-weighted sequences.

Images before and after chemo-radiotherapy should be angled identically. The radiologist found this to be optimal, when comparing the images and when interpreting the diffusion images after chemo-radiotherapy. Likewise, DWI should be performed with the same angulation of T2-weighted images, used as a reference and include high b-values (≥800).

This study has some limitations. A higher validity can be reached using more than two observers, but this was not possible at the time the study was carried out. Furthermore, it is a limitation that we did not perform intra-rater agreement in this study, as this is a way to examine the observers’ measurements quality. Assessing interobserver reliability is not always straightforward when choosing a statistical approach. Measurement always involves some degree of error. Therefore, we included a Bland–Altman plot and found an acceptable mean difference of 5.1°. We had relative wide LoA. LoA estimate the interval within which a proportion of the differences between measurements lie. The limits of agreement include both systematic (bias) and random error (precision), and provide a useful measure for comparing the likely differences between individual results measured by two methods. The 95% limits of agreement as the mean difference 1.96 SD. Our LoA of approximately ±15° seems wide, because ±15° is a 30° maximal difference. This is only 8% of 360 possible degrees. We think this is an acceptable maximal margin of error, but the research in this area is sparse. A new study shows an example with 90° different angles give rise to different conclusions. Results can vary considerably depending on whether the “axial” plane is angled perpendicular to the tumor axis, true axial or parallel to the tumor axis. More clear guidelines are needed on the preferred anatomical plane to assess rectal tumors on MRI since this should be considered as an important technical aspect.

In conclusion, the finding of this study supports healthcare professional engagement in imaging and tumor angulation. We found that the MRI rectal tumor angulation can be performed by a trained radiographer. Using this approach, we can provide a positive impact on the work environment of radiographers.