What is the reliability of non-trained investigators in recognising structural MRI lesions of sacroiliac joints in patients with recent inflammatory back pain? Results of the DESIR cohort.

OBJECTIVE: The objective of this study was to evaluate the reliability of recognising structural lesions on MRI (erosions, fatty lesions, ankylosis) of the sacroiliac joints (MRI-SIJ) in clinical practice compared to a central reading in patients with a possible recent axial spondyloarthritis (axSpA).
METHODS: Patients aged 18-50 years, with recent (<3 years) and chronic (≥3 months) inflammatory back pain, suggestive of axSpA were included in the DEvenir des Spondyloarthrites Indifférenciées Récentes (DESIR) cohort. MRI-SIJ structural lesions were scored by non-trained local readers, and by two trained central readers. Local readers scored each SIJ as normal, doubtful or definite lesions. Central readers scored separately each type of lesion. The central reading (mean of the two central readers' scores) was the external standard. Agreement (κ) was calculated first between local (3 definitions of a positive MRI-SIJ) and central readings (9 definitions), and then between the two central readers.
RESULTS: 664/708 patients with complete available images were included. Agreements between local and central readings were overall 'fair', except when considering at least 2 or 3 fatty lesions and at least 3 erosions and/or fatty lesions where agreement was 'moderate'. Agreement between central readers was similar. MRI-SIJ was positive for 52.6% of patients according to central reading (at least 1 structural lesion) and for 35.4% of patients according to local reading (at least unilateral 'doubtful' or 'definite' structural lesions).
CONCLUSIONS: Agreement on a positive structural MRI-SIJ was fair to moderate between local and central readings, as well as between central readers. The reliability improved when fatty lesions were considered. TRIAL REGISTRATION NUMBER: NCTO 164 8907.

Recent data suggest that structural lesions on MRI of the <span class="Disease">sacroiliac joints (<span class="Disease">MRI-SIJ) may be used instead of structural damage on conventional radiographs in the classification of patients.1

Reliability of non-trained readers as well as trained readers is fair to moderate to recognise structural lesions of SIJ on MRI in <span class="Species">patients with recent <span class="Disease">inflammatory back pain suggestive of spondyloarthritis.

Reliability between local and central readings and between central readers is better when considering at least two <span class="Disease">fatty lesions.

Structural lesions are difficult to identify on <span class="Disease">MRI-SIJ and need to be better defined to be used for <span class="Disease">axial spondyloarthritis diagnosis in clinical practice.

Introduction

Diagnosing <span class="Disease">axial spondyloarthritis (ax<span class="Gene">SpA) in an early stage of the disease is a challenge in clinical practice. Evolution of axSpA can lead to irreversible structural damage and impact the quality of life. There is more and more evidence that early treatment may change the outcome in patients with axSpA.2 3 The Assessment of SpondyloArthritis international Society (ASAS) developed classification criteria relevant for early axSpA.4 According to these criteria, patients with at least one SpA feature may be classified in case of structural lesions on X-ray of the sacroiliac joints (X-SIJ) or active sacroiliitis on MRI.5 Although only inflammatory lesions (bone marrow oedema (BME) and osteitis) have been selected in the ASAS criteria, structural lesions (erosions, fatty lesions, sclerosis, ankylosis) are also visible on MRI of the sacroiliac joints (MRI-SIJ), and have been described by the ASAS/OMERACT MRI group.6 The European League Against Rheumatism recommends taking into account inflammatory lesions as well as structural lesions when diagnosing axSpA.7 Radiographic structural lesions of SIJ are still assessed by the same grading, first described in the New York criteria.8 However, it has been shown to be difficult to identify and discriminate structural lesions on X-SIJ between different observers,9 and training does not seem to improve it.10 These findings were recently confirmed in patients with recent-onset chronic inflammatory back pain (IBP) of the DEvenir des Spondyloarthrites Indifferenciées Récentes; Outcomes in patients with Recent-onset Undifferentiated Spondyloarthritis (DESIR) cohort.11 Furthermore, radiographic sacroiliitis may appear several years after the onset of symptoms,12 delaying the diagnosis of ankylosing spondylitis (AS).13 MRI-SIJ could be an alternative to X-SIJ, especially in early axSpA. It has been recently demonstrated in the DESIR cohort that MRI-SIJ could be reliably used instead of, or in addition to, X-SIJ to assess structural lesions without significant changes in the classification according to ASAS axSpA criteria.1 Previous studies have shown a good reliability of MRI-SIJ to detect structural lesions.14–16 However, in these previous studies, MRI-SIJ were scored by trained readers, blinded for clinical information, while in daily practice scoring is performed by local radiologists or rheumatologists, who have access to clinical data. The ability of these local non-trained readers (ie, not specifically trained for the assessment of structural lesions on MRI-SIJ, but trained as a radiologist or rheumatologist) in various centres to detect structural lesions on MRI-SIJ, instead of a centralised reading, is unknown. The main objective of this study was to evaluate the reliability of non-trained investigators in recognising structural lesions on MRI-SIJ of patients with recent inflammatory back pain, suggestive of axSpA in the DESIR cohort, compared to trained central readers. To put the data in context, the agreements between the two central readers are also presented as well as agreements between ‘MRI-SIJ structural lesions’, according to the local reading, and ‘radiographic sacroiliitis’ according to the local and central reading.

Methods

Study design

This study is a cross-sectional study of baseline data of the multicentre prospective longitudinal DESIR cohort.17

Patients

All <span class="Species">patients of the DESIR cohort with available <span class="Disease">MRI-SIJ and X-SIJ at baseline were included. The DESIR cohort has already been described extensively.17 In short, patients aged between 18 and 50 years and suffering from recent (<3 years) and chronic (>3 months) IBP (in the buttocks, lumbar or thoracic spine) fulfilling either the Calin18 or the Berlin criteria19 were recruited in 25 regional centres in France. Symptoms must be suggestive of axSpA according to the local investigator's assessment with a score ≥5/10 (0=not suggestive; 10=very suggestive for axSpA).17 In total, 708 patients were included between December 2007 and April 2010. The baseline database was locked on 30 October 2012. The cohort was approved by an ethics committee and complied with good clinical practices. A detailed description of the organisation of the cohort, centres, protocol and collected data is available at http://www.lacohortedesir.fr/desir-in-english/. This study was approved by the scientific committee of the DESIR cohort. All patients signed informed consent.

Collected data

Baseline demographic parameters including age and gender, clinical parameters and laboratory test results including acute phase reactants and <span class="Species">human <span class="Gene">leucocyte antigen (HLA)-B27 antigen status were collected using a standardised case report form.

MRI-SIJ assessment

<span class="Disease">MRI-SIJ assessment were performed on a 1 or 1.5 T MRI machine, using coronal oblique T1-weighted fast spin echo and short τ inversion recovery (STIR) sequences, with 12–15 slices of 4 mm thickness. <span class="Disease">MRI-SIJ structural lesions were scored first by the local reader and subsequently by two central readers.

The local readers were radiologists or rheumatologists in each centre with possible access to clinical data. They did not participate in any training session and were instructed to score each SIJ in 3 grades, according to the presence/absence of structural lesions (defined as typical <span class="Disease">sclerosis, erosions, bony bridges or <span class="Disease">ankylosis). Grade 0 was corresponding to normal, grade 1 to doubtful and grade 2 to definite structural lesions.

The central readers first participated in a calibration session (see below). The scoring was mainly based on a T1-weighted sequence, but the readers had free access to the STIR sequence. The central readers were blinded for clinical and other imaging data and for the local readers' results. We have used the adapted <span class="Disease">SpondyloArthritis Research Consortium of Canada scoring system by Weber et al20 to assess <span class="Disease">MRI-SIJ. This method is based on the assessment of lesions (present vs absent) on six consecutive slices through the SIJ, in its cartilaginous compartment (ie, its anteroinferior portion), starting on the slice on which at least 1 cm of the cartilage is visible, from anterior to posterior. Each SIJ is divided into four quadrants. Fatty lesions were only marked present by the readers if appearing with a distinct border and a homogeneous pattern. The readers only marked fatty lesions and erosions present if observed on at least two consecutive slices, resulting in a maximum score of 40 per lesion per patient (20 per SIJ). Ankylosis was considered present if seen on a single slice, with a maximum score of 24 per patient (note that ankylosis always involves two quadrants, either upper iliac and upper sacral or lower iliac and lower sacral quadrants). The presence of sclerosis was not taken into account for this study, since the interest of including sclerosis in structural scores has been debated: most scores did not include it14 20–22 and when it has been included, sclerosis usually showed low reliability.23 The mean score of the two central readers was used for each lesion to obtain the central reading score; no adjudicator has been used.

Several definitions were used to define a positive <span class="Disease">MRI-SIJ for structural lesions by local and central readers, at both the SIJ and <span class="Species">patient levels (table 1).

Table 1

Definitions of positive MRI-SIJ and X-SIJ for structural lesions in local and central readings

Definitions of positive MRI-SIJ
Patient level		Definitions of positive X-SIJ
Local reading		Local reading
Definition 1	At least unilateral grade ≥1	Definition 1	At least unilateral ‘DESIR grade’ 2
Definition 2	At least unilateral grade 2	Definition 2	At least bilateral ‘DESIR grade’ 2 or unilateral ‘DESIR grade’ 3
Definition 3	Bilateral grade 2	Definition 3	Bilateral ‘DESIR grade’ 3
Central reading		Central reading
Definition 1	Presence of ≥1 erosion or fatty lesion or ankylosis (partial or total)	Radiographic sacroiliitis	Bilateral ‘mNY grade ≥2’ or at least unilateral ‘mNY grade≥3’
Definition 2	≥1 erosion
Definition 3	≥2 erosions
Definition 4	≥3 erosions
Definition 5	≥1 fatty lesion
Definition 6	≥2 fatty lesions
Definition 7	≥3 fatty lesions
Definition 8	Presence of ankylosis (partial or total)
Definition 9	≥3 erosions and/or fatty lesions

mNY, modified New York criteria.24 MRI-SIJ, MRI of the sacroiliac joints.

Definitions of positive <span class="Disease">MRI-SIJ and X-SIJ for structural lesions in local and central readings

mNY, modified New York criteria.24 <span class="Disease">MRI-SIJ, MRI of the <span class="Disease">sacroiliac joints.

Definition of a positive MRI-SIJ for structural lesions

Concerning the local reading, the <span class="Disease">MRI-SIJ was considered positive if at least one of the two SIJs was scored ‘doubtful or abnormal’, that is, ‘at least unilateral grade 1’ (definition 1). In sensitivity analyses, we also considered: ‘at least unilateral grade 2’ (definition 2), and ‘bilateral grade 2’ (definition 3) as a positive <span class="Disease">MRI-SIJ (table 1).

For the central reading, in the absence of a generally accepted definition of a positive structural <span class="Disease">MRI-SIJ, we considered the following definition: ‘presence of ≥1 erosion or ≥1 <span class="Disease">fatty lesion or ≥1 ankylosis in at least one SIJ’ (definition 1). We also considered eight additional definitions (table 1).

X-SIJ assessment

<span class="Disease">X-SIJs were performed in the anteroposterior view. Similar to <span class="Disease">MRI-SIJ, baseline X-SIJ were read by the local investigators, and later by two trained central readers. The procedure has been previously described in detail.11 In short, the local readers scored each SIJ according to a method derived from the modified New York criteria (mNY):24 grades 2 and 3 were pooled in one combined grade 2. Thus, grade 0 was corresponding to normal, grade 1 to doubtful, grade 2 to definite sacroiliitis and grade 3 to ankylosis. The central readers used the mNY criteria scoring method, in which sacroiliitis is defined as grade ≥2 bilaterally or grades 3–4 unilaterally. In case of disagreement, an experienced radiologist served as the adjudicator. We considered several definitions of a positive X-SIJ for local and central readings (table 1).

Definition of a patient with radiographic sacroiliitis

For the local reading, radiographic <span class="Disease">sacroiliitis was defined as at least one SIJ scored ‘grade 2 DESIR’ (definition 1). As sensitivity analyses, we also considered two other definitions (table 1). For the central reading, <span class="Disease">sacroiliitis was defined according to the mNY criteria (table 1).

Training of the central readers

The central readers of the <span class="Disease">MRI-SIJ and X-SIJ have participated in calibration sessions. The calibration process with the same readers has been extensively described previously.11 25 After agreements between central readers were moderate to good (κ=0.4–1.0) regarding the various types of structural lesions on <span class="Disease">MRI-SIJ, the two central readers could start reading all available baseline MRI-SIJ of the DESIR cohort.

Statistical analysis

Inter-rater agreements were calculated using Cohen's κ,26 27 and positive and negative percent agreements (PPA/NPA).28 All κs were interpreted according to the standards proposed by Landis and Koch; values<0 indicated no agreement, 0–0.20 slight, 0.21–0.40 fair, 0.41–0.60 moderate, 0.61–0.80 substantial and 0.81–1 almost perfect agreement.27

Central reading was assessed by the mean score of central readers. For sensitivity analyses, we also considered lesions recognised by both central readers and lesions scored by only central reader 1 or only central reader 2.

For the primary objective, agreement was calculated between the local reading and the central reading, which served as an external standard, regarding the different definitions of a positive <span class="Disease">MRI-SIJ. Next, we investigated which type of lesions (reported by the central reading) contributed most to the global assessment of the local readers (3 definitions). As a sensitivity analysis, agreement at the SIJ level was also assessed. Then we calculated the agreement between central readers regarding the nine definitions of a positive structural <span class="Disease">MRI-SIJ with regard to the number of each structural lesion (erosion, fatty lesions and ankylosis) with various cut-off values (≥1 to ≥5) based on a patient's level. Prevalence of each type of lesion (score≥1) was also assessed.

Finally, agreement was calculated between the various definitions of a positive <span class="Disease">MRI-SIJ and a positive <span class="Disease">X-SIJ, first for the local reading and second for the central reading as the external standard. Sensitivity and specificity of MRI-SIJ compared to X-SIJ were calculated, with the X-SIJ as the external standard.

Analyses were performed using Stata SE V.12.

Results

Complete <span class="Disease">MRI-SIJ baseline data were available in 664 <span class="Species">patients. Their mean age was 34.9 (SD 8.7) years and mean symptom duration 24 (SD 22.3) months; 309 (46.5%) patients were men and 388 (51.4%) were HLA-B27 positive; 409/582 (70.2%) were fulfilling the ASAS criteria for axSpA.25

Prevalence of structural lesions scored by central readers

On the basis of the mean scores of the two central readers, 52.6% of <span class="Species">patients had at least one structural lesion including erosions (32.2%), fatty lesions (29.2%) and ankylosis (24.2%) (table 2). When looking at the lesions agreed on by both central readers, 29.1% of patients had at least one structural lesion including erosions (9.3%), fatty lesions (14.6%) and ankylosis (5.2%). For each type of lesion, central reader 1 scored systematically fewer lesions than central reader 2.

Table 2

Prevalence of various MRI structural lesions scored by central readers at the patient level

Patients with at least one of the following types of lesions	Reader 1, N (%)	Reader 2, N (%)	Agreement of both central readers*, N (%)	Mean of the two central readings†, N (%)
Any type of lesions	277 (41.7)	348 (52.4)	193 (29.1)	349 (52.6)
Erosions	82 (12.3)	256 (38.6)	62 (9.3)	214 (32.2)
Fatty lesions	162 (24.4)	186 (28)	97 (14.6)	194 (29.2)
Ankylosis (partial or total)	122 (18.4)	172 (25.9)	39 (5.9)	161 (24.2)

*Number of patients with at least one lesion scored by both readers.

†Number of patients with at least one lesion based on the mean score of the two central readers.

Prevalence of various MRI structural lesions scored by central readers at the <span class="Species">patient level

*Number of <span class="Species">patients with at least one lesion scored by both readers.

†Number of <span class="Species">patients with at least one lesion based on the mean score of the two central readers.

Agreement between local reading and central reading in recognising structural lesions on <span class="Disease">MRI-SIJ.

The κ coefficients, PPA and NPA, between the local reading and the central reading for each definition are shown in table 3 (complete data in supplementary table 1). Local and central readings were concordant for 66.3% of <span class="Species">patients according to the presence/<span class="Disease">absence of abnormalities (definitions 1). Overall, agreements were better for definitions considering at least two or at least three fatty lesions and at least three erosions and/or fatty lesions in central reading and if a more stringent definition was considered in local reading. According to the local reading (definition 1), 35.4% of patients had a positive MRI-SIJ for structural lesions and according to the central reading (definition 1), 52.6% of patients had a positive MRI-SIJ for structural lesions; 8.3% of patients with a positive MRI-SIJ according to the local reading were not recognised by the central reading and 25.5% of patients with a negative MRI-SIJ according to the local reading were marked positive by the central reading.

Table 3

Agreements between local and central readings for various definitions of a positive MRI-SIJ for structural lesions

	MRI central reading (mean of the two central readers)
	Definition 1: ≥1 erosion or ≥1 fatty lesion or ankylosis		Definition 2: ≥1 erosion		Definition 3: ≥2 erosions
MRI local reading	PPA (%)	NPA (%)	PPA (%)	NPA (%)	PPA (%)	NPA (%)
Definition 1: at least unilateral grade ≥1	61.6	69.9	54.3	76.7	48.6	80.5
	κ (95% CI): 0.34 (0.27 to 0.40)		κ (95% CI): 0.31 (0.24 to 0.39)		κ (95% CI): 0.31 (0.24 to 0.39)
Definition 2: at least unilateral grade 2	53.5	69.8	54.3	79.9	49.8	85.0
	κ (95% CI): 0.29 (0.22 to 0.35)		κ (95% CI): 0.35 (0.27 to 0.43)		κ (95% CI): 0.35 (0.27 to 0.43)
Definition 3: bilateral grade 2	44.3	69.5	48.0	80.8	50.0	88.3
	κ (95% CI): 0.24 (0.18 to 0.29)		κ (95% CI): 0.31 (0.24 to 0.39)		κ (95% CI): 0.38 (0.30 to 0.47)
	Definition 4: ≥3 erosions		Definition 5: ≥1 fatty lesion		Definition 6: ≥2 fatty lesions
Definition 1	38.0	80.2	55.9	79.0	52.3	81.8
	κ (95% CI): 0.24 (0.17 to 0.30)		κ (95% CI): 0.35 (0.28 to 0.43)		κ (95% CI): 0.36 (0.29 to 0.43)
Definition 2	40.8	85.6	56.5	83.3	54.2	86.3
	κ (95% CI): 0.28 (0.20 to 0.37)		κ (95% CI): 0.40 (0.32 to 0.48)		κ (95% CI): 0.41 (0.33 to 0.49)
Definition 3	46.4	90.1	52.1	85.1	56.9	89.9
	κ (95% CI): 0.37 (0.28 to 0.46)		κ (95% CI): 0.38 (0.30 to 0.46)		κ (95% CI): 0.47 (0.38 to 0.55)
	Definition 7: ≥3 fatty lesions		Definition 8: ankylosis		Definition 9: ≥3 erosions and/or fatty lesions
Definition 1	46.9	82.9	30.3	70.4	56.8	81.5
	κ (95% CI): 0.34 (0.27 to 0.41)		κ (95% CI): 0.02 (-0.05 to 0.10)		κ (95% CI): 0.39 (0.32 to 0.47)
Definition 2	50.2	87.7	26.3	75.1	57.6	85.6
	κ (95% CI): 0.40 (0.31 to 0.48)		κ (95% CI): 0.01 (-0.06 to 0.09)		κ (95% CI): 0.43 (0.35 to 0.51)
Definition 3	56.2	91.8	24.1	79.5	57.0	88.3
	κ (95% CI): 0.48 (0.39 to 0.57)		κ (95% CI): 0.04 (-0.04 to 0.12)		κ (95% CI): 0.46 (0.38 to 0.54)

κ in bold represent κ >0.40.

MRI-SIJ, MRI of the sacroiliac joints; NPA, negative percent agreement; PPA, positive percent agreement.

Agreements between local and central readings for various definitions of a positive <span class="Disease">MRI-SIJ for structural lesions

<span class="Disease">MRI-SIJ, MRI of the <span class="Disease">sacroiliac joints; NPA, negative percent agreement; PPA, positive percent agreement.

Agreement between the two central readers for various definitions of a positive MRI-SIJ for structural lesions

Agreement between the two central readers was ‘slight’ to ‘fair’ (κ=0.19–0.29) when considering definitions including at least one structural lesion (<span class="Disease">fatty lesion, erosion or <span class="Disease">ankylosis) (definition 1) or only erosions (definitions 2, 3 and 4; table 4). For definitions considering at least two or at least three fatty lesions and at least three erosions and/or fatty lesions, agreement was better but still only moderate (κ=0.44–0.51). With an increasing number of fatty lesions (from ≥1 to ≥5 fatty lesions), agreement increased (κ=0.40–0.59) (supplementary table 4). Conversely, with an increasing number of erosions (from ≥1 to ≥5 erosions), the agreement decreased (κ=0.22–0.07) (supplementary table 5). When considering definition eight including only ankylosis, agreement was ‘slight’.

Table 4

Agreement between central readers for various definitions of a positive MRI-SIJ for structural lesions on patient level

	Definition 1: ≥1 erosion or ≥1 fatty lesion or ankyloses	Definition 2: ≥1 erosion	Definition 3: ≥2 erosions	Definition 4: ≥3 erosions	Definition 5: ≥1 fatty lesion
PPA (%)	61.8	36.7	32.7	25.7	55.7
NPA (%)	66.0	78.4	84.3	88.4	84.3
κ (95% CI)	0.29 (0.22 to 0.36)	0.22 (0.16 to 0.29)	0.23 (0.15 to 0.30)	0.19 (0.11 to 0.27)	0.40 (0.32 to 0.48)
	Definition 6: ≥2 fatty lesions	Definition 7: ≥3 fatty lesions	Definition 8: ankylosis	Definition 9: ≥3 erosions and/or fatty lesions
PPA (%)	55.3	58.8	26.5	56.8
NPA (%)	88.9	92.2	79.1	85.7
κ (95% CI)	0.45 (0.36 to 0.53)	0.51 (0.42 to 0.60)	0.06 (−0.01 to 0.14)	0.44 (0.36 to 0.51)

κ in bold represent κ >0.40.

MRI-SIJ, MRI of the sacroiliac joints; NPA,negative percent agreement; PPA, positive percent agreement.

Agreement between central readers for various definitions of a positive <span class="Disease">MRI-SIJ for structural lesions on <span class="Species">patient level

<span class="Disease">MRI-SIJ, MRI of the <span class="Disease">sacroiliac joints; NPA,negative percent agreement; PPA, positive percent agreement.

Sensitivity analyses

At the SIJ level, agreements between local and central readings and between the two central readers were similar between the left and right SIJ (data not shown).

Comparing local reading and each central reader separately, central reader 1 had lower agreement for definitions 3 and 4 (≥2 or ≥3 erosions) than the mean of both central readers (supplementary tables 2 and 3).

Agreement between a positive MRI-SIJ for structural lesions by local Reading and radiographic sacroiliitis

In total, 649 <span class="Species">patients had complete data for this analysis (ie, local reading of <span class="Disease">MRI-SIJ and local and central readings of X-SIJ). About 21.1% of patients had AS according to mNY criteria in the central reading. Compared to the radiographic local reading, agreements were ‘moderate’ at best (from 0.38 to 0.55) when considering the two less stringent radiographic definitions (definitions 1 and 2) (table 5). Compared to the radiographic central reading, at the patient level, κs were moderate for the various MRI-SIJ definitions (from 0.40 to 0.51). In both comparisons, MRI-SIJ sensitivity was better for a less stringent MRI-SIJ definition and specificity was better for a more stringent MRI-SIJ definition.

Table 5

Agreement between various definitions of a positive MRI-SIJ for structural lesions (local reading) and radiographic sacroiliitis

	X-ray local reading						X-ray central reading (2/3)
	Definition 1: at least unilateral ‘DESIR grade’ 2		Definition 2: at least bilateral ‘DESIR grade’ 2 or unilateral ‘DESIR grade’ 3		Definition 3: bilateral ‘DESIR grade’ 3		bilateral ‘mNY grade ≥2’ or at least unilateral ‘mNY grade≥3’
MRI local reading	PPA (%)	NPA (%)	PPA (%)	NPA (%)	PPA (%)	NPA (%)	PPA (%)	NPA (%)
Definition 1: at least unilateral grade ≥1	63.9	83.5	51.9	83.3	2.6	78.5	56.0	82.6
	κ (95% CI): 0.48 (0.41 to 0.55)Se=73.9%. Sp=78.6%		κ (95% CI): 0.38 (0.31 to 0.45)Se=83.7%. Sp=73.6%		κ (95% CI): 0.01 (−0.01 to 0.03)Se=75.0%. Sp=64.7%		κ (95% CI): 0.40 (0.33 to 0.47)Se=75.2%. Sp=75.0%
Definition 2: at least unilateral grade 2	66.9	87.9	56.7	88.4	3.4	85.0	60.4	87.7
	κ (95% CI): 0.55 (0.48 to 0.62)Se=65.9%. Sp=88.4%		κ (95% CI):0.46 (0.38 to 0.54)Se=75.0%. Sp=82.9%		κ (95% CI): 0.02 (−0.01 to 0.05)Se=75.0%. Sp=74.0%		κ (95% CI): 0.48 (0.40 to 0.56)Se=67.9%. Sp=84.8%
Definition 3: bilateral grade 2	59.0	87.9	61.9	92.1	4.9	90.0	60.9	90.4
	κ (95% CI): 0.48 (0.40 to 0.55)Se=49.4%. Sp=93.2%		κ (95% CI):0.54 (0.45 to 0.63)Se=66.3%. Sp=90.8%		κ (95% CI): 0.04 (−0.01 to 0.08)Se=75.0%. Sp=82.0%		κ (95% CI): 0.51 (0.40 to 0.60)Se=56.9%. Sp=92.0%

κ in bold represent κ >0.40.

MRI-SIJ, MRI of the sacroiliac joints; NPA,  negative percent agreement; PPA, positive percent agreement; Se, sensitivity; Sp, specificity.

Agreement between various definitions of a positive <span class="Disease">MRI-SIJ for structural lesions (local reading) and radiographic <span class="Disease">sacroiliitis

<span class="Disease">MRI-SIJ, MRI of the <span class="Disease">sacroiliac joints; NPA,  negative percent agreement; PPA, positive percent agreement; Se, sensitivity; Sp, specificity.

Discussion

In <span class="Species">patients with recent chronic IBP suggestive of ax<span class="Gene">SpA, the agreement between readings of structural lesions on MRI-SIJ by two trained central readers and by local readers is ‘moderate’ at best. The agreement between local readers and the various definitions of the central reading were always low (‘slight’ or ‘fair’) except for the definitions considering ‘2’ or ‘3’ fatty lesions or ‘at least 3 erosions and/or fatty lesions’, where the agreements reached a ‘moderate’ level. These results show that the reliability of the routine reading, in this context, is overall not good. Although local readers did not independently score each type of lesion, these results also suggest that local readers may recognise fatty lesions better than other types of structural lesions. However, we have to keep in mind that local and central scorings were different, which may have impacted the assessment of agreement.

Agreement between central readers is overall not very different from agreement between local and central readings. Surprisingly, a ‘fair’ agreement between the two central readers was found for the first definition, although it was the loosest definition (‘presence of at least one erosion or at least one fat deposition, or presence of partial or total <span class="Disease">ankylosis’) and the lowest agreement was found for definition 8 regarding the presence of <span class="Disease">ankylosis. Despite the calibration sessions of the readers, reader 1 systematically scored fewer lesions than reader 2, in particular with regard to erosions. Thus, reader 1 was more specific and reader 2 more sensitive, leading to a low agreement between the two readers. Reader 2 may have considered physiological abnormalities as structural lesions (especially for erosions), which might question the validity of the calibration process. However, the same two central readers also scored BME lesions in the DESIR cohort with a similar calibration method and obtained a much better κ (0.73).25 Moreover, despite this limitation, considering the agreements between local reading and each central reader separately, the agreement for erosions was better with reader 2 than with reader 1 and agreement for fatty lesions was similar. This suggests that the fairly low agreement may be rather due to the general difficulty in distinguishing structural lesions from physiological abnormalities and not (only) to the calibration and reading methods. Moreover, global agreement for structural lesions (including erosions, fatty lesions and ankylosis) seems better in studies including a high proportion of patients with AS (κ=0.84),14 or when the median symptoms duration is longer (κ=0.76–0.80).29 Puhakka et al23 included patients with a short symptom duration (median disease duration=19 months) excluding patients with AS, and found an agreement similar to our study (κ=0.42).

Contrary to erosions and <span class="Disease">ankylosis, fatty lesions contributed most to a better agreement between central readers and between central and local MRI readings. The consideration of at least two or, even better, three fatty lesions increased the reliability between readers. Those results are in agreement with the findings of a recent study30 which defined a cut-off of at least three fatty lesions or 5 fatty lesions and/or erosions to define a positive MRI-SIJ with <5% of patients without axSpA fulfilling this definition. In previous studies, agreement seemed usually better for erosions in AS and for fatty lesions in recent axSpA.14 16 23 31 32 Weber et al33 suggested that assessment of fatty lesions in the SIJ may have a diagnostic utility in early axSpA with doubtful erosions or BME, but that only fatty lesions with a distinct border or a homogeneous pattern (and not subchondral lesions) should be considered. In our study, we only marked fatty lesions if there were a distinct border and a homogeneous pattern. Recently, a T1-weighted opposed-phase gradient-echo (opGE) sequence has shown promising results and may improve reliability in detecting erosions.34

Prevalence of structural lesions was low in our study compared to most of the previously published studies. This difference can be partly explained by the lower proportion of <span class="Species">patients with AS in our study (21.1% according to mNY criteria by central readers, and 70.2% of ax<span class="Gene">SpA according to the ASAS criteria25), whereas other studies included 60.7–73.5% of patients with AS.15 32 35 As expected, the prevalence of structural lesions is higher in patients with AS than in non-radiographic patients with axSPA.29 30 32 Part of these results can be explained by the pathophysiology of axSpA: fatty lesions appear to follow resolution of inflammation and of erosions in axSpA.36 37

Previous studies have compared MRI central readings to X-ray central readings with heterogeneous results (Se=49–84% and Sp=61–98%).12 19 Agreements between <span class="Disease">MRI-SIJ local readings and <span class="Disease">X-SIJ local readings and between MRI-SIJ local readings and X-SIJ central readings in our study were moderate. These results confirm that structural lesions are difficult to assess by MRI in clinical practice (local reading) when compared to the gold standard (X-SIJ central reading) and to what is currently performed in clinical practice (local X-SIJ reading). However, the low reliability of X-SIJ readings, which has been confirmed in the DESIR cohort,11 may interfere with the current analysis. Furthermore, this analysis was limited by the use of non-validated definitions for the X-SIJ local reading.

A limitation of this study was the absence of comparison with CT. CT is known to be reliable for the assessment of the bony structures, including erosions, <span class="Disease">sclerosis and ankylosis and is a better external standard than radiographs.38 39 However, it has been chosen in the DESIR cohort not to add CT scan because of time, costs and irradiation dose.

It should also be noted that both local and central readers had access to the STIR sequences and that the presence of BME lesions may have biased their scoring by influencing the global appraisal. However, this is also the situation in which the <span class="Disease">MRI-SIJ will be used in clinical practice. Moreover, the local readers also might have had access to the radiographs while scoring the <span class="Disease">MRI-SIJ, while the central readers were completely blinded for the other imaging modality.

Finally, this study may have been limited by the restriction of structural lesions scoring to their presence on two consecutive slices (which is the rule for BME lesions), contrary to previous studies in which their presence on a single slice was often sufficient. This may have hampered the comparison with previous studies.

This study also has several strengths. The DESIR cohort has included a large number of <span class="Species">patients in whom a standardised <span class="Disease">MRI-SIJ was systematically performed at baseline. Previous studies reporting on this topic mostly included <100 patients with axSpA.14 16 23 29 31 40 The DESIR patients had recent disease onset with symptoms suggestive of axSpA at baseline, but the diagnosis was not necessarily confirmed. The assessment of structural lesions on MRI-SIJ is of particular interest in this type of patients to assist in diagnosing axSpA. Results from previous studies assessing structural lesions in patients with a confirmed diagnosis and long-standing disease14 29 31 cannot be extrapolated to this category of patients.

In conclusion, the reliability of non-trained readers as well as trained readers to recognise structural lesions on <span class="Disease">MRI-SIJ in <span class="Species">patients with recent IBP suggestive of SpA is overall ‘fair’ to ‘moderate’. Fatty lesions may contribute most to a positive structural MRI-SIJ assessment by non-trained and trained readers.