The influence of contrast enhancement and experience of observers on the assessment of mediastinal lymph nodes in sarcoidosis patients.

Purpose: The aim of this study was to assess the influence of contrast enhancement (CE) and experience of observers on the assessment of chest lymph nodes in patients with sarcoidosis. Material and methods: A retrospective analysis of chest lymph nodes on computed tomography (CT) examinations, including CE and non-contrast-enhanced (non-CE) phase, was performed on 40 patients with proven diagnosis of sarcoidosis. Phases were separated, anonymized, and randomized. The assessment was performed by 5 observers: 2 general radiologists, 2 residents, and a senior chest CT expert.
Results: There were no significant differences between radiologists and residents, apart from the determination of the 4R node short diameter on CE images. Agreement between the reference observer and both residents and specialists was equally high, without any significant difference in the assessment all chest nodes and hilar nodes, and between non-CE and CE images. There was a significant difference between all observers in the determination of the largest 4R node short diameter on non-CE images, but not on CE images. The number of affected node levels was found to be significantly higher when evaluated on CE images than on non-CE images. Compared to CE images, non-CE computed tomography has sensitivity of 0.94-1.00 and specificity of 0.98-1.00, depending on the observer. Conclusions: The application of contrast medium has a limited impact on the quality of assessment of the chest lymph nodes in patients with sarcoidosis, regardless of the experience of the observer.

Introduction

Sarcoidosis is a granulomatous disease of an unknown aetio-logy that most commonly involves the respiratory system. In the middle of the 20th century, a classification of pulmonary sarcoidosis based on radiographic findings was introduced by Scadding [1]. It is still in use and has not been modified despite its numerous diagnostic limitations [2-4]. Of course, computed tomography (CT) allows for a more accurate assessment of the lung parenchyma and thoracic lymph nodes than radiography. The high sensitivity and specificity of CT allow for differentiation between reversible and irreversible lesions in the pulmonary interstitium, which is the basis for prediction of the course of sarcoidosis. However, the cost of this greater accuracy is much higher exposition to ionizing radiation in CT than in conventional radiography.

The assessment of enlarged mediastinal and hilar lymph nodes is a part of sarcoidosis diagnostics and follow-up. Computed tomography of the chest is usually performed in 2 parts: as contrast-enhanced (CE) and non-contrast-enhanced (non-CE) acquisition. There are no guidelines on the use of pulmonary CT in the diagnosis and monitoring of sarcoidosis, despite the fact that CT is widely used in clinical practice. Therefore, it is uncertain whether the routine use of intravenous contrast agent for assessing nodal stage is beneficial [5,6]. Another question is whether contrast medium administration, because it increases CT image contrast, is helpful for less experienced observers, i.e. radiology residents [7].

The aim of this study was to assess the influence of contrast enhancement and experience of observers on the assessment of chest lymph nodes in patients with sarcoidosis.

Material and methods

The study was approved by the local Bioethical Committee.

A retrospective analysis of chest CT examinations of 40 patients with proven diagnosis of sarcoidosis was performed. Examinations consisted of non-enhanced and contrast-enhanced acquisitions that were acquired at the same time using the same CT unit (SIEMENS Somatom Emotion 16, Siemens Healthcare, Germany). The computed tomography protocol was as follows: collimation 16 × 1.2 mm, slice thickness 3 mm, and contrast medium at a dose of 1 ml/kg (Omnipaque 350 GE Healthcare, Oslo, Norway). Image reading was performed blindly to the patient’s personal data. Contrast-enhanced and non-enhanced datasets were disconnected, separately anonymized, and then loaded into a dedicated PACS folder in a random order.

The shorter diameter of mediastinal lymph nodes was measured. Image analysis was performed on a dedicated workstation (Syngo MMWP VE36A, Siemens AG, Germany) by 2 board-certified radiologists (S1, S2), and 2 residents who had completed their general radiology fellowship (O2, O3). A reference method (R) was provided by a senior thoracic radiologist who had 10 years of experience in CT of sarcoidosis imaging. In the case when her results were discordant with those of other observers, the cases were resolved by consensus. The use of both axial source images as well as MPR and MIP reconstructions was allowed.

Parametric data were given as mean values with their standard deviations (SD). Concordance between observers regarding non-parametric data was assessed using Cohen’s κ with its 95% confidence interval (95% CI). To compare an overall index of accuracy between radiologists and resi-dents, Light’s method was applied, suggesting computing κ for all coder pairs and then using the arithmetic mean of these estimates [8]. Concordance between observers regarding parametric data was determined using the intraclass correlation coefficient (ICC) as a 2-way model of absolute agreement of average measures. Significance of differences in parametric data was tested using repeated measures ANOVA with Bonferroni correction for pairwise analysis and with Student’s paired t-test. Test characteristics of non-CE CT in the detection of enlarged chest lymph nodes were calculated based on ROC curve analysis. Statistical significance was considered with p < 0.05. Calculations were performed using MedCalc v. 14.12.0 (MedCalc Software bvba, NL).

Results

A total analysis included 40 non-CE and 40 CE datasets for ANOVA analysis, i.e. 80 non-CE and CE datasets for comparisons between radiologists and residents, and 160 datasets for comparison between non-CE and CE images. Images were acquired from patients aged 43.7 ± 11.4 years on average, of whom 42.1% were males. The mean duration of sarcoidosis was 815 ± 1450 days, and 62.4% of subjects were symptomatic. There were no significant differences between radiologists and residents, apart from the determination of the 4R node short diameter on CE images: results of residents were lower than those of specialists (p = 0.0005) (Table 1). Interestingly, concordance between the reference method and both residents and specialists was equally high, without any significant difference in the assessment of all nodes and hilar nodes, and between non-CE and CE images (Table 2).

Table 1

The influence of contrast enhancement on the results of determination of the largest mediastinal chest node short diameter (mm), the largest 4R node short diameter (mm), and the number of affected node levels, by specialists and residents. Mean values with their 95% CIs

		non-CE	CE
Largest chest node short diameter
	p-value for ANOVA	0.010	0.219
	Radiologists	16.44 (15.57-17.30)	16.42 (15.48-17.37)
	Residents	15.96 (15.03-16.89)	16.65 (15.57-17.74)
	p-value for difference	0.3309	0.6194
Largest 4R node short diameter (mm)
	p-value for ANOVA	< 0.001	< 0.001
	Radiologists	19.88 (18.79-20.98)	19.55 (18.39-20.71)
	Residents	20.42 (19.20-21.65)	21.50 (20.26-22.74)
	p-value for difference	0.1465	0.0005
Number of affected node levels
	p-value for ANOVA	0.003	0.04
	Radiologists	3.81 (3.46-4.15)	3.56 (3.27-3.86)
	Residents	4.19 (3.87-4.51)	3.59 (3.31-3.87)
	p-value for difference	0.0885	0.8641

Table 2

Concordance between the reference method and other observers

		Light’s κ
		non-CE	CE
All nodes
	Radiologists	0.988	0.916
	Residents	0.964	0.952
Hilar nodes
	Radiologists	0.913	0.988
	Residents	0.976	1.00

There was a significant difference between all observers regarding measurement of the largest chest node diameter (ANOVA, Table 1). In a pairwise comparison of results of non-CE images, the only significant differences were between observer O2 and all other observers: O1 (p = 0.0005), S1 (p = 0.0042), S2 (p = 0.0080), and R (p = 0.0304). The pairwise comparison of results of CE images revealed that the only significant differences were between observer O2 and all other observers: O1 (p = 0.0003), S1 (p < 0.0001), S2 (p = 0.0022), and R (p = 0.0001).

There was a significant difference between all observers in the determination of the largest 4R node short diame-ter on non-CE images but not on CE images (ANOVA, Table 1). In a pairwise comparison of results based on non-CE-images, the only significant differences were between observer O1 and observer O2 (p = 0.0032). We also found a significant difference between all observers (ANOVA) regarding determination of affected node level (Table 1). In a pairwise comparison of results of non-CE-images, the only significant differences were between observer S1 and observers S2 (p = 0.0461) and R (0.0195). In a pairwise comparison of results of CE-images, the only significant difference was between observers O1 and S1 (p = 0.026).

Overall, contrast enhancement did not influence the results of short diameter measurement of the largest 4R node. However, the number of affected node levels was found to be significantly higher when evaluated on CE images than on non-CE images (Table 3). Compared to CE images, non-CE computed tomography had a sensitivity of 0.94-1.00 and specificity of 0.98-1.00 depending on the observer (Table 4).

Table 3

The influence of contrast enhancement on the determination of the largest mediastinal or hilar node short diameter (mm), the largest 4R node short diameter (mm), and the number of affected node levels. Mean values with their 95% CIs

Parameter	non-CE	CE	difference	p-value
Largest chest node short diameter	20.15 (19.34-20.97)	20.53 (19.67-21.38)	0.37 (–0.22 to 0.97)	0.2178
Largest 4R node short diameter	16.20 (15.57-16.83)	16.54 (15.83-17.25)	0.34 (–0.23 to 0.90)	0.2368
Number of affected node levels	3.58 (3.38-3.78)	4.00 (3.77-4.23)	0.42 (0.20-0.65)	0.0003

Table 4

Test characteristics of non-contrast enhancement (CE) vs. CE computed tomography in the detection of enlarged lymph nodes in particular observers

	Sensitivity	Specificity	AUC	p-value
O1	1.00	1.00	1.000	< 0.0001
O2	0.97	1.00	0.985	< 0.0001
S1	0.94	1.00	0.971	< 0.0001
S2	0.94	0.97	0.952	< 0.0001
R	1.00	0.98	0.992	< 0.0001

AUC – area under receiver operator characteristics curve

Discussion

Our results indicate that the application of contrast medium has a limited influence on the quality of assessment of chest lymph nodes in sarcoidosis patients, regardless of the experience of the observer. We found that for almost all parameters evaluated in this study, board- certified radiologists and radiology residents presented similar efficacy. Moreover, assessment by both specialists and residents showed a similar, very high concordance with the reference method. On the other hand, when considering all observers, assessment of lymph nodes usually presented significant inter-reader variance. However, sources of variance were disseminated within the group of readers.

The necessity of contrast enhancement for imaging of the chest lymph nodes is still a matter of debate and although systematic evidence for the use of CE CT for evaluation of the chest lymph nodes seems to be weak [5,9], there is still a tendency to use the CE phase in routine protocols. The reasons for that may include a tendency to a strict standardization of workflow, cautiousness regarding possible reading errors, and increasing use of teleradiology without proper on-site control. The influence of contrast medium administration on the assessment of chest lymph nodes has been studied since 1990 [9,10], but because of the rapid development of technology those results cannot be compared to contemporary imaging. Recently, the most comprehensive analysis of the problem was presented by Takahashi et al., who used a 16-slice scanner [5]. They found the highest inter-reader reproducibility for the analysis of mediastinal nodes in transverse projections after administration of the contrast medium (κ = 0.81), and the lowest in the assessment of lymph nodes in the non-CE lateral projections (κ = 0.36). Therefore, they showed that the application of the contrast agent was helpful in assessing the lymph nodes of the hila but not the mediastinum and that coronal reconstructions were beneficial for non-CE images [12]. In our study, MPR were less useful because the slice thickness was 3 mm. Despite that, we proved that the use of CM adds little to the final diagnosis.

The decision to use contrast medium is especially important in patients with chronic pathologies, including sarcoidosis, who require repeated follow-up CT examinations because the addition of the CE phase is related to a significant risk of contrast-induced acute kidney injury (CI-AKI) [11] and a potentially cancerogenous cumulative radiation dose [12]. Patients with sarcoidosis due to an increased risk of chronic kidney disease [13] also present potentially increased risk for CI-AKI. However, we could not find any study published on the post-contrast kidney injury prevalence in this group. The harmful effect of the radiation dose is much more difficult to assess due to the stochastic nature of carcinogenesis. A typical thoracic CT scan can give a radiation dose equivalent to 50-450 pairs of chest radiographs, depending on the scan protocol, which relates to a dose of 3-27 mSv [14,15]. However, state-of-the-art techniques can reduce the dose to sub-mSv values [12,16] or even to a dose of 2 chest X-rays [17]. The estimation of risk associated with radiation dose assumes that a linear relationship exists between radiation and subsequent risk of development of cancer [15]. Still, the risk cannot be excluded completely, and the ALARA rule still has to be applied.

Our second important observation is that observers’ experience does not correlate significantly with accuracy of chest lymph node evaluation, even without contrast medium administration. Theoretically, professional experience should influence the quality of CT-image analysis and experienced radiologists should achieve similar results for lymph node detection and measurement. A comprehensive assessment of the repeatability of measurements in the CT was conducted by McErlean et al. [18] In their study, 17 observers of various experience evaluated chest lymph nodes, lung nodules, and focal liver lesions. For lesions smaller than 10 mm, the variability was 7-12% in the short axis and 6-11% in the long axis. In the case of changes ≥ 20 mm, the variability was 5-6% and 4-6%, respectively. The reproducibility was related to the analysed anatomic region: the highest consistency was found in the lungs, then in the liver, and the lowest in the lymph nodes. The reproducibility was also dependent on the experience of observers and was slightly higher for specialists (ICC: 0.95-0.97) than for residents (0.93-0.95) [18].

Our study has some limitations that must be addressed. Firstly, this study included a specific population of patients, i.e. with sarcoidosis. In this disease, lymph node involvement is diagnosed based on the node size without necessity of evaluation of the contrast enhancement pattern. Therefore, our results may not be generalized for pathologies with significant nodal vascularization or necrosis. Secondly, our study of MPR images was not based on the isotropic voxel data. Therefore, the z-axis resolution of the MPR was suboptimal. Thirdly, the retrospective nature might introduce some bias into the results.

Conclusions

The application of contrast medium has limited impact on the quality of assessment of the chest lymph nodes in patients with sarcoidosis, regardless of the experience of the observer.