The diagnostic performance of routinely acquired and reported computed tomography imaging in patients presenting with suspected pleural malignancy.

OBJECTIVES: Contrast-enhanced computed tomography (CT) provides essential cross-sectional imaging data in patients with suspected pleural malignancy (PM). The performance of CT in routine practice may be lower than in previously reported research. We assessed this relative to 'real-life' factors including use of early arterial-phase contrast enhancement (by CT pulmonary angiography (CTPA)) and non-specialist radiology reporting.
MATERIALS AND METHODS: Routinely acquired and reported CT scans in patients recruited to the DIAPHRAGM study (a prospective, multi-centre observational study of mesothelioma biomarkers) between January 2014 and April 2016 were retrospectively reviewed. CT reports were classified as malignant if they included specific terms e.g. "suspicious of malignancy", "stage M1a" and benign if others were used e.g. "indeterminate", "no cause identified". All patients followed a standard diagnostic algorithm. The diagnostic performance of CT (overall and based on the above factors) was assessed using 2×2 Contingency Tables.
RESULTS: 30/345 (9%) eligible patients were excluded (non-contrast (n=13) or non-contiguous CT (n=4), incomplete follow-up (n=13)). 195/315 (62%) patients studied had PM; 90% were cyto-histologically confirmed. 172/315 (55%) presented as an acute admission, of whom 31/172 (18%) had CTPA. Overall, CT sensitivity was 58% (95% CI 51-65%); specificity was 80% (95% CI 72-87%). Sensitivity of CTPA (performed in 31/315 (10%)) was lower (27% (95% CI 9-53%)) than venous-phase CT (61% (95% CI 53-68%) p=0.0056). Sensitivity of specialist thoracic radiologist reporting was higher (68% (95% CI 55-79%)) than non-specialist reporting (53% (95% CI 44-62%) p=0.0488). Specificity was not significantly different.
CONCLUSION: The diagnostic performance of CT in routine clinical practice is insufficient to exclude or confirm PM. A benign CT report should not dissuade pleural sampling where the presence of primary or secondary pleural malignancy would alter management. Sensitivity is lower with non-thoracic radiology reporting and particularly low using CTPA.

Introduction

Contrast-enhanced Computed Tomography (CT) remains the key radiological investigation in patients with suspected pleural malignancy (PM). [1] Morphological features of PM identifiable on CT include pleural thickening >1 cm, nodular or mediastinal pleural thickening, inter-lobar fissural nodularity and infiltration of the chest wall or diaphragm. [2], [3] Reports regarding the performance of CT in this setting show considerable heterogeneity. Previous prospective studies utilising research study-specific reporting in small populations have reported sensitivities of 84–96% and specificities of 80–100% for PM [4], [5] However, a recent retrospective review of the diagnostic performance of CT in 370 patients referred for Local Anaesthetic Thoracoscopy (LAT) found lower sensitivity (68%) and specificity (78%). [6] In the population studied this translated into a negative predictive value (NPV) of only 65%, implying that approximately one third of patients with PM may have a benign CT report in routine practice. However, this study may have selected a particularly difficult to diagnose population since all cases had non-diagnostic or negative pleural aspiration cytology, by definition. We performed a retrospective cohort study in which the primary aim was to reassess the performance of routinely acquired and reported CT in a lessselected population We retrospectively reviewed CT reports in consecutive patients recruited to the DIAPHRAGM study (ISRCTN10079972). This is a multi-centre observational study of mesothelioma biomarkers, recruiting patients at first presentation of suspected pleural malignancy. In this study, patients are generally recruited before, or at diagnostic pleural aspiration, in an ‘intention-to-diagnose’ setting. Most patients with suspected PM are likely to be eligible (see Section 2.1) and CT scans are undertaken as part of routine care. These are therefore subject to the variations in CT acquisition and radiology reporting that exist in clinical practice, including the use of arterial phase contrast CT (as CTPA) and non-specialist thoracic radiology reporting. Importantly, the DIAPHRAGM study incorporates robust diagnostic assessment of all cases, including access to LAT, where indicated, and mandatory follow-up of benign cases.

Materials and methods

Study population

CT reports were reviewed retrospectively for consecutive patients recruited prospectively to DIAPHRAGM at 4 centres in the West of Scotland (Southern General Hospital (now Queen Elizabeth University Hospital), Glasgow Royal Infirmary, Victoria Infirmary and Gartnavel General Hospital) between January 2014 and April 2016. The inclusion criteria were suspected pleural malignancy (defined by a unilateral pleural effusion or pleural mass lesion), sufficient fitness for at least a pleural aspiration and informed written consent. The exclusion criteria were an inter-costal chest drain in-situ, or one within the preceding 3 months.

CT analyses

All CT scans were reported as part of routine clinical activity and underwent no study-specific reporting. Each report was reviewed by a respiratory physician (ST) who classified these as malignant or benign based on specific terms in the report. Malignant reports included terms that increased the level of pre-CT clinical suspicion regarding PM, specifically, “suspicious of malignancy”, “probable malignant effusion”, “disseminated malignancy”, “suggestive of malignancy”, “stage M1a” or “suggest sampling of effusion for cytology”. CT reports without terms that increased the level of pre-CT clinical suspicion were classified as benign. Specific terms that positively identified benign reports were “indeterminate”, “no cause identified”, “no evidence of malignancy”, “differential diagnosis is wide”, “appearances not obviously malignant” or if alternative pathology was suggested as the likely diagnosis, e.g. tuberculous pleurisy. Thoracic radiologists were defined as radiologists with a primary subspecialty interest in chest imaging, including involvement in thoracic oncology multidisciplinary teams. The phase of CT contrast enhancement was recorded as CTPA (pulmonary arterial phase) or non-CTPA (venous phase).

Diagnostic methods

All patients were subject to robust diagnostic assessment based on a pre-specified algorithm (see Fig. 1). PM was cytologically or histologically confirmed where possible. Where not possible PM was diagnosed by MDT consensus and demonstration of progression on follow-up imaging. Patients with benign cytology (+/− histology if pursued) and evidence of resolution or lack of progression of pleural disease after a minimum of 6 months of follow-up were classified as benign. Patients with less than 6 months of clinical follow-up were excluded from analyses.

Fig. 1

Summary of the diagnostic algorithm following by all subjects recruited to the DIAPHRAGM (Diagnostic and Prognostic Biomarkers in the Rational Assessment of Mesothelioma) study, including the 315 patients reported here. DIAPHRAGM is a prospective, multi-centre observational study assessing novel potential blood biomarkers of Malignant Pleural Mesothelioma (MPM). DIAPHRAGM incorporates a Magnetic Resonance Imaging (MRI) sub-study allowing correlation between biomarker level and MPM tumour volume.

Statistical analysis

Data are presented as mean (±standard deviation (SD)) or median (±Interquartile Range (IQR)) depending on the distribution of the data. CT classification (malignant or benign) and the final pleural diagnosis (malignant or benign) were compared using 2 × 2 Contingency Tables, generating sensitivity, specificity and associated 95% confidence intervals (CI). Negative and positive predictive values (NPV and PPV) were calculated using standard definitions. Sensitivity and specificity of CTPA vs. non-CTPA and specialist vs. non-specialist radiology reporting were compared using the Chi-Squared test. Analyses were performed using Graphpad Prism v6.0 (San Diego, USA)) and R v2.1.2 (Lucent Technologies, New Jersey, USA) by ST and CK (Senior Biostatistician, CRUK Glasgow Clinical Trials Unit).

Results

The identification, exclusion and selection of patients is summarised in Fig. 2. 65% (204/315) of the patients were male, 39% (123/315) were asbestos-exposed. 73% (231/315) were current or ex-smokers, 28% (89/315) had a known prior or current non-pleural malignancy. Median age was 74 (65–79) years. 172/315 (55%) presented as an acute admission to hospital.

Fig. 2

Flow chart of patients identified, excluded and recruited to the current study. CT; Computed Tomography, HRCT; High Resolution CT.

CT acquisition

CT scans were acquired on a variety of machines (GE Medical Systems BrightSpeed, LightSpeed or Optima 660; Toshiba Aquilion; Siemens Sensation 4; Philips Brilliance 64). 10% (31/315) of CT scans were CTPAs, with image acquisition using bolus tracking in the pulmonary arterial phase. All CTPAs were performed in patients admitted as an emergency to hospital. 31 of the 172 patients (18%) who presented as an acute admission had their CT performed as a CTPA. All other patients had venous phase imaging, with images acquired 60–70 s post-contrast.

CT results

CT scans were classified as malignant in 43.5% (137/315) and benign in 56.5% (178/315). 77 radiologists were involved in reporting; 22% (17/77) were thoracic radiologists. 37% (115/315) of scans were reported by a thoracic radiologist, of which 50% (57/115) were classified as malignant. 40% (80/200) of scans reported by non-thoracic radiologists were reported as malignant.

Pleural diagnoses

These data are summarised in Table 1. The prevalence of PM was 62% (195/315). 90% (176/195) of PMs were cyto-histologically confirmed. The median follow-up for all cases was 334 (212–577) days. Median follow-up for cases of Benign Asbestos Pleural Effusion (BAPE) was 298 (107–840) days.

Table 1

Summary of Pleural Diagnoses in 315 patients subject to routinely acquired and reported Computed Tomography imaging for suspected Pleural Malignancy.

Pleural Malignancy (n = 195, 62%)	Benign Disease (n = 120, 38%)
Mesothelioma (n = 59, 19%)	BAPE (n = 23, 7%)
Secondary Malignancies (n = 137, 43%)	Pleural Infection (n = 12, 4%)
Lung Cancer (n = 74, 23%)	Reactive assoc. with Lung Cancer (n = 12, 4%)
Breast Cancer (n = 14, 4%)	Cardiac or Liver Transudates (n = 23, 7%)
Renal Cancer (n = 8, 3%)	Fibrothorax (n = 5, 2%)
Haematological Cancer (n = 7, 2%)	Tuberculous Pleuritis (n = 7, 2%,)
Gynaecological Cancer (n = 8, 3%)	Inflammatory Pleuritis (n = 9, 3%)
Other defined malignancy (n = 17, 5%)	Pulmonary Thromboembolism (n = 1, 0.3%)
Unknown Primary (n = 9, 3%)	Drug-related (n = 1, 0.3%)
	Chylothorax (n = 1, 0.3%)
	Post-cardiac surgery (n = 1, 0.3%)
	Pancreatitis (n = 1, 0.3%)
	Reactive secondary to Fibroids (n = 1, 0.3%)
	Post-lobectomy (n = 2, 0.6%)
	No specific diagnosis madea (n = 21, 6.6%)

BAPE; Benign Asbestos-related Pleural Effusion.

with reassuring clinical follow-up of at least 6 months.

Diagnostic performance of CT

The diagnostic performance characteristics of CT for PM are summarised in Table 2, the 2 × 2 Contingency tables used to generate these data are summarised in Table 3. The sensitivity of CTPA (27% (95% CI 9–53%)) was significantly lower than CT acquired in the venous phase of contrast enhancement (61% (95% CI 53–68%), p = 0.0056). The specificity of CTPA appeared lower (69% (95% CI 38–90%) than venous phase CT (82% (95% CI 73–88%)) but this difference did not reach statistical significance (p = 0.2712). The sensitivity of thoracic radiology reporting was higher than non-thoracic reporting (68% (95% CI 55–79%) versus 53% (95% CI 44–62%), p = 0.0488). The specificity of thoracic radiology reporting appeared lower (75% (95% CI 61–86%) versus 84% (95% CI 73–92%)), but this difference did not reach statistical significance (p = 0.2184).

Table 2

The diagnostic performance of routinely acquired and reported Computed Tomography (CT) imaging in 315 patients with suspected Pleural Malignancy. Results are stratified based on relevant image acquistion and reporting factors.

	Sensitivity (95% CI)	Specificity (95% CI)	PPV (95% CI)	NPV (95% CI)
All patients (n = 315)	58% (51–65%)	80% (72–87%)	83% (75–89%)	54% (46–61%)
Image Acqusition
Venous-phase contrast CT (n = 284)	61% (53–68%)	82% (73–88%)	85% (77–90%)	56% (48–64%)
CTPA (arterial-phase contrast) (n = 31)	27%* (9–53%)	69% (38–90%)	55% (21–86%)	40% (20–63%)
Image Reporting
Specialist Thoracic Radiologist (n = 115)	68% (55–79%)	75% (61–86%)	78% (66–88%)	63% (50–76%)
Non-specialist Thoracic Radiologist (n = 200)	53%** (44–62%)	84% (73–92%)	86% (76–92%)	50% (40–59%)
Optimal Acquistion and Reporting
Venous-phase contrast CT reported by a thoracic radiologist (n = 108)	69% (56–80%)	73% (58–85%)	78% (65–88%)	63% (48–76%)

CI; Confidence Interval, PPV; Positive Predictive Value, NPV; Negative Predictive Value, CTPA; CT pulmonary angiography.

p < 0.01 (CTPA vs. venous-phase CT).

p < 0.05 (thoracic vs. non-thoracic radiology reporting).

Table 3

2 × 2 Contingency Tables describing results of routinely acquired and reported Computed Tomography (CT) imaging and diagnostic assessment according to a standardised algorithm in 315 patients with suspected Pleural Malignancy.

		Final Pleural Diagnosis
		Malignant	Benign
All patients (n = 315)
CT report	Malignant	114	23
	Benign	81	97
Venous-phase contrast CT (n = 284)
CT report	Malignant	109	19
	Benign	68	88
Arterial-phase contrast CT (CTPA) (n = 31)
CT report	Malignant	5	4
	Benign	13	9
Thoracic Radiologist Reporting (n = 115)
CT report	Malignant	45	12
	Benign	21	37
Non-thoracic Radiologist Reporting (n = 200)
CT report	Malignant	69	11
	Benign	60	60
Optimal CT (Venous-phase and Thoracic Radiologist Reporting) (n = 108)
CT report	Malignant	44	12
	Benign	19	33

CT; Computed Tomography, CTPA; CT Pulmonary Angiography.

108/315 (34%) patients had CT acquired using venous-phase contrast enhancement and had subsequent thoracic radiology reporting. These ‘optimal’ conditions were associated with a sensitivity of 69% (95% CI 56–80%) and specificity of 73% (95% CI 58–85%).

Discussion

The overall sensitivity and specificity of contrast-enhanced CT for PM in this study were 58% (95% CI 51–65%) and 80% (95% CI 72–87%), respectively, resulting in a PPV of 83% (95% CI 75–89%) and a NPV of 54% (95% CI 46–61%). Almost half of the patients with PM had a benign CT report. This strongly suggests that a benign CT report should not dissuade clinicians from pursuing invasive pleural investigations if the presence of primary or secondary pleural malignancy would alter management. The sensitivity (27% (95% CI 9–53%)) and NPV (40% (20%–63%)) of arterial phase CT (as CTPA) was particularly low. Even allowing for the small numbers and wide confidence intervals around these data, the use of CTPA images to diagnose benign pleural disease must seriously be questioned.

The performance of CT reported in the current study is significantly inferior to results from some earlier studies. For example, Hierholzer et al. reported sensitivity of 93% and specificity of 87% in 42 patients [5], while Metintas reported up to 70% sensitivity and 96% specificity in 215 patients [3]. These differences may reflect the involvement of specialist thoracic radiologists in these research projects. We certainly found that thoracic radiology reporting was associated with greater sensitivity, albeit at the potential cost of a reduction in specificity (which did not reach statistical significance). The results of the current study are concordant with those of similarly sized (n = 370) study reported by Hallifax et al. in which CT performance was retrospectively assessed in patients referred for local anaesthetic thoracoscopy (LAT) for suspected PM (sensitivity 68% (95% CI 62–75%), specificity 78% (95% CI 72–84%), NPV 65% (95% CI 58–72%), PPV 80% (95% CI 75–86%). [6] This study used similar report terms to classify patients into malignant and benign CT cohorts but described a more selected population than our own, since only those referred for LAT were eligible. LAT is indicated when initial pleural diagnostics are non-specific, the suspicion of malignancy is high and the patient is fit for the procedure. The population studied in this previous study could therefore be considered to be a more difficult to diagnose cohort to the current study cohort, in which 89/315 (28%) patients had diagnostic pleural cytology and only 35% (111/315) required a subsequent pleural biopsy. A confident clinical diagnosis of benign disease was achieved in the remaining 37% (115/315) without the need for further sampling. The incidence of Mesothelioma was also significantly lower in the current study (19%) than in the earlier Hallifax study (54.5%) reflecting the earlier recruitment required in DIAPHRAGM. These factors all define the current study cohort as an easier to diagnose population than the Hallifax study [6] and one that is more representative of the intention-to-diagnose population in which CT is routinely used in patients with suspected PM. Despite this and the use of similar methods, CT sensitivity and NPV were both lower in the current study (58% (95% CI 51–65%) and 54% (95% CI 46–61%)) than in Hallifax et al. with similar PM prevalence (62% herein vs. 57% [6]) and specificity.

Possible explanations for this may include differences in the type of CT images acquired and the reporting performed. The current study was recruited across 22 UK and Irish hospitals (including a mixture of academic and clinical centres to ensure generalizability of DIAPHRAGM’s results). This may have differed to recruitment from two tertiary pleural disease units (Oxford and Preston) in the Hallifax study. Our design may have led to greater use of ‘non-optimised’ CT including more use of CTPA and non-thoracic radiology reporting, although these factors are not reported by Hallifax et al. and cannot be directly compared.

We found that the sensitivity of CTPA (27% (95% CI 9–53%)) for PM was significantly inferior to that of venous-phase CT (sensitivity 61% (95% CI 53–68%), p = 0.0056). The NPV of CTPA for PM was only 40% (95% CI 20–63%). To our knowledge, this is the first report to directly compare the diagnostic performance of arterial phase CTPA to later-phase enhanced CT in the assessment of PM. In a previous prospective study Hooper et al. performed CTPA and late arterial phase CT (optimised for pleural imaging) in 141 patients with suspected PM. However the primary objective of this study was to assess the incidence of pulmonary emboli [7] and the diagnostic performance of CTPA for PM was not reported. In a small randomised study, Raj et al. previously demonstrated better enhancement of the pleura utilising delayed phase (at 60–90 s post-contrast) CT acquisition [8], Therefore venous-phase CT should be requested in all patients with suspected PM, and this may have been used more frequently in the Hallifax study. Our study clearly demonstrates that this approach is frequently not taken in a more generalizable population and may be even less frequently used than reported here, since our centres did have a stated interest in pleural disease and were recruiting to DIAPHRAGM. A major reason for this is the frequency of emergency presentation with suspected PM. Previous Mesothelioma series have reported up to 50% of cases presenting in this manner [9], [10] and this is corroborated here, with 172 of the 315 patients (55%) presenting as an emergency to an acute medical receiving unit. 31/172 (almost 1 in 5) of these case had CT performed as a CTPA to exclude concomitant pulmonary thrombo-embolism, presumably driven by non-specifically high D-Dimer results, although this was not specifically assessed here. Despite the small numbers and resulting wide confidence intervals around our results (particularly a sensitivity of 27% (95% CI 9–53%)) the use of CTPA for pleural diagnostics should be seriously questioned and repeat imaging with optimised ‘pleural’ contrast enhancement in the venous phase should be considered where a positive result will alter management.

The diagnostic sensitivity of CT was higher in the current study when thoracic radiologists were involved in reporting (69% (95% CI 55–79%) vs. 53% (95% CI 44–62%) for non-thoracic radiology reporting, p = 0.0488). Optimal imaging conditions (i.e. venous-phase acquisition and thoracic radiology reporting) were associated with a sensitivity of 69% (95% CI 56–80%), although the number of patients imaged under sub-optimal conditions (i.e. CTPA reported by non-thoracic radiologists) was too small (n = 24) to allow a meaningful comparison between these extremes. These data suggest that increased specialist radiology reporting may improve the performance of CT in the routine assessment of patients presenting with PM. However, a limited workforce with job-plans already committed to MDT provision and image-guided biopsy services make this difficult to deliver. For example, in this study thoracic radiologists comprised only 22% of the radiologists involved and reported only 37% of the scans. As a result, optimal imaging conditions were only achieved in 108/315 (34%) of the patients studied. The development of objective imaging biomarkers of PM may therefore provide a better long-term method of reducing inter-observer disagreement in CT reporting, as has previously been demonstrated in breast cancer [11], [12] Novel objective imaging markers of PM are also under development including Pleural Pointillism [13] and Early Contrast Enhancement [14], both defined using MRI.

Study weaknesses and strengths

Although all patients included in this study were recruited prospectively, the classification of each CT report as malignant or benign was performed retrospectively. However, pre-specified criteria were set in advance as to which terms would constitute malignant and benign reports and these were used consistently. The final pleural diagnosis end-point used in the study is likely to be robust given the rigorous prospective assessment required by the DIAPHRAGM study protocol. This included ready access to thoracoscopy (and/or image-guided biopsy), a lung and regional mesothelioma MDT review (where appropriate) and a specialist pleural clinic follow-up for all cases. This was reflected in a high cyto-histological confirmation rate of PM (90%). However, the minimum follow-up period required for inclusion in this current study was only 6 months. Theoretically, this could result in an underestimation of the true Mesothelioma rate since it is normal practice to follow up patients with Benign Asbestos Pleural Effusion (BAPE, 23/315 (7%) of the patients studied) for at least 2 years, because of a potential false negative pleural biopsy rate of up to 12% in previous case series. [15] However, all cases of BAPE were subject to mandatory clinical follow-up as per the DIAPHRAGM study protocol (see Fig. 1) and the mean follow-up for the patients diagnosed with BAPE in this current study was 367 (SD 220) days. Furthermore, it is not clear whether Mesothelioma in follow-up represents a new diagnosis at that point or a genuine false negative based on the original assessment.

Conclusions

In clinical practice, CT plays a major role in the assessment of patients with suspected pleural malignancy. The interpretation of the CT scan by the referring clinician and the reporting radiologist frequently directs the patient towards, or away from further diagnostics, including invasive pleural sampling. This is the second study assessing the diagnostic performance of CT for pleural malignancy in routine clinical practice, and demonstrates significantly lower sensitivity and specificity than previously reported in smaller research studies. Given the NPV reported here (54% (95% CI 46–61%)) it is important to consider invasive (cytological ± histological) sampling in patients where proven pleural malignancy would alter management.

The low sensitivity and NPV of CTPA strongly suggest that this modality is inadequate for the exclusion of PM. Either repeat CT imaging with pleural contrast timing, alternative imaging (e.g. magnetic resonance imaging) or further investigation if there are other indicators of pleural malignancy should be considered after a benign CTPA report. For patients with pleural effusion, revision of existing guidelines advocating routine imaging of the thorax in the arterial phase for oncological purposes [16] should be considered. Given the high level of skill required to interpret pleural CT images, increased specialty reporting may immediately improve the performance of CT in routine clinical practice. In addition, the development of more objective imaging biomarkers of PM may result in better performance by reducing operator dependence.

Conflicts of interest

None declared.