Screening Performance Characteristic of Ultrasonography and Radiography in Detection of Pleural Effusion; a Meta-Analysis.

INTRODUCTION: The role of ultrasonography in detection of pleural effusion has long been a subject of interest but controversial results have been reported. Accordingly, this study aims to conduct a systematic review of the available literature on diagnostic value of ultrasonography and radiography in detection of pleural effusion through a meta-analytic approach.
METHODS: An extended search was done in databases of Medline, EMBASE, ISI Web of Knowledge, Scopus, Cochrane Library, and ProQuest. Two reviewers independently extracted the data and assessed the quality of the articles. Meta-analysis was performed using a mixed-effects binary regression model. Finally, subgroup analysis was carried out in order to find the sources of heterogeneity between the included studies.
RESULTS: 12 studies were included in this meta-analysis (1554 subjects, 58.6% male). Pooled sensitivity of ultrasonography in detection of pleural effusion was 0.94 (95% CI: 0.88-0.97; I2= 84.23, p<0.001) and its pooled specificity was calculated to be 0.98 (95% CI: 0.92-1.0; I2= 88.65, p<0.001), while sensitivity and specificity of chest radiography were 0.51 (95% CI: 0.33-0.68; I2= 91.76, p<0.001) and 0.91 (95% CI: 0.68-0.98; I2= 92.86, p<0.001), respectively. Sensitivity of ultrasonography was found to be higher when the procedure was carried out by an intensivist or a radiologist using 5-10 MHz transducers.
CONCLUSION: Chest ultrasonography, as a screening tool, has a higher diagnostic accuracy in identification of plural effusion compared to radiography. The sensitivity of this imaging modality was found to be higher when performed by a radiologist or an intensivist and using 5-10MHz probes.

Introduction:

The role of ultrasonography in detection of pleural effusion was first discovered in the 1960s (1). One decade later the diagnostic value of A-mode ultrasonography in identification of pleural effusion was reported to be 95% (2). Thus, the impression that ultrasonography is not a suitable modality for detection of chest lesions was rejected at that time. Afterwards, various surveys were conducted evaluating the diagnostic accuracy of ultrasonography and radiography in identification of pleural effusion and most of them found a higher diagnostic value for ultrasonography compared to radiography (3-5). However, ultrasonography is still not considered as the first diagnostic tool for detection of pleural effusion and the majority of physicians use radiography for this propose (6).

Although a radiogram captured in lateral decubitus position with horizontal rays has a high sensitivity and specificity in detection of pleural effusion but it cannot be obtained from all the patients especially critically ill and trauma victims (4). Other views of chest radiogram have low sensitivities in this regard (7, 8). Vast improvements of technology have led to development of hand-held and pocket-size ultrasound devices (9, 10). A meta-analysis was performed in 2010 on four studies that reported the sensitivity and specificity of ultrasonography in detection of pleural effusion to be 93% and 96%, respectively (11). Yet, inclusion of few surveys and lack of sensitivity analysis and publication bias confirm the necessity of an update on this subject. Therefore, this study aims to conduct a systematic review of the available literature determining the diagnostic value of ultrasonography and radiography for detection of pleural effusion through a meta-analytic approach.

Methods:

This study was designed according to the consensus statement of Meta-analysis of Observational Studies in Epidemiology (MOOSE) (12). Extended systematic search was carried out in databases of Medline (via PubMed), EMBASE (via OvidSP), ISI Web of Knowledge, Scopus, Cochrane Library, and ProQuest based on the keywords obtained from Medical Subject Heading (MeSH) terms and EMTREE. The keywords included “Ultrasonography” OR “Sonography” OR “Ultrasound” OR “Chest Film” OR “Chest Radiograph” combined with “Pleural Effusion” OR “Effusion” OR “Pleural Free Fluid”. The directive was to find prospective and retrospective studies assessing the diagnostic value of ultrasonography or chest radiography in detection of pleural effusion. Two of the authors (M.Y, P.G) independently searched for sources and contacted the authors of relevant articles to obtain further articles and data. Only original articles were included. The studies that had used radiography as their reference test were excluded and only surveys in which a final diagnosis of pleural effusion had been confirmed by computed tomography scan or surgery were included. The search yielded three surveys which had compared the diagnostic value of pocket-size ultrasonography machines with high-end ones. Since these high-end ultrasonography machine had been performed by cardiologists and their results had been assessed by the final diagnosis proposed by two specialists as the reference test, these three surveys were also included in the meta-analysis.

The articles were entered into Endnote X7 software and after elimination of duplicates, two reviewers (M.Y, P.G) independently screened the articles. Full texts of the articles were studied and summarization was done. Quality of the surveys were evaluated according to the guidelines proposed by 14-Item Quality Assessment of Diagnostic Accuracy Studies (QUADAS2) tool (18). Each article was assessed regarding presence of various biases including selection, performance, recording, and reporting and they were classified as three grades of poor, fair or good. Only fair and good surveys were included in meta-analysis. Data on distribution of the study population regarding age and gender, sample size, the number of patients with/without pleural effusion according to reference tests, technique of ultrasonography, the specialty of the ultrasound device’s operator, sonographic definition of pleural effusion, probe’s frequency, blinding status, sampling method (consecutive, convenience) and finally number of true and false, positive and negative cases were gathered. A third author (M.H) was in charge of solving disagreements. In cases where data were not presented in the article web-based programs were used to compute true positive, false positive, true negative and false negative figures according to sensitivity and specificity. When available, data for each hemi-thorax were included in the analysis separately.

STATA 11.0 statistical software was used to analyze the data via MIDAS module. To evaluate the adequacy of ultrasonography and radiography in detection of pleural effusion, summary receiver operative curves (SROC), sensitivity, specificity, positive likelihood ratio, and negative likelihood ratio with 95% confidence interval (95% CI) were calculated using a mixed-effects binary regression model. Heterogeneity between the included surveys was assessed through utilization of I2 and χ2 tests and a p value of less than 0.1 along with an I2 greater than 50% were considered as positive heterogeneity (19). Consequently, subgroup analysis was performed on sampling method (consecutive/ convenience), operator (emergency physician/ other specialists) or interpreting physician, frequency of the probe (1-5 MHz/ 5-10 MHz), type of ultrasound device (pocket-size/other) and sample size (less than 100 patients/ more than 100 patients).

Results:

Search process and the number of articles obtained in each step are presented in Figure 1. Twenty one studies met the inclusion criteria. One survey was also found through manual search. After detailed evaluation of these surveys 12 studies were included in meta-analysis (3-5, 7-10, 13-17). The characteristics of included studies are presented in Table 1. The mean age of the subjects was 63.3 years (ranging from 14 to 101 years old) and 58.6% were male. All the studies were single-center (3-5, 7-10, 14-17)except one which was performed in two center (13). A total of 646 patients with pleural effusion and 912 subjects without it were evaluated. Diagnostic accuracy of both these modalities was assessed in nine surveys (3-5, 8-10, 13-17), radiography was evaluated individually in one study (7) and ultrasonography was assessed individually in two articles (10, 17). 11 articles were in English (3-5, 7-10, 13-15, 17) and one was in Farsi (16). The presence of heterogeneity was confirmed between the studies (P<0.1), but no publication bias was detected (Figure 2).

Figure 1

Flowchart of the study

Table 1

Charecteristics of included studies

Study	No. of patient (+ / -) 1	Age 2 (years)	Male (%)	Reference / Index	Transducer / Operator	Sampling	Weaknesses
Ma 1997 ( 5 )	26 / 214	≥ 18	NR	CT/ US, CXR	3.5-to 2.5-MHz / EP	Consecutive /Trauma	Retrospective design
Kataoka 2000 ( 3 )	60 / 22	76.0 ± 12.9	38	CT/ US, CXR	3.5 MHz /Internist	Consecutive /Critically ill	Low sample sizeOnly evaluating of CHF patient
Lichtenstein 2004 ( 4 )	100 / 284	58±15	NR	CT/ US, CXR	5 MHz /Intensivist	Consecutive /Critically ill
Rocco 2008 ( 8 )	38 / 142	42±14	66.7	CT/ US, CXR	3.5 MHz /Intensivist	Consecutive /Trauma	Low sample size
Kitazono 2010 ( 7 )	117 / 83	54 (14-91)	60	CT/ CXR	NA /Radiologist	Consecutive /Critically ill	Retrospective design
Zanobetti 2011 ( 13 )	25 / 11	73 (21-101)	51	CT/ US, CXR	5– to 8-MHz / EP	Consecutive /Critically	CT was performed in some patients
Xirouchaki 2011 ( 14 )	63 / 21	57.1±21.5	81	CT/ US, CXR	5– to 9-MHz / Intensivist	Convenience /Critically ill	Low sample size
Schleder 2012 ( 15 )	35 /13	NR	NR	Outcome / US, CXR	5- to 9-MHz / Radiologist	Consecutive /Critically ill	Low sample size
Kasraei 2014 ( 16 )	27 / 7	18-70	53.8	CT / US, CXR	5- to 9-MHz / Radiologist	Consecutive /Critically ill	Low sample sizePossibility of selection bias
Pocket-size ultrasonography
Dalen 2015 ( 10 )	39 / 85	74 (35–91)	52	High-end US / US	1.7- to 3.8-MHz / Nurse	Consecutive /Heart failure	Low sample size
Graven 2015 ( 9 )	95 / 23	67 (35-86)	66	High-end US / US, CXR	1.7- to 3.8-MHz / Nurse	Convenience /Cardiac surgery	Low sample sizePossibility of selection bias
Stock 2015 ( 17 )	21 / 7	68 (24-94)	42.8	High-end US / US	2- to 4-MHz / Internist	Convenience /Surgery	Low sample sizePossibility of selection bias

1, (+ / -): Number of patient with pleural effusion / number of patient without pleural effusion; 2, Number are presented as mean ± standard deviation or (range). CT: Computed tomography; CXR: Chest radiography; EP: Emergency physician; NA: Not applicable; NR: Not Reported; US: Ultrasonography

Figure 2

Deeks’ funnel plot asymmetry test for assessment of publication bias. P values < 0.05 were considered as significant. Ultrasonography (A); Radiography (B). ESS: Effective sample sizes

Area under the curve of SROCs for ultrasonography and chest radiography in detection of pleural effusion were 0.98 (95% CI: 0.97-0.99) and 0.73 (95% CI: 0.69-0.77), respectively (Figure 3). Pooled sensitivity and specificity of ultrasonography in detection of pleural effusion were 0.94 (95% CI: 0.88-0.97; I2= 84.23, p<0.001) and 0.98 (95% CI: 0.92-1.0; I2= 88.65, p<0.001), respectively. Also, its pooled positive and negative likelihood ratios were calculated to be 53.96 (95% CI: 11.46-254.05; I2= 88.12, p<0.001) and 0.06 (95% CI: 0.03-0.12; I2= 84.44, p<0.001), respectively (Figure 4).

Figure 3

Summary receiver operative curves (SROC) for ultrasound (A) and chest radiography (B) in detection of plural effusion. AUC: Area under the curve; SENS: Sensitivity; SPEC: Specificity

Figure 4

Forest plot of screening performance characteristics of chest ultrasonography in detection of plural effusion. Sensitivity and specificity (A); Diagnostic likelihood ratio (DLR) (B). CI: Confidence interval

Pooled sensitivity and specificity of radiography in pleural effusion diagnosis were 0.51 (95% CI: 0.33-0.68; I2= 91.76, p<0.001) and 0.91 (95% CI: 0.68-0.98; I2= 92.86, p<0.001), respectively. In addition, its pooled positive and negative likelihood ratios were also found to be 5.60 (95% CI: 1.14-27.42; I2= 88.14, p<0.001) and 0.54 (95% CI: 0.35-0.84; I2= 84.44, p<0.001), respectively (Figure 5).

Figure 5

Forest plot of screening performance characteristics of chest radiography in detection of plural effusion. Sensitivity and specificity (A); Diagnostic likelihood ratio (DLR) (B). CI: Confidence interval

Subgroups analyses were performed regarding study design, patients’ type (critically ill / other), operator of ultrasound machine, ultrasound model and sample size. The results showed that the sensitivity was higher when the procedure was performed via an intensivist or a radiologist, using 5-10 MHz transducers, whereas it was lower when carried out via pocket-size devices. On the other hand, the specificity of this modality was not influenced by any of these factors.

The diagnostic value of radiography was affected by the sample size and type of patients. In this regard, the sensitivity and specificity of chest radiography was found to be higher in studies with sample sizes of greater than 100 patients while both of these measures were calculated to be lower in critically ill patients (Table 2).

Table 2

Subgroup analysis of diagnostic accuracy for chest radiography and ultrasonography in detection of plural effusion

Covariate	No. of studies	Bivariate random-effect model
		Sensitivity (95% CI)	P	Specificity (95% CI)	p	heterogeneity, I 2	P *
Ultrasonography
Patient enrollment
Consecutive	8	0.95 (0.90-0.99)	0.08	0.97 (0.92-1.00)	<0.001	15.0 %	0.31
Nonconsecutive	4	0.92 (0.84-1.00)		1.00 (0.98-1.00)
Operator
Radiologist/Intensivist	6	0.97 (0.83-0.96)	<0.001	0.99 (0.98-1.00)	0.34	68.0 %	0.05
Other physician	6	0.90 (0.83-0.96)		0.94 (0.86-1.00)
Sample size
< 100	8	0.94 (0.89-0.99)	0.22	0.96 (0.90-1.00)	<0.001	0.0 %	0.38
≥ 100	4	0.94 (0.88-1.00)		0.99 (0.98-1.00)
Frequency
1-5 MHz	9	0.92 (0.86-0.97)	0.04	0.98 (0.95-1.00)	0.14	42.0 %	0.18
5-10 MHz	3	0.98 (0.95-1.00)		0.99 (0.96-1.00)
Patients
Critically ill	7	0.94 (0.89-0.99)	0.29	0.98 (0.95-1.00)	0.12	15.0 %	0.31
Other	5	0.93 (0.87-1.00)		0.98 (0.94-1.00)
Machine type
Pocket-size	3	0.90 (0.80-1.00)	0.09	0.92 (0.76-1.00)	0.55	16.0 %	0.30
Other	9	0.95 (0.91-0.99)		0.97 (0.97-1.00)
Radiography
Patient enrollment
Consecutive	7	0.50 (0.30-0.71)	0.93	0.81 (0.31-1.00)	0.94	0.0 %	0.77
Nonconsecutive	3	0.53 (0.13-0.92)		0.93 (0.81-1.00)
Operator
Radiologist/Intensivist	6	0.50 (0.27-0.73)	0.95	0.87 (0.65-1.00)	0.95	0.0 %	0.73
Other	4	0.51 (0.22-0.81)		0.96 (0.84-1.00)
Sample size
< 100	6	0.45 (0.22-0.67)	0.44	0.79 (0.51-1.00)	0.99	8.0 %	0.34
≥= 100	4	0.60 (0.33-0.87)		0.97 (0.90-1.00)
Patients
Critically ill	7	0.48 (0.26-0.69)	0.62	0.82 (0.59-1.00)	0.29	12.0 %	0.32
Other	3	0.58 (0.25-0.91)		0.98 (0.93-1.00)

P value < 0.1 was considered as significant for heterogeneity; CI: Confidence interval.

Discussion

The present meta-analysis showed that ultrasonography has high sensitivity (94%) and specificity (98%) in detection of pleural effusion. When the analysis was limited to the surveys in which the procedure was carried out by an intensivist or a radiologist, the presented sensitivity increased (98%). While, the diagnostic value of radiography was reported to be lower than ultrasonography (sensitivity 51%, specificity 91%). When we limited the analyses to studies in which the etiology of pleural effusion was trauma, surgery or congestive heart failure, sensitivity of radiography slightly increased (58%). On this basis it can be concluded that ultrasonography is a better diagnostic tool for pleural effusion compared to radiography.

In comparison with the results of the present study, Grimberg et al. reported similar results in their review of four articles. They found the sensitivity and specificity of ultrasonography to be 93% and 96% respectively while these figures for radiography were calculated to be 24% and 100% (11). Our results were almost similar to Grimberg’s et al. study. The higher sensitivity of radiography presented in this study might be due to inclusion of more surveys in the analyses. Grimberg’s survey lacked subgroup analysis which prevents us from further comparisons. In another meta-analysis Chavez et al. reviewed 10 articles aiming to evaluate the diagnostic value of ultrasonography in detection of pneumonia and they found a sensitivity of 94% and a specificity of 96% for this modality (20). These researchers stated that ultrasonography is a suitable diagnostic tool for ruling out pneumonia in patients referring to medical centers and emergency departments. Congruent with this survey we also found a considerable diagnostic value for ultrasonography in detection of pleural effusion.

One of the factors that influence sensitivity of ultrasonography is the operator of ultrasound device which has been verified by various surveys in the past years (21-24). An ultrasonography carried out by an intensivist or a radiologist is able to detect pleural effusion more effectively and precisely.

Pocket-size ultrasonography is a new technology recently being used in some medical centers and reported to be an effective modality in diagnostic procedures (25-27). This machine transmits high resolution pictures that help physicians to make more precise decisions regarding the patients’ pathology (28, 29). Accordingly, we found three studies which assess the diagnostic accuracy of pocket-size ultrasonography for pleural effusion (9, 10, 17), in two of which the procedure was performed by a trained nurse (9, 10) and in one it was carried out by an internal medicine specialist (17). The first two reported a sensitivity of approximately 92-98% and the latter survey found a sensitivity of 62% in detection of pleural effusion. This inconsistency might have been due to the different study populations. Graven et al. and Dalen et al. only included cardiac patients while Stock et al. evaluate patients with various diseases. Nevertheless, the sample sizes of all the three surveys were small and so further investigation is required for application of this ultrasonography device in clinical settings.

In this meta-analysis the extended search in databases and bibliographies yielded 12 relevant articles. Although few studies were included in this study, but the large sample population of 1554 subjects assured the validity of performed analyses to a great extent. Presence of no publication bias confirms this declaration. Heterogeneity between the articles was another limitation of this survey which was attempted to overcome through application of mixed-effects binary regression model and subgroup analysis. Simultaneous inclusion of retrospective and prospective surveys was probable limitation of this study, but the scatterplot developed to evaluate the outlier studies according to standardized predicted random effects showed that retrospective surveys were not the source of diversity between the included articles.

Conclusion:

The present meta-analysis found chest ultrasonography to have a considerably higher screening value in detection of pleural effusion compare to radiography. Being performed by an intensivist or a radiologist and using probes with frequencies of 5-10MHz improves the sensitivity of this imaging modality.