Development and validation of a novel scoring system developed from a nomogram to identify malignant pleural effusion.

BACKGROUND: This study aimed to establish and validate a novel scoring system based on a nomogram for the differential diagnosis of malignant pleural effusion (MPE) and benign pleural effusion (BPE).
METHODS: Patients with PE and confirmed aetiology who underwent diagnostic thoracentesis were included in this study. One retrospective set (N = 1261) was used to develop and internally validate the predictive model. The clinical, radiological and laboratory features were collected and subjected to logistic regression analyses. The primary predictive model was displayed as a nomogram and then modified into a novel scoring system, which was externally validated in an independent set (N = 172).
FINDINGS: The novel scoring system was composed of fever (3 points), erythrocyte sedimentation rate (4 points), effusion adenosine deaminase (7 points), serum carcinoembryonic antigen (CEA) (4 points), effusion CEA (10 points) and effusion/serum CEA (8 points). With a cutoff value of 15 points, the area under the curve, specificity and sensitivity for identifying MPE were 0.913, 89.10%, and 82.63%, respectively, in the training set, 0.922, 93.48%, 81.51%, respectively, in the internal validation set and 0.912, 87.61%, 81.36%, respectively, in the external validation set. Moreover, this scoring system was exclusively applied to distinguish lung cancer with PE from tuberculous pleurisy and showed a favourable diagnostic performance in the training and validation sets.
INTERPRETATION: This novel scoring system was developed from a retrospective study and externally validated in an independent set based on six easily accessible clinical variables, and it exhibited good diagnostic performance for identifying MPE. FUNDING: NFSC grants (no. 81572942, no. 81800094).

Evidence before this study

Differentiating malignant pleural effusion (MPE) from benign pleural effusion (BPE) remains challenging in clinical practice; thus, developing a useful method that can precisely identify MPE as early as possible is highly important. Previous studies have attempted to differentiate MPE from BPE using various indexes. In addition to the limited sample sizes, most clinical studies have not established a quick and novel scoring system for physicians to use in clinical practice. More importantly, no investigations have used another independent set to externally validate their predictive model.

Added value of this study

We attempted to develop a predictive model using multivariate logistic regression based on a combination of the most widely available clinical features and radiological parameters and a variety of laboratory indexes. Then, the primary predictive model was displayed as a nomogram and modified into a novel scoring system for convenient clinical use. Finally, we not only internally validated the diagnostic performance of the modified scoring model in a retrospective study but also externally verified the scoring system in an independent set.

Implications of all the available evidence

In the present study, we selected the most significant indexes (fever, ADA, ESR, serum CEA, effusion CEA and effusion/serum CEA) to construct a predictive model. With the aim of establishing a novel scoring system, we converted the nomogram into a scoring system. This scoring system showed good diagnostic performance. Furthermore, we also validated the scoring system in an independent set to evaluate its predictability in an independent population, and the result was encouraging. This feasible and novel scoring system for differentiating patients with MPE and BPE should be further validated in multicentre prospective studies in the near future.

Alt-text: Unlabelled box

Introduction

Pleural effusion (PE) is a common but difficult problem in clinical settings. Approximately 1.5 million patients with PE are diagnosed each year in the United States [1]. PE is a symptom caused by more than 50 diseases and is usually classified as malignant pleural effusion (MPE) or benign pleural effusion (BPE). The incidence of MPE is higher than 200,000 cases/year in the United States [2], with lung cancer (36%) and breast cancer (26%) accounting for the vast majority of cases due to metastasis to the pleura [3,4]. With regard to BPE, tuberculous pleurisy effusion (TPE) is the most common cause [5] and a prominent problem in developing countries, including China [6]. The prognosis for patients with MPE is extremely poor in clinical practice [7,8], and delaying treatment due to misdiagnosis will directly lead to an increase in mortality. Therefore, developing a useful method that can identify MPE as early as possible with precision is highly important.

Cytology is most commonly used in clinical practice to identify tumour cells [9], which are the most powerful evidence in support of MPE. In a recent clinical study that included 725 patients with MPE, the sensitivity of cytology for diagnosing MPE was only 63% [10]. Moreover, the assessment of cytology results is relatively subjective, and the diagnostic accuracy of this method largely depends on the expertise of the pathologist. Additionally, reactive mesothelial cells are easily confused with tumour cells due to similar morphological features [11]. Although cell block plays an important role in the identification of MPE, this method is usually time-consuming and has high costs, and its sensitivity is less than 50% [12]. Although pleural biopsy has a relatively higher diagnostic accuracy [13], it is invasive and associated with more complications, as well as higher costs. Hence, it is important to design a cost-effective method that is more accurate and less invasive to identify MPE [14].

A variety of biological markers have been investigated in several studies for the differential diagnosis of MPE and BPE, but most of the biomarkers have been used individually, yielding an unfavourable diagnostic performance [15,16]. The results from a few clinical studies showed that no single index from blood or effusion biochemistry could provide sufficient evidence to accurately distinguish MPE from BPE in clinical practice. Of course, some studies have also attempted to differentiate MPE from BPE using various indexes. In addition to the limited sample sizes, most clinical studies have not established a quick and novel scoring system for physicians to use in clinical practice [14]. More importantly, no investigations used another independent cohort to externally validate their predictive model [17,18]. Hence, the results from the previous studies were inconclusive, and the predictive models for diagnosing MPE were not good enough to be recommended in routine clinical practice.

In the present study, our primary goal was to develop a predictive model using multivariate logistic regression based on a combination of clinical features to distinguish MPE from BPE. Then, the primary predictive model was displayed as a nomogram and modified into a novel scoring system for convenient clinical use. In addition, we not only internally validated the diagnostic performance of the modified scoring model in a retrospective study but also externally verified the scoring system in an another tertiary hospital. Our second aim was to exclusively apply the scoring system to differentiate lung cancer with PE from TPE and to evaluate the system's discriminative and calibration abilities for these two diseases.

Materials and methods

Patients and study design

This clinical study was performed at Renmin Hospital of Wuhan University and Union Hospital, Tongji Medical College, Huazhong University of Science and Technology (Wuhan Union Hospital). This clinical study consisted of two stages. The first stage (training set and internal validation set) retrospectively identified patients with PE in Renmin Hospital of Wuhan University from January 2014 to April 2018. The second stage (external validation set) included PE patients in Wuhan Union Hospital from August 2019 to December 2019. All patients who underwent diagnostic thoracentesis at the Departments of Respiratory, Oncology and Chest Surgery were included in this clinical study. All patients needed to have a confirmed cause of PE for final inclusion in this study.

Participants who met the following inclusion criteria were included in this study: (a) confirmed to have PE by chest CT, X-ray or ultrasonography; and (b) underwent diagnostic thoracentesis or, in some cases, pleural biopsy. The exclusion criteria were as follows: (a) indeterminable cause of PE; (b) younger than 18 years old; (c) no information available; and (d) reluctant to participate in this study.

The primary aim of the present study was to develop a scoring system with high predictive accuracy to accurately differentiate MPE from BPE, and the secondary aim was to differentiate lung cancer with PE from TPE. The patients were divided into three groups (training set, internal validation set and external validation set). The first group (the training set) included 70% of the patients with PE (N = 883) from the retrospective study carried out at Renmin Hospital of Wuhan University to develop the scoring system to discriminate MPE patients from patients with BPE. The second group (the internal validation set) included the remaining 30% participants with PE (N = 378) from Renmin Hospital of Wuhan University to validate the diagnostic performance of this scoring system. The third group (the external validation set) included prospectively recruited patients with PE (N = 172) in Wuhan Union Hospital, independent of the patients in Renmin Hospital of Wuhan University, to further validate the predictive model. This study was conducted in line with the Declaration of Helsinki. The institutional ethics committees of Renmin Hospital of Wuhan University (No. WDRY 2019-K014) and Wuhan Union Hospital (No. 2019-S075) reviewed and approved this study protocol. Moreover, the second-stage study performed at Wuhan Union Hospital was registered on the Clinical Trials website (No. NCT03997669). All patients were required to provide written informed consent.

Data collection

The clinical information, including demographic data, objective symptoms, and radiological and laboratory features, of the included patients were obtained from the electronic medical record (EMR). The following clinical features were abstracted from the platform: demographic characteristics (age and gender), objective symptom (fever), radiological features (site of PE, volume of PE) and laboratory indexes. Additionally, some ratios commonly used in clinical practice were automatically calculated and incorporated into this study. All laboratory indexes were converted to categorical variables according to the reference range, and the combined ratios were converted to binary variables according to the optimal cutoff values via receiver operating characteristic (ROC) analyses. Additionally, fever was defined as a body temperature over 37.5 °C. The volume of PE was categorized as mild (<500 ml), moderate (500–1000 ml) or severe (>1000 ml).

Diagnostic criteria for MPE and BPE

If cancer cells were detected in the PE with cytological smears, cell blocks, or pleural biopsy, then the patient was diagnosed with MPE. In contrast, BPE was identified by a known aetiology, such as TPE or parapneumonic effusion, without any signs of cancer. TPE was established when Lowenstein–Jensen cultures or acid-fast stains of PE, sputum, bronchoalveolar lavage fluid, or pleural biopsy were positive. Parapneumonic effusion was diagnosed when PE coincided with pneumonia and was resolved after antibiotic treatment. Moreover, other aetiologies of PE followed strict clinical criteria (Table S1).

Statistical analysis

SPSS 22.0 (IBM Inc., Chicago, IL, USA) was used to conduct the statistical analyses. Categorical data (laboratory indexes) were expressed as frequencies with percentages. The differences between MPE and BPE groups were analysed with the X2 test or Fisher's exact test, whereas the continuous variable (age) was presented as the mean with standard deviation, and differences between the MPE and BPE groups were compared with Student's t-test. Logistic regression analyses were performed in the training set, and the statistically significant variables with an area under the curve (AUC)>0.6 were further selected for multivariable logistic regression analyses. The regression coefficients were regarded as the weights for the variables in the predictive model. The nomogram applied to create the scoring system was developed with independent risk factors based on multivariate logistic analysis using the rms package in R. A model score for each patient was automatically calculated and evaluated by a ROC curve. The optimal cutoff value was attained based on the Youden index, and specificity, sensitivity, positive likelihood ratio (PLR) and negative likelihood ratio (NLR) were applied to assess the diagnostic performance of the nomogram both in the training set and validation sets. Two-sided P <0.05 was regarded as statistically significant.

Results

Study populations

A total of 1261 participants from Renmin Hospital of Wuhan University were included in this research and 70% (N = 883) were randomly assigned to the training set while the remaining participants (N = 378) were included in the internal validation set. Furthermore, 172 patients from Wuhan Union Hospital were included in the external validation set. A detailed flow diagram of patient selection is listed in Fig. 1. The distributions of age, gender and MPE/BPE ratio in the training and validation sets are presented in Table 1.

Fig. 1

Flowchart of participant selection: (a) Renmin Hospital of Wuhan University set; and (b) Wuhan Union Hospital set.

Table 1

Baseline characteristics of the training set and validation set.

Features	Training set (N = 883)	Internal validation set (N = 378)	External validation set (N = 172)	Score (points)median (IQR)
Age (years)	60.62±16.43	60.44±16.78	57.78±1.13	–
Gender:
Female	312 (35.33%)	143 (37.83%)	62 (36.05%)	–
Male	571 (64.67%)	235 (62.17%)	110 (63.95%)	–
MPE
Lung cancer	316 (35.80%)	131 (34.66%)	51 (29.65%)	32.0 (24.0–33.0)
Breast cancer	12 (1.36%)	3 (0.79%)	2 (1.16%)	34.0 (29.0–36.0)
Lymphoma	24 (2.72%)	6 (1.59%)	1 (0.58%)	21.0 (8.0–26.0)
Mesothelioma	6 (0.68%)	1 (0.26%)	1 (0.58%)	28.5 (22.0–32.0)
Ovarian cancer	5 (0.57%)	4 (1.06%)	0 (0%)	26.5 (14.0–32.0)
Other cancers	57 (6.45%)	30 (7.93%)	4 (2.33%)	28.5 (18.0–32.0)
BPE
Tuberculous pleurisy	149 (16.87%)	77 (20.37%)	56 (32.56%)	3.0 (0–7.0)
Parapneumonic effusion	202 (22.87%)	83 (21.96%)	35 (20.35%)	10.0 (4.0–14.0)
Heart failure	37 (4.19%)	12 (3.17%)	7 (4.07%)	12.5 (10.0–14.0)
Pulmonary embolism	2 (0.22%)	1 (0.26%)	0 (0%)	14
Empyema	13 (1.47%)	7 (1.82%)	5 (2.91%)	3.0 (0–4.0)
Other benign diseases	60 (6.8%)	23 (6.08%)	10 (5.81%)	10.0 (3.0–14.0)

MPE=malignant pleural effusion; BPE=benign pleural effusion; IQR= interquartile range.

Other benign diseases: cirrhosis, nephrotic syndrome, pericardial disease, hypoproteinaemia, parasitic infection, systemic lupus erythematosus, rheumatoid arthritis, chylothorax.

Other cancers: oesophageal cancer, gastric cancer, renal cancer, liver cancer, cervical cancer, colorectal cancer, cholangiocarcinoma, endometrial cancer, nasopharyngeal cancer, pancreatic cancer, adrenal carcinoma, prostate cancer, thyroid cancer and cancer of unknown origin.

Construction of a nomogram and novel scoring system in the training set

In the training set, most of the indexes included in this study were significantly different between the patients with BPE and MPE (Table S2). Moreover, 23 parameters remained significant after the univariate regression analysis (Table S3). To construct a highly accurate predictive model, the significant parameters with an AUC >0.6 were further subjected to a multivariate regression model. Six valuable factors (fever, ESR, effusion ADA, serum CEA, effusion CEA and effusion/serum CEA) were selected to establish the predictive model (Fig. 2). Then, a nomogram (Fig. 3a) to discriminate MPE from BPE was built on the basis of a multivariate logistic regression model. This diagnostic nomogram possessed good discriminative ability, as reflected by an AUC of 0.914 (95% CI=0.888–0.935) (Figure S1a). The calibration curve showed that this diagnostic nomogram exhibited good calibration (Fig. 3b). Moreover, decision curve analysis (DCA) was applied to assess the clinical utility of the diagnostic nomogram. As shown in Fig. 3c, if the threshold probability is 0.45, patients with PE would benefit more from using this diagnostic nomogram than the treating all or treating none scenarios.

Fig. 2

Forest plot of the significant parameters in the multivariate regression analysis.

Fig. 3

Calibration and clinical use of a diagnostic nomogram for the discrimination of MPE and BPE. (a) Diagnostic nomogram for identifying MPE from BPE. (b) Calibration curve of the diagnostic nomogram. (c) DCA of the diagnostic nomogram.

To make this predictive model more convenient for physicians to use in clinical practice, we modified the nomogram into a scoring system with integer points: fever (3 points), ESR (4 points), effusion ADA (7 points), serum CEA (4 points), effusion CEA (10 points) and effusion/serum CEA (8 points) (Table 2).

Table 2

A novel scoring system developed from a nomogram of the training set.

Parameters	Score generated from nomogram (points)	Score modified from nomogram (points)
Fever (No)	2.92	3
ESR (≤43 mm/h)	4.42	4
ADA (≤25.74 U/L)	6.7	7
Serum CEA (>5 ng/mL)	3.93	4
Effusion CEA (>5 ng/mL)	10	10
Effusion/serum CEA (>1.66)	7.58	8

ESR=erythrocyte sedimentation rate; ADA=adenosine deaminase;.

CEA=carcinoembryonic antigen.

Diagnostic performance of the scoring system in the training set

In the training set, based on the cutoff value of 15 points (Table 3), patients with PE were more likely to be diagnosed with MPE when the total number of points was more than 15 while the PE patients were less likely to be diagnosed with MPE when the total number of points was less than 15. Moreover, this scoring system showed good discriminative power in differentiating between MPE and BPE, as reflected by an AUC of 0.913 (95% CI=0.887–0.935) (Fig. 4a). When the optimal cutoff points were set at 15, the corresponding specificity, sensitivity, PLR and NLR values were 89.10%, 82.63%, 7.58 and 0.20, respectively (Table 4). Additionally, this scoring system also showed good calibration, implying that no significant difference existed between the predicted probability of the scoring system and the actual probability (Fig. 4c).

Table 3

ROC analysis of the scoring system for identifying MPE in the training set.

Cutoff score	Youden index	Sensitivity% (95%CI)	Specificity% (95%CI)	PLR (95%CI)	NLR (95%CI)
>12	0.6425	90.73 (86.5–94.0)	73.52 (68.3–78.3)	3.43 (2.8–4.1)	0.13 (0.09–0.20)
>14	0.7101	83.78 (78.7–88.1)	87.23 (83.1–90.7)	6.56 (4.9–8.8)	0.19 (0.14–0.25)
>15	0.7173	82.63 (77.5–87.0)	89.1 (85.2- 92.3)	7.58 (5.5–10.4)	0.20 (0.15–0.25)
>17	0.7165	82.24 (77.0–86.7)	89.41 (85.5–92.6)	7.76 (5.6–10.7)	0.20 (0.15–0.26)
>18	0.6881	77.22 (71.6–82.2)	91.59 (88.0–94.4)	9.18 (6.4–13.3)	0.25 (0.20–0.31)

CI=confidence interval; PLR=positive likelihood ratio; NLR=negative likelihood ratio.

Fig. 4

Discrimination and calibration of the scoring system for discrimination of MPE and BPE. ROC curves of the scoring system in the training set (a), internal validation set (b) and external validation set (c). Calibration curves of the scoring system in the training set (d), internal validation set (e) and external validation set (f).

Table 4

Diagnostic performance of the scoring system in differentiating MPE from BPE and lung cancer with PE from TPE in the training and validation sets.

Variables	MPE/BPE			Lung cancer with PE/TPE
	Training set	Internal validation set	External validation set	Training set	Internal validation set	External validation set
AUC (95%CI)	0.913 (0.887–0.935)	0.922 (0.883–0.952)	0.912 (0.859–0.950)	0.978 (0.954–0.991)	0.966 (0.922–0.989)	0.968 (0.914–0.992)
Sensitivity (95%CI)	82.63% (77.5%−87.0%)	81.51% (73.4%−88.0%)	81.36% (69.1%−90.3%)	92.42% (87.8%−95.7%)	95.56% (89.0%−98.8%)	90.2% (78.6%−96.7%)
Specificity (95%CI)	89.10% (85.2%- 92.3%)	93.48% (88.0%−97.0%)	87.61% (80.1%−93.1%)	94.17% (91.9%−99.4%)	89.47% (78.5%−96.0%)	89.29% (78.1%−96.0%)
PLR (95%CI)	7.58 (5.5–10.4)	12.5 (6.6–23.6)	6.57 (4.0–10.9)	32.35 (10.59–98.74)	9.08 (4.30–19.42)	8.42 (3.89–18.02)
NLR (95%CI)	0.20 (0.15–0.25)	0.20 (0.14–0.29)	0.21 (0.12–0.36)	0.078 (0.05–0.13)	0.05 (0.02–0.14)	0.11 (0.05–0.26)

AUC=area under curve; CI=confidence interval; PLR=positive likelihood ratio; NLR=negative likelihood ratio; TPE=tuberculous pleural effusion.

Diagnostic performance of the scoring system in the validation set

In the internal validation set, this scoring model exhibited favourable discriminative power, as reflected by an AUC of 0.922 (95% CI=0.883–0.952) (Fig. 4b). When the optimal cutoff points were also set at 15, the corresponding specificity, sensitivity, PLR and NLR values were 93.48%, 81.51%, 12.5 and 0.20, respectively (Table 4). Moreover, this scoring system also showed good calibration in the internal validation set, as displayed in Fig. 4e.

External validation was performed in an independent set to further validate the diagnostic performance of the scoring system in discriminating MPE from BPE. Good discrimination (AUC=0.912, 95% CI=0.859–0.950) was also observed in the external validation set (Fig. 4c). When the optimal cutoff score was also set at 15, the corresponding specificity, sensitivity, PLR and NLR values were 87.61%, 81.36%, 6.57 and 0.21, respectively (Table 4). Furthermore, as shown in Fig. 4f, this scoring system possessed good calibration in the external validation set.

Diagnostic significance of the scoring system for differentiating between lung cancer with PE and TPE

As TPE with atypical symptoms and laboratory indexes is usually confused with lung cancer with PE in a clinical setting, we exclusively applied the scoring system to differentiate between these two diseases. Encouragingly, the scoring system modified from the nomogram exhibited favourable diagnostic performances in the training and validation sets. As shown in Fig. 5a-c, the AUC values of this scoring system for distinguishing lung cancer with PE from TPE were 0.978 (95% CI=0.954–0.991) in the training set, 0.966 (95% CI=0.922–0.989) in the internal validation set, and 0.968 (95% CI=0.914–0.992) in the external validation set. The values of the other diagnostic parameters, such as specificity, sensitivity, PLR and NLR, when the total score was over 11 points are listed in Table 4. In addition, as shown in Fig. 5d-f, this scoring system used for the differential diagnosis of lung cancer with PE and TPE was also well calibrated in all three data sets.

Fig. 5

Diagnostic ability and calibration of the scoring system for discrimination of lung cancer with PE and TPE. ROC curves of the scoring system in the training set (a), internal validation set (b) and external validation set (c). Calibration curves of the scoring system in the training set (d), internal validation set (e) and external validation set (f).

Discussion

MPE is a common symptom of lung cancer, breast cancer and other cancer-related pleural lesions and indicates an advanced stage of cancer [19,20]. Moreover, the presence of MPE can seriously impair quality of life and shorten the survival of patients; the median survival time of patients with MPE ranges from 3 to 12 months [21,22]. However, patients with BPE, such as TPE and parapneumonic effusions, if treated in time, can usually be clinically cured. Therefore, promptly discriminating MPE from BPE with high accuracy is essential for therapeutic decisions and thus improving the prognosis of patients with MPE.

There have been a variety of attempts to create a more effective method to discriminate between MPE and BPE. Unfortunately, no specific method for differentiating MPE from BPE is perfect in clinical settings. Yang et al. [23] established a scoring system based on five PET-CT parameters for the differential diagnosis of MPE and BPE, and the sensitivity and specificity in the derivation cohort were 83.3% and 92.2%, respectively, with a cutoff value of 4. In addition, Porcel et al. [24] constructed a CT scan-based scoring system, and this scoring model achieved a sensitivity of 88% and a specificity of 94% with a cutoff value of 7 points in the validation set. Moreover, Liu et al. [25] combined Raman spectra bands of PE with orthogonal partial least squares discriminant analysis to distinguish MPE from BPE, and its sensitivity and specificity reached 92.2% and 93.8%, respectively. Despite the relatively higher diagnostic accuracy, the above methods could not be applied in most hospitals, especially primary hospitals, due to the strict requirements for medical equipment. Furthermore, these methods are not good enough to be recommended in routine clinical practice due to inconvenience. Hence, designing a method that is not only feasible and simple but that could also accurately differentiate MPE from BPE with high calibration is important. In contrast, the indexes involved in our scoring system were easily obtained, and more importantly, the total cost of the indexes involved in the scoring model was acceptable (180 RMB) and lower than that of the commonly used cell block (220 RMB).

Although numerous clinical features are different between MPE patients and patients with BPE, the clinical significance of a single index in the discrimination between the two conditions is quite limited due to either its low sensitivity or specificity. In recent years, with the development of analytical approaches, the construction of mathematical models based on multiple markers has been increasingly applied in the field of medicine [26], [27], [28], [29], [30]. This approach combines a series of significant parameters to generate a predictive model to achieve a better diagnostic performance. In the present study, we selected the most significant indexes (fever, ADA, ESR, serum CEA, effusion CEA and effusion/serum CEA) based on the β-coefficient generated by multivariate regression analysis to construct a predictive model. With the aim of establishing a novel scoring system, we converted the nomogram into a scoring system. This scoring system created in this study showed good diagnostic performance in the derivation and validation sets. Our study aimed to design a novel quantitative tool so that clinicians can predict the probability of MPE and BPE. Thus, our scoring system was based on various clinical and laboratorial indexes, which are accessible in most hospitals, even primary hospitals. This present study integrated a total of thirty-four indexes, including not only primary indexes but also informative ratios reported in other studies, such as effusion/serum CEA and effusion NC/LC. A recent study revealed that the effusion NC/LC was lower in patients with TPE than in MPE patients and exhibited a relatively lower diagnostic significance [31]. Despite a significant difference in effusion NC/LC (P<0.0001) in MPE and BPE in our training set, this difference changed insignificantly after the multivariate regression analysis (OR=0.753, 95%CI=0.543–1.044), P = 0.0887). Moreover, Hackner et al. [15] demonstrated that effusion/serum CEA was a useful marker in predicting MPE, with a sensitivity of 85% and specificity of 92%. Our study also found that effusion/serum CEA had higher diagnostic significance in the discrimination between MPE and BPE, as reflected by an AUC of 0.787. To the best of our knowledge, our study incorporated the most laboratory parameters easily accessible in clinical practice to identify MPE from BPE with the largest sample to date.

The differential diagnosis between MPE and BPE can be challenging in a clinical setting because these conditions sometimes share similar clinical symptoms or biochemical indexes, thus leading to high difficulty. Therefore, many researchers have attempted to solve this clinical problem. Ren et al. [14] used four machine learning algorithms to build TPE diagnostic models based on 28 different features and obtained an acceptable diagnostic performance. However, this retrospective study did not construct a scoring system that could be applied in a clinical setting. In addition, a retrospective study [17] reported that a scoring model generated by logistic regression was successfully applied to identify TPE from MPE. However, no additional set was used to validate the scoring system, and this study did not include tumour biomarkers (neither serum CEA nor effusion CEA) in the scoring system. Our study not only considered serum and effusion CEA but also included other tumour markers, such as SCC and NSE-1. Our results showed that serum CEA, effusion CEA and effusion/serum CEA were significantly different between MPE and BPE according to multivariate regression analysis; thus, they were incorporated into the scoring system. Additionally, based on the diagnostic nomogram, we proposed a novel scoring system for the differential diagnosis of MPE and BPE. Very slight changes in the AUC (Fig. S1) were observed between the diagnostic nomogram and the novel scoring system in the training and validation sets, which suggests that the scoring system that was developed from the nomogram is not only convenient to use but also has robust diagnostic significance.

We recommend the wide application of the novel scoring model in most hospitals for a quick and presumptive discrimination between MPE and BPE, as this system is based on six easily accessible clinical variables. For PE patients with an unknown aetiology in a primary hospital and no hardware conditions for PET-CT or pleural biopsy, if the total score is over 15 points, then a high probability of malignancy exists. Our scoring model could provide an important reference for referrals to tertiary hospitals for further examinations, such as cell block, PET-CT, pleural biopsy or molecular tests, which are helpful for the selection of therapeutic regime and prediction of prognosis [32,33].

This present study is not exempt from limitations. First, this study was based on retrospective data, and no published data were used to assess the validity of the data, and thus the validity of the retrospective data was limited. Moreover, the variable of time to diagnosis was not included in the logistic regression because of the lack of data in the retrospective analysis. Next, although we verified the scoring system to discriminate between MPE and BPE in an independent validation set, the sample size in the external validation set was relatively small, and thus the external validation was only performed at a single medical centre. Finally, given that testing the fluid/serum levels of CEA is not routine in all parts of the world, our novel scoring system is limited in terms of the broader applicability in those countries. Therefore, multicentre validation of the scoring system with a large study population is urgently needed to obtain high-level evidence for its clinical application in the future.

In conclusion, the indexes of fever, ESR, effusion ADA, serum CEA, effusion CEA and effusion/serum CEA are significant for discriminating between MPE and BPE. This novel scoring system that was developed from a nomogram shows a favourable diagnostic performance and good calibration in the discrimination between MPE and BPE, especially in distinguishing between lung cancer with PE and TPE. This feasible and novel scoring system used to identify patients with MPE from those with BPE should be further validated in multicentre prospective studies in the near future.

Authors’ contributions

Authors’ contributions: WGD and YJ are the guarantors and take responsibility for the manuscript, including the data and analysis. SFW, ST, YL and YL conceived and designed the study. NZ, YYG, JJX, YLM, SJZ, SWS, WG, HX, PM, XW and TTL acquired the data. YRD and ST analysed the data. SFW and ST drafted the manuscript. All authors approved the final version for submission.

Declarations of Competing Interests

All authors have no competing interests to report.