Developing a Prediction Score for the Diagnosis of Malignant Pleural Effusion: MPE Score.

BACKGROUND: The objective of this study was to develop a diagnostic prediction model for diagnosis of malignant pleural effusion (MPE) from pleural fluid cytology (MPE score).
MATERIALS AND METHODS: Retrospective analysis of pleural fluid cytology was conducted in patients with MPE between 2018 and 2020. Multivariable logistic regression was used to explore the potential predictors. The selected logistic coefficients were transformed into a diagnostic predictive scoring system. Internal validation was done using the bootstrapping procedure.
RESULTS: The data of pleural fluid cytology from 155 MPE patients were analyzed. Seventy-eight positive pleural cytology patients were found (50.32%). Lung cancer was the cancer most commonly sent for pleural fluid testing, with 66.67% positive cytology.  The predictive indicators included pleural fluid protein > 4.64 g/dL, pleural fluid LDH > 555 IU/L, and pleural fluid sugar > 60 mg/dL. Lung mass from imaging and double tap for pleural cytology were used for the derivation of the diagnostic prediction model. The score-based model showed that the area under the receiver operating characteristic curve was 0.74 (95% CI 0.66-0.82). The developed MPE score ranged from zero to 17. The cut-off point was 15 with 88.31% of specificity, 37.18% of sensitivity, positive predictive value of 0.76, and negative predictive value of 0.58. The measurement of the calibration was illustrated using a calibration plot (p-value = 0.49 for the Hosmer-Lemeshow based goodness of fit). Internal validation with 1,000 bootstrap resampling showed a good discrimination.
CONCLUSIONS: The MPE score, as the diagnostic prediction model can be used in planning for more efficient diagnosis of MPE in patients with cancer under MPE.

Introduction

Malignant pleural effusion (MPE) refers to finding cytology pleural fluid caused by the metastasis of malignant mesothelioma, which is mostly due to lung cancer in men and breast cancer in women (Psallidas et al., 2016; Agrawal et al., 2015; Mongardon et al., 2011; Aydin et al., 2009). The MPE is also the cause of exudative pleural effusion from 42% up to 77% (Valdes et al., 1996).

Diagnosis of MPE through cytology initially showed 60% of positive cytology depending on the type of cancer cells and cancer severity (Antonangelo et al., 2015; Loddenkemper and Boutin, 1993). Later, the diagnostic accuracy of MPE was improved by using pleuroscopy to enhance the efficiency of testing metastasis to the pleura (Ali et al., 2019; Ferreiro et al., 2017). However, this method is an invasive procedure. Therefore, less invasive ones are used such as metabolic imaging with 18-fluoro-deoxy glucose positron emission tomography (FDG-PET). The sensitivity was increased to 90% (Nakajima et al.,2015; Toaff et al.,2005); nonetheless, this method could not determine the types of cancer cells.

In addition, epigenetic analysis of the pleural fluid was used to distinguish malignant DNA from methylation-specific PCR (MSP). This could help the diagnosis of MPE and efficiently specify the types of cancer cells (Herman et al., 1996; Brock et al., 2005; Zhang et al., 2007). Nevertheless, this method is expensive and is not used widely.

Cytology is still a key method with 60% sensitivity depending on the type of cancer (Johnston, 1985; Starr and Sherman, 1991; and Hsu, 1987). Mostly, positive cytology pleural fluid is found in lung cancer and breast cancer. According to studies (Garcia et al., 1994; Desai and Lee, 2017), the repetition pleural fluid cytology can increase the diagnostic opportunities by 24%. However, more than double of the repetition is impractical for the diagnosis of MPE (Garcia et al., 1994). Thus, pleuroscopy is also required for confirmation to conduct a pleural biopsy, which is an invasive procedure.

The clinical features and pleural fluid profile should be used to assist MPE diagnosis as a routine clinical practice and a diagnostic prediction score to facilitate decision-making on whether to wait for cytology results or perform an invasive procedure for efficient and rapid MPE diagnosis. This is because some hospitals still have limited diagnosing capabilities, or patients may have to wait for the cytologic results for weeks. Moreover, not all hospitals have the facilities to perform pleuroscopy, which make the test inaccessible for many patients. Therefore, this diagnostic research aimed to develop a diagnostic prediction model to help the decision-making in the diagnosis of MPE and plan appropriate and efficient diagnostic guidelines in the future.

Materials and Methods

Clinical characteristics of patients with suspected MPE and cytology results of pleural fluid in Buddhasothorn Hospital, Chachoengsao, and Thailand were collected between 2018 and 2020.. The inclusion criteria were as follows:

1) Patients older than 18 years.

2) The results of the pleural fluid comprised of biochemical tests and key serum tests, i.e., lactate dehydrogenase (LDH) and protein.

3) Cancer data based on the radiology findings.

The exclusion criteria consisted of:

1) No results of a pathological diagnosis in the case of negative cytology pleural fluid based on the cytologic results.

Data analysis

Step 1: The data were analyzed to find the potential factors in the diagnosis of positive cytology pleural fluid (MPE) using univariate regression analysis and multivariate logistic regression analysis.

Step 2: Multiple imputation for missing data: Three predictor variables (pleural fluid white blood cell, pleural fluid lymphocyte, pleural fluid sugar) had more than 10% missing values, which could lead to biased estimates of the diagnostic model with the complete-case analysis. Multiple imputation with chained equation via mi impute chained command was used to generate missing values prior to model derivation. The logit model was chosen for the imputation of multivariable missing predictors.

Step 3: The predictive variables from the multivariate logistic regression analysis were brought into the transformation of the risk score. A logistic regression coefficient was used to develop the MPE score to help diagnosing MPE.

Step 4: The area under the receiver operating characteristic curve (AUC) based on the MPE score for the diagnosis of MPE was calculated and showed the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), likelihood ratio for a positive test (LR +) and likelihood ratio for a negative test (LR -).

Step 5: The accuracy was tested through calibration curve and the Hosmer-Lemeshow goodness of fit test. Internal validation was tested by using the bootstrapping procedure (1,000 replicates).

This study was approved by the Institutional Review Board of Buddhasothorn Hospital under the codes BSH-IRB 036/2563

Results

The data of the cytologic results of pleural fluid of 166 patients were collected. Eleven patients were excluded; including six patients with incomplete data of the biochemical tests and five patients did not have the pathological results to confirm the malignancy diagnosis. Therefore, the data were collected for 155 patients.

Seventy-eight patients (50.32%) had positive cytology whereas seventy-seven patients (49.68%) had negative cytology. In terms of the pathological diagnosis and among different cancers, lung cancer was the most frequent cancer (61.9%) that needed the pleural fluid test. It was also the cancer with 66.67% of positive cytology (Table1).

Table 1.

Types of Cancer Confirming the Pathological Diagnosis with the Cytologic Results of the Pleural Fluid

Types of Cancer	Positive Cytology (%)	Negative Cytology (%)	Total
Lung	64 (66.67)	32 (33.33)	96
Breast	10 (38.46)	16 (61.54)	26
Colorectal	0 (0.00)	4 (100.00)	4
Lymphoma	0 (0.00)	9 (100.00)	9
Gastric	1 (14.29)	6 (85.71)	7
Prostate	0 (0.00)	3 (100.00)	3
Ovary	1 (20.00)	4 (80.00)	5
Head and Neck	1 (100.00)	0 (0.00)	1
Melanoma	1 (25.00)	3 (75.00)	4

Based on the univariate analysis of the clinical characteristics and pleural fluid profile on MPE, it was found that lung mass detected by clinical imaging, lung cancer, breast cancer, and lung cancer with extrathoracic metastasis were the factors significantly affecting the predictive variables on MPE (Table2).

Table 2

Univariate Logistic Regression Analysis of MPE and Variable Factors

Variables	Odds Ratio	95% Confidence Interval	p-value
Pleural fluid protein <4.64 g/dL	1	reference
Pleural fluid protein >4.64 g/dL	1.56	0.82 - 2.94	0.171
Pleural fluid LDH <555 IU/L	1	reference
Pleural fluid LDH >555 IU/L	1.14	0.61 - 2.14	0.686
Pleural fluid sugar <60 mg/dL	1	reference
Pleural fluid sugar >60 mg/dL	2.01	0.83 - 4.88	0.122
Low protein ratio (<0.5)	1	reference
High protein ratio (>0.5)	1.38	0.56 - 3.37	0.478
Low LDH ratio (<0.6)	1	reference
High LDH ratio (>0.6)	1.47	0.63 - 3.45	0.372
No detected lung mass	1	reference
Lung mass detected by clinical imaging	6.06	2.65- 13.87	<0.001*
No clinical extrathoracic metastasis	1	reference
Clinical extrathoracic metastasis	1.08	0.54 - 2.19	0.819
Lung cancer without extrathoracic metastasis	1	reference
Lung cancer with extrathoracic metastasis	3.24	1.66 - 6.30	0.001*
Single tap for pleural cytology	1.23	0.51 - 2.95	0.647
Double tap for pleural cytology	1.88	0.68 - 5.15	0.223
Multiple tap for pleural cytology	1	reference
Lung cancer	13.26	4.31 - 40.78	<0.001*
Breast cancer	4.26	1.16 - 15.73	0.029*
Other cancers	1	reference

protein ratio, Pleural fluid protein / serum protein; LDH ratio, Pleural fluid LDH / serum LDH; *, statistical significant

Model development

After analyzing the variable factors by univariate logistic regression analysis, the potential predictors affecting the diagnosis of MPE were selected for the multivariate logistic regression analysis of the scoring system derivation. The area under the receiver operating characteristic curve (AUC) for the final model was equal to 0.74 (95% CI 0.66-0.82).

Score transformation

Each potential predictor in the multivariable model was assigned with a specific score derived from the logistic regression coefficient (Table 3). The scoring scheme had a total score ranging from zero to 17. For the discriminative ability, the area under the parametric ROC curve for the score-based logistic regression model was equal to 0.74 (95% CI 0.66-0.82) (Figure 1). The measurement of the calibration is illustrated with a calibration plot, and the p-value via the Hosmer-Lemeshow goodness of fit test is equal to 0.49 (Figure 2).

Table 3

Risk Score Derivation Using Multivariate Logistic Regression Coefficients

Potential Predictors	Odds Ratio	95% Confidence Interval	p-value	Coefficients	Score
Pleural fluid protein <4.64 g/dL	1	reference	-	-	0
Pleural fluid protein >4.64 g/dL	1.27	0.63- 2.56	0.509	0.24	1
Pleural fluid LDH <555 IU/L	1	reference	-	-	0
Pleural fluid LDH >555 IU/L	1.27	0.59 - 2.72	0.536	0.24	1
Pleural fluid sugar <60 mg/dL	1	reference	-	-	0
Pleural fluid sugar >60 mg/dL	3.31	1.15 - 9.52	0.026	1.2	5
No detected lung mass	1	reference	-	-	0
Lung mass detected by clinical imaging	6.38	2.72- 14.98	<0.001	1.85	8
No double tap for pleural cytology	1	reference	-	-	0
Double tap for pleural cytology	1.55	0.68 - 3.54	0.298	0.44	2

Figure 1

Performance of the Clinical Risk Score, Area under the Receiver Operating Characteristics Curve (AUC), and 95% Confidence Band (Above). The calibration plots (pmcalplot) comparing the observed probabilities (y) and predicted probabilities (x) of the use of the MPE score to predict MPE (Below).

Figure 2

Observed risk (circle) versus the score predicted risk (solid line) of the positive cytology pleural effusion (malignant pleural effusion). The size of the circle represents the frequency of MPE in each score (Left). Well-fitting model shows non-significance difference between the model and the observed data on the Hosmer-Lemeshow goodness of fit test (p-value 0.49) (Right)

According to the sensitivity and specificity in each cut-off point, the point at 15 had 88.31% of specificity and 37.18% of sensitivity. This point displayed appropriate specificity that could be used as a diagnosis tool (Table 4). With the cut-off point of the MPE score at 15, to help the diagnosis of MPE, the odds ratio was equal to 3.18 (95% CI 1.35-8.11; p-value 0.004), positive predictive value (PPV) was equal to 0.76, negative predictive value (NPV) was equal to 0.58, positive likelihood ratio (LR+) was equal to 3.18, and negative likelihood ratio (LR-) was equal to 0.71.

Table 4

The Sensitivity, Specificity, Positive Likelihood Ratio (LR+), and Negative Likelihood Ratio (LR-) of Each Cut-Off Point Value of the MPE Score

Cut-off Point	Sensitivity	Specificity	Positive Likelihood Ratio	Negative Likelihood Ratio
1	100.00%	0.00%	1.00
2	98.72%	3.90%	1.03	0.33
5	97.44%	3.90%	1.01	0.66
6	93.59%	18.18%	1.14	0.35
7	89.74%	31.17%	1.3	0.33
8	88.46%	36.36%	1.39	0.32
9	88.46%	41.56%	1.51	0.28
10	87.18%	50.65%	1.77	0.25
11	85.90%	55.84%	1.94	0.25
12	83.33%	57.14%	1.94	0.29
13	79.49%	61.04%	2.04	0.34
14	61.54%	75.32%	2.49	0.51
15	37.18%	88.31%	3.18	0.71
16	15.38%	96.10%	3.95	0.88
17	7.69%	100.00%		0.92
>17	0.00%	100.00%		1

Internal validation

Through conducting the internal validation using the predictive model with 1,000 resampling bootstrap method data set, the mean of the AUC of the apparent curve was obtained equal to 0.75, the test curve was equal to 0.72 (bootstrap estimator), and average estimates of the optimism curve was equal to 0.03 (Table 5).

Table 5

Internal Validation via 1,000 Resampling Bootstrap Method

Variable	Observe	Mean AUC	Standard Deviation	Min	Max
Apparent C-statistics	1,000	0.75	0.04	0.63	0.88
Test C-statistics	1,000	0.72	0.02	0.62	0.75
Optimism C-statistics	1,000	0.03	0.04	-0.1	0.15

Discussion

Pleural fluid found in malignant disease was due to two major conditions, i.e., paramalignant pleural effusion (PMPE) and malignant pleural effusion (MPE) (Wong et al., 1963; Epelbaum and Rahman, 2019). The PMPE is not a consequence of a malignant disease spreading to the pleura. The probability that an effusion is paramalignant is higher when the effusion is transudative, while MPE is exudative. Therefore, understanding the differentiation between PMPE and MPE is necessary.

There are studies on the use of the cancer ratio using the ratio of serum lactate dehydrogenase (LDH) to adenosine deaminase (ADA) in pleural fluid. The ratios used were based on the cut-off level > 20 to help diagnose the causes of exudative pleural fluid between benign and MPE. It was found that sensitivity and specificity were high because the relationship of the levels of serum LDH was usually high in a malignant disease (Verma et al., 2016; Korczyński et al., 2018; Verma et al., 2016). This was a result of using glycolysis for energy in tumor cells instead of oxidative phosphorylation, a switch in the adenosine triphosphate (ATP) generating pathways, which was mediated by LDH (Pfeiffer et al., 2001; Goldman et al., 1964; Mansouri et al., 2017). Likewise, the infection caused by tuberculosis in the pleural fluid usually had a higher ADA secreted by mononuclear cells, lymphocytes, neutrophils, and red blood cells (Liang et al., 2008; Jiménez Castro et al., 2003).

However, the meta-analysis of using the cancer ratio for the diagnosis of MPE was based on the data from the PubMed and EMBASE databases The cancer ratio had a high diagnostic accuracy for predicting MPE. The pooled sensitivity and specificity of the cancer ratio were equal to 0.97 (95% CI 0.92-0.99) and 0.89 (0.69-0.97) respectively; with AUC equal to 0.98 (95% CI 0.97-0.99). Nevertheless, there were some limitations due to the bias of patient selection and potential partial verification (Han et al., 2019). Yet, it was frequently found that serum LDH may not be raised in the case of cancer. A high LDH may be related to poorer overall survival. Some minor studies (Chantharakit, 2018) also found that high LDH was related to cancer under liver metastasis; however, the data still contained a few limitations.

Porcel et al., (2004) used a panel of tumor markers, i.e., carcinoembryonic antigen (CEA), cancer antigen (CA) 125, carbohydrate antigen (CA) 15-3, and cytokeratin 19 fragments in pleural fluid for the differential diagnosis of benign and malignant effusions. The combination of the four tumor markers reached a sensitivity of 54%, whereas the combined use of the cytology and the tumor marker panel increased the diagnostic yield of the former by 18% (95% CI; 13-23%). Yang et al., (2017) reported about a updated meta-analysis of patients with undiagnosed pleural effusion and showed that the combinations of positive pleural CEA + CA 15-3 and CEA + CA 19-9 were highly suspicious for pleural malignancy. Still, the sensitivity of these tests was poor.

Clive et al., (2014) studied prognostic indicators and found that the ones affecting the survival of MPE patients were pleural fluid LDH, the Eastern Cooperative Oncology Group (ECOG) performance status, and neutrophil-to-lymphocyte ratio (NLR). It was also found that the tumor type could be developed by the LENT scoring system as a prognostic prediction model. The levels of pleural fluid LDH were key markers of inflammation or cellular injury. LDH levels greater than three times the upper limit of normal (often >1,000 U/L) are often indicative of pleural infection. This can also be associated with rheumatoid pleurisy, tuberculous pleurisy or malignancy.

This study is different from previous studies as it focused only on PME. Pleural fluid cytology results may be positive malignant cells or negative malignant cells and it is not the intention of this study to distinguish MPE from benign disease. Therefore, all the pleural fluid was exudative pleural fluid according to Light’s criteria. The clinical information fitting the malignant disease was used to find the predictive indicators affecting the diagnosis of MPE using pleural fluid cytology to develop a diagnostic prediction model (MPE score). This was the first diagnostic prediction model used to assist in the diagnosis of MPE in patients with cancer. The data from the pleural fluid biomarkers were used along with the clinical data of the patients rather than using only the data from the biomarkers.

However, the standard diagnosis of MPE features pleural fluid cytology supported by testing for confirmation by pleural biopsy in the case of negative pleural fluid cytology; still, with suspected MPE. This is an invasive procedure. Despite the effort to use a non-invasive technique, e.g., biomarker tests or molecular analysis from the pleural fluid for the diagnosis of MPE, the less invasive methods have not become popular yet. The validated clinical data found that there were still some limitations of use; thus, further studies are required.

Therefore, using the MPE score at the cut-off point of 15, which has high specificity, may help in predicting MPE diagnosis to make decisions about planning for investigation while waiting for pleural fluid cytology results.This would enhance better efficiency of the diagnosis of MPE.

Author Contribution Statement

Chaichana Chantharakhit: Designed the study, reviewed the paper, collected data, analyzed data, and edited the final version. Nantapa Sujaritvanichpong: Collected data. All authors read and approved the final version.

Funding Statement

The authors confirm that there are no relevant financial or non-financial competing interests to report and no conflicts of interest to declare.

Data Availability

The data used to support the findings of this study have been deposited in the repository [https://drive.google.com/drive/folders/1Enrqg5Zq_co3rY73-epiqymNe_njLL7W?usp=sharing].

We confirm that there are no relevant financial or non-financial competing interests to report and no conflicts of interest to declare