Pleural homocysteine for malignant pleural effusion: A prospective and double-blind diagnostic test accuracy study.

OBJECTIVE: To assess the accuracy of pleural fluid homocysteine for discriminating malignant pleural effusion (MPE) and benign pleural effusion (BPE).
METHODS: A total of 194 patients from two cohorts (Hohhot and Changshu) with undiagnosed pleural effusion were prospectively enrolled. Their pleural homocysteine was measured, and its diagnostic accuracy and net benefit for MPE were analyzed by receiver operating characteristic (ROC) curve analysis and decision curve analysis, respectively.
RESULTS: In the Hohhot cohort (n = 136) and the Changshu cohort (n = 58), MPE patients had significantly higher homocysteine levels than BPE patients. The areas under the ROC curves of homocysteine for the diagnosis of MPE were 0.61 (p = 0.027) and 0.59 (p = 0.247), respectively. The decision curves of homocysteine were close to the reference line in both the Hohhot cohort and the Changshu cohort.
CONCLUSION: The diagnostic accuracy of pleural fluid homocysteine for MPE was low.

INTRODUCTION

Malignant pleural effusion (MPE) is a common sign in late‐stage cancer patients. It is estimated that one in six cancer patients will develop MPE during the course of their disease. The most frequent etiologies of MPE are lung cancer, breast cancer, ovarian cancer, and lymphoma. The prognosis of MPE is poor, with a median survival of <1 year. A timely and accurate diagnosis is crucial to improving the quality of life and survival of MPE patients. However, pleural effusion is not a specific sign of cancer. It can also be caused by pneumonia, tuberculosis, and heart failure, therefore it is challenging for clinicians to differentiate MPE from benign pleural effusion (BPE).

Currently, pleural fluid cytology, imaging‐guided pleural biopsy, and thoracoscopy are the gold standards for diagnosing MPE. However, these diagnostic tools have limitations. Cytology has the advantages of low cost, rapidity, and high specificity, but its sensitivity is only 60% and its diagnostic accuracy is observer‐dependent. , , Pleural biopsy and thoracoscopy have high accuracy, but they are invasive tools and can cause complications such as infection and bleeding. In addition, special training is needed for thoracoscopy, limiting its use in remote areas. Tumor markers in pleural fluid represent promising diagnostic tools for MPE because they have the advantages of minimal invasiveness, low cost, rapidity, and objectiveness. The evidence from a systematic review and meta‐analysis indicated that pleural tumor markers had specificities of >90% for MPE, but their sensitivities were only approximately 50%. , , , It is therefore necessary to explore novel tumor markers for MPE.

Homocysteine is an amino acid intermediate formed in the metabolism of methionine, folic acid, and vitamin B12. Previous studies have revealed that elevated serum homocysteine is a risk factor for cardiovascular and cerebrovascular diseases, fractures, and cancers. , Two previous studies have indicated that homocysteine in pleural fluid was a promising diagnostic marker for MPE, with an area under the curve (AUC) of >0.80. , However, these two studies are from the same center and their results have not been validated. In this study, we investigated the diagnostic accuracy of pleural fluid homocysteine for MPE. Our results indicate that pleural fluid homocysteine has low diagnostic value for MPE. We also discuss the possible explanations for the inconsistency between previous studies and our work. We reported this work following the Standards for Reporting of Diagnostic Accuracy Studies (STARD) guidelines.

MATERIALS AND METHODS

Participants

There were two cohorts in this study. The first cohort was from the SIMPLE study, a prospective, preregistered, and double‐blind diagnostic test accuracy study. The design details of the SIMPLE study have been described previously. Briefly, we recruited patients with undiagnosed pleural effusion who visited the Department of Respiratory and Critical Care Medicine, the Affiliated Hospital of Inner Mongolia Medical University, between September 2018 and July 2021 (Hohhot cohort). The other cohort was from Changshu, China (Changshu cohort). The participants in the Changshu cohort were enrolled at the Affiliated Changshu Hospital of Xuzhou Medical University between June 2020 and July 2021. The inclusion criteria for both the Hohhot and Changshu cohorts were (i) patients admitted with undiagnosed pleural effusion and (ii) patients in whom thoracentesis was needed to determine the etiology. The presence of pleural effusion was confirmed by medical imaging methods (e.g. X‐ray, computed tomography, ultrasound). The exclusion criteria for the Hohhot cohort and the Changshu cohort were as follows: (i) patients with pleural effusion and a clear etiology in the past 3 months; (ii) <18 years old; (iii) pregnant; (iv) patients with insufficient pleural fluid specimens; (v) patients who developed pleural effusion during treatment; and (vi) trauma or operation‐induced pleural effusion.

This study was approved by the Ethics Committees of the Affiliated Hospital of Inner Mongolia Medical University (No: 2018011) and the Affiliated Changshu Hospital of Xuzhou Medical University (No: KY2021014). Informed consent was obtained from all participants.

Diagnostic criteria

MPE was diagnosed with pleural biopsy or cytology. Tuberculosis pleural effusion (TPE) was diagnosed with pleural fluid Mycobacterium tuberculosis (Mtb) culture, acid‐fast staining, response to antituberculosis treatment, or pleural biopsy. Parapneumonic effusion (PPE) was diagnosed with pleural fluid bacterial culture, biopsy, imaging characteristics, and response to antibiotic treatment. Pleural effusion caused by heart failure was diagnosed based on the clinical picture, imaging features, laboratory tests (e.g. serum natriuretic peptides) and treatment response. The homocysteine concentration was blinded to clinicians who made the final diagnosis in both cohorts.

Homocysteine and routine biochemical index assays

A pleural fluid specimen was collected into an anticoagulant‐free tube at the time of patient admission. After centrifugation, the supernatant of the pleural fluid was collected and stored at −70°C until analysis. In both cohorts, homocysteine was measured by a Beckman AU5800 biochemical analyzer in November 2021. The coefficient variations (CVs) of homocysteine were 2.83% and 8.92% at concentrations of 23.35 and 5.53 μmol/L, respectively. The laboratory technician who measured the homocysteine level did not know the patients' clinical details.

Pleural fluid concentrations of glucose, adenosine deaminase (ADA), white blood cells (WBCs), lactate dehydrogenase (LDH), and protein were collected from the medical records of the participants. Pleural glucose was determined by the hexokinase method. LDH was determined by the lactate‐to‐pyruvate method. ADA was determined by the peroxidase method. Total protein was determined by the biuret method. LDH, ADA, and glucose were determined by a Beckman AU5800 biochemical analyzer. WBC counts in the pleural fluid were determined by a Sysmex XN 2000 hematology analyzer.

Statistical analysis

Continuous data are expressed as the median (quartile range), and categorical data are expressed as absolute numbers and percentages. We used the Kolmogorov–Smirnov method to test the normal distribution of the continuous data. The Mann–Whitney U (comparison of two groups) and the Kruskal–Wallis H (comparison of more than two groups) tests were used for continuous data comparisons. The chi‐square test was used to compare categorical variables. We used receiver operating characteristic (ROC) analysis to evaluate the diagnostic accuracy of homocysteine for MPE. Decision curve analysis (DCA) was used to assess the net benefit of homocysteine measurement. All statistical analyses and graphs were performed using SPSS (Version 23.0), GraphPad Prism 6.0 (GraphPad Software), and Stata SE 16 (StataCorp). p < 0.05 was considered statistically significant.

RESULTS

Characteristics of the study population

A total of 232 participants were recruited in the two cohorts. Among them, 38 participants were excluded and 194 participants (Hohhot n = 136, Changshu n = 58) were included in the current study. According to the results of the pleural biopsy, microbiology, cytology, acid‐fast staining, and treatment response, all participants were categorized as BPE or MPE. Figure 1 is a flowchart of participant selection. Table 1 lists the baseline characteristics of the participants.

FIGURE 1

A flowchart of the participant selection process

TABLE 1

Characteristics of the participants

	Hohhot cohort (n = 136)			Changshu cohort (n = 58)
	MPE (n = 57)	BPE (n = 79)	p	MPE (n = 26)	BPE (n = 32)	p
Age, years	71 (46–85)	72 (18–90)	0.870	76 (53–88)	69 (18–96)	0.003
Male, n (%)	33 (58)	55 (70)	0.219	14 (54)	20 (63)	0.691
Appearance			0.044			0.231
Watery or serous	40	66		16	26
Bloody or blood tinged	17	11		9	5
Purulent or turbid	0	2		1	1
Glucose, mmol/L	5.8 (4.6–6.8)	5.7 (4.6–7.1)	0.675	6.4 (5.8–8.0)	5.8 (4.7–7.0)	0.052
ADA, U/L	8 (5–12)	10 (5–25)	0.126	12 (9–16)	26 (15–54)	0.004
WBC, 106/ml	925 (692–1516)	814 (363–2065)	0.377	909 (722–1709)	1781 (705–3834)	0.171
LDH, U/L	214 (174–417)	171 (96–397)	0.025	344 (243–539)	316 (188–644)	0.975
Protein, g/L	37 (31–41)	29 (17–41)	0.006	42 (38–46)	46 (36–50)	0.131
Diagnostic		PPE (n = 31)			PPE (n = 10)
		TPE (n = 16)			TPE (n = 13)
		HF (n = 20)			HF (n = 5)
		Others (n = 12)			Others (n = 4)
Type of MPE
	Lung cancer (n = 42)			Lung cancer (n = 21)
	Mesothelioma (n = 5)			Mesothelioma (n = 1)
	Gastric cancer (n = 2)			Metastatic cancer (n = 4)
	Lymphoma (n = 1)
	Pleural synovial sarcoma (n = 1)
	Metastatic cancer (n = 6)

Note: Data are presented as the median (interquartile range) or absolute number. Age is presented as medians and range.

Abbreviations: ADA, adenosine deaminase; BPE, benign pleural effusion; LDH, lactate dehydrogenase; MPE, malignant pleural effusion; WBC, white blood cell.

Comparison of homocysteine levels between the MPE and BPE patients

We analyzed the participants from the Hohhot cohort and the Changshu cohort independently because (i) they are two independent cohorts and (ii) we intended to test whether the findings in one cohort could be validated by another cohort. Figure 2 shows the pleural homocysteine levels in the MPE and BPE patients. In the Hohhot cohort, the median homocysteine levels in the MPE patients and BPE patients were 20.7 (interquartile range 15.7–30.1) μmol/L and 16.4 (interquartile range 13.6–25.3) μmol/L, respectively. In MPE patients, the median homocysteine concentration was significantly higher than that in BPE patients (p = 0.027). In the Changshu cohort, the median homocysteine levels in the MPE patients and BPE patients were 13.0 (interquartile range 10.0–16.5) μmol/L and 11.3 (interquartile range 9.2–14.5) μmol/L, respectively. The difference was not statistically significant (p = 0.247). These insignificant differences in homocysteine in the Changshu cohort may be due to its smaller sample size and inadequate statistical power. In addition, the Hohhot cohort had significantly higher homocysteine levels than the Changshu cohort (p < 0.001).

FIGURE 2

Comparison of pleural fluid homocysteine levels between MPE and BPE patients in the Hohhot and Changshu cohorts. BPE, benign pleural effusion; MPE, malignant pleural effusion; PF, pleural fluid

Next, we analyzed whether the etiologies of BPE could affect pleural fluid homocysteine levels. The pleural fluid homocysteine levels in patients with different etiologies in both cohorts are presented in Figure 3. In the Hohhot cohort, homocysteine levels in all types of pleural effusion were significantly different (p = 0.011). In the Changshu cohort, homocysteine was not significantly different in all types of pleural effusion (p = 0.164).

FIGURE 3

Homocysteine in patients with various types of pleural effusion in the two cohorts. HF, heart failure; MPE, malignant pleural effusion; PF, pleural fluid; PPE, parapneumonic pleural effusion; Others, other types of pleural effusion; TPE, tuberculosis pleural effusion

Diagnostic accuracy and net benefit of homocysteine for MPE

Figure 4 shows the ROC curve of homocysteine for diagnosing MPE. In diagnostic test accuracy studies, there are many available metrics for estimating the diagnostic accuracy of a given test (e.g. sensitivity, specificity, likelihood ratios, predictive values). Sensitivity and specificity are the basic metrics because they reflect two aspects of an index test: ruling in or ruling out the target disease. However, both sensitivity and specificity have limitations because they are threshold dependent. , In contrast, the area under the ROC curve (AUC) is a threshold‐independent metric used for estimating the overall diagnostic accuracy of an index test. , In the Hohhot cohort, the AUC was 0.61 (95% confidence interval [CI] 0.52–0.71, p = 0.027). At the threshold of 16.73 μmol/L, the sensitivity was 0.74 (95% CI 0.60–0.84) and the specificity was 0.57 (95% CI 0.45–0.68). In the Changshu cohort, the AUC was 0.59 (95% CI 0.44–0.74, p = 0.247). The decision curves of homocysteine in both cohorts were close to the two reference lines (Figure 5), indicating that the net benefit of homocysteine was low.

FIGURE 4

Receiver operating characteristic curve of homocysteine for malignant pleural effusion. AUC, area under the curve

FIGURE 5

Decision curve of homocysteine for malignant pleural effusion

DISCUSSION

For the two cohorts, we analyzed the diagnostic accuracy of homocysteine for MPE. In the Hohhot cohort, MPE patients had significantly higher homocysteine levels than BPE patients. In the Changshu cohort, although the MPE patients had a higher homocysteine level than BPE patients, we failed to observe a statistical significance, which may be due to the small sample size and insufficient statistical power. The AUCs of homocysteine in both cohorts were only approximately 0.60. In addition, the decision curve of homocysteine was close to the reference lines. These results indicate that the diagnostic accuracy of pleural homocysteine for MPE is low.

To date, many pleural tumor markers have been investigated to detect MPE, such as carcinoembryonic antigen (CEA), carbohydrate antigen 15‐3 (CA15‐3), and carbohydrate antigen 125 (CA125). Systematic reviews of these tumor markers revealed that they had sensitivities of approximately 0.50, specificities of >0.90, and AUCs of >0.80. , , Compared with these conventional tumor markers, homocysteine had an AUC of approximately 0.60. Because AUC is a global indicator of diagnostic accuracy, we concluded that the diagnostic accuracy of homocysteine for MPE is limited, and its diagnostic accuracy is inferior to conventional pleural tumor markers.

To date, two studies have investigated the diagnostic accuracy of pleural homocysteine for MPE. , These two studies concluded that the AUC of homocysteine was approximately 0.85. Although increased homocysteine in MPE patients was observed in the present study, the AUCs were only approximately 0.60 in the two independent cohorts. The reason for the inconsistency between previous studies and our study remains unknown. We noticed that the rates of TPE in the Hohhot cohort and Changshu cohort were 11.7% and 22.4%, respectively. In previous studies, the prevalence of TPE was only approximately 4%. , Furthermore, increased homocysteine was also observed in the TPE patients of the Hohhot cohort. It therefore seems that the prevalence of TPE in the studied population may affect the diagnostic accuracy of homocysteine.

Homocysteine may not be useful to diagnose MPE in countries and regions with a high tuberculosis burden. Notably, the homocysteine levels in our cohorts were higher than those in previous studies. , The Hohhot cohort had higher homocysteine than the Changshu cohort (p < 0.001) although their homocysteine assays were identical. These results suggest that homocysteine may be affected by geographic region or individual ethnicity. Previous studies also revealed that the consistency between different homocysteine measurement platforms was poor, , indicating that the homocysteine assay is a possible explanation for the inconsistency between our study and previous studies. , In addition, study design may be another possible explanation for the inconsistency between previous studies and our study. In our study, only adult participants were enrolled, while in previous studies, some children were enrolled. ,

The sources of increased pleural homocysteine in MPE patients remain unclear. We propose a possible explanation here. General population cohort studies revealed that serum B vitamins were protective against various cancers. , , , Notably, B vitamins are involved in homocysteine metabolism. , Therefore, decreased serum B vitamins may lead to increased serum homocysteine. Indeed, higher serum homocysteine has been observed in various cancers. We hypothesize that pleural homocysteine is passively diffused from serum to the pleural cavity. However, it cannot be excluded that pleural homocysteine is released locally by cancer cells or immune cells residing in the pleural cavity. Further studies are needed to investigate the sources of pleural homocysteine.

Our study has some limitations. First, the homocysteine concentration was missing for some patients due to a lack of specimens, which may bias the results. Second, due to financial restrictions and insufficient serum specimens, we did not measure homocysteine and B vitamins in the serum. This limitation hinders us from thoroughly investigating the sources of pleural homocysteine. Third, we hypothesized that PE was caused by a single etiology and not all diagnostic tools were used on every patient, but it has been reported that some undiagnosed PE patients have more than one apparent etiology. , However, to the best of our knowledge, nearly all available published works suffer from this limitation. Fourth, we used the stored pleural fluid specimen to measure homocysteine, and the long‐term stability of homocysteine in pleural fluid specimens has not been investigated. In addition, the homocysteine assay used in this work was designed for serum rather than pleural fluid. The effect matrix effect on pleural fluid homocysteine determination remains unclear. However, a previous study indicated that the stability of serum homocysteine is rather good, indicating that the storage conditions have little effect on homocysteine in vitro.

In summary, our study indicates that the diagnostic accuracy of pleural homocysteine for MPE is low. Considering the missing data and the small sample size of our study, it remains necessary to perform studies with large sample sizes to investigate the diagnostic accuracy of homocysteine for MPE in the future.

AUTHOR CONTRIBUTIONS

Zhi‐De Hu designed and supervised the study. Xi‐Shan Cao analyzed the data and drafted the manuscript. Xu‐Hui Wen, Wen Zhao, and Yu‐Ling Han added intellectual contributions to the manuscript. Ting‐Wang Jiang, Jin‐Hong Huang, and Hong Chen enrolled the participants in the Changshu cohort. Li Yan enrolled the participants in the Hohhot cohort. Zhi‐De Hu and Wen‐Qi Zheng critically reviewed and edited the manuscript. All authors have accepted responsibility for the entire content of this manuscript and approved its submission.

DISCLOSURE

None.