Diagnostic value of circulating free DNA for the detection of EGFR mutation status in NSCLC: a systematic review and meta-analysis.

Epidermal growth factor receptor (EGFR) mutation is a reliable and sensitive biomarker for EGFR-TKI therapy in non-small-cell lung cancer (NSCLC). However, detection of EGFR mutation in tissues has obvious limitations. Circulating free DNA (cfDNA) has been reported as an alternative approach for the detection of EGFR mutations. This systematic review and meta-analysis was designed to assess the diagnostic performance of cfDNA, compared with tissues. True-positive (TP), false-positive (FP), false-negative (FN), and true-negative (TN) values were extracted or calculated for each study. Pooled sensitivity, specificity, positive likelihood ratio (PLR), negative likelihood ratio (NLR), and diagnostic odds ratio (DOR) were calculated. A summary receiver operating characteristic curve (SROC) and area under curve (AUC) were used to evaluate the overall diagnostic performance. 20 eligible studies involving 2012 cases were included in this meta-analysis. The pooled sensitivity, specificity, PLR, NLR, and DOR were 0.674 (95%CI: 0.517-0.800), 0.935 (95%CI: 0.888-0.963), 10.307 (95%CI: 6.167-17.227), 0.348 (95%CI: 0.226-0.537), and 29.582 (95%CI: 4.582-60.012), respectively. The AUC was 0.93 (95% CI: 0.90-0.95). The meta-analysis suggests that detection of EGFR mutation by cfDNA is of adequate diagnostic accuracy and cfDNA analysis could be a promising screening test for NSCLC.

Non-small cell lung cancer (NSCLC) is the most common type of lung cancer, accounting for about 80% cases of lung cancer, and most NSCLC patients are at advanced stage when diagnosed1. Epithermal growth factor receptor-tyrosine kinase inhibitors (EGFR-TKIs), such as gefitinib and erlotinib have been used as targeted therapy in NSCLC. However, only a part of NSCLC patients with female gender, never smokers, adenocarcinoma and Asian ethnicity are sensible to EGFR-TKIs2. In 2009, the landmark clinical trial, IPASS (Iressa Pan-Asia Study) demonstrated that gefitinib showed better survival in NSCLC patients with activating EGFR mutations3. After that, many clinical trials confirmed that the selection of EGFR-TKIs should be based on EGFR mutation status not on clinical characteristics456.

EGFR mutation status is a sensible and reliable biomarker for the responsiveness to EGFR-TKIs78. The deletion in exon 19 and point mutation in exon 21 (L858R) predict good response to EGFR-TKIs8, while the point mutation in exon 20 (T790M) indicates resistance to EGFR-TIKs and poor prognosis9. Additionally, it was found that chemotherapy could affect EGFR mutation status, and patients whose EGFR mutations switched from positive to negative after chemotherapy had a better partial response10. Thus, detection of EGFR mutation status is critical for the application of EGFR-TKIs and monitoring chemotherapy response in clinical practice.

Since most NSCLC patients are diagnosed at advanced stage, surgery is no longer possible and it is hard to get sufficient tissues for molecular testing. On the other hand, for real-time monitoring of EGFR mutation status, repeat biopsy is impossible. Thus, it is needed for a feasible and sensitive biomarker for the detection of EGFR mutation. Circulating free DNA (cfDNA) has been proposed as an alternative approach for the detection of EGFR mutation1112. Numerous studies have investigated the diagnostic performance of cfDNA and a wide range of the concordance rates between cfDNA and tissues have been reported13141516.

With accumulating evidence, varied results raise concern about the diagnostic value of cfDNA for the detection of EGFR mutation. To address this issue, we performed this meta-analysis and systematic review to compare the diagnostic accuracy of cfDNA to tissues for the detection of EGFR mutations.

Methods

Literature search

This meta-analysis was performed and reported according to the guideline about diagnostic studies17. Potentially relevant studies were identified by searching PubMed, EMBASE, and the Cochrane library. A systematic and comprehensive search was performed for the 3 databases using combination of key words and medical subheadings: “lung neoplasms” or “lung cancer”, “Epidermal Growth Factor Receptor” or “erbB1”, “serum” or “plasma” or “circulating”, and “mutations”. Alternative spellings and abbreviations were also considered. To identify additional studies, reference lists of included studies and relevant reviews were also manually searched. The literature search was conducted without any limitations and the last search was performed on March 3, 2014.

Inclusion and exclusion criteria

Records retrieved from databases and reference lists were first screened by titles and abstracts and then full-text articles of relevant studies were retrieved for further review. Eligible studies were selected according to the following inclusion criteria: 1) all NSCLC patients involved should be diagnosed histopathologically or cytologically; 2) EGFR mutation status should be detected by circulating free DNA; 3) EGFR mutations were verified by detection of tumor tissues; 4) enough data to construct the diagnostic 2 × 2 table.

Studies met the following criteria were excluded: 1) tumor tissues and cfDNA were not paired; 2) EGFR mutation status were not verified by detection of tumor tissues; 3) insufficient data to construct the 2 × 2 table; 4) duplicate reports from the same patients (the latest or the one with most NSCLC patients were included). All records were reviewed by two authors independently and reached consensus at each eligible study.

Data extraction

The following data were extracted by 2 authors independently: name of author, year of publication, country where the study was conducted, percentage of female, percentage of ever-smokers, histological type, TNM stage, methods for EGFR mutation status detection in cfDNA, true positive (TP), false positive (FP), false negative (FN), and true negative (TN). When multiple methods were used for EGFR mutation detection in cfDNA, the methods with best sensitivity or specificity was extracted. According to the media of sample size, eligible studies were classified as large (≥median sample size) or small (

Quality assessment

Methodological quality of eligible studies was evaluated by QUADAS-2 (quality assessment of diagnostic accuracy studies 2) by two investigators18. QUADAS-2 is an improved tool, designed to evaluate the quality of primary diagnostic accuracy studies, which consists of 4 key domains (patient selection, index test, reference standard, and flow and timing). With signaling questions, risk of bias and concerns regarding applicability (except for the “flow and timing” domain) were judged as “low”, “high”, or “unknown”. Summary of QUADAS plot was generated by Review Manager software (version 5.2.9, the Cochrane Collaboration).

Statistical analysis

The pooled sensitivity, specificity, positive likelihood ratio (PLR), negative likelihood ratio (NLR), positive predicted value, negative predicted value, diagnostic odds ratio (DOR) and corresponding 95% confidence intervals (95% CI) were calculated by the accuracy data (TP, FP, FN, and TN) extracted from each eligible studies. The PLR is calculated as: sensitivity/(1-specificity) and the NLR is calculated as: (1-sensitivity)/specificity. A clinically useful test was defined with a PLR > 5.0 and a NLR < 0.2. DOR is a measure that combined sensitivity and specificity, which is calculated as: PLR/NLR19. The summary receiver operative curve (SROC) was generated and the area under the curve (AUC) was calculated.

The Spearman correlation between the logit of sensitivity and logit of 1-specificity was calculated to determine the effect of threshold, and a P value < 0.05 indicated significant threshold effect. The heterogeneity caused by non-threshold effect was measured by Q test and the inconsistency index (I2), and a P value ≤ 0.05 and a I2 value ≥50% indicated significant heterogeneity caused by non-threshold effect. In the presence of significant heterogeneity, the DerSimonian Laird method was used to calculate the estimates20 and meta-regression was performed to detect the source. Sub-group analyses were performed for sample size, countries, detection methods, and TNM stages. Publication bias was detected by the Deek's funnel plot21 and a P vale< 0.05 indicated the presence of publication bias.

All statistical analyses were performed using the STATA software (version 11.2, STATA Corp., Texas USA) with the MIDAS module and Meta-DiSc.

Results

Study selection

As shown in Figure 1, after primary screening, 34 full-text articles10131415162223242526272829303132333435363738394041424344454647484950 were selected for further evaluation of eligibility. By rigorous evaluation, 20 eligible studies were identified and included in meta-analysis1314151622232425262728293031323334353637. The main reasons for exclusion were: EGFR mutation status was not detected in tissues1038394041(5), insufficient data to construct 2 × 2 tables42434445(4), duplicate reports464748(3), cfDNA not detected49 (1), and not matched tissues and cfDNA50(1). No additionally studies were identified by searching the references of eligible studies or relevant review.

Figure 1

Flow diagram of study selection.

Characteristics of eligible studies

Baseline characteristics of eligible studies were shown in Table 1. All eligible studies were published between 2006 and 2013. 2012 NSCLC patients were included in this meta-analysis, and most of them were at advanced stage (TNM III–IV) with adenocarcinoma. There were various kinds of methods applied for detection of EGFR mutation in cfDNA, while the ARMS was the most common method. Notably, only one study was carried in USA33and the other 19 studies were all performed in Asia. QUADAS-2 summary plot was presented in Figure S1. As shown, methodological quality of eligible studies were adequate and not significantly affected by bias.

Table 1

Characteristics of eligible studies

Author	Year	Country	Ethnicity	Number	Female	Ever Smoker	AC	TNM (I/II/III/IV)	Assay
Wang S	2013	China	Asian	74	48.5%	46.3%	80.6%	0/0/9/115	ARMS
Jing CW	2013	China	Asian	120	42.5%	NA	58.3%	38/82†	HRM
Zhang H	2013	China	Asian	86	43.0%	51.2%	75.6%	0/0/16/70	MEL
Kim HR	2013	Korea	Asian	40	35.0%	28.3%	88.3%	0/0/7/53	PNA
Liu X	2013	China	Asian	86	34.9%	54.7%	98.8%	0/0/4/82	ARMS
Lv C	2013	China	Asian	6	54.5%	45.5%	NA	0/0/11/0	DHPLC
Kim ST	2013	Korea	Asian	57	38.6%	56.1%	70.2%	0/0/7/50	PNA-LNA
Xu F	2012	China	Asian	34	39.2%	NA	84.3%	0/0/6/45	ARMS
Hu C	2012	China	Asian	24	50.0%	45.8%	58.3%	11/8/4/1	HRM
Zhao X	2012	China	Asian	111	31.5%	51.4%	65.8%	22/10/33/46	ME-PCR
Yam I	2012	China	Asian	35	60.0%	14.3%	94.3%	0/0/8/27	AS-APEX
Goto K	2012	Japan	Asian	86	87.6%	9.0%	NA	NA	ARMS
Jiang B	2011	China	Asian	58	31.0%	62.1%	72.4%	NA	ME sequencing
He C	2009	China	Asian	18	36.6%	53.0%	75.4%	19/115†	ME-PCR
Bai H	2009	China	Asian	230	46.5%	44.8%	74.3%	0/0/80/150	DHPLC
Kuang Y	2009	USA	Caucasian	43	81.5%	NA	NA	NA	ARMS
Yung TK	2009	China	Asian	29	NA	NA	NA	NA	digital PCR
Kimura H	2007	Japan	Asian	42	33.3%	66.7%	73.8%	NA	ARMS
Kimura H	2006	Japan	Asian	11	37.3%	NA	85.2%	0/0/3/24	ARMS
Huang Z	2012	China	Asian	822	46.7%	41.4%	78.0%	NA	DHPLC

AC: adenocarcinoma; NA: not available; † data present as I–II/III–IV; ARMS: Scorpion Amplification Refractory Mutation System; HRM: High Resolution Melting; MEL: Mutant Enriched Liquidchip; PNA: Peptide Nucleic Acid (PN A)-mediated PCR clamping; DHPLC: Denaturing High-Performance Liquid Chromatography; PNA-LNA: Peptide Nucleic Acid-Locked Nucleic Acid PCR; ME-PCR: mutant-enriched PCR; AS-APEX: Allele-Specific Arrayed Primer Extension.

Accuracy of cfDNA for the detection of EGFR mutation

Results of this meta-analysis were shown in Table 2. Compared with NSCLC tumor tissues, the pooled sensitivity and specificity of cfDNA for the detection of EGFR mutation status were 0.674 (95%CI: 0.517–0.800) and 0.935 (95%CI: 0.888–0.963), respectively (Figure 2). The PLR and NLR of cfDNA were 10.307 (95%CI: 6.167–17.227) and 0.348 (95%CI: 0.226–0.537), respectively (Figure S2). The DOR was 29.582 (95%CI: 4.582–60.012) (Figure S3). Figure 3A showed the SROC with AUC of 0.93 (95% CI: 0.90–0.95), indicating cfDNA had high diagnostic accuracy. Fagan plot was generated for the visual presentation of diagnostic performance (Figure 3B). Sub-group analyses were performed to assess the influence of sample size, countries, detection methods, and TNM stages (Table 2). The diagnostic accuracy data were consistent across different sub-groups.

Table 2

Meta-analyses Results

	Sensitivity	Specificity	PLR	NLR	DOR	AUC
Overall	0.674 (0.517–0.800)	0.935 (0.888–0.963)	10.307 (6.167–17.227)	0.348 (0.226–0.537)	29.582 (4.582–60.012)	0.93 (0.90–0.95)
Sample Size
Small	0.776 (0.459–0.934)	0.898 (0.785–0.955)	7.621 (3.906–14.871)	0.250 (0.088–0.709)	30.509 (9.471–98.279)	0.92 (0.90–0.94)
Large	0.591 (0.461–0.709)	0.954 (0.901–0.979)	12.820 (5.494–29.915)	0.429 (0.312–0.590)	29.897 (10.474–85.340)	0.89 (0.86–0.91)
Country
China	0.721 (0.511–0.865)	0.930 (0.861–0.966)	10.349 (5.336–20.069)	0.300 (0.159–0.566)	34.526 (13.371–89.151)	0.93 (0.91–0.95)
Japan	0.590 (0.349–0.794)	0.962 (0.824–0.993)	15.591 (3.319–73.233)	0.426 (0.243–0.749)	36.556 (6.948–192.339)	0.87 (0.84–0.90)
Korea	0.378 (0.111–0.747)	0.928 (0.758–0.981)	5.240 (0.966–28.417)	0.670 (0.363–1.238)	7.816 (0.874–69.889)	0.92 (0.90–0.94)
Detection Methods
ARMS	0.525 (0.357–0.688)	0.947 (0.860–0.981)	9.948 (3.181–31.107)	0.501 (0.342–0.734)	19.855 (4.839–81.468)	0.86 (0.83–0.89)
DHPLC	0.658 (0.396–0.849)	0.854 (0.783–0.904)	4.499 (2.119–9.550)	0.401 (0.186–0.865)	11.221 (2.474–50.894)	0.86 (0.83–0.89)
HRM	0.887 (0.402–0.989)	0.736 (0.042–0.994)	3.360 (0.197–57.314)	0.153 (0.035–0.673)	21.974 (1.522–317.223)	0.91 (0.88–0.93)
TNM Stages
III–IV	0.576 (0.333–0.787)	0.934 (0.862–0.970)	8.702 (3.760–20.144)	0.454 (0.255–0.807)	19.163 (5.560–66.042)	0.92 (0.89–0.94)
I–IV	0.831 (0.380–0.975)	0.883 (0.563–0.978)	7.072 (1.904–26.260)	0.192 (0.040–0.929)	36.856 (9.199–147.668)	0.93 (0.90–0.95)

Data were present as accuracy data with 95% confidence intervals. PLR: positive likelihood ratio; NLR: negative likelihood ratio; DOR: diagnostic odds ratio; AUC: area under curve; ARMS: Scorpion Amplification Refractory Mutation System; DHPLC: Denaturing High-Performance Liquid Chromatography; HRM: High Resolution Melting.

Figure 2

Forest plots of sensitivity and specificity of cfDNA.

The pooled sensitivity was 0.691 (95% CI: 0.569–0.790) and the pooled specificity was 0.922 (95% CI: 0.878–0.951).

Figure 3

The summary operative receiver characteristic curve indicated high diagnostic accuracy (A, the area under summary receiver characteristics curve was) and Fagan plot presents the clinical utility of cfDNA (B).

Threshold effect and heterogeneity

Threshold effect is a major source of between study heterogeneity. Visual assessment of ROC plane revealed no significant threshold effect (Figure S4). Spearman correlation coefficient and P value were calculated to assess the threshold effect. The Spearman correlation coefficient was 0.114 and the P value was 0.652 (>0.05), confirming that the threshold effect was not significant. As shown in the forest plots of accuracy data (sensitivity, specificity, PLR, NLR, and DOR), significant heterogeneity was detected. Thus, meta-regression was performed to detect the source of heterogeneity, and sample size, TNM stage, detection methods, and country were analyzed for each accuracy data. However, none of the above covariates contributed heterogeneity.

Sensitivity analysis and publication bias

Publication bias was tested by the Deek's funnel plot. As shown, the funnel plot and P value 0.243 (>0.05) suggested no evidence of publication bias (Figure 4A). Sensitivity analysis was performed and the results showed the pooled results were not affected by individual studies (Figure 4B).

Figure 4

Deek's funnel plot showed no significant publication bias (A) and sensitivity analysis showed that the pooled results were robust and not affected by individual studies (B).

The P value of Deek's funnel plot was 0.243, indicating no significant publication bias.

Discussion

The need for a feasible, reliable, and minimally invasive approach for EGFR mutation detection has been a limiting factor in clinical research and practice. Although EGFR mutation could be detected by tumor tissues, its limitations are well known. As a more feasible and less invasive alternative, cfDNA has received more and more interest5152. However, a wide range of diagnostic accuracy values of cfDNA have been reported.

We performed this meta-analysis and systematic review to determine the diagnostic accuracy of cfDNA for EGFR mutation detection. The pooled sensitivity for cfDNA was 0.674 and the specificity was 0.935. The sensitivity of cfDNA is not high enough as a diagnostic method. However, as a cancer screening test, sensitivity is not vital and a high specificity is more important if it triggers invasive diagnostic procedures53. In the circumstance of real-time monitoring of EGFR mutation status in regimens of NSCLC, cfDNA might be a suitable screening test for EGFR mutation status, due to the high specificity and non-invasive nature33. It is worth noting that the high DOR and AUC indicate an overall high diagnostic accuracy of cfDNA.

Under the background that a feasible biomarker with optimal performance is needed and the uncertainty of whether cfDNA provides satisfying diagnostic accuracy, this meta-analysis adds important evidence to the literature.

This is the first meta-analysis of the diagnostic performance of cfDNA for the detection of EGFR mutation status and represents an attempt to provide guidance for future studies. However, some issues worth noting when reviewed current literature. First, most studies were retrospective studies and the tissue samples were formalin-fixed paraffin-embedded, which lead to significant DNA degradation and increase detection bias52. Second, chemotherapy could affect EGFR mutation status, thus, the timing of tissue collection and peripheral blood collection matters the concordance rate. It is believed that more studies using standardized handling and detecting procedures may contribute to more robust diagnostic performance.

A lot of method have been developed to detect EGFR mutations in cfDNA, such as direct sequencing, the scorpion-amplified refractory mutation system (ARMS), denaturing high-performance liquid chromatography (DHPLC), peptide nucleic acid mediated polymerase chain reaction clamping method, high resolution melting (HRM), digital PCR, and so on. The diagnostic performance has increased with the development of detection methods. Our sub-group analyses also confirmed that DHPLC and HRM showed higher sensitivity than ARMS. It is highly possible that the diagnostic performance of cfDNA will improve in the future.

Limitation of this meta-analysis should also be highlighted. First, most studies analyzed were small-sized, which might lead to bias. To evaluate the effects of small-sized studies, sub-group and sensitivity analyses were performed and results revealed that the pooled results were stable and not affected by bias. Second, significant heterogeneity was observed. Spearman correlation and ROC plane suggested that the heterogeneity was not caused by threshold effect. Multi-variables meta-regression was also performed, and unfortunately, none of the analyzed covariates was the source of heterogeneity. Third, only English databases were searched in this meta-analysis, while most eligible studies were conducted in Asia (China, Japan, and Korea). Thus, it was possible that there were some non-English studies which were not included in this meta-analysis. On the other hand, PubMed and EMBASE are the two most comprehensive databases of medicine, and most high power full trials tend to publish in English. Additionally, Deek's funnel plot was applied to detect the publication bias and no evidence of publication bias was found.

In conclusion, in our meta-analysis of 20 studies including >2000 participants, detection EGFR mutations in cfDNA appears to be of adequate diagnostic value in NSCLC. Due to its high specificity and non-invasive nature, cfDNA might be a promising screening test for NSCLC.