The Diagnostic Performance of Stool DNA Testing for Colorectal Cancer: A Systematic Review and Meta-Analysis.

This meta-analysis was designed to evaluate the diagnostic performance of stool DNA testing for colorectal cancer (CRC) and compare the performance between single-gene and multiple-gene tests.MEDLINE, Cochrane, EMBASE databases were searched using keywords colorectal cancers, stool/fecal, sensitivity, specificity, DNA, and screening. Sensitivity analysis, quality assessments, and performance bias were performed for the included studies.Fifty-three studies were included in the analysis with a total sample size of 7524 patients. The studies were heterogeneous with regard to the genes being analyzed for fecal genetic biomarkers of CRC, as well as the laboratory methods being used for each assay. The sensitivity of the different assays ranged from 2% to 100% and the specificity ranged from 81% to 100%. The meta-analysis found that the pooled sensitivities for single- and multigene assays were 48.0% and 77.8%, respectively, while the pooled specificities were 97.0% and 92.7%. Receiver operator curves and diagnostic odds ratios showed no significant difference between both tests with regard to sensitivity or specificity.This meta-analysis revealed that using assays that evaluated multiple genes compared with single-gene assays did not increase the sensitivity or specificity of stool DNA testing in detecting CRC.

INTRODUCTION

Colorectal cancer (CRC) is the third leading cause of cancer-related death worldwide.[1] The 5-year survival rate of CRC patients with localized disease is about 90% following curative surgery and about 60% for patients with lymph node metastasis.[2] Hence, early diagnosis is critical and of utmost importance in reducing CRC-related mortality.[3] In fact in countries with active CRC screening programs, there has been a decrease CRC mortality.[4]

CRC develops from the transformation of normal bowel epithelium into a precancerous state and finally into a malignancy. The changes in the epithelium are results of complex molecular alterations, including mutations and epigenetic changes in the genomic DNA.[5-7] Current CRC screening options include colonofibroscopy, barium enema, flexible sigmoidoscope, and fecal occult blood testing (FOBT).[5,8] However, these methods are not ideal screening tool in the clinical setting as these screening methods are invasive, unpleasant, and nonoptimal for patients.[8] More importantly, most of these tests are of poor sensitivity or specificity.[8] Fecal DNA testing has the advantage of being noninvasive, technically easy, and convenient. This strategy stems from the fact that malignant cells continuously shed into the colonic lumen, creating a source for disease-specific DNA biomarkers in a patient's stool.

Numerous studies evaluated different potential fecal DNA biomarkers for screening CRC.[5] These biomarkers comprised a wide variety of genetic alterations, including DNA mutations and changes in the methylation status of a gene.[5] Some tests were based on detection single genetic changes while others evaluated multiple genes. The difference assays varied significantly in sensitivity and specificity, and the relative diagnostic performance among the assays is unclear. We performed a systematic review and meta-analysis to evaluate the diagnostic performance of single-gene assays compared with multiple-gene assays that utilized stool DNA for screening for CRC.

METHODS

The meta-analysis was performed in accordance with the PRISMA 2009 guidelines.[9] MEDLINE, Cochrane, EMBASE databases were searched until August 7, 2015 using following keywords CRC, stool/fecal, DNA, screening, sensitivity, and specificity. Studies reporting results in treatment-naive patients with confirmed diagnosis of primary CRC were included. The included studies had a control group of normal healthy subjects. All included studies used stool DNA testing as CRC screening tool, and employed colonofibroscopic or surgical pathology examination as the reference standard. Studies involving patients with the diagnoses of secondary or metastatic instead of primary colon cancers, precancerous lesions (such as metaplasia, dysplasia, etc.), and other chronic inflammatory diseases mimicking malignancy (such as inflammatory bowel disease) were excluded. Studies with incomplete patients’ profiles, missing essential data, questionable diagnosis or disease status, trials lacking appropriate informed consent, and articles not reporting quantitative data of primary study endpoints of interest were also omitted. L667etters, commentaries, editorial, case reports, expert opinions, and articles not published in English were excluded.

Study Selection and Data Extraction

All potential studies were reviewed thoroughly by 2 independent reviewers. A third reviewer was consulted to resolve any discrepancy between reviewers. All essential data and relevant information, including the name of the first author, year of publication, study design, subject demographics, pathology and cancer stages, targeted genes, and detection method of targeted genes, were extracted from the included studies.

Quality Assessment

The Revised Quality Assessment of Diagnostic Accuracy Studies (QUADAS 2) tool was utilized for quality assessment for the included studies.[10] The QUADAS 2 tool consists of 4 key domains that cover patient selection, index tests, reference standard, and flow of patients through the study and timing of the index tests and reference standard (flow and timing). The quality assessment was also performed by the independent reviewers and a third reviewer was consulted for any uncertainties.

Statistical Analysis

The outcomes of the meta-analysis were the diagnostic performance, denoted as sensitivity, specificity, the positive likelihood ratio, negative likelihood ratio, and diagnostic odds ratio of single-gene and multigene tests. Representation of accuracy estimates from each study in a receiver operating characteristic (ROC) space and computation of Spearman correlation coefficient between the log (SEN) and log (1 − SPE) were assessed for threshold effect. A typical pattern of “shoulder arm” plot in an ROC space and a strong positive correlation would suggest a threshold effect.[11,12]

Heterogeneity among the studies was assessed by the Cochran Q and the I2 statistic. For the Q statistic, P < 0.10 was considered statistically significant for heterogeneity. For the I2 statistic, which indicates the percentage of the observed between-study variability due to heterogeneity rather than chance, the following ranges were used: no heterogeneity (I2 = 0%–25%), moderate heterogeneity (I2 = 25%–50%), large heterogeneity (I2 = 50%–75%), and extreme heterogeneity (I2 = 75%–100%). If a Q statistics (P < 0.1) or I2 statistic (I2 > 50%) indicated heterogeneity between studies, the random-effects model was preferred (DerSimonian–Laird method). Otherwise, the fixed-effect model (Mantel–Haenszel method) was recommended. We pooled the results of single-gene test in each study. A 2-sided P value <0.05 was considered statistically significant. The homogeneity test, pooled estimates for sensitivity, specificity, the positive likelihood ratio, negative likelihood ratio diagnostic odds ratio (DOR), and summary ROC curve were performed by using Meta-Disc version 1.4.[12] Pairwise comparison of ROC curves was performed based on model of Hanley and McNeil.[13] In addition, publication bias was inspected by a Deeks funnel plot of the diagnostic odds ratio against study size. Deeks funnel plot was conducted using Stata software (version 14.0, StataCorp, College Station, TX).[14]

RESULTS

Literature Search

Of the 363 articles initially identified, 267 were excluded for being irrelevant (Figure 1). The remaining 96 studies were fully reviewed, of which 43 were excluded for not precisely meeting all the inclusion criteria. The resultant 53 studies were included in the analysis.[8,15-66]

FIGURE 1

Flow diagram of study selection.

Study Characteristics

The total number of participants in the studies was 7524 (Table 1  ), with the percentage of males ranging from 35% to 83%. The numbers of patients in the CRC group and control group ranged from 14 to 116 and from 5 to 1426, respectively. The staging system applied, stages of cancer, frequency of each cancer stage in a given study, targeted genes assessed, and the analysis methods to detect the presence genetic alterations of the targets genes were heterogeneous varied significantly across the studies. Across the studies a wide type or different genes were analyzed (Table 2  ). The sensitivity and specificity of a given assay for detecting CRC in stool samples also varied across the studies; for each assay, the range for sensitivity was 2% to 100% and for specificity was 81% to 100%.

TABLE 1

Summary of Basic Characteristics of Studies Included in Meta-Analysis

TABLE 1 (Continued)

Summary of Basic Characteristics of Studies Included in Meta-Analysis

Performance of Single-Gene and Multi-Gene Test

A Spearman rank correlation was performed as a further test for threshold effect. The Spearman correlation coefficient was 0.080 (P = 0.205) for a single-gene assay and 0.257 (P = 0.126) for multigene test, The results indicate that other factors than threshold are causing variations in accuracy estimates among individual studies.

For evaluation of the sensitivity of a stool-based biomarker test using a single-gene or multiple-genes, the homogeneity tests found Q = 1244.70 (d.f. = 47, P < 0.001) and I2 = 96.2% for single-gene test; and Q = 115.35 (d.f. = 20, P < 0.001) and I2 = 82.7% for multigene test, indicating significant heterogeneity existed between these studies. Consequently, the random-effects model was used for the pooled analysis. The pooled sensitivities of single-gene and multigene tests were 48.0% and 78%, respectively (Figures 2A and 3A).

FIGURE 2

The summary of single-gene assay (A) sensitivity, (B) specificity, (C) positive likehood ratios, (D) negative lokehood ratios, (E) diagnostic odds ratio, (F) summary ROC curves.

The summary of multiple-gene assays (A) sensitivity, (B) specificity, (C) positive likehood ratios, (D) negative lokehood ratios, (E) diagnostic odds ratio, (F) summary ROC curves.

The homogeneity tests for assessing the specificity of a single-gene or a multigene stool-based biomarker assay indicated Q = 470.28 (d.f. = 47, P < 0.001) and I2 = 90.0% for single-gene test; and Q = 46.17 (d.f. = 20, P = 0.001) and I2 = 56.7% for multigene test. These findings indicated the presence of significant heterogeneity among the studies; hence, the random-effects model was used for the pooled analysis. The pooled specificities of single-gene and multigene tests were 97.0% and 93%, respectively (Figures 2B and 3B).

In addition, the pooled positive likelihood ratios of single-gene and multigene tests were 9.17 and 7.94, respectively (Figures 2C and 3C), while the negative positive likelihood ratio of single-gene and multigene tests were 0.44 and 0.24, respectively (Figures 2D and 3D).

Summary ROC Curves and Diagnostic Odds Ratio

For all studies, the pooled DOR was 20.35 (95% CI: 17.63–23.49) for the single-gene assay and 31.64 (95% CI: 25.13–39.84) for multigene assay (Figures 2E and 3E). Moreover, the area under the summary ROC curves was 0.908 (standard error = 0.013) for the single-gene assay and 0.934 (standard error = 0.011) for multigene assays (Figures 2F and 3F). No significant difference was observed between the ROC curves for single-gene and multigene tests (P = 0.063). These findings suggest both types of assays have good diagnostic discrimination for the presence of absence of CRC.

Subgroup Analyses According to Duke and AJCC Classification as Cancer Staging System

Further subgroup analyses were performed according to Duke criteria and AJCC classification of primary CRC. DOR and summary ROC curves were represented only. According to the Duke criteria, the pooled DOR was 18.55 for the single-gene assay and 36.42 for multigene assays (Figure 4A and B); the summary ROC was 0.939 for the single-gene assay and 0.980 for multigene assays, respectively (Figure 5A and B). According to AJCC classification, the pooled DOR was 20.03 for the single-gene assay and 37.45 for multigene assays (Figure 4C and D); the summary ROC was 0.890 for the single-gene assay and 0.933 for multigene assays, respectively (Figure 5C and D).

FIGURE 4

Pooled DOR according to Duke criteria for (A) single-gene assay, (B) multiple-gene assays; pooled DOR according to AJCC classification for (C) single-gene assay, (D) multiple-gene assays.

FIGURE 5

Summary ROC according to Duke criteria for (A) single-gene assay, (B) multiple-gene assays; summary ROC according to AJCC classification for (C) single-gene assay, (D) multiple-gene assays.

Publication Bias

The results via Deeks funnel plot showed that there was publication bias for the single-gene assay in regards to DOR value (P < 0.001, Figure 6A). However, there no publication bias for multigene assays was found (P = 0.11, Figure 6B).

FIGURE 6

Publication bias for (A) single-gene assay, (B) multiple-gene assays.

Quality assessment of the different studies found the greatest potential risk of bias came from patient selection as most of the studies did not collect a consecutive or random sample (Figure 7). Furthermore, some of the included studies did not prespecify the threshold of the index test.

FIGURE 7

Results of quality assessment. A, Potential risk of bias of individual study. B, Summarized risk of bias of all included studies.

DISCUSSION

This meta-analysis evaluated the diagnostic performance of stool DNA testing for screening for CRC and compared the diagnostic performance or single-gene and multiple-genes assays. The pooled sensitivities were found to be 48.0% for single-gene and 77.8% for multiple-gene assays, while the pooled specificities for the single-gene and multiple-gene assays were 97.0% and 92.7%, respectively. There was no significant difference between single- and multigene tests regarding the pooled sensitivity and specificity by ROC curve analysis. However, multiple-gene assays were noted to have higher sensitivity than the single-gene assays, implying that the former testing may have advantages for screening CRC. Although previous studies reported that assays based on a combination of biomarkers had a high detection rates of both CRC and advanced adenomas,[67] no large-scale comparison between the single-gene and multigene stool DNA testing has previously been reported. Our study updated and expanded the prior information by including recent studies, and to the best of our knowledge, is the first study to compare the diagnostic accuracy of single-gene and multiple-gene testing for CRC using stool DNA.

A possible reason why the multiple-gene assay has higher sensitivity but similar specificity as the single-gene assay is that the multiple-gene assay has the ability to detect methylation or genetic changes across multiple CRC-related genes, so the chance of detecting CRC-related changes is higher than detecting changes of only 1 gene. The similar but high specificity indicates that both assays have few false positives suggesting the individual assays used across the studies for detecting the stool DNA have high accuracy for only evaluating the particular genes of interest.

Several prior meta-analyses have assessed the diagnostic value of stool DNA testing.[68-70] Yang et al[68] performed a meta-analysis that evaluated the diagnostic abilities of testing stool for multiple DNA markers of CRC. They included 20 studies that comprised 5876 patients. They found that multiple marker tests had a sensitivity for CRC of 0.676 (95% confidence interval [CI]: 0.642–0.708) and a specificity of 0.928 (95% CI: 0.917–0.939). Subgroup analysis indicated that the detection sensitivity and specificity for advanced adenoma was 0.329 (95% CI: 0.294–0.365) and 0.939 (95% CI: 0.927–0.949). In addition, subgroup analysis indicated methylation DNA testing had a significantly higher sensitivity (0.753 [95% CI: 0.685–0.812]) for CRC and a relatively similar specificity (0.913 [95% CI: 0.860–0.95] than evaluating genetics. The authors concluded that the methylated markers may have better diagnostic value than genetic markers. The sensitivity, we found in our multiple-gene biomarker analysis for CRC, was higher than that reported by Yang et al. The difference between the 2 meta-analyses may reflect differences in included studies, as our study included a larger number of studies than that of Yang et al.

Zhang et al[69] evaluated the accuracy of gene methylation analysis of DNA in stool samples for diagnosing CRC. They included 37 articles that comprised 4484 patients. The sensitivity and specificity for detection of CRC were 73% (95% CI: 71%–75%) and 92% (90%–93%), respectively. They also found that for adenoma the sensitivity was 51% (95% CI: 47%–54%) and the specificity was 92% (95% CI: 90%–93%). Pooled diagnostic performance for methylation of the SFRP2 gene indicated the sensitivity of an SFRP2-based methylation assay was 70% (95% CI: 75%–82%) and the specificity was 93% (95% CI: 90%–96%).

Luo et al[70] also evaluated the use of measuring the methylation state of biomarkers for detecting CRC. Their analysis included 19 studies comprising 2356 patients. They found that the sensitivity and specificity for detecting CRC were 0.62 (95% CI: 0.51–0.71) and 0.89 (95% CI: 0.86–0.92), respectively. The sensitivity and specificity for adenoma were 0.54 (95% CI: 0.39–0.68) and 0.88 (95% CI: 0.83–0.92). Luo et al found a lower sensitivity of methylation-based assays for detecting CRC than did Yang et al and Zhang et al. Luo et al concluded that the use of hypermethylated gene panels was not yet currently sufficiently accurate to be used alone for CRC screening, and future studies and evaluation of additional biomarkers were mandatory to improve sensitivity and specificity.

This analysis has several limitations that should be considered when interpreting the results. We did not perform subgroup analysis to compare the diagnostic abilities of methylation-based and mutation-based assays or the sensitivity or specificity of the single-gene or multigene assay for detecting advanced adenoma or adenoma highly potential for neoplastic change. All these issues are important and deserve additional studies. Most of the included studies were not prospective randomized controlled trials and there was a great heterogeneity across the studies, including CRC staging criteria and methods targeted genes, and analysis methods, which may confound study findings. Finally, the cost effectiveness of the clinical use of fecal DNA testing for CRC screening was not evaluated. A prior meta-analysis that included 7 studies found that fecal DNA testing was not cost-effective if compared with other CRC-screening tools; it would only become cost-effective when compared with no screening.[71]

In conclusion, no statistically significant difference was observed from the ROC curves between both tests for detecting CRC. Compared with the single-gene testing, multiple-gene stool DNA testing was shown to confer no better diagnostic performance in the screening of CRC. The high specificity of the assaying stool DNA for CRC-related genes suggested these assays may not only be of benefit to diagnosing CRC but also for evaluation recurrence of the disease.

TABLE 1 (Continued)

Summary of Basic Characteristics of Studies Included in Meta-Analysis

TABLE 2

Summary of Performance of Studies Included in Meta-Analysis

TABLE 2 (Continued)

Summary of Performance of Studies Included in Meta-Analysis

TABLE 2 (Continued)

Summary of Performance of Studies Included in Meta-Analysis