Reliability of BRAF mutation detection using plasma sample: A systematic review and meta-analysis

BACKGROUND: Testing of B-Raf proto-oncogene (BRAF) mutation in tumor is necessary before targeted therapies are given. When tumor samples are not available, plasma samples are commonly used for the testing of BRAF mutation. The aim of this study was to investigate the diagnostic accuracy of BRAF mutation testing using plasma sample of cancer patients.
METHODS: Databases of Pubmed, Embase, and Cochrane Library were searched for eligible studies investigating BRAF mutation in paired tissue and plasma samples of cancer patients. A total of 798 publications were identified after database searching. After removing 229 duplicated publications, 569 studies were screened using the following exclusion criteria: (1) BRAF mutation not measured in plasma or in tumor sample; (2) lacking BRAF-wildtype or BRAF-mutated samples; (3) tissue and plasma samples not paired; (4) lacking tumor or plasma samples; (5) not plasma sample; (6) not cancer; (7) un-interpretable data. Accuracy data and relevant information were extracted from each eligible study by 2 independent researchers and analyzed using statistical software.
RESULTS: After pooling the accuracy data from 3943 patients of the 53 eligible studies, the pooled sensitivity, specificity, and diagnostic odds ratio of BRAF mutation testing using plasma sample were 69%, 98%, and 55.78, respectively. Area under curve of summary receiver operating characteristic curve was 0.9435. Subgroup analysis indicated that BRAF mutation testing using plasma had overall higher accuracy (diagnostic odds ratio of 89.17) in colorectal cancer, compared to melanoma and thyroid carcinoma. In addition, next-generation sequencing had an overall higher accuracy in detecting BRAF mutation using plasma sample (diagnostic odds ratio of 63.90), compared to digital polymerase chain reaction (PCR) and conventional PCR, while digital PCR showed the highest sensitivity (74%) among the 3 techniques.
CONCLUSION: BRAF testing using plasma sample showed an overall high accuracy compared to paired tumor tissue sample, which could be used for cancer genotyping when tissue sample is not available. Large prospective studies are needed to further investigate the accuracy of BRAF mutation testing in plasma sample.

Introduction

During the development of cancer, tumor cells accumulate hundreds of mutations, a subset of which was found to play key roles in cancer development and progression.[ As one of those so-called “driver mutations,” B-Raf proto-oncogene (BRAF) mutation was observed in many types of cancer, which is most prevalent in thyroid carcinoma, melanoma, colorectal cancer (CRC), and non-small cell lung cancer (NSCLC).[ On the basis of those findings, targeted therapies on BRAF-mutant cancer have been developed. Two specific inhibitors for BRAF, Vemurafenib, and Dabrafenib, have been approved for treatment of advanced-stage melanoma patients with BRAF V600E mutation in 2011 and 2013, respectively.[ In combination with mitogen-activated protein kinase inhibitor, dual inhibition on BRAF and mitogen-activated protein kinase kinase showed significant improvement of patient prognosis and was approved by Food and Drug Administration for treatment of BRAF-mutant advanced melanoma and BRAF-mutant advanced NSCLC.[

Before those targeted therapies are given, it is required to determine the BRAF mutation status of tumor.[ When available, tumor tissue is a more reliable sample type for the testing of BRAF mutation status due to its high abundance of tumor DNA.[ However, tissue sample is sometimes not available (e.g., in metastatic or recurrent cancer patients), and liquid biopsy sample (e.g., plasma, urine, etc) could serve as an alternative.[ Liquid biopsy sample contains circulating tumor DNA (ctDNA) which derives from tumor cells and carries tumor-specific mutations,[ making it possible to determine the gene mutation status in tumor using liquid biopsy samples.

Due to the low abundance of ctDNA,[ measurement of tumor-specific mutations using liquid biopsy samples requires highly-sensitive techniques (e.g., digital polymerase chain reaction [PCR]), and their reliability is still under debate. Many studies have investigated the accuracy of BRAF mutation testing using liquid biopsy samples.[ In this systemic review and meta-analysis, we aimed to investigate the diagnostic accuracy of BRAF mutation testing using ctDNA in plasma samples, with BRAF mutation status in paired tissue sample as reference.

Methods

Literature searching and selection of publication

Literature search was performed independently by PY and PC in April 2020. Databases including Pubmed, Embase, and Cochrane Library were searched using keywords “BRAF,” “cell-free DNA,” “circulating tumor DNA,” “plasma,” and “cancer,” and alternative spelling or abbreviations were also searched. After obtaining the searching results, duplicates were firstly removed and irrelevant studies were excluded after carefully reviewing the title and abstract of publications using the following criteria. Inclusion criteria: all original studies describing accuracy of BRAF mutation testing using plasma samples from patients with cancer, with tissue sample as reference. Exclusion criteria:

not a human study;

not describing BRAF mutation;

no plasma or tissue samples included;

not from patients with cancer;

reviews, abstracts, letter to the editor, comments, case reports, or studies with un-interpretable data.

Full text of the rest publications were then downloaded and examined carefully by 2 investigators. Publications were further excluded due to:

BRAF mutation was not measured in plasma or in tumor sample;

lacking BRAF-wildtype or BRAF-mutated samples;

tissue and plasma samples were not paired;

lacking tumor or plasma samples;

not plasma sample;

not cancer;

un-interpretable data (data were mixed with other genes, or difficult to extract accuracy data from the results).

For the rest eligible studies, accuracy data were extracted from BRAF mutation results from paired plasma and tissue samples, which included true positive, false positive, false negative, true negative, and sample size. Other relevant information was also extracted, including cancer type, technique used to detect BRAF mutation in plasma and tissue samples, region of the study. When several techniques were used to detect BRAF mutation in plasma sample from the same cohort of patients, only 1 of those techniques was used for data extraction and the selection criteria was:

technique used for a larger number of samples;

technique with similar detection region with the one used for paired tissue sample.

When a series of plasma samples were collected at multiple time points, results of plasma sample collected at the time point which was closest to the collection time point of tissue samples (usually at baseline) were used. Quality assessment of diagnostic accuracy studies 2 was also used to evaluate every eligible studies.[ When there was disagreement between the 2 investigators (PY and PC), it was solved by a third investigator (JZ). Ethical approval was not necessary for this study because all the data obtained and analyzed were extracted from previously-published literature and not on individual patients.

Statistical analysis

The accuracy parameters of the eligible studies were pooled or calculated using Meta-DiSc 1.4, including sensitivity, specificity, positive likelihood ratio (PLR), negative likelihood ratio (NLR), diagnostic odds ratio (DOR), and area under curve (AUC) of summary receiver operating characteristic (SROC) curve. When significant heterogeneity was observed (I ≥ 50% and P ≤ .05) during the pooling, random effects model (DerSimonian-Laird model) was used; otherwise, fixed effects model (Mantel-Haenszel model) was used. When significant inter-study heterogeneity was observed after evaluating Cochran-Q and I, threshold analysis and meta-regression were used to investigate potential source of heterogeneity using Meta-DiSc 1.4. Deek funnel plot asymmetry test was used to evaluate potential publication bias using STATA 12.0 (STATA Corp.). Results were considered statistical significant if P < .05.

Results

Search results

As shown in Figure 1, a total of 798 publications were identified after searching Pubmed (n = 395), Embase (n = 354), and Cochrane Library (n = 49). After removing duplicates, titles and abstracts of 569 publications were screened and another 445 irrelevant publications were excluded. Full text of the rest 124 studies were downloaded and evaluated, and another 71 studies were further excluded due to lacking of BRAF-wildtype or -mutated samples, or due to un-interpretable data. Data from the rest 53 eligible studies were extracted (see Table S1, Supplemental Digital Content which summarizes the extracted data from eligible studies), and meta-analysis was performed.

Figure 1

Preferred Reporting Items for Systematic Reviews and Meta-Analyses 2009 flow diagram.

Review of eligible publications

In the 53 eligible studies, 21 studies used next-generation sequencing (NGS) to test BRAF mutation in plasma sample, while this number was 13 for digital PCR, 16 for conventional PCR, and 3 for MassARRAY (Table 1). For the testing of BRAF mutation in paired tissue sample, more than half (30 out of 53) of the eligible studies used the same technique as plasma sample (15/21 for NGS, 3/13 for digital PCR, 10/16 for conventional PCR, and 2/3 for MassARRAY). In the rest 23 studies, 16 studies used standard of care instead (4/21 for NGS, 6/13 for digital PCR, 5/16 for conventional PCR, and 1/3 for MassARRAY), 4 studies used Sanger sequencing (1/21 for NGS, and 3/13 for digital PCR), 2 studies used conventional PCR (1/21 for NGS, and 1/13 for digital PCR), and 1 study used pyrosequencing (1/16 for conventional PCR). Overall, for the testing of BRAF mutation in tissue sample, 15 studies used NGS, 3 used digital PCR, 12 used conventional PCR, 2 used MassARRAY, 16 used standard of care, 4 used Sanger sequencing, and 1 used pyrosequencing.

Table 1

Summary of studies comparing BRAF mutation status in plasma and tumor tissue samples from cancer patients.

Author, year	Sample size	Type of cancer	Detection method (plasma)	Detection method (tissue)	Region
Gupta et al, 2020[15]	75	Colorectal cancer	NGS	NGS	America
Tzanikou et al, 2020[16]	34	Melanoma	Digital PCR	Sanger sequencing	Europe
Nguyen et al, 2020[26]	50	Colorectal cancer	NGS	NGS	Asia
García-Romero et al, 2019[46]	13	Central nervous system tumors	Digital PCR	Sanger sequencing	Europe
Maurel et al, 2019[51]	178	Colorectal cancer	PCR	PCR	Europe
Wong et al, 2017[32]	52	Melanoma	NGS	NGS	Australia
Iyer et al, 2018[19]	44	Thyroid carcinoma	NGS	NGS	America
Diefenbach et al, 2019[27]	10	Melanoma	NGS	NGS	Australia
Li et al, 2019[47]	59	Thyroid carcinoma	Digital PCR	Digital PCR	Asia
Choi et al, 2019[20]	61	Colorectal cancer	NGS	NGS	America
Sakai et al, 2015[33]	15	Colorectal cancer	NGS	NGS	Asia
Lin et al, 2014[17]	191	Colorectal cancer	MassARRAY	MassARRAY	Asia
Spindler et al, 2013[55]	94	Colorectal cancer	PCR	PCR	Europe
Leighl et al, 2019[21]	92	Lung cancer	NGS	Standard of care	America
Mas et al, 2019[28]	405	Colorectal cancer	NGS	Standard of care	Europe
Haselmann et al, 2018[49]	187	Melanoma	Digital PCR	Sanger sequencing	Europe
Liebs et al, 2019[39]	53	Colorectal cancer	Digital PCR	Digital PCR	Europe
Tang et al, 2018[48]	57	Melanoma	Digital PCR	Standard of care	Asia
Mohrmann et al, 2018 [40]	41	Mixed type	Digital PCR	Standard of care	America
Gangadhar et al, 2018[34]	25	Melanoma	NGS	NGS	America
Long-Mira et al, 2018[52]	19	Melanoma	PCR	Pyrosequencing	Europe
Sclafani et al, 2018[41]	97	Colorectal cancer	Digital PCR	PCR	Europe
Thierry et al, 2017[63]	97	Colorectal cancer	PCR	Standard of care	Europe
Mithraprabhu et al, 2017[30]	48	Multiple myeloma	NGS	NGS	Australia
Sandulache et al, 2017[22]	23	Thyroid carcinoma	NGS	NGS	America
Wang et al, 2017[35]	103	Lung cancer	NGS	PCR	Asia
Yang et al, 2017[65]	107	Lung cancer	PCR	PCR	Asia
Kidess-Sigal et al, 2016[36]	3	Colorectal cancer	NGS	Sanger sequencing	America
Jovelet et al, 2016[29]	283	Mixed type	NGS	NGS	Europe
Janku et al, 2016[53]	160	Mixed type	PCR	Standard of care	America
Andersen et al, 2016[42]	11	Cholangiocarcinoma	Digital PCR	Standard of care	Europe
Beranek et al, 2016[31]	32	Colorectal cancer	NGS	staNdard of care	Europe
Janku et al, 2015[50]	137	Mixed type	digital PCR	Standard of care	America
Gonzalez-Cao et al, 2015[62]	92	Mixed type	PCR	PCR	Europe
Kim et al, 2015[23]	27	Mixed type	NGS	Standard of care	Asia
Thierry et al, 2014[64]	95	Colorectal cancer	PCR	Standard of care	Europe
Oxnard et al, 2014[43]	13	Melanoma	digital PCR	Standard of care	America
Perkins et al, 2012[67]	85	Mixed type	MassARRAY	MassARRAY	Europe
Solit et al, 2008 [58]	13	Melanoma	PCR	PCR	America
Yancovitz et al, 2007[59]	17	Melanoma	PCR	PCR	America
Arnold et al, 2020[66]	28	Mixed type	PCR	Standard of care	America
Khatami et al, 2020[56]	57	Thyroid carcinoma	PCR	PCR	Asia
Liu et al, 2019[57]	175	Colorectal cancer	PCR	PCR	Asia
Kato et al, 2019[24]	76	Colorectal cancer	NGS	NGS	America
Janku et al, 2019[25]	22	Histiocytosis	NGS	NGS	America
Gray et al, 2019[68]	51	Melanoma	MassARRAY	Standard of care	Australia
Burjanivova et al, 2019[44]	87	Melanoma	Digital PCR	digital PCR	Europe
Jin et al, 2018[37]	14	Colorectal cancer	NGS	NGS	Asia
Kidess et al, 2015 [38]	38	Colorectal cancer	NGS	NGS	America
Hyman et al, 2015[45]	13	Histiocytosis	Digital PCR	Standard of care	America
Aung et al, 2014[54]	108	melanoma	PCR	Standard of care	Europe
Cradic et al, 2009 [60]	56	Thyroid carcinoma	PCR	PCR	America
Lilleberg et al, 2004[61]	20	Colorectal cancer	PCR	PCR	America

Detailed accuracy results of those studies are summarized below.

NGS

In the 21 studies using NGS for plasma sample, 8 studies by Gupta,[ Iyer,[ Choi,[ Leighl,[ Sandulache,[ Kim,[ Kato,[ and Janku[ used commercial Guardant NGS panel (Guardant Health) and the sensitivity ranged from 50.0%[ to 100%,[ and specificity were all high (from 89.5%[ to 100%[). The concordance rate ranged from 72.7%[ to 100%.[ In the study by Leighl et al,[BRAF V600E mutation was tested in 92 paired plasma and tissue samples of patients with metastatic NSCLC, and results showed complete agreement between plasma and tissue. Similarly, study by Kim et al[ also showed 100% agreement in BRAF V600E mutation statuses between 22 paired plasma and tissue samples of patients with CRC or melanoma.

Another 6 studies also used commercial NGS panel for BRAF mutation testing in plasma sample. Nguyen et al[ used commercial xGen predesigned gene capture pools (Integrated DNA Technologies) and obtained complete agreement of BRAF mutation results between plasma and tumor tissue sample from 50 CRC patients. Diefenbach et al[ used whole exome sequencing panel (SureSelect, Agilent) in 10 melanoma patients and the calculated sensitivity and specificity were 66.7% and 100%, respectively, with concordance rate at 80%. Mas et al[ used AmpliSeq Colon and Lung Cancer Panel V2 (Life Technology) and tested BRAF mutation in plasma samples from 405 CRC patients, and the sensitivity, specificity, and overall concordance rate were 76.7%, 98.9%, and 97.3%, respectively. Jovelet et al[ also used commercial panel from Life Technology (Cancer Hotspot Panel V2) in plasma samples from 283 patients with various types of cancer, and results showed sensitivity of only 25%, but high specificity (100%) and overall concordance rate (98.9%). Mithraprabhu et al[ used OnTarget Mutation Detection platform (Boreal Genomics, Canada) for plasma samples from 48 patients with multiple myeloma, and the sensitivity was 50%, and specificity and concordance rate were 97.6% and 91.7%, respectively. Beranek et al[ used Somatic 1 Master Kit (Multiplicom, Belgium) for BRAF mutation testing in plasma samples from 32 CRC patients, and results showed a complete agreement between plasma and paired tissue sample results.

The rest 7 studies used customized targeted NGS panels instead. Wong et al[ sequenced 15 genes using Access ArrayTM system (Fluidigm) in plasma samples from 52 melanoma patients and results showed sensitivity of 75.7%, specificity of 100%, and concordance rate of 82.7%. Sakai et al[ used a customized NGS panel targeting Kirsten rat sarcoma viral oncogene homolog, neuroblastoma ras oncogene, and BRAF in plasma samples of 15 CRC patients, and achieved 100% agreement between plasma and tissue results. Gangadhar et al[ used a customized 61-gene panel to test BRAF mutation in plasma samples from 25 melanoma patients, and the sensitivity was 20% only, with high specificity of 93.3% and concordance rate of 64%. Wang et al[ used a highly sensitive NGS-based technique, cSMART, and obtained complete agreement between plasma and tissue samples of 103 patients with advance stage lung adenocarcinoma. The rest 3 studies by Kidess-Sigal et al,[ Jin et al,[ and Kidess et al[ all used a multiplexed synchronous coefficient of drag alteration mutation enrichment and detection platform, and all achieved 100% agreement between plasma and tissue samples from CRC patients.

Digital PCR

Eight of the 13 studies using digital PCR used droplet digital PCR (Bio-Rad) for BRAF mutation testing in plasma samples of cancer patients.[ Results showed a highly variable sensitivity from 20% to 100%. The specificity of the 8 studies was all high, ranging from 89.3% to 100%, with concordance rate from 72.7% to 100%.

In the rest 5 studies, 3 studies by García-Romero et al,[ Li et al,[ and Tang et al[ used QuantStudioTM 3D digital PCR system (ThermoFisher Scientific), and the calculated sensitivity was 25.0%, 61.5%, and 76.0%, respectively. The specificity was 77.8%, 90.9%, and 28.6%, with concordance rate at 61.5%, 78.0%, and 70.2%, respectively. The rest 2 studies used BEAMing instead. Haselmann et al[ tested BRAF mutation in plasma samples of 187 melanoma patients using BEAMing, and the sensitivity and specificity were 86.2% and 93.4%, with concordance rate at 90.9%. Study by Janku et al[ also used BEAMing in 137 cancer patients and results showed calculated sensitivity, specificity, and concordance rate of 76.3%, 96.0%, and 90.5%, respectively.

Conventional PCR

The conventional PCR discussed in this section included real-time PCR, amplification refractory mutation system, mutation/allele-specific PCR, and quantitative PCR. In those 16 studies using conventional PCR for BRAF mutation testing in plasma sample, 3 of them[ used real-time PCR performed on IdyllaTM platform (Biocartis, Belgium), and the calculated sensitivity ranged from 64.3%[ to 98.0%[, with specificity ranging from 88.1%[ to 99.4%,[ and concordance rate from 84.2%[ to 96.6%.[

Four studies used amplification refractory mutation system for BRAF mutation testing in plasma.[ The sensitivity was from 94.1% to 100%, specificity was from 64.8% to 100%, and concordance rate was from 64.8% to 100%. Spindler et al[ tested BRAF mutation in plasma samples from 94 CRC patients, and obtained 100% agreement between plasma and tissue results.

Five studies used mutation/allele-specific PCR to detect BRAF mutation in plasma samples.[ Solit et al[ detected BRAF mutation in plasma samples from 13 melanoma patients and results showed sensitivity, specificity, and concordance rate of 66.7%, 76.9%, and 76.9%, respectively. Yancovitz et al[ tested BRAF mutation in 17 melanoma patients and the calculated sensitivity, specificity, and concordance rate were 60%, 58.8%, and 58.8%, respectively. Gonzalez-Cao et al[ measured BRAF mutation in plasma of 92 patients and got a 100% sensitivity, 73.9% specificity, and 73.9% concordance rate. Plasma samples from 56 thyroid carcinoma patients were tested for BRAF mutation using allele-specific real-time PCR, and results showed 92.9% sensitivity, 37.5% specificity, and 37.5% overall concordance rate.[ Lilleberg et al[ used allele-specific PCR combined with denaturing high-performance liquid chromatography, and achieved complete agreement in BRAF mutation results between plasma and tissue samples of 20 CRC patients.

In the rest 4 studies, Thierry et al used an optimized quantitative PCR method to detect BRAF mutation in plasma samples from 97 CRC patients, and obtained sensitivity, specificity, and concordance rate of 88.9%, 86.6%, and 86.6%.[ Another study by Thierry et al used the same method in 95 CRC patients and achieved 100% agreement between plasma and tissue results.[ Yang et al used CastPCR and the calculated sensitivity and specificity were 93.0% and 88.8%, with overall concordance rate of 88.8%.[ Arnold et al used a real-time PCR-based Target Selector ctDNA platform and results showed calculated sensitivity of 100%, specificity of 92.9%, and concordance of 92.9%.[

MassARRAY

Only 3 studies used MassARRAY to test BRAF mutation in plasma sample of cancer patients.[ Specificity of the 3 studies were all 100%, with sensitivity ranging from 37.5%,[ 75%,[ to 92.5%,[ and concordance rate from 94.1%[ to 97.6%.[

In summary, the 53 studies comprised 3943 cancer patients with paired plasma and tumor tissue samples. High concordance rate (≥ 80%) was observed in majority (42/53) of the studies, while 46 studies (86.8%) showed high specificity (≥ 80%). High sensitivity was observed in more than half of the studies (31/53).

Quality assessment of eligible studies

Quality of each eligible study was assessed using quality assessment of diagnostic accuracy studies 2, as shown in Table 2. In the assessment of risk of bias, the percentage of high risk ranged from 0% (n = 0, patient selection, reference standard) to 6% (n = 3, flow and timing), while percentage of low risk ranged from 19% (n = 10, flow and timing) to 36% (n = 19, patient selection). Flow and timing showed the highest risk of bias (6% high risk and 19% low risk) among the 4 aspects in risk of bias assessment. In applicability concerns, index test showed the highest risk (2% high risk and 55% low risk), while reference standard showed the lowest risk (100% low risk).

Table 2

QUADAS-2 assessment of eligible studies.

	Risk of bias				Applicability concerns
Author, year	Patient selection	Index test	Reference standard	Flow and timing	Patient selection	Index test	Reference standard
Gupta et al, 2020[15]	Unclear	Unclear	Unclear	Unclear	Low	Low	Low
Tzanikou et al, 2020[16]	Unclear	Unclear	Unclear	Unclear	Unclear	Low	Low
Nguyen et al, 2020[26]	Unclear	Unclear	Unclear	Unclear	Low	Unclear	Low
García-Romero et al, 2019[46]	Unclear	Unclear	Unclear	Unclear	Unclear	Low	Low
Maurel et al, 2019[51]	Low	Unclear	Unclear	Unclear	Low	Unclear	Low
Wong et al, 2017[32]	Unclear	Unclear	Unclear	Unclear	Low	Unclear	Low
Iyer et al, 2018[19]	Unclear	Unclear	Unclear	Unclear	Low	Low	Low
Diefenbach et al, 2019[27]	Unclear	Unclear	Unclear	Unclear	Low	Unclear	Low
Li et al, 2019[47]	Unclear	Unclear	Unclear	Unclear	Low	Unclear	Low
Choi et al, 2019[20]	Low	Unclear	Unclear	Unclear	Low	Unclear	Low
Sakai et al, 2015[33]	Unclear	Unclear	Unclear	Unclear	Low	Low	Low
Lin et al, 2014[17]	Unclear	Unclear	Unclear	Low	Low	Unclear	Low
Spindler et al, 2013[55]	Unclear	Unclear	Unclear	Unclear	Low	Unclear	Low
Leighl et al, 2019[21]	Unclear	Unclear	Unclear	Low	Low	Unclear	Low
Mas et al, 2019[28]	Low	Low	Low	Unclear	Low	Low	Low
Haselmann et al, 2018[49]	Low	Low	Low	Unclear	Low	Low	Low
Liebs et al, 2019[39]	Unclear	Unclear	Unclear	Low	Low	Low	Low
Tang et al, 2018[48]	Unclear	Unclear	Unclear	Unclear	Low	Unclear	Low
Mohrmann et al, 2018[40]	Low	Unclear	Unclear	Unclear	Low	Unclear	Low
Gangadhar et al, 2018[34]	Unclear	Low	Unclear	High	Low	Low	Low
Long-Mira et al, 2018[52]	Low	Unclear	Low	Unclear	Low	Low	Low
Sclafani et al, 2018[41]	Low	Low	Low	Unclear	Low	Low	Low
Thierry et al, 2017[63]	Low	Low	Low	Unclear	Low	Low	Low
Mithraprabhu et al, 2017[30]	Low	Unclear	Unclear	Unclear	Low	Unclear	Low
Sandulache et al, 2017[22]	Low	Unclear	Unclear	Low	Low	Low	Low
Wang et al, 2017[35]	Unclear	Unclear	Unclear	Unclear	Low	Unclear	Low
Yang et al, 2017[65]	Unclear	Unclear	Unclear	High	Low	Low	Low
Kidess-Sigal et al, 2016[36]	Unclear	Unclear	Unclear	Unclear	Low	Unclear	Low
Jovelet et al, 2016[29]	Low	Low	Low	Unclear	Low	Low	Low
Janku et al, 2016[53]	Unclear	Unclear	Unclear	Unclear	Low	Low	Low
Andersen et al, 2016[42]	Unclear	High	Low	Unclear	Unclear	High	Low
Beranek et al, 2016[31]	Unclear	Unclear	Unclear	Unclear	Low	Unclear	Low
Janku et al, 2015[50]	Unclear	Low	Low	High	Low	Low	Low
Gonzalez-Cao et al, 2015[62]	Unclear	Unclear	Unclear	Unclear	Low	Unclear	Low
Kim et al, 2015[23]	Low	Low	Low	Low	Low	Low	Low
Thierry et al, 2014[64]	Unclear	Low	Low	Unclear	Low	Low	Low
Oxnard et al, 2014[43]	Unclear	High	Unclear	Low	Low	Unclear	Low
Perkins et al, 2012[67]	Unclear	Low	Low	Low	Low	Low	Low
Solit et al, 2008[58]	Unclear	Unclear	Unclear	Unclear	Low	Unclear	Low
Yancovitz et al, 2007[59]	Unclear	Unclear	Unclear	Unclear	Low	Unclear	Low
Arnold et al, 2020[66]	Unclear	Low	Low	Unclear	Unclear	Low	Low
Khatami et al, 2020[56]	Low	Unclear	Unclear	Low	Low	Unclear	Low
Liu et al, 2019[57]	Unclear	Unclear	Unclear	Low	Low	Low	Low
Kato et al, 2019[24]	Low	Unclear	Unclear	Unclear	Low	Low	Low
Janku et al, 2019[25]	Unclear	Unclear	Unclear	Unclear	Low	Low	Low
Gray et al, 2019[68]	Unclear	Low	Low	Unclear	Low	Low	Low
Burjanivova et al, 2019[44]	Low	Unclear	Unclear	Unclear	Low	Unclear	Low
Jin et al, 2018[37]	Low	Unclear	Unclear	Unclear	Low	Unclear	Low
Kidess et al, 2015[38]	Unclear	Unclear	Unclear	Low	Low	Low	Low
Hyman et al, 2015[45]	Low	Low	Low	Unclear	Low	Low	Low
Aung et al, 2014[54]	Low	Low	Low	Unclear	Low	Low	Low
Cradic et al, 2009[60]	Low	Low	Low	Unclear	Low	Low	Low
Lilleberg et al, 2004[61]	Unclear	Unclear	Unclear	Unclear	Low	Unclear	Low

Meta-analysis of the accuracy of BRAF mutation testing using plasma samples

The BRAF mutation results in paired tissue and plasma samples from 3943 cancer patients were pooled using Meta-DiSc v1.4 statistical software. As shown in Figure 2, results showed pooled sensitivity of 0.69 (95% confidence interval [CI]: 0.66–0.72) and pooled specificity of 0.98 (95% CI: 0.97–0.98). Pooled PLR, NLR, and DOR were 16.84 (95% CI: 10.59–26.78), 0.35 (95% CI: 0.28–0.44), and 55.78 (95% CI: 33.62–92.54), respectively. AUC of the SROC curve was 0.9435 (also see Figure S2, Supplemental Digital Content which illustrates the detailed and pooled PLR, NLR, and SROC curve).

Figure 2

Detailed and pooled sensitivity, specificity, and DOR of the eligible studies. DOR = diagnostic odds ratio.

Since the forest plots indicated significant inter-study heterogeneity (I ≥ 50% and P ≤ .05), we further looked for possible sources of heterogeneity. Analysis of diagnostic threshold showed a Spearman correlation coefficient of –0.093 (P = .51), indicating no significant threshold effect. We then performed meta-regression analysis, and results indicated that inter-study heterogeneity was not associated with cancer type (P = .84), technique used for plasma sample (P = .86), technique used for tissue sample (P = .84), or region of the study (P = .76).

Subgroup analysis was performed on different cancer types. Eight of the 53 eligible studies were performed on patient cohorts of mixed types of cancer.[ For those studies, we successfully separated the data by cancer types from 2 studies,[ and the rest 6 studies[ were excluded from subgroup analysis since we cannot separate their data by cancer type. After data separation, cancer types other than melanoma, CRC, and thyroid carcinoma were further excluded from subgroup analysis due to limited number of studies. As shown in Table 3, among the 3 cancer types, melanoma showed the highest pooled sensitivity (0.74 [95% CI: 0.69–0.79]), while CRC showed the highest specificity (0.99 [95% CI: 0.98–0.99]), PLR (32.79 [95% CI: 17.16–62.68]), and DOR (89.17 [95% CI: 50.65–156.97]), and thyroid carcinoma showed the highest AUC of SROC curve (0.9896).

Table 3

Meta-analysis results.

	No. of studies/ patient cohorts	Sensitivity	Specificity	PLR	NLR	DOR	AUC of SROC curve
Overall	53	0.69 (0.66–0.72)	0.98 (0.97–0.98)	16.84 (10.59–26.78)	0.35 (0.28–0.44)	55.78 (33.62–92.54)	0.9435
Type of cancer
Melanoma	15	0.74 (0.69–0.79)	0.91 (0.88–0.94)	6.06 (2.74–13.39)	0.32 (0.19–0.52)	23.29 (9.13–59.39)	0.8962
Colorectal cancer	21	0.71 (0.62–0.78)	0.99 (0.98–0.99)	32.79 (17.16–62.68)	0.34 (0.24–0.50)	89.17 (50.65–156.97)	0.9195
Thyroid carcinoma	5	0.58 (0.50–0.67)	0.96 (0.90–0.99)	12.21 (5.26–28.33)	0.35 (0.13–0.92)	25.85 (9.95–67.15)	0.9896
Techniques used for plasma sample
NGS	21	0.71 (0.63–0.77)	0.99 (0.98–0.99)	23.61 (14.29–39.02)	0.36 (0.25–0.51)	63.90 (33.24–122.83)	0.9336
Digital PCR	13	0.78 (0.72–0.82)	0.94 (0.92–0.96)	9.28 (3.66–23.54)	0.32 (0.18–0.57)	35.38 (12.81–97.71)	0.9128
Conventional PCR	16	0.60 (0.55–0.65)	0.97 (0.96–0.98)	14.39 (6.39–32.42)	0.38 (0.26–0.56)	45.18 (16.82–121.31)	0.8537
Techniques used for plasma sample (for studies using standard of care for tissue sample)
NGS	4	0.82 (0.66–0.92)	0.99 (0.98–1.00)	66.25 (27.32–160.69)	0.21 (0.12–0.38)	331.93 (107.84–1021.68)	0.9889
Digital PCR	6	0.80 (0.72–0.87)	0.94 (0.89–0.97)	9.61 (1.19–77.69)	0.23 (0.15–0.35)	37.22 (5.52–250.91)	0.8516
Conventional PCR	5	0.63 (0.55–0.70)	0.96 (0.93–0.98)	17.59 (5.08–60.88)	0.37 (0.22–0.61)	51.62 (12.05–221.04)	0.2550
Techniques used for plasma sample versus tissue sample
Matched	30	0.63 (0.58–0.67)	0.98 (0.97–0.99)	15.39 (9.15–25.86)	0.41 (0.31–0.54)	51.25 (26.39–101.47)	0.9193
Unmatched	23	0.75 (0.71–0.79)	0.97 (0.96–0.98)	17.10 (7.71–37.92)	0.29 (0.20–0.40)	61.07 (28.03–133.07)	0.8702

Subgroup analysis was also performed on techniques used for plasma sample. MassARRAY was excluded due to limited number of studies. In the rest 3 types of techniques (NGS, digital PCR, and conventional PCR), digital PCR showed the highest pooled sensitivity (0.78 [95% CI: 0.72–0.82]), and NGS showed the highest specificity (0.99 [95% CI: 0.98–0.99]), PLR (23.61 [95% CI: 14.29–39.02]), DOR (65.90 [95%CI: 33.24–122.83]), and AUC of SROC curve (0.9336).

Considering the different techniques used for paired tissue samples among the studies, we further analyzed the performance of the 4 techniques in plasma sample when a certain technique was used for tissue sample. When standard of care was used for tissue sample, NGS also had the best performance by showing the highest pooled sensitivity (0.82 [95% CI: 0.66–0.92]), specificity (0.99 [95% CI: 0.98–1.00]), PLR (66.25 [95% CI: 27.32–160.69]), DOR (331.93 [95% CI: 107.84–1021.68]), and AUC of SROC curve (0.9889), compared to digital PCR and conventional PCR (Table 3). MassARRAY was excluded from the analysis due to limited number of studies. When NGS was used for tissue sample, all of the studies (15/15) used NGS for plasma sample, and further analysis was not applicable. Similarly, when conventional PCR was used for tissue sample, majority of the studies (10/12) used conventional PCR for plasma sample, and further analysis was not performed due to limited number of studies using other techniques. For the rest techniques (digital PCR, MassARRAY, Sanger sequencing, and pyrosequencing), further analysis was also not performed due to limited number of studies.

Furthermore, we also divided the studies into 2 groups based on whether the study used the same technique in plasma and tissue samples (matched/unmatched). However, limited difference was observed in the performance of BRAF mutation testing in plasma sample between the matched and unmatched groups.

Deek funnel plot asymmetry test was used to evaluate publication bias since our study is investigating diagnostic accuracy. The test results showed no significant publication bias (P = .43, Fig. 3).

Figure 3

Deek funnel plot.

Discussion

Precise measurement of BRAF mutation status in tumor is essential for the success of anti-BRAF targeted therapy, for example, Vemurafenib and Dabrafenib.[ Tumor tissue samples (resection or biopsy) are commonly used for tumor genotyping, which is abundant in tumor-derived DNA.[ When tumor tissue samples are not available (e.g., in recurrent or metastatic cancer), liquid biopsy samples (e.g., plasma, urine, and etc) are mostly used as an alternative to determine the mutation status in tumor.[ However, the reliability of tumor genotyping using liquid biopsy samples needs to be validated. In this systemic review and meta-analysis, we investigated the accuracy of BRAF mutation detection using plasma sample, compared to paired tumor tissue sample.

In many previous studies, the accuracy of BRAF mutation detection in plasma samples has been validated using tissue sample as reference. In all, we involved 53 eligible studies in our systemic review and meta-analysis after database searching and screening. After pooling, BRAF mutation detection using plasma sample showed a moderate sensitivity (69%) and a high specificity (98%) as compared to tissue sample. The DOR, an important indicator of diagnostic test, was also quite high (55.78), and AUC of SROC curve was 0.9435. Those results indicated an overall high accuracy of BRAF mutation detection using plasma sample. Esagian et al compared tumor genotyping results using NGS in paired liquid biopsy and tissue biopsy samples of NSCLC patients, and reported a positive percent agreement of 53.9% for BRAF.[ Since the study by Esagian et al only involved studies using NGS as the detection method and did not report sensitivity and specificity,[ it is difficult to compare their results with findings of our meta-analysis.

During the data pooling, we observed significant inter-study heterogeneity. Therefore, we performed diagnostic threshold analysis and meta-regression. The analysis results did not shown significant threshold effect, and meta-regression also showed no significant association between inter-study heterogeneity and the covariates (cancer type, technique used for plasma sample, technique used for tissue sample, and region of the study). We further performed subgroup analysis based on cancer type and techniques used for plasma sample. For subgroup analysis on cancer type, we separated and pooled the results among melanoma, CRC, and thyroid carcinoma. Among the 3 types of cancer, CRC showed the highest specificity (99%), PLR (32.79), and DOR (89.17), indicating that an overall higher accuracy of plasma testing for BRAF mutation in CRC, although melanoma showed the highest sensitivity (74%) and thyroid carcinoma had the highest AUC of SROC curve (0.9896). Among the different techniques used for plasma sample, NGS showed the highest specificity (99%), PLR (23.61), DOR (63.90), and AUC of SROC curve (0.9336), while digital PCR had the highest sensitivity (78%). In addition, in studies using standard of care for tissue samples, NGS also showed the highest sensitivity (82%), specificity (99%), PLR (66.25), and DOR (331.93), and AUC of SROC curve (0.9889) for the detection of BRAF mutation in plasma samples, compared to digital PCR and conventional PCR. Those results indicate an overall higher accuracy of NGS in BRAF mutation testing using plasma sample. The differences in diagnostic accuracy among the subgroups might partially explain the inter-study heterogeneity observed in data pooling. Publication bias was also investigated using Deek funnel plot asymmetry test, and results indicated no significant publication bias.

In all, our study results indicated moderate sensitivity and high specificity and DOR of BRAF mutation testing using plasma sample. Overall, the testing of BRAF status using plasma sample showed high accuracy compared to paired tumor tissue sample of cancer patients, and could be used as an alternative when tissue sample is not available. Among the cancer types which most frequently carry BRAF mutation (melanoma, CRC, thyroid carcinoma), plasma sample showed the highest accuracy in CRC. Among different techniques used for plasma sample, NGS showed the highest accuracy and is more recommended for BRAF mutation testing using plasma sample. On the other hand, digital PCR showed the highest sensitivity and therefore is recommended if high sensitivity is expected. Limitation of this study may include the small number of studies in some subgroups (thyroid carcinoma) which should be treated carefully. In addition, although the performance of BRAF mutation testing between different techniques does not differ much in tissue sample due to high abundance of tumor DNA, difference in technique may cause potential bias. Large prospective studies are needed to further validate the accuracy of BRAF mutation testing using plasma sample.

Author contributions

Conceptualization: Peng Ye, Jie Zhang.

Data curation: Peng Ye, Peiling Cai.

Formal analysis: Peng Ye, Peiling Cai, Jing Xie.

Funding acquisition: Peng Ye.

Supervision: Jie Zhang.

Writing – original draft: Peng Ye.

Writing – review & editing: Peng Ye, Peiling Cai, Jing Xie, Jie Zhang.