Diagnostic value of BRAF (V600E)-mutation analysis in fine-needle aspiration of thyroid nodules: a meta-analysis.

Fine-needle aspiration (FNA) is a reliable method for preoperative diagnosis of thyroid nodules; however, about 10%-40% nodules are classified as indeterminate. The BRAF (V600E) mutation is the most promising marker for thyroid FNA. This meta-analysis was conducted to investigate the diagnostic value of BRAF (V600E) analysis in thyroid FNA, especially the indeterminate cases. Systematic searches were performed in PubMed, Web of Science, Scopus, Ovid, Elsevier, and the Cochrane Library databases for relevant studies prior to June 2015, and a total of 88 studies were ultimately included in this meta-analysis. Compared with FNA cytology, the synergism of BRAF (V600E) testing increased the diagnostic sensitivity from 81.4% to 87.4% and decreased the false-negative rate from 8% to 5.2%. In the indeterminate group, the mutation rate of BRAF (V600E) was 23% and varied in different subcategories (43.2% in suspicious for malignant cells [SMC], 13.77% in atypia of undetermined significance/follicular lesion of undetermined significance [AUS/FLUS], and 4.43% in follicular neoplasm/suspicious for follicular neoplasm [FN/SFN]). The sensitivity of BRAF (V600E) analysis was higher in SMC than that in AUS/FLUS and FN/SFN cases (59.4% vs 40.1% vs 19.5% respectively), while specificity was opposite (86.1% vs 99.5% vs 99.7% respectively). The areas under the summary receiver-operating characteristic curve also confirmed the diagnostic value of BRAF (V600E) testing in SMC and AUS/FLUS rather than FN/SFN cases. Therefore, BRAF (V600E) analysis can improve the diagnostic accuracy of thyroid FNA, especially indeterminate cases classified as SMC, and select malignancy to guide the extent of surgery.

Introduction

Thyroid cancer is the most common endocrine malignancy, with favorable outcome after early detection and treatment.1,2 Fine-needle aspiration (FNA) guided by ultrasound is a routine and reliable approach for preoperative evaluation of thyroid nodules. Approximately 10%–40% of FNA specimens yield indeterminate results, and the majority of them turn out to be benign after diagnostic surgery, and thus a sizable portion of indeterminate specimens lead to unnecessary thyroidectomy.3–7 The Bethesda System for Reporting Thyroid Cytopathology divides indeterminate nodules into three subgroups: atypia of undetermined significance/follicular lesion of undetermined significance (AUS/FLUS), follicular neoplasm/suspicious for follicular neoplasm (FN/SFN), and suspicious for malignant cells (SMC).8 The indeterminate thyroid nodule is the most intractable problem in clinical management, which highlights the urgency to develop effective ancillary testing to identify cancerous nodules for timely and appropriate management.

Great progress has been achieved in the understanding of molecular mechanisms of thyroid cancer, and various mutations have been identified in the early stage of thyroid cancer, such as BRAF, RAS, PI3K, and PTEN.9 These genetic alterations are excellent candidates for disease hallmarks, since 60%–70% of thyroid cancers harbor at least one genetic mutation.9 The BRAFV600E mutation appears to be the most promising biomarker specific for papillary thyroid cancer (PTC),9 which aberrantly activates the tumor-initiating MAPK pathway and drives the carcinogenesis and progression of thyroid cancer.9,10

Whether BRAFV600E analysis could be routinely used in clinical practice is still controversial. Numerous researchers have proved that BRAFV600E-mutation testing is an effective diagnostic approach for thyroid FNA,11 while others believe that its utility is limited by low prevalence of BRAFV600E mutation in indeterminate nodules.12 Therefore, we conducted a structured meta-analysis to estimate the additional diagnostic yield of BRAFV600E-mutation analysis in thyroid FNA, and further evaluated the malignancy rate, BRAFV600E-mutation frequency, and diagnostic value of BRAFV600E testing in different categories of indeterminate nodule.

Materials and methods

Search strategy and selection criteria

Systematic searches were performed in the PubMed, Web of Science, Scopus, Ovid, Elsevier, and Cochrane Library databases for relevant articles prior to June 2015. The search terms were: ([thyroid cancer] or [thyroid neoplasm] or [thyroid tumor]), (BRAF), and ([FNA] or [fine needle aspiration]). The references of available articles were also reviewed. Study selection consisted of initial screening of titles or abstracts and second screening of full texts. Studies were included if they met the following criteria: 1) research article rather than review, system review, case report, editorial, or comments; 2) the material for BRAFV600E-mutation analysis was obtained by FNA; 3) the final diagnosis was based on a definite gold standard, such as surgical histology, unequivocal histocytopathology, or reliable clinical follow-up; 4) the data were available to construct 2×2 tables or analyze malignancy rate or BRAFV600E-mutation prevalence.

Data extraction and quality assessment

The following items were extracted: study by author name(s), country, number of centers, enrollment period, study design, mean age of patients, mean diameter of nodules, reference standard of final diagnosis, and genotyping method. Most research classified cytological results according to the Bethesda system8 or the British Thyroid Association,13,14 as shown in Table 1. In this meta-analysis, FNA cases classified as AUS/FLUS (Thy3a) and FN/SFN (Thy3f) were regarded as cytologically negative and lesions diagnosed as SMC (Thy4) were cytologically positive. Final diagnosis was based on histopathologic examination after surgery or a combination of cytological examination and clinical follow-up. Then, patient numbers for true-positive, false-positive, false-negative, and true-negative results were extracted to construct the 2×2 tables.

Table 1

Comparison between the British and Bethesda systems for classification of thyroid cytopathology

Bethesda	British
Nondiagnostic or unsatisfactory	Thy1 (nondiagnostic)
Benign	Thy2 (nonneoplastic)
AUS/FLUS (atypia of undetermined significance/follicular lesion of undetermined significance)	Thy3a (neoplasm possible, atypia/nondiagnostic)
FN/SFN (follicular neoplasm/suspicious for follicular neoplasm)	Thy3f (neoplasm possible, suggesting follicular neoplasm)
SMC (suspicious for malignancy)	Thy4 (suspicious of malignancy)
Malignant	Thy5 (malignant)

The methodological quality of studies eligible for diagnostic analysis of FNA cytology and/or BRAFV600E testing was assessed according to the Quality Assessment of Diagnostic Studies 2, which comprises four domains: patient selection, index test, reference standard, and flow and timing.15 A series of questions was used to judge the risk of bias and applicability concerns as low, high, or unclear risk.

Statistical analysis

The threshold effect was calculated by the Spearman correlation coefficient, and P<0.05 indicated the existence of a threshold effect. Nonthreshold heterogeneity was assessed by the Cochran Q test and inconsistency index (I2). I2>50% suggested significant heterogeneity, and a random-effect model (DerSimonian–Laird method) was chosen.16,17 Metaregression analysis was used to identify the possible sources of nonthreshold heterogeneity. The following covariates were considered in the metaregression analysis: country, number of centers (single or multiple), sample size (<100, 100–500, 500–1,000, or >1,000), study design (prospective or retrospective), reference standard (histology or cytology plus clinical follow-up), and genotyping method. If P<0.05, the covariate was to be regarded as the source of nonthreshold heterogeneity.

The pooled sensitivity, specificity, positive likelihood ratio (PLR), negative likelihood ratio (NLR), and diagnostic odds ratio (DOR) with 95% confidence interval (CI) were computed to estimate diagnostic accuracy. DOR combined the data of sensitivity and specificity into a single indicator ranging from 0 to infinity, reflecting the discriminatory performance of testing. The summary receiver-operating characteristic (SROC) curve was a mathematical model for the plot of sensitivity (1 – specificity). The Q index indicated the point at which sensitivity was equal to specificity. The areas under the SROC curve (AUCs) calculated the inherent capacity of the diagnostic test. If the AUC closed to 1, the diagnostic method was thought to be perfect.

The threshold effect, pooled diagnostic features, and metaregression were calculated by Meta-Disc (version 1.4; Ramony Cajal Hospital, Madrid, Spain). Pooled rates of malignancy and BRAFV600E mutation were calculated by R statistical software (version 3.2.1; R Foundation for Statistical Computing, Vienna, Austria). Quality assessment was conducted using Review Manager (version 5.2; Cochrane Collaboration). P<0.05 was considered statistically significant.

Results

Search results and quality assessment

The search process is shown in Figure 1. A total of 1,261 articles were initially identified, and 1,130 of these were excluded after reviewing titles and abstracts. The remaining 131 articles were investigated in detail. In accordance with the selection criteria mentioned in the Materials and methods section, 43 articles were excluded after reading the full texts. Finally, 88 studies published from 2004 to 2015 were included in this meta-analysis. Among these, 51 studies were included in the analysis of diagnostic accuracy, and at the same time 37 studies and 62 studies were available for analysis of malignancy rate and BRAFV600E-mutation rate, respectively.

Figure 1

Flowchart of study-selection process.

Abbreviation: FNA, fine-needle aspiration.

The characteristics of studies eligible for diagnostic analysis of FNA cytology and BRAFV600E testing are summarized in Table 2. As shown in Figure 2, about a third of studies had a high risk of bias in patient selection, because 14 of them did not enroll the samples consecutively or at random and eleven excluded a number of patients inappropriately. Twelve studies did not receive the same reference standard, since some patients were diagnosed by histopathology and others by FNA cytology plus clinical follow-up. Also, 17 studies did not include all patients, due to the unsatisfactory FNA or failure of BRAFV600E testing. As a result, nearly half of the studies harbored a high risk of bias in flow and timing. Fortunately, the risk of bias in the index test and reference standard was relatively low.

Table 2

Characteristics of studies eligible for the diagnostic analysis of FNA cytology and BRAFV600E testing

Study	Country	Centers, n	Enrollment period	Design	Mean age, years	Mean diameter, cm	Final diagnosis	Genotyping method
Cohen et al18	USA	1	Jan 2001–Jan 2003	Retroa	–	–	A	Direct sequencing + mutector assay
Xing et al19	USA	1	–	Prob	–	–	B	Direct sequencing + colorimetric method
Domingues et al20	Portugal	1	–	Retro	–	–	A	PCR-RFLP
Pizzolanti et al21	Italy	1	Sep 2005–Jun 2006	Pro	–	–	A	Real-time AS-PCR
Sapio et al22	Italy	2	–	Retro	–	–	B	Direct sequencing
Sapio et al23	Italy	2	–	Retro	–	–	B	MASA
Kim et al24	South Korea	1	Aug 2005–Jul 2006	Retro	–	–	A	Pyrosequencing
Bentz et al25	USA	1	1994–2004	Retro	40.9	–	A	LCPCR + FMCA
Jo et al26	South Korea	1	June 2006–Dec 2006	Pro	–	1	A	Pyrosequencing
Marchetti et al27	Italy	1	1996–2008	Retro	–	–	A	Direct sequencing
Nikiforov et al28	USA	2	–	Pro	–	–	B	LCPCR + FMCA
Zatelli et al29	Italy	1	Oct 2008–Dec 2009	Pro	50.7	1.1	A	Direct sequencing
Cantara et al30	Italy	1	–	Pro	51.2	–	A	DHPLC + direct sequencing
Girlando et al31	Italy	1	–	Pro	–	–	A	Direct sequencing
Kim et al32	South Korea	1	–	Pro	50.6	1.29	A	DPO-based multiplex PCR + direct sequencing
Kwak et al33	South Korea	1	Mar 2008–Jun 2008	Retro	45.6	1.17	A	DPO-based multiplex PCR
Moses et al34	USA	1	Jun 2006–Jul 2008	Pro	51	–	B	Direct sequencing
Musholt et al35	Germany	6	Jan 2008–Jul 2009	Pro	–	–	A	Direct sequencing
Adeniran et al36	USA	1	Sep 2009–Nov 2010	Pro	52.6	–	A	SSCP analysis
Kim et al37	South Korea	1	Mar 2007–Feb 2009	Pro	–	–	A	Pyrosequencing
Lee et al38	South Korea	1	July 2007–Dec 2009	Pro	50.3	1.46	A	Pyrosequencing
Moon et al39	South Korea	1	Sep 2008–May 2009	Retro	49.4	0.95	B	Direct sequencing
Pelizzo et al40	Italy	1	Oct 2008–Sep 2009	Pro	47.8	–	A	Direct sequencing + MASA
Smith et al41	USA	1	–	Retro	–	–	A	MCA
Yeo et al42	South Korea	1	Jul 2009–Jan 2010	Pro	51.27	1.3	B	Pyrosequencing
Cañadas-Garre et al43	Spain	1	Jun 2006–Dec 2009	Pro	49.8	–	A	PCR-RFLP
Kang et al44	South Korea	1	Apr 2008–Jul 2009	Pro	–	–	A	AS-PCR + direct sequencing
Kwak et al45	South Korea	1	Jun 2009–Oct 2010	Retro	48	0.92	A	DPO-PCR + real-time PCR
Lee et al46	South Korea	1	Aug 2008–Mar 2011	Pro	49.5	–	A	MEMO-PCR + direct sequencing
Mancini et al47	Italy	1	–	Pro	55.1	2.38	A	High-resolution melting analysis
Rossi et al48	Italy	1	–	Pro	52	–	B	Direct sequencing
Tomei et al49	Italy	1	–	Retro	–	–	A	Pyrosequencing
Brahma et al50	Indonesia	3	Aug 2010–Jun 2011	Pro	46	.1	A	PCR-RFLP
Di Benedetto et al51	Italy	1	–	Pro	–	–	A	Direct sequencing
Koh et al52	South Korea	1	Jan 2009–Oct 2010	Pro	48.6	1.05	B	DPO-PCR
Park et al53	South Korea	1	Jan 2011–May 2011	Retro	–	–	B	Real-time PCR + pyrosequencing
Beaudenon-Huibregtse et al54	USA	5	Jul 2010–Oct 2012	Pro	–	–	A	Multiplex PCR
Crescenzi et al55	Italy	1	–	Pro	–	–	A	Real-time sequencing
Eszlinger et al56	Germany	1	1995–2009	Retro	–	–	A	High-resolution melting PCR + pyrosequencing
Guo et al57	PRC	1	Nov 2010–Jul 2011	Pro	–	–	A	Direct sequencing
Johnson et al58	UK	1	Sep 2011–Oct 2012	Retro	–	–	A	High-resolution MCA
Liu et al59	PRC	1	Sep 2012–Dec 2013	Pro	–	–	B	Pyrosequencing
Seo et al60	South Korea	1	Dec 2010–Jan 2011	Pro	48.4	1.11	A	Real-time PCR
Seo et al61	South Korea	1	Dec 2010–Feb 2012	Pro	50.3	1.9	B	Real-time PCR
Wan et al62	PRC	1	Mar 2013–Sep 2013	Pro	49		A	–
Zeck et al63	USA	1	Apr 2011–Jan 2013	Pro	–	–	A	miRInform test
Eszlinger et al64	Italy	1	1995–2009	Retro	–	–	A	High-resolution melting analysis + pyrosequencing
Krane et al65	Germany	1	May 2011–Mar 2012	Pro	–	–	A	High-resolution melting PCR + pyrosequencing
Park et al66	South Korea	1	Jul 2011–Mar 2012	Pro	–	–	A	Real-time PCR/AS-PCR + MEMO sequencing
Shi et al67	USA	1	Jan 2011–Feb 2013	Retro	–	–	A	Real-time PCR

Notes:

Retrospective;

prospective; A, histopathologic examination after surgery; B, combination of cytological examination and clinical follow-up.

Abbreviations: PCR-RFLP, polymerase chain reaction–restriction fragment-length polymorphism; AS, allele-specific; MASA, mutant allele-specific amplification; LCPCR, LightCycler PCR; FMCA, fluorescent melting-curve analysis; SSCP, single-strand conformational polymorphism; DHPLC, denaturing high-performance liquid chromatography; DPO, dual-priming oligonucleotide; MEMO, 3′-modified oligonucleotide; PRC, People’s Republic of China; FNA, fine-needle aspiration; –, data not available.

Figure 2

Methodological quality of studies included, assessed by the Quality Assessment of Diagnostic Studies 2 criteria.

Synthesis of analysis results

Diagnostic value of FNA cytology, BRAFV600E-mutation analysis, and combined strategy in all the thyroid FNA specimens

Spearman correlation coefficients for FNA cytology, BRAFV600E testing and combined strategy were 0.032 (P=0.826), 0.254 (P=0.078), and 0.064 (P=0.661), respectively; therefore, no threshold effect existed in the analysis. However, there was substantial nonthreshold heterogeneity (I2>50%, P<0.05), so the random-effect model was chosen to pool the diagnostic features. A total of 51 studies were included in this part of the analysis,18–68 but one was excluded because it had no false-positive or true-negative case to calculate the diagnostic index (Table 3).68

Table 3

Diagnostic analysis of FNA cytological examination and BRAFV600E-mutation analysis in all the FNA specimens

Study	Year	FNA				BRAF				FNA + BRAF
		TP	FP	FN	TN	TP	FP	FN	TN	TP	FP	FN	TN
Cohen et al18	2004	25	0	34	32	23	0	36	32	30	0	29	32
Xing et al19	2004	10	0	19	12	8	0	22	14	12	0	17	12
Domingues et al20	2005	10	0	3	11	3	0	10	11	10	0	3	11
Pizzolanti et al21	2007	13	0	4	32	11	0	6	32	15	0	2	32
Sapio et al22	2007	24	23	2	95	10	0	16	118	25	23	1	95
Sapio et al23	2007	6	0	2	67	4	0	4	123	6	0	2	67
Kim et al24	2008	60	0	21	22	63	0	18	22	73	0	8	22
Bentz et al25	2009	22	0	18	5	17	0	20	5	24	0	16	5
Jo et al26	2009	30	0	9	58	30	0	10	58	38	0	2	58
Marchetti et al27	2009	88	2	4	17	59	0	32	19	88	2	4	17
Nikiforov et al28	2009	27	2	21	36	18	0	30	38	33	2	15	36
Zatelli et al29	2009	66	5	24	373	48	0	42	378	73	5	17	373
Cantara et al30	2010	46	8	16	112	33	0	45	157	50	8	12	112
Girlando et al31	2010	38	0	22	2	41	0	19	2	51	0	9	2
Kim et al32	2010	251	2	6	690	221	5	47	688	253	6	4	686
Kwak et al33	2010	108	10	1	10	87	0	22	20	109	10	0	10
Moses et al34	2010	71	13	30	337	23	0	78	95	75	13	27	336
Musholt et al35	2010	19	13	11	50	9	0	21	63	23	13	7	50
Adeniran et al36	2011	47	0	13	12	40	0	20	12	55	0	5	12
Kim et al37	2011	146	0	27	21	154	1	19	20	167	0	6	21
Lee et al38	2011	127	0	70	29	174	1	24	28	183	0	15	29
Moon et al39	2011	98	0	10	191	57	0	51	191	105	0	3	191
Pelizzo et al40	2011	133	5	6	117	98	0	59	113	138	5	3	124
Smith et al41	2011	10	0	5	5	10	0	5	5	11	0	4	5
Yeo et al42	2011	183	1	9	709	99	0	93	710	185	1	7	709
Cañadas-Garre et al43	2012	12	0	31	132	17	0	31	160	23	0	25	162
Kang et al44	2012	289	1	15	8	226	2	78	7	291	3	13	6
Kwak et al45	2012	318	0	33	86	–	–	–	–	192	85	1	169
Lee et al46	2012	382	1	47	33	342	0	87	34	398	1	31	33
Mancini et al47	2012	13	1	10	32	12	0	11	33	16	1	7	32
Marchetti et al68	2012	85	0	5	0	63	0	22	0	32	0	15	0
Rossi et al48	2012	159	3	73	1,621	114	0	172	93	193	4	42	1,672
Tomei et al49	2012	44	0	5	38	28	0	21	38	44	0	5	38
Brahma et al50	2013	23	0	26	21	17	0	32	21	25	0	24	21
Di Benedetto et al51	2013	15	1	3	239	13	0	5	240	17	1	1	239
Koh et al52	2013	277	0	27	194	176	3	141	198	287	3	30	198
Park et al53	2013	71	5	8	31	44	1	37	35	76	5	3	31
Beaudenon-Huibregtse et al54	2014	36	4	18	49	21	0	35	53	37	4	19	49
Crescenzi et al55	2014	20	0	1	9	8	0	13	9	20	0	1	9
Eszlinger et al56	2014	57	0	28	225	22	0	43	188	57	0	28	225
Guo et al57	2014	55	1	8	19	41	0	22	20	57	1	6	19
Johnson et al58	2014	31	3	19	44	16	0	28	42	29	3	17	44
Liu et al59	2014	109	8	11	171	88	0	32	179	113	8	7	171
Seo et al60	2014	115	0	17	7	98	0	34	7	121	0	11	7
Seo et al61	2014	42	4	18	36	32	0	28	36	45	4	15	36
Wan et al62	2014	18	0	23	7	25	0	16	7	30	0	11	7
Zeck et al63	2014	7	2	6	6	5	0	8	8	7	2	6	6
Eszlinger et al64	2015	69	1	68	201	57	0	80	201	80	1	57	201
Krane et al65	2015	54	2	19	77	32	0	41	79	60	2	13	77
Park et al66	2015	111	0	13	34	101	1	23	33	116	1	8	32
Shi et al67	2015	20	0	3	7	11	0	12	7	20	0	3	7

Abbreviations: FNA, fine-needle aspiration; TP, true positive; FP, false positive; FN, false negative; TN, true negative; –, data not available.

Based on the feasible FNA cytology results from 50 studies, pooled sensitivity, specificity, PLR, NLR, and DOR were 0.814 (95% CI 0.803–0.824), 0.981 (95% CI 0.978–0.985), 23.868 (95% CI 14.139–40.293), 0.216 (95% CI 0.172–0.273), and 127.73 (95% CI 75.082–217.28) (Table 4). The AUC of the SROC curve was 0.9551 (standard error [SE] 0.0127), with a Q-value of 0.8975 (SE 0.0178) (Figure 3A). Data for the BRAFV600E-mutation test were unavailable in one study,45 and 49 studies with 9,361 patients were finally analyzed. Pooled sensitivity, specificity, PLR, NLR, and DOR were 0.619 (95% CI 0.605–0.633), 0.997 (95% CI 0.995–0.998), 34.982 (95% CI 23.801–51.415), 0.433 (95% CI 0.384–0.489), and 96.570 (95% CI 63.932–145.87) (Table 4). The AUC of the SROC was 0.9207 (SE 0.0233), with a Q-value of 0.8542 (SE 0.0268) (Figure 3B). Also, the positive predictive value of BRAFV600E testing was 99.5% (2,886 of 2,900). After BRAFV600E analysis was combined with FNA cytology, sensitivity increased to 0.874 (95% CI 0.865–0.884), the DOR and AUC improved to 187.92 (95% CI 110.24–320.35) and 0.9744 (SE 0.0062), respectively, with a Q-value of 0.9271 (SE 0.0107) (Table 4, Figure 3C). The synergism between FNA cytology and BRAFV600E testing also decreased the false-negative rate from 8% in FNA cytology to 5.2%, but increased the false-positive rate from 3% to 5% at the same time.

Table 4

Results of meta-analysis for diagnostic value of FNA cytology, BRAFV600E-mutation analysis, and the combined strategy in all FNA specimens

Parameter	FNA			BRAF			FNA + BRAF
	Result	95% CI	Heterogeneity, I2	Result	95% CI	Heterogeneity, I2	Result	95% CI	Heterogeneity, I2
Pooled sensitivity	0.814	0.803–0.824	93.5%	0.619	0.605–0.633	93%	0.874	0.865–0.884	92.5%
Pooled specificity	0.981	0.978–0.985	86.4%	0.997	0.995–0.998	14.1%	0.968	0.963–0.972	92.5%
Pooled LR, +	23.868	14.139–40.293	87.7%	34.982	23.801–51.415	19.5%	22.353	13.027–38.355	93.1%
Pooled LR, −	0.216	0.172–0.273	94.2%	0.433	0.384–0.489	91.8%	0.146	0.111–0.192	93%
Pooled DOR SROC	127.73	75.082–217.28	76.1%	96.570	63.932–145.87	21.4%	187.92	110.24–320.35	76.4%
AUC	0.9551			0.9207			0.9744
Q*	0.8975			0.8542			0.9271

Note:

The Q index indicates the point at which sensitivity is equal to specificity.

Abbreviations: FNA, fine-needle aspiration; CI, confidence interval; LR, likelihood ratio; DOR, diagnostic odds ratio; SROC, summary receiver-operating characteristic; AUC, area under the curve.

Figure 3

Summary receiver-operating characteristic (SROC) curve and area under the curve (AUC).

Notes: FNA cytology (A), BRAFV600E-mutation analysis (B), and combination of BRAFV600E mutation and FNA cytology (C). *The Q index indicates the point at which sensitivity is equal to specificity.

Abbreviations: FNA, fine-needle aspiration; SE, standard error.

Diagnostic value of BRAFV600E-mutation analysis in indeterminate cases (Bethesda categories III–V)

There were 43 studies included in the diagnostic analysis of BRAFV600E testing in the indeterminate thyroid nodules (Table 5).18,19,21,22,24,26–37,40,42–44,46–48,50–54,57–62,64–67,69–72 Our data showed that 23% of indeterminate nodules harbored the BRAFV600E mutation. No threshold effect was detected, so the random-effect model was chosen to pool the diagnostic features: sensitivity 0.442 (95% CI 0.416–0.468), specificity 0.997 (95% CI 0.994–0.999), PLR 12.267 (95% CI 8.175–18.406), NLR 0.613 (95% CI 0.551–0.683), and DOR 23.939 (95% CI 15.388–37.242) (Table 6; Figure 4A and B). The AUC of the SROC was 0.8711 (SE 0.0414), with a Q-value of 0.8015 (SE 0.0410) (Figure 4C).

Table 5

Diagnostic analysis of BRAFV600E-mutation analysis for indeterminate cases

Study	Year	BRAF
		TP	FP	FN	TN
Cohen et al18	2004	5	0	27	23
Xing et al19	2004	2	0	15	9
Pizzolanti et al21	2007	2	0	2	15
Sapio et al22	2007	1	0	1	45
Kim et al24	2008	13	0	8	6
Jo et al26	2009	7	0	2	15
Marchetti et al27	2009	18	0	15	19
Nikiforov et al28	2009	7	0	14	31
Zatelli et al29	2009	1	0	17	71
Cantara et al30	2010	2	0	5	34
Girlando et al31	2010	10	0	8	2
Kim et al32	2010	50	1	24	5
Kwak et al33	2010	16	0	4	10
Moses et al34	2010	13	0	30	94
Musholt et al35	2010	1	0	5	13
Adeniran et al36	2011	10	0	12	12
Kim et al37	2011	52	1	9	12
Nikiforov et al69	2011	17	0	104	392
Patel et al70	2011	2	0	18	10
Pelizzo et al40	2011	30	0	30	104
Yeo et al42	2011	14	0	39	10
Cañadas-Garre et al43	2012	5	0	10	32
Kang et al44	2012	57	0	38	7
Lee et al46	2012	79	0	27	33
Mancini et al47	2012	6	0	11	30
Rossi et al48	2012	14	0	29	157
Brahma et al50	2013	5	0	6	12
Di Benedetto et al51	2013	4	0	2	13
Koh et al52	2013	32	1	49	9
Park et al53	2013	21	1	23	15
Beaudenon-Huibregtse et al54	2014	1	0	24	28
Guo et al57	2014	16	0	7	4
Johnson et al58	2014	5	0	22	42
Liu et al59	2014	6	0	8	49
Poller et al71	2014	6	0	6	14
Seo et al60	2014	22	0	14	4
Seo et al61	2014	10	0	21	17
Wan et al62	2014	12	0	11	5
Eszlinger et al64	2015	37	0	51	119
Krane et al65	2015	6	0	27	35
Le Mercier et al72	2015	1	0	6	27
Park et al66	2015	17	0	13	4
Shi et al67	2015	1	0	8	4

Abbreviations: TP, true positive; FP, false positive; FN, false negative; TN, true negative.

Table 6

Results of meta-analysis for diagnostic value of BRAFV600E mutation in indeterminate cases

Parameter	Indeterminate			SMC			AUS/FLUS			FN/SFN
	Result	95% CI	Heterogeneity, I2	Result	95% CI	Heterogeneity, I2	Result	95% CI	Heterogeneity, I2	Result	95% CI	Heterogeneity, I2
Pooled sensitivity	0.442	0.416–0.468	86.4%	0.594	0.556–0.631	76%	0.401	0.328–0.477	77.4%	0.195	0.128–0.278	73.1%
Pooled specificity	0.997	0.994–0.999	0	0.861	0.784–0.918	70.8%	0.995	0.982–0.999	17.9%	0.997	0.983–1.000	11.8%
Pooled LR, +	12.267	8.175–18.406	0	3.434	1.625–7.259	64.1%	7.001	3.336–14.691	0	9.573	3.611–25.379	0
Pooled LR, −	0.613	0.551–0.683	84.8%	0.542	0.462–0.637	29.8%	0.694	0.576–0.835	56.1%	0.733	0.522–1.030	85%
Pooled DOR SROC	23.939	15.388–37.242	0	7.588	3.944–14.598	0	14.469	6.100–34.320	0	14.808	4.966–44.156	2.2%
AUC	0.8711			0.7674			0.7999			–
Q*	0.8015			0.7079			0.7358			–

Note:

The Q index indicates the point at which sensitivity is equal to specificity. “–’’ indicates the AUC of the SROC was not significant in FN/SFN cases, since the lower limit of the AUC was less than 0.5.

Abbreviations: FNA, fine-needle aspiration; CI, confidence interval; LR, likelihood ratio; DOR, diagnostic odds ratio; SROC, summary receiver-operating characteristic; AUC, area under the curve.

Figure 4

Forest plots.

Notes: Sensitivity (A), specificity (B), and summary receiver-operating characteristic (SROC) curve and area under the curve (AUC) (C) of BRAFV600E-mutation analysis in cases classified as indeterminate by FNA cytology. *The Q index indicates the point at which sensitivity is equal to specificity.

Abbreviations: FNA, fine-needle aspiration; CI, confidence interval; SE, standard error.

To evaluate the diagnostic value of BRAFV600E testing in different categories of indeterminate nodules, we separated the indeterminate cases into three different and more specific categories according to the Bethesda system. Studies with sample sizes fewer than ten were excluded to avoid potential bias. The malignancy rates of FN/SFN and AUS/FLUS were 30.55% and 34.99%, while 90.35% of SMC cases turned out to be malignant (Table 7). Besides that, the BRAFV600E-mutation rate varied among these groups: it existed in 43.2% of SMC cases, but only 13.77% in AUS/FLUS and 4.43% in FN/SFN patients (Table 7). Furthermore, the sensitivity of BRAFV600E testing was higher in SMC (0.594, 95% CI 0.556–0.631) than AUS/FLUS (0.401, 95% CI 0.328–0.477) and FN/SFN (0.195, 95% CI 0.128–0.278), while specificity was higher in the AUS/FLUS (0.995, 95% CI 0.982–0.999) and FN/SFN (0.997, 95% CI 0.983–1.000) groups than the SMC group (0.861, 95% CI 0.784–0.918) (Table 6). The AUC of the SROC was 0.7674 (SE 0.0564) with a Q-value of 0.7079 (SE 0.0474) in the SMC group, and 0.7999 (SE 0.0897) with a Q-value of 0.7358 (SE 0.0783) in the AUS/FLUS group, but was not significant in FN/SFN cases, since the lower limit of the AUC was less than 0.5 (Figure 5).

Table 7

Malignancy rate and BRAFV600E-mutation prevalence in three categories of indeterminate cases

Category	Malignancy rate					BRAFV600E-mutation rate
	n	Event	Pooled	95% CI	Heterogeneity, I2	n	Event	Pooled	95% CI	Heterogeneity, I2
SMC	1,214	1,067	0.9035	0.8769–0.9301	83.62%	2,382	1,074	0.4320	0.3340–0.5299	98.22%
FN/SFN	509	158	0.3055	0.2394–0.3715	54.6%	1,758	101	0.0443	0.0292–0.0594	64.02%
AUS/FLUS	594	198	0.3499	0.2956–0.4042	83.01%	2,304	310	0.1377	0.0989–0.1765	95.93%

Abbreviations: CI, confidence interval; SMC, suspicious for malignant cells; FN/SFN, follicular neoplasm/suspicious for FN; AUS/FLUS, atypia of undetermined significance/follicular lesion of undetermined significance.

Figure 5

Summary receiver-operating characteristic (SROC) curve and area under the curve (AUC) of SMC cases (A), AUS/FLUS cases (B) and FN/SFN cases (C).

Note: *The Q index indicates the point at which sensitivity is equal to specificity.

Abbreviations: SMC, suspicious for malignant cells; AUS/FLUS, atypia of undetermined significance/follicular lesion of undetermined significance; FN/SFN, follicular neoplasm/suspicious for FN; SE, standard error.

Heterogeneity test

Heterogeneity was present in our meta-analysis, and Spearman correlation coefficients suggested no significant threshold effect. To explore sources of heterogeneity, we assessed multiple variables by metaregression, including country, number of centers, sample size, study design, reference standard, and genotyping method. The results indicated that country and sample size were possible sources of heterogeneity (data not shown). Other covariates that may have caused heterogeneity, such as enrollment period, age, sex, nodule diameter, size of needle, use of blinding method, and differences in operating protocol, were not analyzed here, due to the loss of partial data.

Discussion

Thyroid cancer is on a rapid increase these days, partially due to advancing diagnostic methods. The majority of cases have an excellent prognosis, with 30-year survival rate exceeding 90% after thyroidectomy and/or radioiodine ablation.2 Preoperative diagnosis is of indisputable value in distinguishing thyroid cancer from benign nodules. FNA biopsy is a conventional technique to identify malignant thyroid nodules preoperatively and effectively, which has also been demonstrated in our meta-analysis. However, the extensive use of this approach is influenced by its inherent limitations, such as size or location of nodule, quantity and quality of obtained material, technical skill of the cytopathologist, and the overlap of cytomorphological features between malignant and benign nodules. Therefore, a fraction of cases are classified as nondiagnostic or indeterminate, and about 15%–30% of them get malignant pathology after diagnostic surgery.8,73 Since the occurrence of malignancy is too high for just watchful waiting, numerous patients with indeterminate diagnosis accept unnecessary surgical intervention. BRAFV600E mutation is the most promising marker for thyroid nodules. A similar meta-analysis conducted by Jia et al of 16 studies suggested that BRAFV600E analysis had diagnostic value in indeterminate thyroid nodules,11 but another analysis of eight eligible studies found a low BRAFV600E-mutation rate within indeterminate cases, and thus the value of BRAFV600E-mutation testing remains controversial.12 However, the number of studies these two analyses included was limited, and did not systematically stratify the indeterminate categories. Therefore, we designed a more comprehensive meta-analysis to evaluate the diagnostic yield of BRAFV600E analysis in thyroid FNA, especially those specific categories of indeterminate cases.

Consistent with previous research, our meta-analysis showed that BRAFV600E analysis had high specificity and positive predictive value. As a rule-in test, a positive result of BRAFV600E analysis indicates a high probability of malignancy so that therapeutic surgery is recommended, but the negative result cannot exclude malignancy, and further evaluations, such as follow-up ultrasound or repeat FNA, are needed. When we combined BRAFV600E-mutation testing with FNA cytological examination, sensitivity increased by 6% and the false-negative rate decreased from 8% to 5.2%, while the false-positive rate increased from 3% to 5% at the same time. However, BRAFV600E testing had relatively low sensitivity of 44.2% in the indeterminate group. Also, the yield and usefulness of BRAFV600E analysis can be greatly varied with the prevalence of BRAFV600E mutation in different subcategories of indeterminate nodules. BRAFV600E mutation was present in 43.2% of SMC cases regarded as cytologically positive in our meta-analysis, but only 13.77% in AUS/FLUS and 4.43% in FN/SFN cases. Therefore, it was reasonable that BRAFV600E analysis did best in SMC lesions (sensitivity 59.4%, specificity 86.1%) and also had certain diagnostic value in AUS/FLUS nodules (sensitivity 40.1%, specificity 99.5%), but no significant benefit in the FN/SFN group, which needs other diagnostic approaches with high sensitivity.

BRAFV600E mutation is specific to PTC or anaplastic thyroid cancer arising from PTC, and more common in conventional and tall-cell PTC than follicular-variant PTC (FVPTC), which results in the discrepancy of BRAFV600E test in different indeterminate subgroups. The FN/SFN category is mainly constituted of FVPTC, follicular thyroid cancer (FTC), adenomatoid hyperplasia, and follicular adenoma,74 which harbors low prevalence of BRAFV600E mutation and is hard for BRAFV600E testing to determine malignancy, so FVPTC and FTC may be the main source of false-negative results. The molecular profiles of FVPTC and FTC are similar, with frequent RAS and rare BRAF mutation.75,76 RAS mutation, mutually exclusive with BRAF mutation, is the most frequent genetic mutation in indeterminate nodules, and provides important diagnostic information for BRAFV600E-negative nodules.69,77 An et al reported that single RAS-mutation analysis had a sensitivity of 93.3% and specificity of 75.0% in indeterminate nodules, and the combination of RAS and BRAF mutation provided additional diagnosis value for 60%–70% indeterminate thyroid nodules.78 Other genetic alterations, such as RET/PTC and PAX8/PPARG, also contribute to the definite diagnosis of indeterminate nodules.69,79,80 Therefore, an expanded panel can be more effective, which is also recommended by the revised American Thyroid Association management guidelines.73 As some mutations also present in benign nodules, the accompanying increase in false-positive rate should not be neglected. For instance, RAS mutation and PAX8/PPARG translocation are also found in follicular adenoma.79,81 Additionally, some thyroid cancer does not have definitive molecular mutation, and other efficient rule-out testing with high negative predictive value should be explored.

The clinical management decision is directly based on the malignant risk, ranging from repeat FNA to diagnostic lobectomy to total thyroidectomy. Uncertain diagnosis may lead to delayed treatment or unnecessary intervention. Based on the Bethesda classification, malignancy rates for FN/SFN and SMC nodules are 15%–30% and 60%–75%, respectively, and are much more variable in AUS/FLUS cases (7%–48%).8 In our analysis, the malignancy rate of the SMC group was higher than that recorded in the Bethesda classification, and this discrepancy might have resulted from continuous improvement in FNA technique, since the data for the Bethesda system were collected several years ago. BRAFV600E mutation is a strong indicator for malignancy, and total thyroidectomy should be proposed as the first-line treatment for BRAFV600E-positive nodules to decrease the recurrence and avoid complications caused by standard two-stage surgery. Nevertheless, BRAFV600E testing is relatively insufficient for AUS/FLUS and even has no effect in FN/SFN patients, due to the low prevalence of BRAFV600E mutation, but their malignant occurrence (30.55% and 34.99%) was too high to perform clinical observation. Other approaches, such as core-needle biopsy and immunohistochemistry, are also required to confidently guide the management. Several multicenter studies have reported that BRAFV600E mutation is associated with aggressive clinicopathological characteristics and predicts recurrence and mortality for PTC patients.82–89 Therefore, more aggressive surgery, such as prophylactic central lymph-node dissection and closer follow-up, should be considered in the management of BRAFV600E-positive thyroid cancer.

Despite its achievements, our meta-analysis had several limitations. Firstly, there was significant nonthreshold heterogeneity, partly caused by country and sample size of different studies, but other possible covariates were unable to be analyzed due to the paucity of data. The heterogeneity from country may be due to the different BRAFV600E prevalence in worldwide populations, eg, it is up to 80% in South Korea, which is much higher than other regions.24 Secondly, about a third of the studies had a high risk of bias in patient selection, and nearly half had a high risk of bias in flow and timing, which may affect the reliability of our results.

Conclusion

This meta-analysis demonstrated that BRAFV600E analysis using residual material obtained from routine FNA could improve diagnostic accuracy and reduce false-negative rates. Besides, BRAFV600E analysis had certain diagnostic value in SMC and AUS/FLUS cases, especially the SMC group, selecting cases with high malignancy possibility and guiding intraoperative or postoperative management, though its value in FN/SFN cases was doubtful, and expanded panels containing other diagnostic markers are recommended. Therefore, more studies of high quality are needed to balance the advantages and disadvantages of BRAFV600E testing for patients and to select the most suitable population for this diagnostic method.