High-resolution melting analysis for accurate detection of BRAF mutations: a systematic review and meta-analysis.

The high-resolution melting curve analysis (HRMA) might be a good alternative method for rapid detection of BRAF mutations. However, the accuracy of HRMA in detection of BRAF mutations has not been systematically evaluated. We performed a systematic review and meta-analysis involving 1324 samples from 14 separate studies. The overall sensitivity of HRMA was 0.99 (95% confidence interval (CI) = 0.75-0.82), and the overall specificity was very high at 0.99 (95% CI = 0.94-0.98). The values for the pooled positive likelihood ratio, negative likelihood ratio, and diagnostic odds ratio were 68.01 (95% CI = 25.33-182.64), 0.06 (95% CI = 0.03-0.11), and 1263.76 (95% CI = 393.91-4064.39), respectively. The summary receiver operating characteristic curve for the same data shows an area of 1.00 and a Q* value of 0.97. The high sensitivity and specificity, simplicity, low cost, less labor or time and rapid turnaround make HRMA a good alternative method for rapid detection of BRAF mutations in the clinical practice.

BRAF, a signal transduction protein which is downstream of KRAS in the mitogen-activated protein kinase (MAPK) pathway, is one of three members of the RAF kinase family: A-RAF, B-RAF, and C-RAF12. It acts as direct effector of RAS and through the activation of MEK, promotes tumour growth and survival3. Since mutations in BRAF gene were originally identified in melanomas1, now they have been found in different kinds of cancers, predominantly in melanoma, metastatic colorectal cancer and papillary thyroid cancer456. In recent years, BRAF mutation has been found to correlate with an aggressive clinical outcome in patients with PTC and drug resistance of patients receiving anti-EGF-receptor (EGFR) therapies for colorectal cancer7. These findings not only make BRAF an attractive therapeutic target but also demonstrate the significant diagnostic and prognostic implications of BRAF mutation status in a wide variety of clinical settings7. Therefore, the detection of BRAF mutation is increasingly important and frequent in clinical practice for both diagnostics and screening of the patients who treatment with BRAF inhibitor.

Up to date, a number of methods for detection of BRAF mutations in different kinds of cancer have been described, including DNA sequencing, TaqMan array, single-strand conformation analysis, allele-specific PCR, pyrosequencing89101112. Although DNA sequencing, including direct DNA sequencing and pyrosequencing, is considered as the “gold standard” for known/unknown mutation scanning, but it is limited by high cost and low sensitivity for the clinical screening of BRAF mutation13. Other methods mentioned above are time consuming and require manipulation of amplified PCR products, which is a common source of sample contamination13. The most important is that BRAF mutation testing requires a much higher analytical sensitivity when dealing with cytological specimens where the cells in question may be a minor population among the vast majority of background non-neoplastic cells7. However, the methods above could not overcome this shortcoming. Furthermore, some of the methods need the expensive equipment or fluorescence-labeled probes may not be economical for all the patients with BRAF mutation.

In recent years, high-resolution melting curve analysis (HRMA) has been proved to be applicable for detection of various clinically relevant human mutations13. It is a cost-efficient, closed-tube system and very sensitive scanning method that allows rapid detection of DNA sequence variations without cumbersome post–polymerase chain reaction (PCR) methods, which is not achievable by other methods including direct sequencing14.

Therefore, HRMA might be a good alternative method for rapid detection of BRAF mutations in the clinical practice. However, the accuracy of HRMA in the detection of BRAF mutations has not been systematically evaluated. Thus, we performed this meta-analysis to evaluate the overall accuracy of HRMA for rapid detection of BRAF mutations.

Results

Selection characteristics

A total of 143 abstracts and titles were obtained by searching multiple databases and sources. 53 records were excluded because of duplicates. And then the remaining 90 records were screened by the titles and abstracts. After 63 studies were excluded, 27 full-text papers were deemed to be relevant and were examined in detail. 13 of full-text articles were excluded with the reasons described in Figure 1. Finally, 14 studies met the inclusion criteria, and were included in our meta-analysis.

Figure 1

A flow chart highlighting study selection.

Study characteristics and quality of reporting

The main characteristics of the studies included were summarized in Table 1. A total 14 studies1314151617181920212223242526 with 1324 samples were included in our study and the average sample number was 94.5 (range 13 to 201). The earliest study was in March 200515, while the latest was in April 20132526. Thirteen studies were from western world country, while only one study is from China,which is in Asia20. Type of disease included colorectal cancers (CRCs), melanoma, thyroid nodules, endometrial cancers, gastrointestinal stromal tumors and lung cancers, and the CRCs accounts for more than 57.1%. Specimen were mainly formalin-fixed paraffin-embedded (FFPE) (n = 8), the remaining were paraffin-embedded (PE) (n = 3) and fresh frozen tissue (FF) (n = 3). Instrument types included HR-1 (n = 2), LightCycler480 (n = 8), RotorGene6000 (n = 3) and Rotor-Gene Q (n = 1). The dye included LCGreen(n = 1), Syto 9(n = 5), LightCycler LC480 High Resolution Melting Master(n = 6),EvaGreen(n = 1),and one study didn't mention about it17. The final volumes (μL) of HRM were 10 μL (n = 8), 20 μL (n = 4), 25 μL (n = 1), while one study didn't mention about it19. The quality of individual studies was relatively high, as all studies met 8 or more of the QUADAS criteria. All the studies were Cross-sectiona and used the DNA sequencing as the reference method.

Table 1

Characteristics of the 14 studies included in the meta-analysis

Author/Year	Country	Disease	Study design	Reference method	QUADAS score	Total nnumber	Specimen Source	DNA Extraction kit	Instrument	Dye	Final volume(μL)
Willmore-Payne 200517	USA	melanoma	CS	SEQ	12	90	PE	NA	HR-1	LCGreen	10
Simi 200814	Italy	CRCs	CS	SEQ	9	116	FF	Qiagen	RG 6000	Syto 9	10
Pichler 200913	Austria	CRCs	CS	SEQ	10	13	FFPE	Qiagen	LC 480	HRMM	20
Seth 200918	UK	CRCs	CS	SEQ	9	29	FFPE	Qiagen	HR-1	Syto 9	10
Bennani 201019	Morocco	CRCs	CS	SEQ	10	62	PE	Qiagen	LC 480	NA	10
Borras 201120	Spain	CRCs/LCs	CS	SEQ	9	54	FFPE	Qiagen	LC 480	HRMM	10
Bozzao 201221	Italy	CRCs	CS	SEQ	8	117	PE	Qiagen	RG 6000	Syto 9	NA
Hsieh 201222	China	CRCs	CS	SEQ	11	182	FFPE	Qiagen	LC 480	HRMM	10
Krol 201223	Netherlands	CRCs	CS	SEQ	11	126	FF	Qiagen	LC 480	HRMM	25
Mancini 201224	Italy	TNs	CS	SEQ	11	17	FF	Qiagen	RG 6000	Syto 9	20
Ney 201225	Germany	Mixed	CS	SEQ	11	194	FFPE	Qiagen	LC 480	HRMM	20
Yang 201226	Germany	melanoma	CS	SEQ	13	31	FFPE	Qiagen	RGQ	EvaGreen	20
Guedes 201327	Portugal	CRCs	CS	SEQ	8	201	FFPE	Qiagen	LC 480	HRMM	10
Richter 201328	Australia	melanoma	CS	SEQ	13	92	FFPE	NA	LC 480	Syto 9	10

CRCs, colorectal cancers; LCs: lung cancers; TNs, thyroid nodules; Mixed, colorectal cancers, endometrial cancers, melanomas, gastrointestinal stromal tumors; CS, Cross-sectiona; SEQ, DNA sequencing; PE, paraffin-embedded; FF, fresh frozen tissue; FFPE, formalin-fixed paraffin-embedded; NA, not available; LC 480, LightCycler 480; RG 6000, RotorGene 6000; RGQ, Rotor-Gene Q; HRMM, LightCycler LC480 High Resolution Melting Master.

Diagnostic accuracy

The Spearman correlation coefficient is −0.130 (p = 0.657), which indicated absence of threshold effect. Therefore, we pooled the sensitivity, specificity, PLR, NLR, and DOR by the data extracted from the included studies. Fig. 2 showed the results of the pooling sensitivity and specificity. The sensitivity ranged from 0.93 to 1.00 and the pooled sensitivity was 0.99 (95% confidence interval (CI) = 0.75–0.82) while the specificity from 0.88 to 1.00 and the pooled specificity was 0.99 (95% CI = 0.94–0.98). Fig. 3 showed the results of the pooling PLR and NLR. PLR was 68.01 (95% CI = 25.33–182.64), NLR was 0.06 (95% CI = 0.03–0.11). Fig. 4 showed that DOR was1263.76 (95% CI = 393.91–4064.39). The chi-square and I2 tests for heterogeneity in the summary results suggested significant heterogeneity for specificity (P = 0.00, I2 = 68.8%) and PLR (P = 0.00, I2 = 63.5%). The SROC curve, which is intended to represent the relationship between sensitivity and specificity across all included studies, is shown in Fig. 4. The SROC curve from our data showed that the Q value was 0.97, while the area under the curve (AUC) was close to 1.00 (0.995).

Figure 2

Forest plots estimates of sensitivity (a) and specificity (b) for high-resolution melting curve analysis with 95% confidence intervals (CI).

Figure 3

Forest plots estimates of positive likelihood ratio (PLR) (a) and negative likelihood ratio (NLR) (b) for high-resolution melting curve analysis with 95% confidence intervals (CI).

Figure 4

Forest plots estimates of diagnostic odds ratio (DOR) (a) and summary receiver operating characteristic (SROC) curves (b) for high-resolution melting curve analysis.

Multiple-regression analysis and publication bias

To investigate the source of the heterogeneity for specificity and PLR, Multivariate meta-regression analysis with the covariates including quality, specimen, instrument, dye, final volume of HRM and disease was performed. The quality for every study was scored using QUADAS criteria, where a 1 or a 0 was given if all criteria were fulfilled or not achieved, respectively27. The result of the Multivariate meta-regression analysis showed that there was no statistical significance in RDOR values between studies (Table 2). The subgroup was not performed because the source of the heterogeneity was not found. To our knowledge, biases are inherent within any meta-analysis during the process of locating, selecting, and combining studies. In our study, the Deek's Funnel Plot Asymmetry Test showed that the biases were not very significant (P = 0.23). (Fig. 5)

Table 2

Result of the multivariable meta-regression model for the most important characteristics with backward regression analysis (Inverse Variance Weights)

Variable	Coeff.	Std. Err.	p-value	RDOR	[95%CI]
Cte.	6.146	2.7445	0.0664	----	----
S	−0.454	0.4282	0.3302	----	----
QUADAS	−0.272	1.165	0.823	0.76	(0.04;13.18)
Specimen	0.249	0.7097	0.7375	1.28	(0.23;7.29)
Instrument	−0.759	0.7302	0.3384	0.47	(0.08;2.79)
Dye	0.174	0.5577	0.765	1.19	(0.30;4.66)
Volume	0.757	0.6199	0.2679	2.13	(0.47;9.72)
Disease	0.076	1.0092	0.9422	1.08	(0.09;12.75)

Figure 5

Deek's Funnel Plot Asymmetry Test for the assessment of potential publication bias.

Discussion

Mutations in BRAF have been researched for many years in different kinds of the cancers282930. Now, BRAF mutation has been found to correlate with an aggressive clinical outcome in patients with PTC and drug resistance of patients receiving anti-EGF-receptor (EGFR) therapies for colorectal cancer7. There is obviously an urgently need for fast, reliable, and inexpensive methods for screening of BRAF mutation in the clinical practices. HRMA might be a good alternative method for rapid detection of BRAF mutations. However, the accuracy of HRMA in the detection of BRAF mutations has not been systematically evaluated. We performed a meta-analysis including 14 studies with 1324 samples to evaluate the overall accuracy of HRMA for rapid detection of BRAF mutations.

Our meta-analysis showed that the pooled values of the sensitivity and specificity of HRMA were 0.99 (95% CI = 0.75–0.82) and 0.99 (95% CI = 0.94–0.98), respectively. And the SROC showed that Q value was 0.97 and the AUC was 1.00. The results obtained here indicate a very high level of overall accuracy of HRMA. It suggests that the HRMA was a more appropriate method for screening of the BRAF mutation compared with the DNA sequencing. Although DNA sequencing is accepted as the gold standard, but mostly it is limited by higher cost and lower sensitivity in the clinical screening of BRAF mutations. The DOR is a single indicator of test accuracy that combines the data from sensitivity and specificity into a single number and the ratio of the odds of a positive test result for a patient with disease or without disease27. The value of a DOR ranges from 0 to infinity, with higher values indicating better discriminatory test performance27. In our present meta-analysis, we found that the pooled DOR was 1263.76 (95% CI = 393.91–4064.39). It further indicates that the HRMA is ideal methods with a high level of overall accuracy. HRMA also had a very high PLR and a very low NLR for the detection of BRAF mutation in our study, which indicates an excellent ability to both confirm and exclude the presence of BRAF mutation.

Since HRMA appeared as a rapid method for genotyping known variants or scanning for unknown variants, it has been as a screening technique for wider application31. HRMA is a simple, PCR-based method for detecting DNA sequence variation by measuring changes in the melting of a DNA duplex32. Duplex melting is generally monitored using intercalating dyes, which bind to double-stranded but not single-stranded DNA. Therefore, the PCR for the sample, carried out in the presence of a saturating DNA binding dye, is needed before HRMA. After this, the product is heated while the level of fluorescence is measured. As the temperature rises and the duplex passes through its melting transition, dye is released and fluorescence intensity is reduced32. The Tm of duplex is always different among the homozygous sequence, the homozygous sequence with variant and the heterozygous sequence, and the fluorescence is distinct from each other during the product is subjected to increasing temperatures, resulting different shapes of the melting curve. The mutations are identified by a change in the melt curve shape compared with the reference profile. Therefore, the sensitivity of HRMA is very high. A previous study has demonstrated that HRM analysis were able to detect as little as 5%–6% tumor cell line DNA in a background of wild-type DNA and another study has a lower detection limit at 0.5%3334. All categories of substitution are detectable, as are insertions and deletions small enough to be amplified by PCR32.

Therefore, HRMA has been an excellent new tool for BRAF mutation detection. Now, HRMA has been also developed for the detection of DNA sequence variants from different Samples from many sources, such as dried blood spots, buccal cells and frozen tumour specimens, Archived samples in the form of methanol-fixed tissue, formalinfixed paraffin-embedded tissue and cytology slides33. the advantage of simplicity, low cost, ease of use, and a high sensitivity/specificity have made HRMA an attractive tool for genotyping and application in diagnostic labs35.

Though the accuracy of HRMA is very high, some factors had been reported that might affect the HRMA accuracy, such as sample source and preparation, amplicon length, GC content, dye, equipments and so on333637. Takano et al. showed that a higher sensitivity could be achieved when fresh tumor samples were used for HRMA, because the DNA might have degenerated during sampling or the preservation of the archived samples3638. Wittwer et al. found that although HRMA of fragments could be up to 600 bp and more, but the technology is more sensitive for smaller fragments39. When the fragments was below 400 bp, sensitivity and specificity were 100%, while sensitivity decreased to 96.1% and 99.4% for specificity when between 400 and 1000 bp36. And they also found it critical for high sensitivity that the melt profile contains not more than one to two melt domains39. Taylor et al described in their study that Lower GC content might have an association with false negative results32. Besides, dye and equipments are also very important in the accuracy of HRMA. Different dyes detect heteroduplexes to varying degrees and the most common real-time dye, SYBR Green I, does not detect them at all36. And instruments vary in their ability to resolve shape and absolute temperature differences, explained by their signal-to-noise ratios, acquisition rates, data density and software36. Furthermore, Tindall et al proposed that the shape of an HRMA curve in itself is usually not sufficient to type a specific variant, to achieve this either a melt probe should be added or the fragment should be sequenced40. However, we performed this meta-analysis including 14 studies with 1324 samples proved the high accuracy of HRMA for detection of BRAF mutation compared with the DNA sequencing.

An exploration of the reasons for heterogeneity rather than the computation of a single summary measure is an important goal of meta-analysis27. Although the accuracy of the HRMA showed a high level in our study, we found that there was heterogeneity in the pooled result of specificity and PLR. To investigate the source of the heterogeneity, we performed the multivariate meta-regression analysis with the covariates including quality, specimen, instrument, dye, final volume of HRM and disease. The results of the meta-regression analysis indicated there was no significant characteristic influencing diagnostic accuracy. Previous studies reported that the instrument type, sample source and dye may affect the accuracy3739. The inconsistent result between our study and the previous studies may be explained by the sample size. Though we performed the multivariate meta-regression analysis, we still could not find the source of the heterogeneity. Therefore, the subgroup would not be performed according to the results of multivariate meta-regression analysis. Besides, the dependence of scanning accuracy on PCR product length has been studied on the HR-1 and LightScanners instruments, revealing more errors as the length increases above 400 bps39. Therefore, we may infer that another reason responsible for the heterogeneity is the length of the DNA product, but we could not get sufficient information in our included studies.

Our meta-analysis had several strengths. Firstly, we used a comprehensive search strategy with well defined inclusion criteria in such a meta-analysis to date. Secondly, two reviewers performed the study selection and data extraction dependently and discrepancies were resolved by consensus including a third author. Thirdly, we assessed the quality of included studies by the standard criteria and the score of included in our study was relatively higher. Lastly, we used a careful approach to selecting a fixed or random effects model for pooling studies by taking into account the presence or absence of significant heterogeneity. However, our systematic review also had several limitations. Firstly, we could not find the source of the heterogeneity and the subgroup was not performed according to the result of multivariate meta-regression analysis. Secondly, though our meta-analysis included fourteen available studies, but the number is still relatively small, which may affect the pooling accuracy of the HRMA. Lastly, we didn't collect enough information about other factors which may disrupts the accuracy of the HRMA, such as length of DNA products or the GC content.

Despite these limitations, this study proved the higher sensitivity and specificity of HRMA. The advantage high sensitivity and specificity, simplicity, low cost, less labor and time and rapid turnaround make HRMA a good alternative method for rapid detection of BRAF mutations in the clinical practice.

Methods

Search strategy

To identify relevant publications, PubMed (http://www.ncbi.nlm.nih.gov/pubmed), ISI Science Citation Index (http://apps.isiknowledge.com), and EMBASE (http://www.embase.com/home) databases were searched using the following search terms: “BRAF”, “B-RAF”, “HRMA”, “HRM”, “HRMCA”, “HRMC”, “high resolution melting analysis”, “high resolution melting”, “high resolution melting curve analysis”, “high resolution melting curve”. No limitations were placed on the language of publications and the type of studies. We also manually searched the reference lists of the articles identified in the searches for additional eligible studies.

Selection criteria

Studies eligible for inclusion in this meta-analysis met the following criteria: (1) HRMA was applied to the study of BRAF mutations in humans; (2) DNA Sequencing was used as a reference standard; (3) provided enough information to describe true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN); (4) it must be published as a full text in English. While the exclusion critseria were as follows: (1) studies were performed using only HRMA or comparing HRMA with non-sequencing techniques; (2) studies used samples with artificially created sequences or the cell lines; (3) The TP is 0; (4) meeting abstracts, case reports, letters, and reviews were excluded due to limited data; (5) Publications identified as duplicates were excluded.

Data extraction and quality assessment

Information was carefully extracted from all eligible studies. The following data were collected from each study: first author's name, year of publication, study design, country where the study was conducted, type of disease, sources of sample, kits of DNA extraction, instruments, dye and final volume of HRMA, reference method, number of sample, and number of samples with the indicated test result (TP, FP, FN, TN). Data extraction was done independently by two of the authors and discrepancies were resolved by consensus including a third author. We assessed the quality of the individual studies using criteria based on the Quality Assessment of Studies of Diagnostic Accuracy41.

Data synthesis and meta-analysis

We used the standard methods recommended for meta-analysis of diagnostic test evaluations for the measures of test accuracy for each study by Meta-Disc (version 1.4) and Stata (version 11.0) software42. The numbers of TP, FP, FN, and TN were taken directly from the included studies, and 0.5 was added to all cells to handle studies with empty cells.

First of all, we conducted analysis of diagnostic threshold to find weather there was a threshold effect in included studies. Threshold effect arises from different cut-offs used in different studies to define a positive test result, which was which was assessed by Spearman correlation coefficient.

Threshold effect was defined as a positive correlation (P < 0.05). If threshold effect was present, the accuracy data should be pooled by summary receiver operating characteristic (SROC) curve and calculating the area under the curve (AUC). If not, we performed the next pooling.

Study heterogeneity was assessed using the chi-squared test of heterogeneity (Q Cochran's Q statistic), and the Higgin's I2 measure. Taking into account the low statistical power of these tests of heterogeneity, significant heterogeneity was defined as a Q test P value of <0.10, or an I2 measure >30%. The sensitivity, specificity, positive likelihood ratio (PLR), negative-likelihood ratio (NLR), and diagnostic odds ratio (DOR) from the different studies were combined using fixed effects or random effects models with 95% confidence intervals (CIs). The choice of the fixed or random effects model was made on the absence or presence of significant heterogeneity based on the depended on the Q test, respectively.

Source of the heterogeneity were investigated by multivariate meta-regression analysis with the covariates including QUADAS scores, specimen, instrument, dye, final volume of HRM and disease if there was statistically significant heterogeneity across the included studies. The subgroup analysis was followed multivariate meta-regression analysis if factors of heterogeneity could be found. Because publication bias is of concern for meta-analysis of diagnostic studies, we tested the potential presence of this bias using the Deek's Funnel Plot Asymmetry Test.

Author Contributions

Conceived and designed the experiments: D.C., W.F., Q.H. Performed the experiments: Y.Y.W., Y.X.W. Analyzed the data: K.L., J.H. Contributed reagents/material/analysis tools: D.S., Q.L. Wrote the manuscript: D.C., L.Z. Reference collection and data management: Z.C., Z.Y.