Can ultrasound be as a surrogate marker for diagnosing a papillary thyroid cancer? Comparison with BRAF mutation analysis.

PURPOSE: We investigated the merit of ultrasound (US) features and BRAF(V600E) mutation as an additional study of cytology and compared the diagnostic performances of cytology alone, cytology with US correlation, cytology with BRAFV600E mutation, and a combination of cytology, US, and BRAFV600E mutation all together.
MATERIALS AND METHODS: This study included 185 patients (mean age, 48.4 years; range 20-77 years) with 191 thyroid nodules who underwent US-guided fine-needle aspiration (FNA) with an additional BRAFV600E mutation test. Three radiologists highly experienced in thyroid imaging retrospectively reviewed US images and classified each nodule into two categories (positive for malignancy or negative for malignancy). Interobserver variability (IOV) of US assessment between the three readers was estimated using the generalized kappa statistic of Landis and Koch. We also calculated the diagnostic performances of these studies.
RESULTS: There were 131 cases of malignancy (131/191, 68.6%) and 60 cases of benign nodules (60/191, 31.4%). In terms of IOV of US assessment, the generalized kappa value was 0.242, indicating fair agreement was reached. The combination of cytology with BRAFV600E showed higher specificity (100%) and positive predictive value (PPV) (100%) compared to the combination of cytology, BRAFV600E, and US (specificity 28.3%, 66.7%, 68.3%; PPV 74.6%, 86.6%, 86.8%, respectively; p<0.001). However, cytology with BRAFV600E showed lower sensitivity (84.7%) than cytology with BRAFV600E and US (96.2%, 98.5%, 95.4%, respectively; p<0.001).
CONCLUSION: Considering the diagnostic performance and low reproducibility of US, the combination of FNA with BRAFV600E is the most reliable and objective method for diagnosing thyroid malignancy.

INTRODUCTION

The advance of high-resolution ultrasound (US) has led to the discovery of a greater number of thyroid nodules. US-guided fine-needle aspiration (FNA) has been widely used as the main diagnostic method for patients with thyroid nodules. However, this method entails some limitations, such as nondiagnostic, false-negative and false-positive results, the extent of which depends on the skills and accuracy of the operator and interpreting cytologist.1,2,3 To overcome the limitations of FNA, several molecular factors have been added to improve the diagnostic accuracy of US-guided FNA.4,5 The BRAFV600E mutation is one of the most commonly used molecular factors therein and has a high positive predictive value (PPV), especially in patients undergoing indeterminate FNA. The combination of cytologic diagnosis and testing for the BRAFV600E mutation has improved the overall diagnostic performance of US-guided FNA.6 However, checking for the BRAFV600E mutation can cause misinterpretation due to false positive results-although this occurs rarely-as well as an elevation of medical costs.7,8

The correlation of US features with FNA results helps overcome the limitations of FNA.9,10,11 Even when cytologic results were the same, the malignancy rate was higher when nodules had suspicious US findings.2,9 This demonstrates how US features and FNA results can complement each other. US does not involve additional medical costs, but can be a very subjective diagnostic tool and dependent on the skills and experience of the performer.12,13

In this study, we investigated the merit of US features and BRAFV600E mutation as an additional study of cytology and compared the diagnostic performances of cytology alone, cytology with US correlation, cytology with BRAFV600E mutation, and a combination of cytology, US, and the BRAFV600E mutation all together.

MATERIALS AND METHODS

This retrospective study was approved by the Severance Hospital Institutional Review Board, and the need for informed consent for inclusion in this study was waived. Written informed consent for US-FNA and BRAFV600E mutation analysis was obtained from all patients included in this study, prior to all procedures.

Patients

This study was performed at our institution (a referral center) from December 2010 through January 2011. During this period, 373 patients with 385 nodules underwent FNA with an additional BRAFV600E mutation test. Of the 385 nodules, we excluded 26 nodules in 25 patients that were less than 5 mm in the longest diameter, because these nodules inherently have a high false positive rate on US3,14 and, moreover, several guidelines recommend that nodules less than 5 mm without clinical risk factors for thyroid cancer should not undergo FNA even if suspicious malignant US features are observed.3,15,16 Among the 348 patients with 359 nodules, 79 nodules in 76 patients were excluded because they showed cytologic results of nondiagnostic (n=50), atypia (n=13), suspicious for malignancy (n=6), and malignancy (n=10) without further cytopathologic diagnosis. Among the 150 nodules with benign cytologic results, 71 nodules in 69 patients were excluded because they did not undergo follow-up FNA or follow-up US and another 18 nodules in 18 patients were excluded because they showed an increase in size in follow-up USs without further cytologic or pathologic evaluation. Finally, 191 thyroid nodules in 185 patients (mean age, 48.4 years; range 20-77 years) were included in this study (Fig. 1, Table 1). The patients included 153 women (mean age, 49.5 years; range, 20-74 years) and 32 men (mean age, 50.7 years; range, 18-79 years). The mean nodule size (±standard deviation) was 11.1±7.4 mm (range, 5-51 mm). Among the 191 nodules, 139 nodules were confirmed by operation (Surgery group) and the other 52 nodules were observed by follow-up FNA or follow-up US after a year (Observation group).

Fig. 1

Diagram of study group. US, ultrasound; F/U, follow-up.

Table 1

FNA, BRAFV600E, and Final Pathologic Diagnosis with Operation in Thyroid Nodules

PTC, papillary thyroid carcinoma; AH, adenomatoid hyperplasia; AUS/FLUS, atypia undetermined significance/follicular lesion of undetermined significance; FN, follicular neoplasm; HN, Hurthle cell neoplasm; FA, follicular adenoma; FNA, fine-needle aspiration.

Imaging analysis

US imaging was scanned by radiologists with 1 to 13 years of experience. All US images were performed with a 5-12-MHz linear array transducer (iU22; Philips Medical Systems). An investigator (J.Y.S.) selected two gray scale images (both transverse and longitudinal scans) from each US examination on the Picture Archiving and Communication System (PACS, GE Medical System, Milwaukee, WI, USA). This investigator modified the images to jpg files and organized them on PowerPoint XP (Microsoft, Redmond, WA, USA) slides in random order. She was excluded from further image review after arranging the images. A total of 191 slides were made. Three radiologists (E.K.K., J.H.P., and J.H.S.) with 13, 17, and 9 years of experience in thyroid imaging, respectively, participated in this study. They were blinded to the cytologic analysis results and final pathologic diagnosis, and retrospectively reviewed the slides individually on a liquid crystal display monitor.

All readers were asked to classify each nodule into two categories (positive for malignancy or negative for malignancy) using US features to decide whether the thyroid nodule should be aspirated or not. Readers were asked to designate a categorization for each nodule based on their overall impression of the nodule on US. In this study, readers were not asked to define US features such as internal content, nodule echogenicity, presence or absence of calcification, nodule shape, and nodule margin.

US-guided Fine-Needle Aspiration (US-FNA)

US-FNA was performed by the same radiologists who performed real-time US. Free hand US-FNA was performed with a 23-gauge needle attached to a 2-mL disposable plastic syringe. Each lesion was aspirated at least twice. Obtained samples were expelled on glass slides, smeared, and placed immediately in 95% alcohol for Papanicolaou staining. The remaining material in the syringe was rinsed in saline for cell block processing. Cytopathologists were not present during biopsies.3,11

Cytology slides were reviewed by an experienced pathologist to confirm the cytologic diagnosis. Based on the Bethesda System for Reporting Thyroid cytology, FNA cytology results were categorized as nondiagnostic, benign, atypia undetermined significance/follicular lesion of undetermined significance (AUS/FLUS), suspicious for follicular neoplasm or suspicious for a Hurthle cell neoplasm, suspicious for malignancy and malignancy.2

DNA extraction and real time PCR

The BRAFV600E mutation analysis was performed with DNA extracted from FNA cells remaining after cytologic evaluation.

Real-time PCR was performed using the Applied Biosystems 7500 real-time PCR system (Applied Biosystems, Foster City, CA, USA). The Real-Q BRAFV600E Detection Kit (BioSewoom, Korea) was used to carry out PCR reactions. The Real-Q BRAFV600E Detection Kit is a ready-to-use kit that detects the BRAFV600E (1799T>A) somatic mutation in the BRAF oncogene in a background of wild type genomic DNA with a multiplex real time PCR assay based on the TaqMan MGB probe system. The analytical sensitivity was evaluated using the plasmid clone BRAFV600E mutation and the 95% positive cut-off value (limit of detection) was calculated at 21.5 copy/uL by the Probit analysis.8

Data and statistical analysis

Final diagnoses of all malignant nodules were based on pathological results (n=131). Final diagnoses (n=60) of benign nodules were based on pathological results (n=8), with benign cytologic results being confirmed at least twice (n=8), a benign cytologic result and no increase in size (n=34), or a decrease in size being observed on follow-up US (n=10) (Fig. 1).

Interobserver variability (IOV) of US was estimated using the generalized kappa statistic of Landis and Koch.17 The degree of agreement was categorized in terms of kappa values: 0 corresponds to no agreement, 1.00 to complete agreement, less than or equal to 0.20 to slight agreement, 0.21-0.40 to fair agreement, 0.41-0.60 to moderate agreement, 0.61-0.80 to substantial agreement, and 0.81-1.00 to near perfect agreement.17

To determine diagnostic performances, we used logistic regression with the generalized estimating equation or categorical data analysis for repeated measures with the weighted least square method as appropriate. The US, BRAFV600E mutation, and cytology assessments were dichotomized according to the presence of malignant findings. The malignant cytologic diagnoses were considered positive cytology when calculating diagnostic values of FNA. When more than two diagnostic tests were combined, a nodule was considered positive for malignancy when any test was positive. We calculated the diagnostic performances of cytology alone, cytology with US, cytology with BRAFV600E, and cytology with BRAFV600E and US. Also, we compared the diagnostic performances of cytology, BRAFV600E, US, and combinations thereof.

All statistical analyses were performed using SAS software (version 9.1.3; SAS Institute, Cary, NC, USA). Statistical significance was assumed for p-values less than 0.05. All reported p-values are 2-sided.

RESULTS

Among the 185 patients, 126 patients (107 women, 19 men; mean age, 48.3; range 20-77 years) were included in the malignant group and 59 patients (46 women, 13 men; mean age, 48.4, range 28-72 years) were included in the benign group. The mean age and gender of the patients were not associated with malignancy (p=0.569 and p=0.244, respectively). There were 131 cases of malignancy (131/191, 68.6%) and 60 cases of benign nodules (60/191, 31.4%). Benign nodules were significantly larger than the malignant nodules (mean size, 13.9±8.9 mm vs. 9.8±6.2 mm, respectively; p=0.002).

Reproducibility of US assessment

The IOV of US assessment was calculated between the three readers. Reader 1 predicted 156 nodules as positive for malignant, reader 2 predicted 145 nodules as positive for malignant, and reader 3 predicted 101 nodules as positive for malignant nodule. The generalized kappa value was 0.242, indicating fair agreement was observed. Diagnostic performances varied between the three readers (Table 2).

Table 2

Comparison of Diagnostic Performance of Cytology, BRAFV600E, US Assessment, and Combinations Thereof

PPV, positive predictive value; NPV, negative predictive value; All combinations, combinations of cytology, BRAFV600E mutation and US; NA, not applicable; US, ultrasound; F/U, follow-up.

*p-value of cytology vs. cytology with BRAFV600E.

†p-value of cytology with BRAFV600E vs. cytology with US.

‡p-value of cytology with US vs. all combinations.

§p-value of cytology with BRAFV600E vs. all combinations.

Comparison of diagnostic performance between the combination of cytology with BRAFV600E and cytology alone

Of the 52 patients with negative cytologic results, 24 patients harbored the BRAFV600E mutation. One of 6 patients with nondiagnostic cytology, 1 of 6 patients with benign cytology, 4 of 6 patients with AUS/FLUS cytology and 18 of 28 patients with suspicious for malignancy cytology showed positive BRAFV600E mutation results (Table 1).

The sensitivity, accuracy and negative predictive value (NPV) of cytology were 66%, 77%, and 57.7%, respectively (Table 2). When BRAFV600E and cytology were considered in combination, the combination showed higher sensitivity (84.7%; p<0.001), accuracy (89.5%; p<0.001), and NPV (75%; p<0.001). Nevertheless, the positive predictive values (PPVs) were the same between cytology alone and cytology with BRAFV600E.

Comparison of diagnostic performance between the combination of cytology with BRAFV600E and the combination of cytology with US

The combination of cytology with BRAFV600E showed increased specificity and PPV, compared with those of cytology with US (p<0.001). In terms of accuracy, cytology with US showed decreased accuracy for all readers, but only reader 1 showed statistical significance, compared with that of cytology with BRAFV600E. However, the sensitivity of cytology with BRAFV600E was lower than that of cytology with US for all three readers (84.7% vs. 92.4%, 97.7%, and 90.1%, respectively; p≤0.005).

Comparison of diagnostic performance between the combination of cytology, BRAFV600E and US and the combination of cytology and US

In terms of accuracy, PPV and NPV, the combination of cytology, BRAFV600E, and US showed increased diagnostic performances in all three readers compared to the combination of cytology with US; however, only reader 1 and reader 3 showed statistical significance. The combination of cytology, BRAFV600E, and US showed higher sensitivity (96.2%, 98.5%, and 95.4%, respectively according to the reader) than the combination of cytology with US (92.4%, 97.7%, and 90.1%, respectively according to the reader) (p<0.005).

Comparison of diagnostic performance between the combination of cytology and BRAFV600E and the combination of cytology, BRAFV600E, and US

The combination of cytology with BRAFV600E showed higher specificity (100%) and PPV (100%) compared to the combination of cytology, BRAFV600E, and US (specificity 28.3%, 66.7%, and 68.3%; PPV 74.6%, 86.6%, and 86.8%, respectively; p<0.001). In terms of accuracy, the combination of cytology with BRAFV600E showed higher accuracy in all readers; only reader 1 showed statistical significance. However, cytology with BRAFV600E showed lower sensitivity (84.7%) than cytology with BRAFV600E and US (96.2%, 98.5%, and 95.4%, respectively; p<0.001).

DISCUSSION

US-guided FNA is currently the standard diagnostic method for patients with thyroid nodules.1,3 However, up to 40% of cytologic results on thyroid nodules are inconclusive, which results in repeated aspirations and unnecessary surgical interventions.6,18 Such a high incidence of unnecessary procedures results in additional morbidity and higher medical costs.19 To resolve this issue, the scientific community has struggled to translate molecular markers into useful clinical tools for patients with thyroid nodules, and nowadays, several molecular markers are used to improve diagnostic accuracy in cases with inconclusive cytological results.4,5 Mutations or aberrant expressions of genes coding for signaling cascade proteins (RET, RAS, BRAF, PI3K, PTEN, AKT) have been identified in the majority of papillary thyroid carcinoma (PTC) patients.18,19,20 These alterations change the MAPK/ERK pathway and PI3K/Akt pathway, which play important roles in the transmission of cell signals and contribute to the transformation of malignant follicular cells. Several studies have demonstrated that analyzing the combination of BRAF, RAS, RET/PTC, and PAX8/PPARr mutations improves the diagnostic accuracy of thyroid cancer, particularly in samples with indeterminate cytology.19 The BRAFV600E mutation is the most common mutation observed, and the frequency of the BRAFV600E mutation in PTC ranges from 29 to 83%. In Korea, which is a BRAFV600E mutation-prevalent area, the BRAFV600E mutation is present in more than 90% of PTCs.21 The RAS mutation is the second most common finding, but in an FNA sample, it also can be detected in follicular carcinoma and other benign nodules. RET/PTC is found in 20% of adult sporadic PTCs. It usually occurs in patients with a history of radiation exposure (50-80%) and in PTC from children to young adults (40-70%). In many clinical studies, its analysis is difficult and may only be useful in combination with other markers. These findings have made the BRAFV600E mutation a more generally accepted reliable prognostic marker for PTC and as a result, additional molecular studies to improve the cytopathologic diagnosis of PTC have been focused on the BRAFV600E mutation.4,6,22

Various studies have supported the effectiveness of US for diagnosing malignancy in thyroid nodules.10,14,23 US assessment can be usefully adjusted in thyroid nodules because it can reduce the false negative rate of FNA without additional medical cost. However, US is a rather subjective and operator dependent diagnostic method, and there have been some studies reporting various IOVs in US assessments of thyroid nodules.12,13,24 Therefore, it is hard to discuss the diagnostic value of US as a additional method to FNA without a discussion about IOV. In a previous study, moderate to substantial agreement was obtained in regards to IOV among a highly experienced group assessing thyroid nodules, and the high interobserver agreement was thought to be obtained because the readers worked at the same institution, so they had been taught a uniform approach to translating sonographic findings of thyroid nodules.12 In another study, the IOV among faculty members was higher than that of residents; the residents showed poor agreement for interpretation of US findings of thyroid nodules.24 However, in our study, only fair agreement was observed among highly experienced radiologists for interpretation of US findings of thyroid nodules.

A recent study reported that the diagnostic performance of the combination of cytology, BRAFV600E and US was found to be superior to that of BRAFV600E with cytology interms of sensitivity, although, in terms of specificity and accuracy, results showed decreased diagnostic performance.23 However, the study did not include an analysis about the IOVs of US assessment. In this study, we investigated whether the combination of cytology, BRAFV600E, and US can be the best diagnostic method for detecting thyroid malignancy, and if not, which combination is the most effective and objective ancillary method to FNA. To analyze IOV, we evaluated US assessment conducted by three highly experienced radiologists who work at different institutions. We demonstrated a fair agreement of US assessment between the three highly experienced readers. This probably means that even experienced and specialized radiologists have their own diagnostic criteria based on their experience, thus this can cause low interobserver agreement between US assessments of thyroid nodules. Our results showed that the combination of cytology with BRAFV600E showed higher specificity, PPV, and accuracy, compared to the combination of cytology with BRAFV600E and US. The combination of cytology with BRAFV600E and US showed increased sensitivity and NPV, compared with cytology with BRAFV600E, a conclusion similar to one from a previous study.23 To be a good additional test, the tool should be highly reproducible as well as highly accurate. Considering the low reproducibility of US and diagnostic performances, a combination of cytology with BRAFV600E can be a better diagnostic approach than a combination of cytology with BRAFV600E and US.

There were some limitations to our study. First, there was a selection bias because we excluded thyroid nodules without further cytopathologic diagnosis or follow-up US to evaluate diagnostic performance. Second, we retrospectively reviewed still images specifically selected by an investigator, not real-time images. Therefore, these results may be different if the readers performed the US. Third, not all thyroid nodules underwent surgery or cytopathologic diagnosis. Some final references were based on cytology and follow-up US, so this can cause some false negative and false positive cytologic results.3,25 Fourth, diagnostic performance varies according to the methodology of BRAFV600E testing; therefore, the result may not be reproducible if another method is used. Finally, the study population was made up of Korean patients, who are known to have a high prevalence of BRAF mutation; therefore, the conclusion of this study may not be relevant in other countries, especially in areas with a low prevalence of BRAF mutation.

In conclusion, although the combination of cytology, BRAFV600E, and US can increase sensitivity, in terms of accuracy and specificity, they can decrease the diagnostic performance. When also considering the low reproducibility of US, we concluded that the combination of FNA with BRAFV600E is the most reliable and objective method for diagnosing papillary thyroid cancer.