Impact of the Multi-Gene ThyroSeq Next-Generation Sequencing Assay on Cancer Diagnosis in Thyroid Nodules with Atypia of Undetermined Significance/Follicular Lesion of Undetermined Significance Cytology.

BACKGROUND: Fine-needle aspiration (FNA) cytology is a common approach to evaluate thyroid nodules. It offers definitive diagnosis of a benign or malignant nodule in the majority of cases. However, 10-25% of nodules yield one of three indeterminate cytologic diagnoses, leading to suboptimal management of these patients. Atypia of undetermined significance/follicular lesion of undermined significance (AUS/FLUS) is a common indeterminate diagnosis, with the cancer risk ranging from 6% to 48%. This study assessed whether a multi-gene next-generation sequencing (NGS) assay can offer significant improvement in diagnosis in AUS/FLUS nodules.
METHODS: From May 2014 to March 2015, 465 consecutive FNA samples with the cytologic diagnosis of AUS/FLUS underwent prospective molecular testing using the ThyroSeq v2.1 panel. The panel included 14 genes analyzed for point mutations and 42 types of gene fusions occurring in thyroid cancer. In addition, eight genes were assessed for expression in order to evaluate the cell composition of FNA samples. Ninety-eight (21%) of these nodules had definitive surgical (n = 96) or nonsurgical (n = 2) follow-up and were used to determine the assay performance.
RESULTS: Among 465 AUS/FLUS nodules, three were found to be composed of parathyroid cells and 462 of thyroid follicular cells. Of the latter, 31 (6.7%) were positive for mutations. The most frequently mutated genes were NRAS and HRAS, and overall point mutations in seven different genes and five types of gene fusions were identified in these nodules. Among 98 nodules with known outcome, histologic analysis revealed 22 (22.5%) cancers. ThyroSeq v2.1 was able to classify 20/22 cancers correctly, showing a sensitivity of 90.9% [confidence interval (CI) 78.8-100], specificity of 92.1% [CI 86.0-98.2], positive predictive value of 76.9% [CI 60.7-93.1], and negative predictive value of 97.2% [CI 78.8-100], with an overall accuracy of 91.8% [CI 86.4-97.3].
CONCLUSIONS: The results of the study demonstrate that the ThyroSeq v2.1 multi-gene NGS panel of molecular markers provides both high sensitivity and high specificity for cancer detection in thyroid nodules with AUS/FLUS cytology, which should allow improved management for these patients.

Introduction

The introduction of fine-needle aspiration (FNA) biopsy in the early 1980s led to a significant improvement in cancer diagnosis in thyroid nodules and a decrease in thyroidectomies performed for benign nodules (1). With more recent advances including routine use of ultrasound guidance, FNA biopsy allows a definitive benign or malignant diagnosis in 60–80% of all nodules (2,3). However, 10–25% of the cases yield one of three indeterminate cytologic diagnoses, which according to the Bethesda reporting system include atypia of undetermined significance/follicular lesion of undetermined significance (AUS/FLUS), follicular neoplasm/suspicious for follicular neoplasm or Hürthle cell neoplasm/suspicious for Hürthle cell neoplasm (FN/SFN), and suspicious for malignancy (4,5). Based on the definition proposed by the Bethesda system, the diagnosis of AUS/FLUS is made in cytologic samples that contain cells with architectural and/or nuclear atypia that is more pronounced than expected for benign changes, but not sufficient to be classified as one of the higher-risk diagnostic categories (4,5). In a meta-analysis of eight studies that reported >25,000 thyroid FNA samples using the Bethesda system, 9.6% of samples were diagnosed as AUS/FLUS (3). With an estimated number of thyroid FNA in the United States of 526,700 in 2011 (6), it can be predicted that >50,000 thyroid nodules are diagnosed as AUS/FLUS every year in the United States alone.

The risk of cancer in nodules diagnosed by AUS/FLUS was estimated by the Bethesda system to be 5–15% (4,5). In studies by investigators who adopted the Bethesda system, 6–48% (M = 16%) of surgically removed AUS/FLUS nodules were found to be malignant (3). In the absence of ancillary molecular testing, many patients with AUS/FLUS diagnosis undergo repeat FNA, which yields a more definitive cytologic diagnosis in a significant proportion of cases, but many of those nodules are again diagnosed as AUS/FLUS (7–9). Additionally, the rate of malignancy on surgical follow-up appears to be similar in nodules after a single or a second AUS/FLUS diagnosis or in those with benign cytology following initial AUS/FLUS diagnosis, suggesting a low impact of repeat FNA in these patients (10).

More recently, molecular testing has been offered to refine the assessment of cancer risk in nodules with AUS/FLUS cytology to improve management of these patients. Whereas testing for single mutations such as BRAF has a high specificity for cancer, it shows low sensitivity (11,12). Testing for limited panels of mutations, such as a seven-gene panel, offers an improved sensitivity of cancer detection in these nodules, with a negative predictive value (NPV) of 94% and positive predictive value (PPV) of 88% (13). High PPV is helpful to select the appropriate extent of surgery for patients with AUS/FLUS nodules, but the residual cancer risk of 6% may not be sufficiently low to select a nonsurgical approach for many of these patients. Another common diagnostic approach is the use of a gene expression classifier (GEC) test known as Afirma, which has been reported to have a 90% sensitivity and 95% NPV in nodules with AUS/FLUS cytology (14). In populations with a pretest prevalence of cancer of 24%, the Afirma GEC is expected to bring the residual cancer risk in a given nodule down to 5%, which is believed to be sufficient to follow patients without surgery (15). However, the specificity of the test is low, resulting in PPV of 38% in a large validation study (14), and an even lower PPV in a recent smaller series of patients (16). As a result, more than a half of patients tested positive by the Afirma GEC may still have benign disease on surgical pathology. Therefore, a molecular test is needed that would offer both high sensitivity and high specificity for cancer detection in thyroid nodules with AUS/FLUS cytology. Such a test would allow clinicians to optimize management by avoiding many of the diagnostic surgeries currently performed for benign nodules, with associated decreases in health costs and postsurgical complications.

Recent progress in identifying driver mutations in thyroid cancer offers an opportunity to improve significantly the sensitivity of preoperative cancer detection by mutational panels. A large multi-gene panel known as ThyroSeq v2 has been recently shown to provide high sensitivity and high specificity for cancer detection in thyroid nodules with indeterminate FN/SFN cytology (17). This study analyzed the performance of this panel, which was further expanded based on the recently reported TCGA study (18), in nodules with AUS/FLUS cytology.

Materials and Methods

Study cohort

From May 2014 to March 2015, 465 thyroid FNA samples from 441 patients had the cytologic diagnosis of AUS/FLUS established at the University of Pittsburgh Medical Center and underwent molecular analysis. The study was approved by the University of Pittsburgh Institutional Review Board. Cytologic criteria used for the diagnosis of AUS/FLUS were as described by the Bethesda system (4,5). Of the 441 patients, 90 patients with 96 nodules underwent surgery based on the presence of another nodule with the FNA diagnosis of suspicious for malignancy or malignant in five patients, positive molecular testing results in 27 patients, and patient preference in the rest of cases. Three patients with nodules that were initially believed to be thyroidal had a diagnostic biochemical evaluation that confirmed the presence of parathyroid disease, and one of those patients underwent surgery. Thus, altogether 98 nodules from 92 patients had a definitive surgical (n = 96) or nonsurgical (clinical evidence of primary hyperparathyroidism; n = 2) outcome information on their AUS/FLUS nodules and were used to assess the performance of the test.

Sample collection for molecular analysis

At the time of the FNA procedure, a small portion of the first and on occasion second needle pass was collected for molecular analysis as previously described (13). A portion of the second pass was added to the same tube when the first pass had low cellularity by on-site assessment. No extra passes beyond routine clinical protocol were taken for molecular analysis. The collected samples were stored at −20°C until cytologic diagnosis was rendered. In those cases where the cytologic diagnosis of AUS/FLUS was established, the collected samples were submitted for molecular analysis per the protocol established by the UPMC/UPCI Multidisciplinary Thyroid Center.

Molecular testing using ThyroSeq v2.1 assay

Molecular analysis was performed prospectively, that is, before surgery and histopathologic results became available. Total nucleic acids were isolated using the Compact MagNA Pure instrument (Roche). Samples were tested using the previously reported ThyroSeq v2 assay (17) with some modifications. Similar to ThyroSeq v2, two libraries were prepared to study separately (i) point mutations and small indels using isolated DNA, and (ii) gene fusions and gene expression controls using isolated RNA. For the DNA panel, in addition to the 13 genes on the ThyroSeq v2 panel (17), the EIF1AX gene was also studied for hotspots of mutations reported by TCGA (18). For each patient sample, the two libraries were mixed together and sequenced using targeted next generation sequencing on Ion Torrent PGM or Ion Proton (Life Technologies). Although the analytic sensitivity of the assay remained at ∼1% of mutant alleles, the clinical sensitivity was set at 5% for BRAF, TP53, AKT1, CTNNB1, PIK3CA, and RET mutations and at 10% for NRAS, HRAS, KRAS, PTEN, TSHR, and EIF1AX mutations. Finding of GNAS mutation was considered a marker of a benign nodule. The presence of at least 50 reads of the fusion transcript constituting 0.1% or more of all mapped sequencing reads for a given tumor was required to consider the test positive. The expression of eight control genes was interpreted as described for ThyroSeq v2 (17).

Histologic review and immunohistochemistry

Initial histopathologic diagnosis was established by pathologists who were not blinded to the results of molecular testing. Histopathologic slides of all mutation-positive and histologically benign (n = 6) and mutation-negative but histologically malignant (n = 2) nodules were re-reviewed to confirm the nodule sampled by FNA based on the location, size, and microscopic evidence of FNA injury. The initial histopathologic diagnosis for the aspirated nodule was not changed after review in any of the cases. Immunostaining for HBME-1 was performed using a monoclonal antibody (DAKO; M3505) in 1:100 dilution on the Benchmark Ventana Ultra instrument and interpreted as reported previously (19).

Statistical analysis

Calculations of specificity, sensitivity, PPV, NPV, and false positive fraction (1-specificity) and asymptotic 95% confidence intervals were performed. The outcomes of diagnosis based on ThyroSeq were compared to an unpaired set of samples previously evaluated with the Afirma GEC. These comparisons used the empirical ratios of two measures with asymptotic confidence intervals for logged ratios. Computations were conducted using the R statistical package (R Foundation for Statistical Computing; www.R-project.org).

Results

Results of molecular testing

Out of 465 nodules with AUS/FLUS diagnosis, 462 had the expression of the cell lineage genes consistent with thyroid follicular cells, whereas three samples showed strong expression of the parathyroid hormone (PTH) gene, indicating the predominance of parathyroid cells in the aspirated nodule (Fig. 1). These three patients underwent additional laboratory and imaging workup, demonstrating the presence of primary hyperparathyroidism. Out of the remaining 462 nodules that showed the predominance of thyroid follicular cells, 31 (6.7%) were tested positive for thyroid point mutations or gene fusions, whereas the rest were mutation negative. Mutations affecting the NRAS and HRAS genes were most common. Overall, 24 point mutations and seven gene fusions were identified in these nodules (Table 1).

Schematic representation of the study flow and main findings.

1.

Molecular Abnormalities Detected in AUS/FLUS FNA Samples and Associated Cancer Risk

	Number of mutation-positive samples	Nodules with surgical outcome	Cancer on surgery (cancer risk)
Point mutation
NRAS	7	5	3 (60%)
HRAS	7	6	5 (83%)
KRAS	3	3	3 (100%)
PTEN	2	1	0
EIF1AX	2	2	1 (50%)
BRAFV600E	1	1	1 (100%)
BRAFK601E	1	1	1 (100%)
TSHR	1	0	0
Gene fusions
THADA	3	3	2 (67%)
PPARG	1	1	1 (100%)
NTRK3	1	1	1 (100%)
NTRK1	1	1	1 (100%)
ALK	1	1	1 (100%)

AUS/FLUS, atypia of undetermined significance/follicular lesion of undermined significance; FNA, fine-needle aspiration.

Correlation with surgical follow-up

Ninety patients with 96 nodules underwent surgery, including patients with 26 mutation-positive nodules and with 70 nodules that had tested negative for mutations. Total thyroidectomy was performed on 45 of these patients and lobectomy/hemithyroidectomy on 44, and one patient with the nodule that was negative for mutations but showed increased expression of the PTH gene underwent thyroidectomy and parathyroid gland exploration. On histopathologic examination, 22 of the aspirated nodules were malignant (22.5%) and 74 were benign (Fig. 1).

Among 31 mutation-positive nodules, 26 (84%) were surgically removed. Of those, 20 (77%) were malignant and six (23%) were benign. Two of the six benign nodules had NRAS mutations, and single nodules harbored HRAS, EIF1AX, or PTEN mutation or THADA fusion. Among the 20 malignant nodules, two were classic papillary carcinomas and 18 were follicular variant papillary carcinomas.

Among 70 mutation-negative nodules, 68 were benign and two were malignant at surgical resection. The two malignant tumors were a 1.4 cm encapsulated follicular variant papillary carcinoma and a 1.9 cm nonencapsulated follicular variant papillary carcinoma. Both tumors were intrathyroidal and showed no angiolymphatic invasion or overtly infiltrative growth. A nodule that showed high expression of the PTH gene was found to be an enlarged (1.0 cm, 260 mg) parathyroid gland.

Histologic review of mutation-positive histologically benign thyroid nodules

Among six nodules positive for mutations that were found to be benign after surgical excision, four were diagnosed as follicular adenomas and two as hyperplastic nodules. Five of these cases had histologic slides available for review. These included three nodules positive for RAS mutations (two NRAS and one HRAS), one nodule positive for a PTEN mutation, and one for an EIX1AX mutation. The review revealed that the three RAS-positive nodules had a thin capsule and were composed of a mixture of normal to large size follicle with abundant colloid, prompting the diagnosis of a hyperplastic nodule in one of the cases. Two nodules positive for NRAS mutations showed scattered areas composed of small-size follicles lined by cells with nuclear features that were suspicious for papillary carcinoma, but were felt to be insufficient for the diagnosis of cancer (Fig. 2A and B). HBME-1 immunostain, a marker of papillary carcinoma, was also focally positive (Fig. 2C). The HRAS-positive nodule showed scattered formation of rudimentary papillae lined by cells that showed some nuclear features of papillary carcinoma, which nevertheless were considered to be insufficient for the diagnosis of cancer. HBME-1 staining was negative in this case. Two other nodules with an isolated PTEN or EIX1AX mutation had a thick or thin capsule and follicular-pattern growth with no atypical nuclear features.

Microscopic appearance of one of the NRAS-positive nodules histopathologically diagnosed as a hyperplastic nodule. (A) Low-power view showing a moderately thick capsule (arrow) and a nodule (top) composed predominantly of large follicles with abundant colloid intermixed with small follicles (hematoxylin and eosin [H&E], 40×). (B) In focal areas, small follicles are lined by cells with nuclear enlargement, irregular nuclear contours, and some chromatin clearing (arrows) (H&E, 200×). (C) Small follicular structures are focally positive for HBME-1 by immunohistochemistry (arrows), whereas most of the adjacent large follicles are negative (HBME-1 immunostain, 100×). Color images available online at www.liebertpub.com/thy

Performance characteristics of the ThyroSeq test in AUS/FLUS nodules

Overall, between 96 nodules with known surgical outcome and two nodules found to have strong expression of the PTH gene with clinical evidence of primary hyperparathyroidism, 98 nodules were considered to have definitive outcome information. In this group, ThyroSeq allowed correct classification of 91 nodules as either benign (n = 71) or malignant (n = 20), with six false-positive and two false-negative test results. As a result, it showed performance characteristics of 90.9% [confidence interval (CI) 78.8–100] sensitivity, 92.1% [CI 86.0–98.2] specificity, 76.9% [CI 60.7–93.1] PPV, and 97.2% [CI 78.8–100] NPV, with an overall accuracy of 91.8% [CI 86.4–97.3].

In evaluating the performance of molecular tests, whereas sensitivity and specificity are intrinsic characteristics, the PPV and NPV of any test depends on the prevalence of disease in the studied population and will change if the test is applied to a population with different disease prevalence (16). Based on known sensitivity and specificity, NPV and PPV can be predicted for populations with different disease prevalence using Bayes' theorem. In the cohort studied here, the prevalence of cancer in cytologically AUS/FLUS nodules was 22.5%, whereas in the literature it ranges from 6% to 48% (3). To estimate the performance of ThyroSeq v2.1 in populations with variable pretest cancer incidence, the established sensitivity and specificity were used to calculate PPV and NPV corresponding to the whole spectrum of disease prevalence (Fig. 3). The analysis showed that with cancer prevalence in AUS/FLUS nodules ranging between 6% and 48%, the NPV of this test would range between 99% and 92%, and PPV between 42% and 91%.

Negative predictive value (NPV; red line) and positive predictive value (PPV; blue line) of ThyroSeq v2.1 in thyroid nodules with atypia of undetermined significance/follicular lesion of undermined significance cytology found in this cohort with a cancer prevalence of 22.5% (black dotted line) and expected in a range of cancer prevalence rates between 6% and 48% (yellow dotted lines). Color images available online at www.liebertpub.com/thy

Finally, the performance of ThyroSeq v2.1 in AUS/FLUS nodules found in this study was compared with the performance parameters of the Afirma GEC based on the largest study reported to date (14) (Fig. 4). Using the unpaired analysis of the true positive fraction (TPF i.e., sensitivity) and false positive fraction (FPF or 1-specificity), the NPV of ThyroSeq and Afirma were not statistically different (TPF ratio 1.006 [CI 0.844–1.198]). However, the PPV of ThyroSeq was significantly higher (FPF ratio 0.168 [CI 0.076–0.372]).

Predicted NPV and PPV of ThyroSeq v2.1 compared to the Afirma gene expression classifier test in AUS/FLUS nodules based on the sensitivity and specificity of ThyroSeq (solid lines) identified in this study and of Afirma (dashed lines) reported by Alexander et al. (14). Color images available online at www.liebertpub.com/thy

Discussion

This study evaluated the performance of a large multi-gene panel of mutation markers in thyroid nodules diagnosed as AUS/FLUS by cytology. Using a large series of such nodules with known surgical or nonsurgical outcome, it was demonstrated that the assay provides accurate stratification of the vast majority of AUS/FLUS nodules as benign or malignant, showing both high sensitivity and specificity. The residual risk of cancer in AUS/FLUS nodules in the studied patients was as low as 2.8%. Furthermore, due to high sensitivity, the ThyroSeq v2.1 assay is expected to retain high NPV in populations with widely variable disease prevalence.

In managing patients with indeterminate FNA cytology, it has been suggested that if the risk of cancer in a given thyroid nodule is ≤5%, observation instead of surgery can be considered, since the risk is close to that of a lesion with a benign FNA cytology (15). To follow this approach, most patients with AUS/FLUS nodules and negative ThyroSeq test may be followed with observation (20). The exception would be a population with a very high pretest probability of cancer. If one uses a residual cancer risk of 5% as a maximum acceptable risk, ThyroSeq v2.1 can be used to avoid surgery, as long as the probability of cancer associated with AUS/FLUS cytology at a given institution is <38%, which should encompass the vast majority of clinical centers performing thyroid FNA biopsies (2,3).

In addition to a very high NPV, ThyroSeq v2.1 showed a high PPV for cancer detection. The addition of multiple gene markers to the seven-gene panel led to some reduction of the PPV from 88% to 77%, but the PPV remained significantly higher than that of the Afirma GEC (14,16). As a result, ThyroSeq v2.1 has a high performance as both a “rule out” and “rule in” test. Furthermore, the risk of cancer in mutation-positive nodules can be further stratified based on the specific mutations. Indeed, the finding of BRAF mutations or PPARG, NTRK1, NTRK3, and ALK fusions appears to confer a close to 100% risk of cancer, whereas the presence of RAS, PTEN, and EIF1AX mutations or THADA fusions entails a significant but lower risk of cancer.

Among six nodules found to be false positive, three had RAS mutations. Histopathologically, they were diagnosed as benign adenomas or even hyperplastic nodules because of the presence of large follicles containing abundant colloid. However, biologically, these nodules are clonal neoplasms, as they had distinct mutations present in the majority of cells. Furthermore, microscopically, these nodules showed focally present nuclear features suspicious for papillary carcinoma and at least some of them had focal areas of expression of immunohistochemical markers characteristic of papillary carcinoma, suggesting early transformation. This supports other observations, including those in mice with thyroid-specific expression of mutant NRAS (21), suggesting that RAS is an oncogene responsible for gradual transformation of thyroid cells and progression from benign to malignant tumors.

In addition to mutations in oncogenes and tumor suppressor genes, which served as primary diagnostic markers on the panel, additional diagnostic information was provided by analyzing the expression of cell lineage genes. This allowed us to determine whether the aspirated nodule was composed of thyroid follicular cells, C-cells/medullary carcinoma cells, or nonthyroid cells. Indeed, three nodules, initially thought to be of thyroid origin, had a high-level expression of the PTH gene, and the patients were found to have parathyroid disease. Overall, 0.6% of nodules in this series of samples diagnosed as AUS/FLUS by cytology were found to be composed of parathyroid cells, contributing to the diagnostic value of the panel. The incidence of parathyroid lesions observed in this study was generally similar to that previous reported (0.4%) (22). Therefore, ThyroSeq v2.1 not only improves diagnostic accuracy in differentiating benign and malignant thyroid nodules, but can also uncover previously unrecognized parathyroid disease.

The comparison of the predicted performance of ThyroSeq v2.1 with that of the Afirma GEC suggested that the NPV of both assays for AUS/FLUS nodules are expected to be similar, whereas the PPV are dissimilar, with a significantly better performance of ThyroSeq. This comparison analysis, however, has significant limitations, since the two assays were tested in different patient cohorts. Furthermore, the current ThyroSeq analysis was performed on a cohort of 98 nodules from one institution, whereas the results of Afirma GEC were obtained by analyzing 129 AUS/FLUS nodules in a multi-institutional study involving 49 clinical sites (14).

In summary, the results of this prospective study demonstrate that the next-generation sequencing–based ThyroSeq v2.1 panel offers accurate classification of most thyroid nodules with AUS/FLUS cytology into benign or malignant group. This should guide improved management for a large number of patients and allow clinicians to avoid many of the currently required diagnostic surgeries that are associated with significant costs and potential risks.