Ultrasound is helpful to differentiate Bethesda class III thyroid nodules: A PRISMA-compliant systematic review and meta-analysis.

BACKGROUND: Fine-needle aspiration (FNA) is the most dependable tool to triage thyroid nodules for medical or surgical management. However, Bethesda class III cytology, namely "follicular lesion of undetermined significance" (FLUS) or "atypia of undetermined significance" (AUS), is a major limitation of the US-FNA in assessing thyroid nodules. As the most important imaging method, ultrasound (US) has a high efficacy in diagnosing thyroid nodules. This meta-analysis aimed to assess the role of US in evaluating Bethesda class III thyroid nodules.
METHODS: With keywords "Undetermined Significance," "Bethesda Category III," "Bethesda system," "Cytological Subcategory," "AUS/FLUS," "Atypia of Undetermined Significance," and "Ultrasound/US," papers in PubMed, Cochrane Library, Medline, Web of Science, Embase, and Google Scholar from inception to December 2016 were searched. A meta-analysis of these trials was then performed for evaluating the diagnostic value of thyroid ultrasound in Bethesda Category III thyroid nodules.
RESULTS: Fourteen studies including 2405 nodules were analyzed. According to the criteria for US diagnosis of thyroid nodules in each article, with any one of suspicious features as indictors of malignancy, US had a pooled sensitivity of 0.75 (95% CI 0.72-0.78) and a pooled specificity of 0.48 (95% CI 0.45-0.50) in evaluating Bethesda Class III Nodules. The pooled diagnostic odds ratio was 10.92 (95% CI 6.04-19.74). The overall area under the curve was 0.84 and the Q* index was 0.77. With any 2 or 3 of US suspicious features as indictors of malignancy, the sensitivity and specificity were 0.77 (95% CI 0.71-0.83) and 0.54 (95% CI 0.51-0.58), 0.66 (95% CI 0.59-0.73) and 0.71 (95% CI 0.68-0.74), respectively.
CONCLUSIONS: US was helpful for differentiating benign and malignant Bethesda class III thyroid nodules, with the more suspicious features, the more likely to be malignant.

Introduction

Thyroid nodules are very common medical problem with the prevalence of 19% to 68% in general population.[ About 7% to 15% of thyroid nodules are thyroid cancer, it was estimated that 96% of all new endocrine organ cancers originate from the thyroid gland in 2014.[ According to American Thyroid Association (ATA) guidelines, FNA is the most accurate and cost-effective method for evaluating thyroid nodules.[ The reports of FNA are dependable to triage thyroid nodules for medical or surgical management. From 2007 to now, the Bethesda classification system for reporting thyroid FNA has been used world widely. The most important contributions are the risk of malignancy and management recommendation for each category, and the creation of an atypical category for repeat FNA, that is Bethesda class III (atypia of undetermined significance or follicular lesion of undetermined significance in the Bethesda System), with the risk of malignancy 5% to 15%.[ Bethesda class III, which do not provide a differential diagnosis between malignant and benign lesion, is a major limitation of the US-FNA in assessing thyroid nodules.[ Management of Bethesda class III nodule seems to be diverse, including clinical observation, ultrasound follow-up, repeat FNA, molecular test or surgery,[ sometimes depending on the willing of patients or experiences of physicians. Recently, several studies have focused on the feasibility of using thyroid US to predict the malignancy of Bethesda class III thyroid nodules.[ Although its clinical use was uncertain, these studies showed that promising results of US might provide useful information. We conducted this meta-analysis to evaluate the role of US in the diagnosis of Bethesda class III thyroid nodules.

Materials and methods

Search strategy and inclusion criteria

Our systematic review was designed and performed according to the guidelines of the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) statement.[ The ethical approval was not necessary because our meta analysis was based on data from previously published studies. We searched PubMed, Cochrane Library, Medline, Web of Science, Embase, and Google Scholar from inception to December 2016 under the key words relating to “Undetermined Significance,” “Bethesda Category III,” “Bethesda system,” “Cytological Subcategory,” “AUS/FLUS,” “Atypia of Undetermined Significance” and “Ultrasound/US.” The inclusion criteria were studies of using thyroid US to predict malignancy of the Bethesda class III nodules in the initial US-FNA. Informed consent was necessary. References cited in original and review articles were cross checked. Reviews, abstracts, and duplicate data were removed. No language restrictions were applied.

Two authors (LYG and YW) performed the search job and data screening independently, and the discrepancies resolved by consensus (BZ).

Data extraction, quality assessment of articles, and statistical analysis

We attempted to determine the extent to which US is diagnostic that identifies Bethesda class III nodules as malignant or benign. However, US classification schemes for thyroid nodules are diverse, and different reports used various categories. So according to number of the suspicious malignant features of US, data were extracted by 2 independent authors. As the criteria for US diagnosis of thyroid nodules in each article, suspicious malignant features included hypoechogenicity, irregular margins, microcalcifications, taller-than-wide shape or macrocalcifications, increasing size during follow-up, increased vascularization, and disrupted rim calcifications.[ The absolute number of true-positive (TP), false-positive (FP), false-negative (FN), and true-negative (TN) were retrieved or calculated from the articles; other characteristics, including publication year, country or region, average age of patients, proportion of males, nodules number, retrospective or prospective set-up of the study and the reference standard that was used in the study were also recorded.

Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) criteria was applied in the methodological quality of the included studies.[

Sensitivity, specificity, diagnostic accuracy, and diagnostic odds ratio (DOR) were calculated using Meta-Disc version 1.4 statistical software (Meta-Disc, Unit of Clinical Biostatistics Team of the Romany Cajal Hospital, Madrid, Spain). All results were estimated as the pooled odds ratio with 95% confidence interval. To detect heterogeneity, the likelihood ratios and DORs were graphically displayed using forest plots. A summary receiver operating characteristic (SROC) curve was constructed, the Q∗index was calculated. Heterogeneity among studies was assessed by the I2 statistic.

Results

Study characteristics

The flow chart is shown in Fig. 1. According to the searching strategy, the initial search retrieved 188 articles, of those, 146 were excluded based on the title and abstract. Full-text assessment was performed on 42 articles. Of these, 13 articles only analyzed the ultrasonic features which predicted the risk of malignancy, but did not examine US as a diagnostic tool in Bethesda class III nodules; 5 articles were written in Chinese or French; and 10 articles were duplicated articles. Finally, 14 articles including 2405 thyroid nodules were included in the analysis[ (Fig. 1).

Figure 1

The procedure of study selection in our meta-analysis. A total of 14 studies were included in this systematic review, which fulfilled all the inclusion criteria.

The 14 articles were published from 2011 to 2016. The study designs were retrospective (N=11) and prospective (N = 3). The number of analyzed nodules ranged from 29 to 548. The average age of participants in the study was 48.4 years. Males constituted 19.3% of all patients. In 6 studies, surgery and repeat FNA was carried out for histopathological and cytological results. Four studies carried out surgery for histological results. In addition, 3 studies included follow-up US, and 1 study included a genetic test used as a tool to assess the final diagnosis (Table 1).

Table 1

Basic characteristic of 14 studies.

The included studies generally had a low risk of bias. Details about the quality of trials were according to the QUADAS-2.

The funnel plot indicated no likely publication bias, showing a symmetric shape when the log DORs of individual studies, was plotted against their sample sizes.

Summary of sensitivity, specificity, positivity, and SROC curves

When thyroid nodules with any one of US feature of malignancy were considered malignant, all 14 articles were included. The pooled sensitivity of US diagnostic accuracy for differentiating malignant and benign nodules was 0.75 (95% CI 0.72–0.78) (Fig. 2), and the pooled specificity of US was 0.48 (95% CI 0.45–0.50) (Fig. 3). The summary positive LRs (likelihood ratio) were 2.29 (95% CI 1.65–3.18), and the summary negative LRs were 0.27 (95% CI 0.12–0.20), respectively. The pooled DOR was 10.92 (95% CI 6.04–19.74) (Fig. 4). The SROC curve was symmetric. The overall AUC (area under the curve) was 0.8358, and the Q∗ index was 0.7679 (Fig. 5).

Figure 2

Forest plot showed pooled sensitivity of US in the differentiated diagnosis of Bethesda class III thyroid nodules. US = ultrasound.

Figure 3

Forest plot showed pooled positivity of US in the differentiated diagnosis of Bethesda class III thyroid nodules. US = ultrasound.

Figure 4

Forest plot showed diagnostic odds ratio (DOR) of US in the differentiated diagnosis of Bethesda class III thyroid nodules. DOR = diagnostic odds ratio, US = ultrasound.

Figure 5

Summary receiver operating characteristic (SROC) curves of US in the evaluation of Bethesda class III thyroid nodules. SROC = summary receiver operating characteristic, US = ultrasound.

When thyroid nodules with any 2 of suspicious features were considered malignant nodules, 5 articles were included. The pooled sensitivity was 0.77 (95% CI 0.71–0.83), and the pooled specificity was 0.54 (95% CI 0.51–0.58). The pooled DOR was 7.61 (95% CI 3.04–19.07) (see Supplemental Digital Content Fig. 1s, which shows the DOR of US). The overall AUC was 0.8116, and the Q∗ index was 0.7461.

When thyroid nodules with any 3 of suspicious features were considered malignant nodules, 4 articles were included. The pooled sensitivity was 0.66 (95% CI 0.59–0.73), and the pooled specificity was 0.71 (95% CI 0.68–0.74). The pooled DOR was 5.74 (95% CI 3.02–10.93) (see Supplemental Digital Content Fig. 2s which shows the DOR of US). The overall AUC was 0.7425, and the Q∗ index was 0.6873.

Subgroup analysis

Because noticeable heterogeneity was observed in the tests of sensitivity and specificity, the random effects model was used. In the analysis, only 14 studies were available, which prevented the use of a meta-regression. Therefore, we performed a subgroup analysis of 3 subgroups (increased vascularization as one of the US malignant features, number of nodules, and thyroid imaging reporting and data system (TI-RADS) criteria) to determine the potential sources of heterogeneity. Among them, only 1 main factor (increased vascularization as one of the US malignant features) contributed to the heterogeneity of our systematic-review (Table 2). The result indicated that studies including increased vascularization as one of US malignant features had important influence on the overall specificity and sensitivity (P < 0.05).

Table 2

Diagnostic accuracy of subgroups studies.

Discussion

It is necessary to improve the management of AUS/FLUS nodules based on FNA biopsy diagnosis. Our meta-analysis showed that ultrasound classification scheme did facilitate malignancy prediction and helped guide the therapeutic plan for AUS/FLUS nodules.

The systematic review focused on the diagnostic value of US in Bethesda class III nodules. According to our meta-analysis, with 1 US suspicious feature as indictors of malignancy, the pooled sensitivity was 0.75, and the pooled specificity was 0.48. The overall AUC was 0.84. These values indicated that US has very good diagnostic accuracy for the differentiation of Bethesda class III nodules. Moreover, the DOR in our review was 10.92 (95% CI 6.04–19.74), which demonstrated that the US was a good diagnosis test for the differentiation of Bethesda class III nodules. In most studies, thyroid nodules with 1 US feature of malignancy were considered as malignant nodules. For nodules with 2 US features of malignancy, the pooled sensitivity was 0.77 and the pooled specificity was 0.54. The AUC was 0.81, and the DOR in our review was 7.61 (95% CI 3.04–19.07). For thyroid nodules with 3 US features of malignancy, the pooled sensitivity was 0.66 and the pooled specificity was 0.71. The overall AUC was 0.74, and the DOR in our review was 5.74 (95% CI 3.02–10.93). Therefore, it is reasonable that when Bethesda class III nodules showed more US features of malignancy, the specificity of diagnostic malignancy were higher.

The diversity of US classification schemes for nodules may contribute to the heterogeneity of this meta-analysis. Many studies have indicated the US features of thyroid nodal diseases and some malignant features have been generally accepted.[ However, the criteria for the ultrasound features of malignant nodules are still controversial. In our meta-analysis, we found a large range of sensitivity and specificity (0.02–1.00 and 0.13–1.00) with high heterogeneity (P < 0.01). The various criteria for the US malignant features in these publications may contribute to the different sensitivities and specificities. In most studies, nodules diagnosed as malignant were characterized by hypoechogenicity, a taller than wide shape, microcalcifications, and a spiculated margin. However, macro-calcifications, increasing size during follow-up, increasing vascularization and disrupted rim calcifications with extrusive soft tissue components were considered to be malignant US features in several studies, but not in others. Kim et al[ emphasized different US categories for solid thyroid nodules and partially cystic thyroid nodules (PCTNs), but the other studies used the same US classification schemes for solid nodules and PCTNs.

The multiclassifications of US features in different medical institutions may confuse ultrasonologist who perform US examinations. Moreover, the experience differences of cytopathologists in interpreting FNA slides might have resulted in variable cytological diagnoses for cases, and the operator-dependent character of US might contribute to this divergence. In addition, through the subgroup regression analyses, we identify only 1 main factor (increased vascularization) had a significant influence on the specificity and sensitivity, and the factors may be responsible for the heterogeneity. The data demonstrated that only.

The management for AUS/FLUS nodules varies widely among institutions, including US follow-ups, repeat FNA, molecular test, and surgery.[ Layfield et al[ recommended a repeat US-FNA at a specified interval. However, 1 group recommended the limited use of repeat FNA because a discrepancy might be unavoidable in the interpretation of the overlapping cytological criteria.[ The clinical application of molecular test for Bethesda class III nodules is fairly common practice; however, according to ATA, the utility of molecular testing is applicable only when combined with clinical and ultrasonic risk factors for malignancy.[ Although it has been reported that the Veracyte Afirma gene expression classifier (GEC) test had a positive predictive value (PPV) of 38% and an FN rate of 5%, the cost effectiveness of molecular test is still controversial.[

There are several limitations to our meta-analysis. First, only a limited number of studies have reported the accuracy of US diagnosis of AUS/FLUS, which limits the generalizability of our results. Second, the lack of uniform criteria of the US diagnosis scheme may result in evaluation bias of nodules. The diversity of US classification schemes of differentiation of thyroid nodules limits the use of US as an effective diagnostic test. Third, not all cases of included studies were confirmed by pathology. Some patients experienced US follow-ups.

US provides different risks of malignancy for nodules initially classified as Bethesda category III, with the more suspicious features, the more likely to be malignant.