Changes in Diagnostic Performance of Thyroid Cancer Screening before and after the Korean Thyroid Imaging Reporting and Data System Revision.

BACKGROUND: Since the era of "thyroid cancer epidemic," many Korean academic societies discouraged the use of ultrasonography in healthy individuals and revised the Korean Thyroid Imaging Reporting and Data System to address the overscreening and overdiagnosis issues. This study aimed to evaluate the change in the diagnostic effectiveness of thyroid cancer screening over the last decade.
METHODS: This single-center, retrospective observational study analyzed the data of 125,962 thyroid nodules obtained during cancer screening at the health promotion center of Seoul National University Bundang Hospital from 2010 to 2019. Only 327 thyroid cancer cases pathologically confirmed by fine-needle aspiration (FNA) were included in the study. The strength of the association between the number of FNA and (1) the number of thyroid cancer diagnoses, (2) the positive predictive values (PPVs), and (3) the difference in PPV from the previous year were evaluated using Pearson's correlation analysis.
RESULTS: The number of thyroid FNA biopsies as well as the thyroid cancer diagnoses decreased from 2010 to 2019 (166 to 48 [-71.1%] vs. 43 to 22 [-48.8%]). The PPV of FNA biopsies increased from 25.9% to 45.8% (+76.8%) and was negatively correlated with the number of FNA biopsies performed (R=-0.87, P<0.001). The difference in PPV from the previous year increased similarly but without statistical significance (R=-0.59, P=0.09).
CONCLUSION: The diagnostic efficiency of thyroid cancer screening has increased over the last decade, as evidenced by the increasing PPV of FNA biopsies.

INTRODUCTION

In recent decades, the incidence of thyroid cancer in high-income countries has markedly increased [1]. South Korea has been one of the “thyroid cancer epidemic” countries to be publicly criticized in many scholarly articles and has high incidence of overscreening and overdiagnosis of thyroid cancer [2]. According to Statistics Korea, the incidence rate of thyroid cancer in 2012 was almost 12 times higher than that observed in 1999 (incidence rate: 7.2 versus 88.8 cases per 100,000 persons) [3]. Many factors may have accelerated the detection of small, asymptomatic thyroid nodules: (1) the development of imaging modalities (e.g., ultrasonography), (2) wide coverage of public and private health insurance, and (3) improved public awareness of cancer screening by the National Cancer Screening Program and local health promotion centers (HPCs) [4].

Many academic societies have taken tentative steps to implement these measures. In 2015, the Korean Medical Association recommended the use of ultrasonography in healthy individuals [5]. In 2016, the American Association of Clinical Endocrinologists/Association of Clinical Endocrinology (Associazione Medici Endocrinology) introduced the “revised Korean Thyroid Association (KTA) management guidelines for patients with thyroid nodules and thyroid cancer.” These guidelines recommended the establishment of a new malignancy risk stratification according to the Korean Thyroid Imaging Reporting and Data System (K-TIRADS) based on solidity, echogenicity, and presence of suspicious ultrasonographic features of thyroid nodules [6]. In addition, a more conservative size indication for fine-needle aspiration (FNA) for each K-TIRADS category was suggested to reduce the number of unnecessary FNAs.

The thyroid cancer incidence in South Korea begun to decrease after reaching its peak in 2014 [3]. There has been a notable decrease (30%) in the number of thyroid cancer surgeries performed as well as that of claimed insurance cases [7]. Several previous studies have argued that the increasing number of cancer screening recipients is mostly attributed to the high thyroid cancer incidence [2]. However, the decreasing thyroid cancer incidence cannot be reasonably explained by the trend of cancer screening because the number of thyroid cancer screening recipients has continuously been increasing. The National Health and Nutrition Survey showed that the percentage of health screening recipients, aged 19 years and older, has been increasing from 43% (1998) to 66.4% (2018) and has the tendency to the number of thyroid ultrasound performed at low cost [8].

If overscreening remains a major issue, other factors might have caused the decrease in thyroid cancer incidence (e.g., improved diagnostic efficiency, higher ultrasound resolution, greater expertise of physicians, and media effect). The revised KTA guidelines and K-TIRADS might have played a role in solving the problem of overdiagnosis to some extent [9]. Therefore, this study aimed to investigate the change in the diagnostic performance of thyroid cancer screening, especially the efficiency of FNA biopsies, before and after the K-TIRADS revision. By evaluating the change, the study may clinically provide further direction to resolve issues related to overdiagnosis and the medical burden on many asymptomatic patients and future healthcare systems.

METHODS

1. Data Sources

This single-center, retrospective study was conducted from 2010 to 2019, using the electronic medical records (EMRs) of patients who had undergone thyroid cancer screening at Health Promotion Center of Seoul National University Bundang Hospital, South Korea.

The following EMR data were obtained: (1) patients’ hospital numbers, (2) date of thyroid ultrasound examination, (3) date of FNA biopsy, (4) final report of thyroid ultrasonography, and (5) histopathological report obtained from FNA biopsy. Only patients with pathologically confirmed thyroid cancers diagnosed within a year after the initial assessment were included in the study. The text mining method was used to retrieve and classify certain text values from the EMRs using the Microsoft Excel “FIND” function (Microsoft Corp., Redmond, WA, USA). Histopathological results that mention the terms “cancer,” “malignancy,” and “neoplasm” were considered to confirm the thyroid cancer diagnosis.

Patients whose (1) FNA biopsies were recommended by other institutions, (2) FNA biopsies were performed at other institutions, (3) FNA biopsies that were not performed despite the recommendation, and (4) with pathologically confirmed “benign” or “indeterminate” histopathological results that indicated the following terms: “unsatisfactory,” “suggestive of,” or “rule out” were excluded.

Among the 125,962 thyroid cancer screening procedures performed from 2010 to 2019, 1,666 thyroid nodules were recommended for “FNA biopsy” based on the thyroid ultrasonographic report. All recommendations made by well-trained radiologists relied heavily on the conventional three-tier risk categorization system until the middle of 2016 and on the revised KTA guidelines over the following years. Some radiologists may have provided experience-based personal opinions regarding their decision to recommend FNA biopsies. Among the 1,666 thyroid nodules, 1,128 underwent FNA biopsies, and 327 were pathologically confirmed as thyroid cancers. A detailed flowchart is shown in Figure 1.

Figure. 1.

Flowchart showing the total number of population included in the study. US, ultrasonography; FNA, fine-needle aspiration.

2. Statistical Analysis

The degree of linear association was measured using Pearson’s correlation coefficient. The correlation coefficients between the annual number of thyroid FNA and (1) the annual number of thyroid cancer diagnoses confirmed from the FNA biopsy, (2) the positive predictive values (PPVs), and (3) the difference in PPV from the previous year were calculated using RStudio ver. 1.4.1106 (RStudio, Boston, MA, USA) and R-packages (RStudio). The PPV for each year was calculated by dividing the number of diagnosed thyroid cancers by the number of thyroid FNA cases in the year. The difference in the PPV from the previous year was calculated by dividing the annual change in the number of diagnosed thyroid cancers by the annual change in the number of thyroid FNA cases. The calculated annual changes can be both positive and negative. All tests were two sided, and a P-value of <0.05 was considered significant.

3. Ethics Committee Approval

The study protocol was approved by the ethics committee of Seoul National University Bundang Hospital, South Korea (Institutional Review Board no., L-2020-1504). The personal identifiers of all study participants were de-identified. The requirement for obtaining informed consent was waived.

RESULTS

From 2010 to 2019, the total number of thyroid ultrasounds performed for the purpose of cancer screening increased from 14,542 to 10,609 (+27.0%). The total number of thyroid FNA biopsies decreased from 166 to 48 (-71.1%), and the number of thyroid cancer diagnoses decreased from 43 to 22 (-48.8%) (Table 1). A strong positive correlation between the number of thyroid FNA and thyroid cancer diagnoses is shown in Figure 2A (R=0.89, P<0.0005).

Table 1.

Total number of thyroid FNA biopsies performed from 2010 to 2019 in patients visiting the health promotion center and the number of diagnosed thyroid cancer from the biopsies, and the calculated positive predictive values of thyroid FNA each year

Year	No. of thyroid ultrasounds performed (n=125,962)	No. of thyroid FNA[*] (n=1,128)	No. of diagnosed thyroid cancer[†] (n=327)	Positive predictive values (%)[‡]
2010	10,609	166	43	25.9
2011	11,941	230	49	21.3
2012	12,480	151	31	20.5
2013	12,431	129	46	35.7
2014	11,833	150	43	28.7
2015	11,925	93	33	35.5
2016	13,926	58	20	34.5
2017	12,321	64	25	39.1
2018	13,954	39	15	38.5
2019	14,542	48	22	45.8

FNA, fine-needle aspiration.

“No. of thyroid FNA” contains the total number of thyroid FNA performed each year. FNA samples from other institutions were excluded.

“No. of diagnosed thyroid cancer” contains the total number of thyroid cancer diagnoses confirmed by pathologic reports. Pathological reports from other institutions were excluded.

“Positive predictive values” were calculated by dividing the “no. of thyroid cancer diagnosed” and the “no. of thyroid FNA” in such a year.

Figure. 2.

Pearson’s correlation coefficients were calculated to evaluate the correlation between the annual number of thyroid fine-needle aspiration (FNA) biopsy performed in patients visiting from 2010 to 2019 and (A) the annual number of thyroid cancer diagnosis confirmed through FNA biopsy, (B) the positive predictive values (PPV), and (C) the difference in PPV from previous year. The linear lines indicate significant correlations (P<0.05). *The PPV were calculated. The results are shown in the fourth column of Table 1. See the commentary in Table 1 for a detailed equation. †Differences in PPV from that in the previous year were calculated using the following equation: difference in PPV=(number of thyroid cancer diagnoses–number of thyroid cancer diagnoses from previous year)/(number of thyroid FNA–number of thyroid FNA from previous year). The calculated annual changes can be both positive and negative.

The PPVs of thyroid FNA increased from 25.9% to 45.8% (+76.8%) from 2010 to 2019 (Table 1). Figure 3 clearly shows an increasing trend of PPV, especially in a linear pattern from 2016. An inverse correlation was found between the number of thyroid FNA findings and PPVs (R=-0.87, P<0.001) (Figure 2B), and between the number of thyroid FNA cases and the difference in PPV from the previous year (R=-0.59, P=0.09) (Figure 2C).

Figure. 3.

Positive predictive values* (%) of thyroid fine-needle aspirations performed on patients visiting the health promotion center from 2010 to 2019. *Positive predictive values were calculated and shown on the fourth column of Table 1. See commentary on Table 1 for a detailed equation.

The number of histological types determined through FNA biopsy was similar to the distribution of thyroid cancer diagnoses each year (Table 2); papillary carcinoma is the most frequently diagnosed type. During the 10-year study period, only one patient was diagnosed with medullary carcinoma in 2015.

Table 2.

Histological types obtained from thyroid fine-needle aspiration performed on patients visiting the health promotion center from 2010 to 2019

Year	Histological types[*]
	Papillary carcinoma (n=314)	Follicular carcinoma (n=12)	Medullary carcinoma (n=1)
2010	42	1	0
2011	47	2	0
2012	30	1	0
2013	44	2	0
2014	41	2	0
2015	31	1	1
2016	19	1	0
2017	23	2	0
2018	15	0	0
2019	22	0	0

Histological types were classified according to the International Agency for Research on Cancer (International Classification of Diseases for Oncology, third edition).

DISCUSSION

Some researchers have argued that the overuse of diagnostic tools, such as thyroid ultrasound and FNA, has accelerated the thyroid cancer incidence rate in South Korea. A higher number of FNA biopsies performed per 100,000 individuals was positively correlated with an excessively increased thyroid cancer diagnosis in a previous study [10]. In our study, a strong positive correlation (R=0.89, P<0.0005) was observed between the number of FNA and thyroid cancer diagnosis, which was consistent with the results of a previous study, but in a different direction. The number of thyroid cancer diagnoses gradually decreased as the FNA frequency decreased from 2010 to 2019. The number of FNA biopsies dramatically decreased within a 10-year period as opposed to the increased number of performed thyroid ultrasound procedures (-71.1% versus +27.0%). Therefore, the decline in thyroid cancer diagnoses was not associated with the number of performed thyroid ultrasound procedures; instead, the decline was more likely due to the reduced FNA frequency.

The diagnostic efficiency of FNA, based on the PPV, gradually increased over time, even with a reduction in FNA frequency and thyroid cancer diagnoses (Figure 2B). The degree of change in FNA frequency and the PPV within a 10-year period was similar (-71.1% versus +76.8%), and the strength of the association was extremely strong (R=-0.87, P<0.001). Reduced FNA frequency may have been a major driving force for improving the diagnostic efficiency of FNA biopsies. Moreover, a linearly increasing pattern of PPV was observed, particularly from 2016 to 2019 (Figure 3). Interestingly, the trend observed in 2016 coincided with that in the period when some radiologists in our study reported the K-TIRADS stratification of the thyroid ultrasound results based on the revised KTA guidelines. A few possibilities could explain the increasing trend in PPV, which has become more evident since 2016. First, the proportion of “false positives” might have been decreased by avoiding unnecessary FNA biopsies because the updated guideline more conservatively allowed FNA to be performed only in patients with highly malignant nodules. Second, the proportion of “true positives” might have increased every year due to the higher resolution of ultrasonography or the expertise of radiologists who may have reclassified previously ill-defined nodules as benign. The latter could explain why the PPV increased even before 2016. Both scenarios could have enhanced the diagnostic performance of FNA biopsies by effectively reducing the number of FNA biopsies.

Compared with the American College of Radiology and European Thyroid Association guidelines, the revised K-TIRADS has high specificity (87.4%), but is the main cause of the high rate of unnecessary FNA biopsies (32.1%) and has the lowest sensitivity (71.4%) [11]. Presumably, “fewer FNAs” may have further lowered the sensitivity of the KTIRADS and aggravated the fear of missing or not identifying the thyroid cancer on time. In our study, the distributions of papillary, follicular, and medullary carcinomas remained similar throughout the study period (Table 2). Although the data related to the false-negative results or long-term clinical outcomes (e.g., mortality and surgical complications) were not collected during the study, our limited data suggest that the fear of missing or delaying cancer diagnosis could be dismissed to some extent.

To further explore the increasing pattern of diagnostic efficiency, the difference in PPV from the means of the previous year should be explored further. The difference in PPV from the previous year indirectly refers to the annual change in the diagnostic performance of FNA. However, the correlation between the “difference in PPV from the previous year” and “the number of FNA biopsies” were not significant (P=0.09), which was the important key point shown in the Figure 2. However, the Figure 2C should not be overlooked. The annual change in diagnostic efficiency from 2011 to 2014 showed a scattered distribution. However, the diagnostic efficiency from 2015 to 2019 consistently improved throughout the years with positive annual changes, with the greatest improvement observed in 2017. This result could reflect some clinical changes made in the previous year (2016). In 2016, the new revised KTA guideline introduced K-TIRADS, which recommended that FNA should not be used to examine thyroid nodules less than 1.0 cm in size unless tumor progression was highly suspected or the patient preferred to undergo a biopsy [5]. A higher threshold for FNA size indication for each category and a more sophisticated malignancy risk stratification based on echogenicity, solidity, and ultrasonographic features (non-parallel, spiculated/microlobulated margins, and microcalcification) may have effectively enhanced the diagnostic performance of thyroid cancer screening since 2016 [12]. This trend is consistent with the findings of a previous study, which confirmed that the predictive power (Az=0.878) of the K-TIRADS was significantly superior to that (Az=0.805) of the three-tier risk categorization systems (P<0.001) by avoiding 16.2% of unnecessary FNA biopsies [11]. We may also assume that as the number of physicians who applied the new guideline into practice and applied it accurately increased each year, the accumulated positive change could have resulted in an annual improvement in the diagnostic performance of thyroid cancer screening.

In this study, we observed a gradual improvement in the diagnostic performance of thyroid cancer screening, which was affected by the lower FNA frequencies. If further studies were conducted to prove that the proportion of “false-negative” results remained the same, we can conclude that physicians had successfully reduced the number of “unnecessary” FNA biopsies; therefore, their efforts to overcome the thyroid cancer overdiagnosis were not wasted. Moreover, a further increase in the diagnostic performance of thyroid cancer screening could prevent several patients from developing procedural-related complications. In addition to pain or discomfort, FNA procedures can cause hemorrhage or hematoma, which can be severe enough to cause airway obstruction, cervical neuritis, pseudoaneurysm, and serious complications such as infection, recurrent laryngeal nerve palsy, and dysphagia [13]. Therefore, physicians must be aware of the potential complications of thyroid FNA and carefully weigh the individuals’ risks and benefits before recommending it to the patient. In this study, the patient’s benefit from undergoing FNA biopsies is expected to eventually outweigh the risk if the diagnostic efficiency of FNA continues to improve in the future.

This study has several strengths. First, the study was conducted in a large teaching-associated tertiary hospital in South Korea, where a large volume of thyroid cancer screening procedures and FNA biopsies are routinely performed. Thyroid nodule imaging, biopsy, and reporting skills of well-trained radiologists and pathologists have consistently improved over time. Second, since the study included only asymptomatic thyroid nodules from a large-scale HPC, physicians and patients’ factors leading to overdiagnosis are mitigated to some extent. For example, the physicians’ or patients’ desire to avoid missing a diagnosis or widen disease definitions (e.g., lower thresholds for FNA indication) are not easily perceived [14] because two or more experts usually review the report and patients often revisit the center 1 or 2 years later to undergo another cancer screening. Therefore, the study setting is unique and substantial, as it has shown an increase in the diagnostic performance of thyroid cancer screening amidst all the problems related to overscreening.

This study has several limitations. First, it is difficult to evaluate whether the revised K-TIRADS was properly introduced and accurately applied in the actual clinical practice. Many radiologists have started reporting the K-TIRADS in 2016, but some are still reluctant to do so. Further study is necessary to determine whether the overall percentage of sonographic results that reported the K-TIRADS has increased since 2016. Matching the reported K-TIRADS with that retrieved from an automatic computational algorithm that incorporates the actual KTIRADS using natural language processing is also required to confirm the appropriate use of the guidelines. Second, the histopathological features were not determined during the surgical interventions (e.g., thyroidectomy) to confirm the accuracy of FNA biopsies. The overestimation of the diagnostic power could be further confirmed by matching the surgical and FNA biopsy reports. Third, the diagnostic performance of FNA biopsies may have been underestimated, owing to the exclusion of “indeterminate” nodules. Reports containing words such as “in favor of” or “suggestive of” are likely to include a cancer diagnosis. Finally, the intra-observer variations among radiologic and histopathologic results obtained by radiologists and pathologists were not controlled. Technological advances and the training level of physicians may have influenced the efficiency of thyroid cancer screening. The imaging resolution of thyroid ultrasound has gradually improved as well as the skills of sonographers and FNA performers, owing to the increasing number of screening cases each year [15].

In conclusion, since the era of the thyroid cancer epidemic, the overuse of diagnostic tools, such as FNA, has been effectively resolved, while the overuse of thyroid ultrasonography remains a chronic issue due to the excessive number of cancer screening procedures performed. Fortunately, the diagnostic performance of FNA gradually increased, most prominently in 2016, as the number of FNA biopsies and thyroid cancer diagnoses decreased in the last decade. As a positive consequence of K-TIRADS revision, the performance of “fewer FNA biopsies” might have reduced the “unnecessary” FNA biopsies alone and therefore caused less harm to patients. Such a trend could have been achieved not only by the emergence of new guidelines but also by the media, the advancements in technology, and the performance of sonographers and radiologists. The synergistic effect of the abovementioned possibilities may have optimistically helped South Korea overcome the issue of overdiagnosis and resolved the current and future medical burden on thyroid cancer screening.