Differences among the Thyroid Imaging Reporting and Data System proposed by Korean, the American College of Radiology and the European Thyroid Association in the diagnostic performance of thyroid nodules.

The aim of this study was to evaluate the diagnostic performance of the Thyroid Imaging Reporting and Data System (TIRADS) in the forms proposed by Kwak (K-TIRADS), the American College of Radiology (ACR-TIRADS) and the European Thyroid Association (EU-TIRADS). A total of 846 thyroid nodules were evaluated by K-TIRADS, ACR-TIRADS and EU-TIRADS. All the ultrasound data were analyzed and classified according to the criteria of the three systems. In addition, we calculated the risk of malignancy and plotted receiver operating characteristic (ROC) curves. Moreover, the diagnostic efficacy for malignancy were compared. Of the 846 thyroid nodules, 316 were malignant nodules and 530 were benign nodules. The areas under the ROC curves of K-TIRADS (0.827) and ACR-TIRADS (0.817) were not significantly different (P=0.2425); however, they were greater than that of EU-TIRADS (0.758) (P=0.000). The sensitivity of K-TIRADS (94.94%) was higher than that of ACR-TIRADS (61.08%) or EU-TIRADS (58.86%) (P=0.000), while ACR-TIRADS (89.62%) and EU-TIRADS (83.21%) had higher specificity than K-TIRADS (50.75%) (P=0.000). Although all the K-TIRADS, ACR-TIRADS and EU-TIRADS have values in risk stratification for thyroid nodules, they cannot simultaneously achieve high sensitivity and high specificity. Further research should be performed to develop a TIRADS which is more suitable for the population of China. 2020 Translational Cancer Research. All rights reserved.

Introduction

The reported incidence of thyroid cancer (TC) has increased gradually in recent years around the world, and this type of neoplasm is currently the eighth most frequent cancer in China (1). Although the overall prognosis is optimistic, the 5-year survival rate of TC (84.3%) is still lower in China than in developed countries such as the United States (98%) (1); accurate and efficient diagnosis of thyroid nodules is critical for prognostic assessment and treatment strategies determination. As an initial imaging modality, ultrasonography (US) is the first choice for thyroid examination and it can distinguish benign and malignant nodules (2). Ultrasound features like solid, hypoechoic or markedly hypoechoic, irregular or ill-defined margin, taller than wide, microcalcification and extra-thyroidal extension are associated with malignancy. US-guided fine needle aspiration (US-FNA) is the most valuable method for diagnosing thyroid nodules and can reduce unnecessary surgery. Therefore, in order to better identify benign and malignant nodules and optimize the clinical management, a variety of thyroid imaging reporting and data system (TIRADS) were developed in the past decades.

TIRADS was first established by Horvath et al. in 2009 (3), and it was complex and involved in more than 10 patterns of ultrasound features. After that, the TIRADS was further proposed and optimized by Park et al. and Kwak et al. (K-TIRADS) with only five suspicious US features and led to the TIRADS widely used in clinic (4,5). In order to establish a risk stratification system for thyroid nodules, the American College of Radiology (ACR) developed a TIRADS (ACR-TIRADS) (6) based on the breast imaging reporting and data system. And more recently, the European Thyroid Association published its TIRADS for thyroid nodules (7).

At present, many versions of TIRADS, such as K-TIRADS, ACR-TIRADS and EU-TIRADS are used in China, and there have been many studies on K-TIRADS, ACR-TIRADS and EU-TIRADS, however, the differences of diagnostic value among them is not clear. The purpose of this study was to analyze the diagnostic efficacy of the K-TIRADS, ACR-TIRADS and EU-TIRADS, and to provide clinical evidence for the establishment of suitable TIRADS for the population of China.

Methods

Patients

The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). This study was approved by Huadong hospital’s Clinic Institutional Review Board (No.: 2014K058), and all patients signed written informed consent. From January 2016 to December 2017, a total of 1,195 thyroid nodules in 1,087 patients who underwent US and US-FNA or surgery were collected in this study. The cytological results of US-FNA were classified into the six categories according to the Bethesda System for Reporting Thyroid Cytopathology: Bethesda I, Bethesda II, Bethesda III, Bethesda IV, Bethesda V and Bethesda VI (8). The inclusion criteria were as follows: (I) maximum diameter of the nodule was equal to or greater than 10 mm as measured by US; (II) histologic results available regardless of US diagnosis or cytological report; (III) nodules were considered benign by US or classified as Bethesda II by cytological reports, and no increase in nodule size and no change in features at follow-up US for at least one year. An increase in size was defined as a 20% increase in one dimension of nodules with a minimal increase of 2 mm in solid nodules or in the solid portion of mixed cystic solid nodules (6). Patients with the following criteria were excluded: (I) nodules were considered by US without follow-up or US-FNA and nodules were classified as Bethesda II without follow-up; (II) the size of nodules were increased on follow-up; (III) nodules defined by Bethesda I and III were not subjected to repeat US-FNA or surgery; (IV) nodules defined by Bethesda IV, V and VI were not subjected to surgery.

Ultimately, 846 nodules in 839 patients (594 women and 245 men; mean age, 54 years; age range, 14 to 88 years) were enrolled in this study. shows the flowchart of selection of patients and nodules in this study.

Figure 1

The flowchart of patients and nodules. N, number of patients; n, number of nodules; US-FNA, US guided fine needle aspiration.

Ultrasound examination and image review

All US examinations were performed using an Aplio 500 system (Toshiba Medical Systems Corporation, Tokyo, Japan) equipped with a 5–14 MHz high frequency linear array transducer. The scanning protocol in all cases included both transverse and longitudinal real-time imaging of thyroid nodules. In order obtain optimal images, the target nodule was placed at the center of the screen and the machine settings were constantly adjusted. If needed, the position of the probe was marked on the image to help to determine if the mass was on the right or left the thyroid gland. Nodule size was recorded based on the nodule’s largest dimension on US. Images were saved in local systems, and the characteristics of the thyroid nodules were recorded.

The characteristics of nodules that were documented were number of nodules, size, location (left, right, isthmus), composition (solid, cystic, mixed), spongiform (yes, no), echogenicity (anechoic, hyperechoic or isoechoic, hypoechoic, markedly hypoechoic), Orientation (wider-than-tall, taller-than-wide), margins (well defined, ill-defined, lobulated or irregular), extra-thyroidal extension (yes, no), calcification (none, micro-calcifications, macro-calcifications, peripheral-calcification), halo (none, thin halo, thick halo), vascularization (avascular, hypovascular, hypervascular or penetrating vessel) and cervical lymph node (normal, lymphadenopathy).

Two thyroid radiologists with 8 and 10 years of clinical experience in performing and evaluating thyroid US data reviewed all of the US images independently while blinded to patient information and reached a consensus. All thyroid nodules were assessed on the basis of the K-TIRADS, ACR-TIRADS and EU-TIRADS, respectively. Examples of US images of thyroid nodules classified based on the three risk stratification systems are shown in .

Figure 2

US scans illustrate ACR-TIRADS, K-TIRADS and EU-TIRADS categories. (A) K-TIRADS Category 2; ACR-TIRADS category 1; EU-TIRADS Category 2. (B) K-TIRADS Category 3; ACR-TIRADS category 2; EUTIRADS Category 3. (C) K-TIRADS Category 4a; ACR-TIRADS category 3; EU-TIRADS Category 3. (D) K-TIRADS Category 4b; ACR-TIRADS category 4; EU-TIRADS Category 4. (E) K-TIRADS Category 4c; ACR-TIRADS category 4; EU-TIRADS Category 5. (F) K-TIRADS Category 5; ACR-TIRADS category 5; EU-TIRADS Category 5. TIRADS, Thyroid Imaging Reporting and Date System; ACR, American College of Radiology; K, Kwak; EU, the European Thyroid Association.

US-FNA and cytological classification

In our institution, US-FNA was performed in thyroid nodules with US features indicating a high risk for malignancy, and surgery was indicated based on cytological results malignant. US-FNA was performed by the same radiologists after the nodules were evaluated with a 23-gauge needle attached to a 5-mL syringe. Each lesion was aspirated at least three times. Materials obtained from aspiration biopsy were expelled onto glass slides and smeared. In our institution, two experienced pathologists read all the samples.

The cytological results of the US-FNA were broadly classified according to the Bethesda System for Reporting Thyroid Cytopathology (7) into the following categories: Bethesda I, nondiagnostic or unsatisfactory; Bethesda II, benign; Bethesda III, atypia of undetermined significance or follicular lesion of undetermined significance; Bethesda IV, follicular neoplasm or suspicious for a follicular neoplasm; Bethesda V, suspicious for malignancy; Bethesda VI, malignant.

Statistics

Statistical analyses were performed with SPSS software (version 22.0) and MedCalc software (version 15.2.2). The results of categorical data were expressed using frequencies and percentages, and baseline continuous data were expressed as the mean ± standard deviation. Student’s t-test was used to compare continuous variables and the χ2 test or Fisher’s exact test was used to compare categorical variables. The association between the categories of each guideline and the final diagnosis was evaluated with the Cochran-Mantel-Haenszel test. A receiver operating characteristic (ROC) curve analysis was used to compare the diagnostic performance of K-TIRADS, ACR-TIRADS, and EU-TRADS. The areas under the ROC curves (AUCs) of K-TIRADS, ACR-TIRADS, and EU-TRADS were compared using a Z test. The best cut-off levels for sensitivity and specificity were evaluated from the ROCs. Two-tailed P values of <0.05 were considered statistically significant.

Results

Characteristics and pathological findings of 846 nodules

Of the 846 thyroid nodules, 530 (62.6%) were diagnosed as benign and 316 (37.4%) were diagnosed as malignant. The final results of each nodule are presented in . Among the 530 benign nodules, there were 323 (38.2%) benign nodules with follow-up, 156 (18.4%) Nodular goiter hyperplastic nodules, 32 (3.8%) adenomas, 14 (1.7%) Hashimoto’s thyroiditis, and 5 (0.6%) subacute thyroiditis. Among the 316 malignant nodules, there were 304 (35.9%) papillary carcinomas, 8 (0.9%) follicular carcinomas and 4 (0.5%) medullary carcinomas.

Table 1

Final results of the 846 thyroid nodules

Nodule nature	Final results	Number (%) of cases
Benign	Benign nodules with follow-up	323 (38.2)
	Nodular goiter hyperplastic nodules	156 (18.4)
	Adenoma	32 (3.8)
	Hashimoto’ disease	14 (1.7)
	Subacute thyroiditis	5 (0.6)
	Subtotal	530 (62.7)
Malignant	Papillary thyroid carcinoma	304 (35.9)
	Follicular thyroid carcinoma	8 (0.9)
	Medullary thyroid carcinoma	4 (0.5)
	Subtotal	316 (37.3)
Total		846 (100)

Patient demographics and sonographic findings of all nodules

shows the patient demographics and US findings of all nodules based on the final results. The median size of the thyroid nodules was 22 mm (14–32 mm). The benign nodules were significantly larger than the malignant nodules [median size, 26 mm (18–36 mm) vs. 14 mm (11–22 mm), respectively; P=0.000]. Significant differences were found in terms of patient age, composition, spongiform, echogenicity, shape, orientation, margin, calcification, halo, extra-thyroidal extension, and suspicious lymph nodes and vascularization between benign and malignant nodules (P<0.05). Conversely, patient gender and nodule location did not achieve significant differences (all P>0.05).

Table 2

Patient demographics and sonographic findings of all nodules included in this study

Characteristics	Overall	Benign	Malignant	P value
Number of nodules (%)	846 (100)	530 (62.65)	316 (37.35)
Age (years)	53±14	56±13	49±14	0.000
Gender, n (%)				0.39
Male	245 (29.2)	148 (28.3)	97 (30.7)
Female	594 (70.8)	375 (71.7)	219 (69)
Size (mm)	22 [14–32]	26 [18–36]	14 [11–22]	0.000
Location				0.363
Left	441 (52.1)	267 (50.4)	174 (55.1)
Right	388 (45.9)	253 (47.7)	135 (42.7)
Isthmus	17 (2.0)	10 (1.9)	7 (2.2)
Composition, n (%)				0.000
Solid	593 (70.1)	298 (56.2)	295 (93.4)
Cystic	95 (11.2)	94 (17.7)	1 (0.3)
Mixed cystic and solid	150 (17.7)	130 (24.5)	20 (6.3)
Spongiform, n (%)				0.029
Yes	8 (0.9)	8 (1.5)	0(0)
No	838 (99.1)	522 (98.5)	316 (100)
Echogenicity, n (%)				0.000
Anechoic	102 (12.1)	101 (19.1)	1 (0.3)
Hyperechoic or isoechoic	108 (12.8)	71 (13.4)	37 (11.7)
Hypoechoic	628 (74.2)	355 (42)	273 (86.4)
Markedly hypoechoic	8 (0.9)	3 (0.5)	5 (1.6)
Orientation, n (%)				0.000
Wider-than-tall	828 (97.9)	529 (99.8)	299 (94.6)
Taller-than-wide	18 (2.1)	1 (0.2)	17 (5.4)
Margin, n (%)				0.000
Well	482 (57.0)	334 (63.0)	148 (46.8)
Ill-defined	181 (21.4)	125 (23.6)	56 (17.8)
Lobulated/irregular	90 (10.6)	28 (5.3)	62 (19.6)
Extra-thyroidal extension, n (%)				0.001
Yes	93 (11.0)	43 (8.1)	50 (15.8)
No	753 (89.0)	487 (91.9)	266 (84.2)
Calcification, n (%)				0.000
None	476 (56.3)	389 (73.4)	87 (27.5)
Macro-calcification	147 (17.4)	65 (12.3)	82 (25.9)
Micro-calcification	217 (25.7)	62 (11.7)	155 (49.1)
Peripheral-calcification	18 (2.1)	14 (2.6)	4 (1.3)
Halo, n (%)				0.000
None	783 (92.5)	476 (89.8)	307 (97.0)
Thin halo	39 (4.6)	35 (6.6)	4 (1.2)
Thick halo	24 (2.9)	19 (3.6)	5 (1.8)
Vascularization, n (%)				0.000
Avascular	347 (41.0)	217 (40.9)	130 (41.1)
Hypovascular	354 (41.8)	243 (45.8)	111 (35.1)
Hypervascular or penetrating vessel	145 (17.1)	70 (13.3)	75 (23.8)
Cervical lymph node, n (%)				0.000
Normal	792 (93.6)	522 (98.5)	270 (85.4)
Lymphadenopathy	54 (6.4)	8 (1.5)	46 (14.6)

Malignancy risk assessed by the three guidelines

lists the results of a comparison of malignancy risk according to each category in the three TIRADS. The three TIRADS proposed an estimated risk of malignancy in each category. All of them were well matched within the range of the recommended malignancy risk except for the 4a, 4b and 4c in the K-TIRADS and the TR3 and TR4 in the ACR-TIRADS and the EU3 and EU4 in the EU-TIRADS. The malignancy rates tended to increase along with the higher risk categories (P<0.001).

Table 3

Malignant rates in the categories of ACR, EU and Kwak TIRADS

TIRADS	Total (%) (n=846)	Final diagnosis		Recommended malignancy risk (%)	Calculated malignancy rate (%)	P value
		Benign (n=530)	Malignant (n=316)
KWAK						<0.001
2	84 (9.93)	84	0	0	0
3	10 (1.18)	10	0	1.7	0
4a	191 (22.58)	175	16	3.3	8.4
4b	347 (41.02)	216	131	9.2	37.8
4c	172 (20.33)	41	131	44.4–72.4	76.2
5	42 (4.96)	4	38	87.5	90.48
ACR						<0.001
TR1	75 (8.87)	75	0	0	0
TR2	10 (1.18)	10	0	<2	0
TR3	139 (16.43)	131	8	2–5	5.8
TR4	381 (45.04)	259	122	5–20	32
TR5	241 (38.49)	55	186	>20	77.2
EU						<0.001
2	87 (10.28)	87	0	0	0
3	78 (9.22)	61	17	2–4	21.79
4	399 (47.16)	293	106	6–17	26.6
5	282 (33.33)	89	193	26–87	68.4

TIRADS, Thyroid Imaging Reporting and Date System; ACR, American College of Radiology; EU, the European Thyroid Association.

Diagnostic performance in the prediction of thyroid malignancy with the three TIRADS

The ROC curves presented the diagnostic performance in the prediction of thyroid malignancy with the three guideline systems (). Among the guideline systems, there was no significant difference between K-TIRADS (0.827) and ACR-TIRADS (0.817) in the AUC (P=0.2425). However, they both had higher diagnostic performances than the EU-TIRADS (0.758) (both P<0.001). From the ROCs, the best cut-off value of K-TIRADS was category 4a with higher sensitivity (94.94% to 61.08% and 58.86%) than EU-TIRADS or ACR-TIRADS; meanwhile, the best cut-off values of EU-TIRADS and ACR-TIRADS were both category 4, with ACR-TIRADS having higher specificity (89.62% to 83.21% and 50.75%) (All P<0.001) ().

Figure 3

The comparison of the receiver operating characteristics (ROC) curves among K-TIRADS, ACR-TIRADS and EU-TIRADS. TIRADS, Thyroid Imaging Reporting and Date System; ACR, American College of Radiology; K, Kwak; EU, the European Thyroid Association.

Table 4

Diagnostic performance of ACR, EU and Kwak TIRADS

TIRADS	Cutoff	Sensitivity (%)	Specificity (%)	AUC (95% CI)
1KWAK	>4a	94.94	50.75	0.827 (0.800–0.852)
2ACR	>4	58.86	89.62	0.817 (0.789–0.842)
3EU	>4	61.08	83.21	0.758 (0.727–0.786)
P value of 1 vs. 2		0.000	0.000	0.2425
P value of 1 vs. 3		0.000	0.000	<0.0001
P value of 2 vs. 3		0.570	0.002	<0.0001

TIRADS, Thyroid Imaging Reporting and Date System; ACR, American College of Radiology; EU, the European Thyroid Association; AUC, area under the curve.

Discussion

US plays an important role in detecting and diagnosing thyroid nodules. It has been reported that several US features of thyroid nodules are associated with an increased likelihood of malignancy. Taller-than-wide shape, ill-defined margin, marked hypoechogenicity, and micro-calcification were independent risk factors for the PTC (9). Li et al. found that hypoechogenicity and irregular margins combined with solid component or taller-than-wide shape or micro-calcifications have a high predictive value for distinguishing malignant nodules from benign (10). In our study, composition, spongiform, echogenicity, shape, orientation, margin, calcification, halo, extra-thyroidal extension, and suspicious lymph nodes and vascularization were associated with malignant nodules, and solid, no spongiform, markedly hypoechoic, taller-than-wide morphology, lobulated/irregular margin, extra-thyroidal extension, micro-calcification, no halo, hypervascular or penetrating vessel and lymphadenopathy had higher percentages of malignancy. These US features were also included in the K-TIRADS, ACR-TIRADS and EU-TIRADS.

The risk stratification is paramount to appropriate US-FNA for thyroid nodules in clinic (11), however, suspicious US features may be classified in different ways and the size cutoff for US-FNA are not consistent among the TIRADS (4,6,8,12). Thus, it is necessary to validate the risk of malignancy in TIRADS in a large population study. In the present study, the malignancy rates of the thyroid nodules increased with the category in the three TIRADS. The malignancy risks in our study were generally well correlated within the range of the recommended malignancy risk except that category 4a, 4b and 4c in K-TIRADS, category TR3 and TR4 in ACR-TIRADS and category EU3 and EU4 in EU-TIRADS were relatively higher than the recommended malignancy. The overall malignancy rate in our study was also high, at 37.4%. Since this is a retrospective analysis, the patients in this study were highly selected and 61.9% of patients had undergone surgery, which may lead to be classified at a higher risk for malignancy. In addition, some US features like punctate echogenic foci of benign and malignant nodules are not typical, and some US features like macro-calcifications and peripheral calcifications or extra-thyroidal extension are even not defined in K-TIRADS and EU-TIRADS, therefore, such features may lead to the difference and should be taken into consideration when stratifying the thyroid nodules.

Thyroid ultrasound classification systems are divided into three main categories. One is the counting method like K-TIRADS, which is based on the number of characteristics of malignancy. The other is the US patterns method like EU-TIRADS, which classified the thyroid nodules based on the performance patterns on the US. The last one is the weighting method like ACR-TIRADS, which assigns scores to each ultrasound feature to classify the thyroid nodules. In a research of Migda et al., K-TIRADS was very useful to discard the benign cases and to reduce the number of biopsies (13). Skowrońska et al. reported that EU-TIRADS was valuable to assess the risk of malignancy of thyroid nodules (14). Koseoglu Atilla et al. suggested that ACR-TIRADS was an applicable approach to determine thyroid nodules to be biopsied (15). The present study compared the diagnostic performance of K-TIRADS, ACR-TIRADS and EU-TIRADS, and the results found that the K-TIRADS, ACR-TIRADS and EU-TIRADS were all performed well in differentiating thyroid nodules, the K-TIRADS and ACR-TIRADS had the similar highest AUC, whereas the K-TIRADS showed a higher sensitivity and ACR-TIRADS showed a higher specificity. These findings were consistent with a previous study on 2,544 nodules, which has confirmed that ACR-TIRADS were significantly less sensitive and had a higher specificity than K-TIRADS in differentiating nodules >10 mm (2). At present, K-TIRADS, EU-TIRADS and ACR-TIRADS are widely used in different hospitals in China, hence, the criteria and reporting system used to validate and report the results of thyroid nodules imaging may vary among radiologists, therefore, the ambiguity across different institutions poses a substantial burden for the management of thyroid nodules in China. Further research is needed to establish a new TIRADS for accurate and efficient diagnosis of thyroid nodules in Chinese population.

However, there are several limitations to our study. First, this is a retrospective study and the results of our study are from a single institute. All analyses were based on the recorded static images and thus may have led to misdiagnosis. Second, we followed up only a subset of our population. Some thyroid nodules with benign results were diagnosed with follow-up alone, without surgical confirmation, which may have led to bias resulting in the miscalculation of sensitivity, specificity and risk of malignancy for ACR-TIRADS, EU-TIRADS and K-TIRADS. Third, we did not evaluate the nodules with a size <10 mm, and selection bias may have existed. More studies in various organizations are needed in the future to evaluate TIRADS. However, the size thresholds for US-FNA of three TIRADS in our study are larger than 10 mm and we aimed to analyze the risk of malignancy in each TIRADS, thus we evaluated the nodules with a size <10 mm in order to consent with the three TIRADS. Finally, subgroup analysis of patients who went through pathological diagnosis by surgery was not included in this study. The thyroid nodules with pathological results were only TI-RADS 3, 4 and 5, therefore, completely assessment of TI-RADS 1, 2, 3, 4 and 5 in our study maybe feasibility and could ensure the integrity of nodule classification.

Conclusions

In summary, K-TIRADS, ACR-TIRADS and EU-TIRADS all performed well in differentiating thyroid nodules, among them, K-TIRADS and ACR-TIRADS showed a better diagnostic efficiency in differentiating nodules >10 mm, but they cannot simultaneously achieve high sensitivity and high specificity. Therefore, prospective large, and multicenter studies should be carried out in the future in order to obtain suitable TIRADS for accurate diagnosis of thyroid nodules in Chinese population.