Reproducibility of the Bethesda System for Reporting Thyroid Cytopathology: A Methodological Issue.

Dear Editor,

We were interested to read the article by Awasthi et al. published in the March 2018 issue of J Cytol.[1] They aimed to assess the reproducibility of “The Bethesda System for Reporting Thyroid Cytopathology” (TBSRTC) for diagnosing thyroid lesions. Wright Giemsa and Papanicolaou-stained fine-needle aspiration cytology smears along with relevant clinical and radiological details of 107 patients having a non-TBSRTC diagnosis were analyzed by two trained cytopathologists.[1] The interobserver variation and reproducibility of the reporting system was assessed by Cohen's kappa coefficient.[1]

First of all, it is very important to know that there is no value of kappa that can be regarded universally as an indication of good agreement. Two important weaknesses of kappa to assess agreement of a qualitative variable are as follows.First, it depends on the prevalence in each category, which means it can be possible to have different k values having the same percentage for both concordant and discordant cells. Table 1 shows that in both (a) and (b) situations, the prevalence of concordant cells is 90% and of discordant cells is 10%; however, we get different kappa values (0.44 as moderate and 0.80 as very good, respectively). Kappa value also depends on the number of categories.[23456] In this study, since we face a five-state variable, it is better to apply weighted kappa to avoid misinterpretation.

Table 1

Comparison of two cytopathologists’ diagnoses with different prevalence in the two categoriesa

Situation (a)	Cytopathologist 1
		Positive	Negative	Total (%)
Cytopathologist 2	Positive	85	5	90
	Negative	5	5	10
k=0.44moderate	Total	90	10	100
Situation (b)	Cytopathologist 1
		Positive	Negative	Total (%)
Cytopathologist 2	Positive	45	5	50
	Negative	5	45	50
k=0.80very good	Total	50	50	100

aAuthor’s own hypothetical data to demonstrate the limitation of the kappa value to assess reproducibility Courtesy: Siamak Sabour, Tehran, Iran

They concluded that TBSRTC is a distinctive, universal, and homogeneous terminology for reporting thyroid cytology, and its execution should be promoted because of its virtual ease of reproducibility. Due to the above-mentioned limitations of kappa value to assess reproducibility, such a conclusion may be a misleading message.[23456]

In this letter, we discussed limitations of kappa value to assess reproducibility. Therefore, any conclusion especially in clinical practice should be supported by the above-mentioned statistical and methodological issues. Otherwise, mismanagement of the patients cannot be avoided.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.