Validation of the local applicability of the 'TIB' Olfactory Test Device in the era of COVID-19.

OBJECTIVE: To evaluate the validity and test-retest reliability of the novel 'TIB' Olfactory Test Device (TIB) and to determine its normative values.
METHODS: The study stratified the study subjects into normosmic, hyposmic and anosmic groups according to their olfactory function. The olfactory function of the subjects was evaluated using both the traditional Chinese version of the University of Pennsylvania of Smell Identification Test (UPSIT-TC) and the TIB. The normosmic group was used to retest with the UPSIT-TC and TIB at an inter-test interval of at least 7 days. The cut-off scores of TIB among the three different groups were determined by receiver operating characteristic curve analysis.
RESULTS: This study enrolled 180 subjects: 60 in each group. The mean scores of TIB were 44.1 for the normosmic group, 27.5 for the hyposmic group and 10.9 for the anosmic group. The TIB scores were significantly different across the three groups. There was a significant correlation between the first and second TIB tests (r = 0.506). The cut-off scores were 41 for normosmic subjects and 24 for hyposmic subjects.
CONCLUSION: The validity and test-retest reliability results suggest that the TIB is an appropriate olfactory test for the Taiwanese population.

Introduction

The University of Pennsylvania Smell Identification Test (UPSIT), which was developed at the University of Pennsylvania, is one of the most reliable and the most widely used olfactory test.[1,2] It is available commercially and has been administered more than half a million times. UPSIT can be self-administered without the assistance of a test examiner. The test consists of four 10-odorant booklets. Each odorant is embedded in 10 to 50 µm urea-formaldehyde polymer microencapsules fixed in a proprietary binder and positioned on a brown strip that is located at the bottom of each page of the test booklet. When the examinee takes the test, each of the 40 odorants is released by scratching the strip with a pencil tip. The released odorant is sniffed and the examinee identifies the odorant by choosing a name from a set of 4 odour descriptors. The UPSIT results are strongly correlated with those of odour threshold tests. Therefore, the development of UPSIT has allowed for accurate and convenient testing of olfactory function without the use of complex olfactometric equipment, cumbersome bottles or pen-like devices.

While UPSIT has allowed for accurate and convenient testing of olfactory function, it has been shown that cultural factors can influence test scores on odour identification tests. The odorants and the response items have been modified in a number of foreign language versions to make the test scores more congruent with North American norms.[2,6,7] The traditional Chinese version of the University of Pennsylvania Smell Identification Test (UPSIT-TC) has been developed for applicability in Taiwan by replacing eight odorants and changing two response items. For example, the odorant of dill pickle is replaced by jasmine since dill pickle is not a common daily food in Taiwan. The validity, reliability and olfactory diagnosis of UPSIT-TC were confirmed in our previous studies.[4,8,9] Although UPSIT-TC has been widely used to evaluate olfactory function in Taiwan,[10,11] it needs to be imported from the USA and is expensive.

The coronavirus disease 2019 (COVID-19) pandemic currently remains a major global health crisis. Olfactory dysfunction is a common presentation in patients with COVID-19. Hyposmia has been suggested as a potentially reliable indicator of mild COVID-19 and is being used in screening for COVID-19.[13,14] Therefore, a cheap, accurate, convenient and self-administered olfactory test is needed to test the olfactory function of patients that are diagnosed with COVID-19 infection or suspected to be infected with the severe acute respiratory syndrome coronavirus 2.

Recently, a new smell identification test, the ‘TIB’ Olfactory Test Device (Top International Biotech, Taipei) has been developed (Figure 1). It consists of 16 tests with an odorant embedded in fragrant microcapsules positioned on a strip. As in the UPSIT-TC, the examinee scratches the strip to release the odorant. The released odorant is sniffed and the examinee identifies the odorant by choosing a name from a set of four odour descriptors. This process is then repeated with the next odorant on the strip. Although its normative values have been determined using the tenth percentile values of healthy volunteers, its validity, reliability and olfactory diagnosis have not been established. This current study investigated its validity, reliability and olfactory diagnosis; and explored the influence of age and sex on the results.

Figure 1.

The ‘TIB’ Olfactory Test Device used in a study to compare the ‘TIB’ Olfactory Test Device with the traditional Chinese version of the University of Pennsylvania Smell Identification Test in subjects (n = 180) with different olfactory functions.

Subjects and methods

Study subjects

This prospective study recruited consecutive male and female subjects between November 2020 and August 2021 at the ENT outpatient clinic of the Taichung Veterans General Hospital, Taichung. The study participants were assigned to one of three groups based on their olfactory function: a normosmic group, a hyposmic group with partial loss of olfactory function and an anosmic group with complete loss of olfactory function. There were equal numbers of male and female subjects in each of the three groups. The inclusion criteria for the normosmic group were healthy subjects that were 20–59 years old without a history of sinonasal symptoms within 1 week before the test and self-reported absence of deficits in olfaction. The inclusion criteria for the hyposmic group were subjects aged 20–59 years with a loss of olfactory function with a phenyl ethyl alcohol (PEA) odour detection threshold below –1. The inclusion criteria for the anosmic group were subjects aged 20–59 years with a loss of olfactory function with a PEA threshold equal to –1.

All subjects were measured for olfactory function using both the UPSIT-TC and TIB Olfactory Test Device (TIB). In each group, subjects received the TIB and UPSIT-TC in a random order. The normosmic group received a second TIB and UPSIT-TC 1 week later to evaluate test–retest reliability. This study was approved by the Institutional Review Board (I) of Taichung Veterans General Hospital (TCVGH-IRB No. CE20329A). Written informed consent was obtained from each study subject. The clinical trial was registered with Clinicaltrial.gov (registration identifier: NCT05152030).

Olfactory tests

The PEA odour detection threshold test consists of sniff bottles containing a rose-like PEA odorant with concentrations ranging from 10−1 to 10−9 log vol/vol in half-log concentration steps and pure mineral oil. A 2-alternative forced-choice single-staircase procedure is used to measure the odour threshold. Two sniff bottles that contain PEA dissolved in mineral oil or mineral oil alone are opened and placed under the subject’s nose in a random order. The subject chooses which bottle contains the stronger odour. If he/she cannot choose, a guess is required. The test begins with a bottle containing PEA odorant at 10−6 log vol/vol. Correct identification of the bottle that contains the PEA odorant in five successive trials triggers a reversal of the staircase to the next lower concentration, whereas a single incorrect identification triggers the reversal of the staircase to the next higher concentration. When a total of seven reversals are acquired, the test is finished. The geometric mean of the last four reversed concentrations is used as the PEA threshold estimate.

The UPSIT-TC consists of 40 tests. In each test, an odorant is embedded in 10-μm to 50-μm microcapsules fixed in a proprietary binder and positioned on the brown bottom strips of the test page. The subject releases each odorant by scratching the strip with a pencil tip. Then the subject sniffs the released odorant and identifies the odorant by choosing a name from a set of four odour descriptors. The test is scored as the number of odours identified correctly. A guess is required for each test even if no odour is perceived. Hence, the maximum score is 40 for 40 tests.

The TIB consists of 16 tests. Each test contains an odorant and two questions. The first eight tests are the same as the second eight tests but in a different order (Figure 2). Each odorant is embedded in fragrance microcapsules positioned on a “scratch-and-sniff” strip. The fragrance microcapsules are composed of melamine, formaldehyde and fragrant oil, which are combined using condensation polymerization. At the beginning of each test, the subject scratches the strip with a pencil and smells the odorant released from the microcapsules. After the subject sniffs the odour, he/she identifies the odorant by choosing a name from a set of four odour descriptors, and scores 1 point if the answer is correct. He/she then answers the next question, for which there are three possible responses. The first is ‘not detectable’, which means the subject smells nothing at all (scores 0 points), the second is ‘detectable, but not sure’, which means the subject smells something but is unsure (scores 1 point), and the third is ‘detectable’, which means the subject smells and knows exactly what the odour is (scores 2 points). For example, a subject scores 1 point for correctly identifying the odorant and an additional 2 points if he/she chooses ‘detectable’. If the subject does not identify the odorant correctly, but chooses ‘detectable, but not sure’, he/she gets 1 point. However, if the subject chooses ‘detectable’, but does not identify the odorant correctly, he/she scores 0 point. Hence, the maximum score is 48 points for the 16 tests.

Figure 2.

Answer sheet for the ‘TIB’ Olfactory Test Device used in a study to compare the ‘TIB’ Olfactory Test Device with the traditional Chinese version of the University of Pennsylvania Smell Identification Test in subjects (n = 180) with different olfactory functions.

Statistical analyses

All statistical analyses were performed using IBM SPSS Statistics for Windows, Version 22.0 (IBM Corp., Armonk, NY, USA). The age, UPSIT-TC score and TIB score of the three groups were compared using Kruskal–Wallis test to demonstrate any statistical difference between the three groups. Mann–Whitney U-test was applied for post-hoc analyses of between-group differences. The UPSIT-TC score and TIB score were compared between male and female subjects and between those aged 20–39 years and 40–59 years in each group using Mann–Whitney U-test. Spearman’s correlation coefficient was used to measure the strength and direction of any association between the UPSIT-TC test and TIB test and to evaluate the validity of the test. The test–retest reliability was also evaluated using Spearman’s correlation coefficient and a Bland–Altman plot. A Bland–Altman plot is a graphical method to show the agreement between the first and second TIB tests in the normosmic group. The cut-off point between the three groups was determined based on the receiver operating characteristic (ROC) curve. The area under the ROC curve was used to represent the cut-off score for discriminating between the different groups. The values between 1 and 0.9 indicate outstanding discrimination, and those between 0.8 and 0.9 indicate excellent discrimination. A sample size calculation was undertaken according to previous published data to determine the mean and standard deviation values for the TIB and UPSIT-TC tests.[9,15] At least 43 subjects in each group were needed to show significance. Two-tailed P-values <0.05 were considered statistically significant.

Results

This study recruited a total of 180 male and female subjects. Among these participants, 60 healthy subjects were assigned to the normosmic group, 60 subjects with partial loss of olfactory function were assigned to the hyposmic group and 60 subjects with complete loss of olfactory function were assigned to the anosmic group. Table 1 shows the demographic characteristics of all included subjects. Of the 180 study subjects, their ages ranged from 22 to 50 years with a mean ± SD of 35.2 ± 7.5 years in the normosmic group, from 20 to 59 years with a mean ± SD of 41.5 ± 10.7 years in the hyposmic group, and from 20 to 59 years with a mean ± SD of 41.6 ± 11.3 years in the anosmic group. The study subjects were significantly younger in the normosmic group compared with the hyposmic and anosmic groups (normosmic versus hyposmic, P < 0.001; normosmic versus anosmic, P = 0.001).

Table 1.

Demographic characteristics of subjects (n = 180) with three different olfactory functions that were included in a study to compare the ‘TIB’ Olfactory Test Device with the traditional Chinese version of the University of Pennsylvania Smell Identification Test.

	Normosmic group n = 60	Hyposmic group n = 60	Anosmic group n = 60	Statistical analysesa
Male	30	30	30
Female	30	30	30
Age 20–39 years	42	25	26
Age 40–59 years	18	35	34
Age, years	35.15 ± 7.51	41.53 ± 10.67	41.55 ± 11.26	P < 0.0001

Data presented as n of subjects or mean ± SD.

aBetween-group comparisons undertaken using Kruskal–Wallis test.

The TIB and UPSIT-TC scores are shown in Table 2. The mean TIB score was 44.1 for the normosmic group, 27.5 for the hyposmic group and 10.9 for the anosmic group. There was a significant difference in TIB scores between the normosmic group and the hyposmic group (P < 0.001) and between the hyposmic and anosmic groups (P < 0.001). When TIB scores were compared between male and female subjects and between the 20–39 years age group and the 40–59 years age group, no significant differences were found in any of the three groups. There was no correlation between the TIB and UPSIT-TC scores in the normosmic group (r = 0.018), but a significant correlation was observed in the hyposmic group (r = 0.808; P < 0.001) and the anosmic group (r = 0.677; P < 0.001). According to previous literature on the strength of correlation coefficients, there was strong correlation between the TIB and UPSIT-TC scores for hyposmic subjects and a moderate correlation for anosmic subjects.

Table 2.

The scores for the ‘TIB’ Olfactory Test Device (TIB) and the traditional Chinese version of the University of Pennsylvania Smell Identification Test (UPSIT-TC) in subjects (n = 180) with three different olfactory functions.

Score	Normosmic group n = 60	Hyposmic group n = 60	Anosmic group n = 60	Statistical analysesa
TIB	44.1 ± 3.6 (60)	27.5 ± 11.4 (60)	10.9 ± 7.2 (60)	P < 0.0001
UPSIT-TC	31.0 ± 4.7 (60)	20.1 ± 6.7 (60)	13.2 ± 4.3 (60)	P < 0.0001
TIB
Male	43.4 ± 4.2 (30)	26.0 ± 10.1 (30)	10.1 ± 7.7 (30)
Female	44.7 ± 2.8 (30)	29.0 ± 12.6 (30)	11.7 ± 6.8 (30)
Statistical analysesb	NS	NS	NS
UPSIT-TC
Male	29.3 ± 4.9 (30)	18.5 ± 5.3 (30)	12.3 ± 4.2 (30)
Female	32.6 ± 3.8 (30)	21.6 ± 7.6 (30)	14.0 ± 4.3 (30)
Statistical analysesb	P = 0.006	NS	NS
TIB
Age 20–39 years	44.2 ± 3.4 (42)	27.5 ± 11.0 (25)	9.7 ± 6.2 (26)
Age 40–59 years	43.6 ± 4.0 (18)	27.5 ± 11.8 (35)	11.8 ± 7.9 (34)
Statistical analysesb	NS	NS	NS
UPSIT-TC
Age 20–39 years	30.8 ± 4.6 (42)	19.3 ± 7.1 (25)	12.0 ± 3.9 (26)
Age 40–59 years	31.4 ± 4.8 (18)	20.6 ± 6.4 (35)	14.0 ± 4.5 (34)
Statistical analysesb	NS	NS	NS

Data presented as mean ± SD (subject number).

aBetween-group comparisons undertaken using Kruskal–Wallis test.

bBetween-sex or age group comparisons undertaken using Mann–Whitney U-test; NS, no significant between-group difference (P ≥ 0.05).

Regarding the test–retest results, the TIB score did not differ significantly between the first (mean ± SD = 44.1 ± 3.6) and second (mean ± SD = 44.7 ± 4.4) tests and was significantly correlated between the first and second tests (r = 0.506; P < 0.001). For the UPSIT-TC, the score of the first test (mean ± SD = 31.0 ± 4.7) was significantly different from but correlated with that of the second test (mean ± SD = 29.7 ± 3.7; P = 0.004) (r = 0.744; P < 0.001). Figure 3 shows the Bland–Altman plot of the first and the second TIB tests.

Figure 3.

Bland–Altman plot of the scores of the first and second tests of the ‘TIB’ Olfactory Test Device in the normosmic group (n = 60).

The ROC curve was used to determine the cut-off scores between the normosmic and hyposmic groups and between the hyposmic and anosmic groups. The cut-off score was 41 between the normosmic and hyposmic groups and the area under the curve was 0.94 with a sensitivity of 86.7% and specificity of 86.7%. The cut-off score was 24 between the hyposmic and anosmic groups and the area under the curve was 0.88 with a sensitivity of 96.7% and specificity of 71.7%.

The individual test–retest change was determined by a 95% confidence interval. For the TIB, the mean ± SD of the first and second tests was 44.0 ± 3.3. Therefore, an individual test–retest change was a change of 6 for the TIB test.

Discussion

The UPSIT is an odour identification test. Odour identification ability has been found to be influenced by several factors such as age and sex. Therefore, sex and age should be taken into account when making an olfactory diagnosis using an odour identification test. Although olfactory diagnosis for adults has been developed for UPSIT, the cut-off scores were chosen using percentile scores of healthy subjects. When the normative data for the “Sniffin’ Sticks” were established based on a group of 3282 healthy subjects, the tenth percentile was used to separate hyposmia from normosmia.

The TIB is a new odour identification test that was developed in Taiwan. It consists of 16 tests. In the UPSIT, the subject needs to make a guess about the odorant even if he/she does not perceive any odour after releasing and sniffing the odorant. As with the UPSIT, the TIB also requires the subject to make a guess if he/she does not perceive any odour, but the subject is asked to answer a further question to indicate how sure he/she is about the odorant. This question gives physicians additional information about the test results. At present, the provided cut-off scores are also determined using the tenth percentile.

In the current study, the TIB scores showed a strong correlation with the UPSIT-TC scores in the hyposmic group and a moderate correlation in the anosmic group. Therefore, TIB had good validity for differentiating normosmic subjects from hyposmic subjects and for differentiating hyposmic subjects from anosmic subjects. These results were comparable to those obtained using the UPSIT-TC. TIB also demonstrated good reliability based on the test–retest results, consistent with the UPSIT-TC. When the results of TIB and UPSIT-TC were correlated, a significant correlation existed for hyposmic and anosmic subjects but not for the normosmic subjects. A similar phenomenon was found in a previous study. The results of the two different devices for the PEA odour detection threshold test, the Smell Threshold Test and the Snap & Sniff Threshold Test were correlated, and a significant correlation existed for hyposmic and anosmic patients, though not the normosmic subjects. This might be partly explained by the definition of the normosmic subjects in the current study, which were healthy volunteers that self-reported an absence of olfactory deficits without a history of sinonasal symptoms within 1 week before the test. Their olfactory function was not examined to confirm whether they were normosmic using a PEA test because it needs the examinee to take off their facemask in front of the examiner, which is not recommended during the current COVID-19 pandemic status.

To establish the olfactory diagnosis for TIB, the effect of sex and age on the results of TIB were examined. There were no significant differences in the TIB scores between male and female subjects or between the 20–39 years age group and the 40–59 years age group in all three groups. This might have been due to the small number of subjects in each group. However, a previous study reported that when the UPSIT and PEA odour detection threshold tests were administered to adolescent and adult twins, sex and age affected the performance of the UPSIT, but neither sex nor age affected PEA thresholds. Therefore, the olfactory diagnosis of TIB was established using a ROC curve without a sex and age classification scheme. When comparing cut-off scores established by ROC curves and the tenth percentiles, cut-off scores were 41 by ROC curve and 38.2 by the tenth percentile for the normosmic group; and 24 by ROC curve and 8.3 by the tenth percentile for hyposmic subjects. This indicated that cut-off scores established by the tenth percentiles were lower than those established by ROC curves. The difference in cut-off values between using ROC curves and using the tenth percentiles may result from different inclusion criteria and the sample size. The previous study suggested that anosmic subjects would hardly obtain TIB scores more than 11 points according to the authors’ experience. However, in this current study, some anosmic subjects got more than 11 TIB points when using a PEA test to define anosmic subjects. It might imply that some of anosmic subjects when defined by the PEA test were not literally anosmic.

These current results showed that the TIB was effective at distinguishing among the normosmic, hyposmic and anosmic groups, and had good reliability. The TIB consists of only 16 tests and is simpler to execute and is cheaper compared with the UPSIT-TC. In the era of COVID-19, it is important to develop a local, accurate, cheap and self-administered olfactory test to examine the olfactory function of subjects that complain of loss of smell function.

In conclusion, these current results showed that the TIB was an appropriate olfactory test based on its validity and test–retest reliability. When applied in Taiwanese subjects, the cut-off scores were 41 for normosmic subjects and 24 for hyposmic subjects. The individual test–retest results revealed a change of 6. Whether sex and age affect the performance of TIB requires further analysis using large datasets. Analysis of the results of TIB in other ethnic populations to evaluate whether any difference exists is planned for the future.