An olfactory self-test effectively screens for COVID-19.

Background: Key to curtailing the COVID-19 pandemic are wide-scale screening strategies. An ideal screen is one that would not rely on transporting, distributing, and collecting physical specimens. Given the olfactory impairment associated with COVID-19, we developed a perceptual measure of olfaction that relies on smelling household odorants and rating them online.
Methods: Each participant was instructed to select 5 household items, and rate their perceived odor pleasantness and intensity using an online visual analogue scale. We used this data to assign an olfactory perceptual fingerprint, a value that reflects the perceived difference between odorants. We tested the performance of this real-time tool in a total of 13,484 participants (462 COVID-19 positive) from 134 countries who provided 178,820 perceptual ratings of 60 different household odorants.
Results: We observe that olfactory ratings are indicative of COVID-19 status in a country, significantly correlating with national infection rates over time. More importantly, we observe indicative power at the individual level (79% sensitivity and 87% specificity). Critically, this olfactory screen remains effective in participants with COVID-19 but without symptoms, and in participants with symptoms but without COVID-19. Conclusions: The current odorant-based olfactory screen adds a component to online symptom-checkers, to potentially provide an added first line of defense that can help fight disease progression at the population level. The data derived from this tool may allow better understanding of the link between COVID-19 and olfaction.

Introduction

The COVID-19 pandemic has wreaked havoc on world order. A necessary tool for effectively dealing with the pandemic is a wide-scale, rapid, and cheap method for screening. In that national medical systems are overloaded as it is, an ideal screening tool would be one that does not entail transportation, dissemination, and processing of a physical test. One alternative that has received attention is the possibility of AI-enabled diagnosis by the sound of coughing, or hoarse voice, that can be recorded on a phone line (AI-Cough)[1]. A second alternative is online subjective self-reported symptom-checkers, that have indeed generated some remarkable results[2,3]. Although both of these approaches satisfy the need for a wide-scale easily-administered screening scheme[4], they both inherently fail at two critical points: One is in individuals who are ill, but not with COVID-19. If an individual has a cough, fever, and a headache, current AI-Cough and symptom-checkers will most likely estimate them to have COVID-19, even when they do not. The second point of failure is individuals who have COVID-19, but experience no apparent symptoms. By definition, such individuals, who constitute ~30% of all those infected[5,6], will go undetected by current AI-Cough and symptom-checkers. Paradoxically, these very same individuals are possibly the most concerning from an epidemiological perspective, as they may unwittingly spread the disease. An alternative to subjective symptom-reporting alone is an objective performance-based test, and the sense of smell provides for a particularly attractive target in this respect. This is for two reasons: First, loss and/or alterations in the sense of smell have been widely recognized as a highly prevalent early symptom of COVID-19[7-10]. Second, most individuals have household odorants readily available to them for testing. Indeed, a large consortium of clinicians and basic scientists known as GCCR (https://gcchemosensr.org) has set out to investigate olfaction in COVID-19, and have convincingly established olfaction as a marker[11-13]. Moreover, in cases where the olfactory loss is subjectively noticed, that alone is a powerful marker of COVID-19[14]. That said, the power of olfaction as a marker in subjectively asymptomatic individuals (namely the most concerning group from an epidemiological standpoint) who have no subjective sense of an olfactory loss, has yet to be tested as far as we know. Because, by definition, subjectively asymptomatic individuals are unaware of any olfactory change or impairment they may have, the test used needs to be one that is particularly sensitive to perceptual alterations that may be subconscious. The recently developed olfactory perceptual fingerprint (OPF) is precisely such an instrument[15,16]. Because it is not a performance measure per se (e.g., a score for detection/discrimination/identification), but rather a perceptual quantifier, it effectively taps into minute subconscious alterations in perception[17,18]. Thus, our hypothesis was that the OPF may allow for accurate classification of individuals who are COVID-19 positive but without symptoms, or COVID-19 negative but with symptoms of (other) disease. Beyond testing this hypothesis, our aim was to generate a convenient online tool that applies this approach. To address these hypotheses we built an online tool for reporting olfactory perception. Here, we report the results from 13,484 participants (462 COVID-19 positive) from 134 countries who provided 178,820 perceptual ratings of 60 different household odorants. We observe that olfactory ratings are indicative of COVID-19 status in a country, significantly correlating with national infection rates over time. More importantly, we observe indicative power at the individual level (79% sensitivity and 87% specificity). Critically, consistent with our hypothesis, this olfactory screen remaines effective in participants with COVID-19 but without symptoms, and in participants with symptoms but without COVID-19.

Methods

Recruitment

There was no systematic recruitment. We formed a small international consortium, and each participating lab tried to inform the local media in their country of residence. This resulted is several news stories published in several countries, and these led to dissemination. The success of the publicity varied greatly from country to country due to the resources available to participating labs and to public disposition. Participants were directed to the web-tool at www.smelltracker.org where they consented to participate anonymously in a study that was approved by the Wolfson Hospital Helsinki Committee (Approval #0066-20-WOMC). We note that during the initial reported time period we had 12,800 participants, but of these 780 participants reported only partial data, retaining only 12,020 participants for full analysis. In a reported follow-up we then analyzed an added 1464 participants, culminating at 13,484 participants. This study size reflected a balance between two considerations: Web-based questionnaires on odor intensity and pleasantness gain sufficient power at 198 participants per odorant[16]. We wanted this power for at least 40 odorants, and indeed here report on 42 odorants with more than 198 respondents. In turn, we did not wait for this number of respondents on all possible odorants, as we wanted to report on this in a timely manner, given the progression of the pandemic.

Web-tool

We built an online odorant rating tool (www.smelltracker.org). The tool was written in open-source Drupal (drupal.org), and translated into 15 languages by the co-authors who are all respective native speakers. Using the tool, participants first created a unique login to facilitate repeated testing. Otherwise, the tool was completely anonymous to protect user privacy[19]. Next, participants provided details regarding age, sex (female/male), and country of residence (here we made a mistake in that the country pull-down menu did not start from an empty space, but rather from “India”. Thus, participants who failed to answer this question were registered as from India by default. For this reason, we are unable to faithfully include India in the country-specific analyses). Next, participants selected five of 71 possible odorants to rate (Supplementary Table 1). We opted for five odorants, rather than a larger number, to strike a balance between increased reliability, where more assessments render more reliable data[20], versus low burden of participation. Each odorant was selected from a separate category with a fixed list of common household odorants (Supplementary Table 1). This list was generated in coordination with the participating labs, each contributing for their native culture in order to assure cultural diversity. Two odorant categories contained odorants with reduced trigeminal components (e.g., vanilla extract), and three categories had increased trigeminal components (e.g., vinegar). Participants made their odorant selections upon first use of the tool, and were then automatically prompted to use the same odorants on subsequent uses. Participants then smelled and rated each odorant using visual-analog scales (VASs) for perceived intensity and pleasantness, namely the primary dimensions of olfactory perception[21]. VASs ranged from very weak to very strong, and from very pleasant to very unpleasant. These scales were coded in the system as ranging from 0 to 100. Participants could smell the odorant as often as they liked, and there was no time limit applied. Following the ratings, participants were asked whether they had been tested for COVID-19 (No; Yes-Pending; Yes-Positive; and Yes-Negative), and whether they are currently experiencing any COVID-19 symptoms (Fever; Cough; Shortness of breath or difficulty breathing; Tiredness; Aches; Runny nose; Sore throat; Loss of the sense of smell; Loss of taste; and No symptoms). We have recently added to the live site questions on vaccinations, but these were not available when this data was collected. Finally, after completing participation, participants were presented with a graph depicting their olfactory perceptual fingerprint as it related to the average scoring, and if they participated again, the graph depicted the evolution of their perception over time. In addition to the graph, participants were presented with a text informing them whether their perception was within range of most participants, or aberrant. Based on the results now obtained and reported in this manuscript, we have only recently modified the feedback component such that the system now also informs participants to what extent they resemble a person who is COVID-19 positive or COVID-19 negative. Given regulatory restrictions, this is as close as we could get to giving a diagnosis. This extended feedback, however, was not provided to participates reported on in this manuscript.

Statistics and reproducibility

All analyses were conducted using Matlab software, and the complete data file allowing full recreation of these results is in Supplementary Data 1. For initial analysis of intensity and pleasantness, we restricted our analysis to 23 odorants that had more than 25 C19+ raters. This gave rise to 46 distributions of ratings, of which only 18 and 33 for intensity and pleasantness, respectively, were normally distributed. Given non-normal distributions, we applied a two-sided Kolmogorov–Smirnov test to all C19+ and C19− intensity and pleasantness comparisons. In the individual odorant follow-up comparisons we estimated effect size using the Eta squared effect size measure[22].

Country-specific correlations between odorant ratings and rates of COVID-19 were calculated as follows: To produce time-series for rates of COVID-19, we conducted two steps: 1. The number of daily cases in each country was obtained from the Johns Hopkins Coronavirus Resource Center[23]. 2. We calculated a 7-day moving average for the dates between March 15, 2020 and September 30, 2020. For national intensity ratings time-series, we conducted three steps: 1. Average intensity ratings of the five odorants were calculated for each entry. 2. Mean intensity ratings were inverted by subtracting them from 100. This was done so that higher values imply greater smell loss. 3. A 5-day moving average was calculated by averaging all ratings in the span of 7 days. We used this moving average to match the cases span. After obtaining these two values, a cross-correlation between the daily ratings and inverse intensity was then calculated (using the xcorr function in Matlab). The cross-correlation analysis resulted in a correlation between the two signals for different lags (between 14-days earlier to 14-days later response) in the inverse intensity signal. The lag that produced the maximal correlation between the two signals was chosen for the analysis. The Pearson correlation between daily cases and lagged inverse intensity was calculated. Daily cases time-series, inverse intensity signal and lagged inverse intensity signal are shown the related figure.

Receiver operating curves (ROCs) were calculated using standard technique[24]. We used a moving cutoff point on a continuous scale, and at each point measured the true positive (TPR) and false positive (FPR) ratios which result from selecting that cutoff. All confidence intervals in ROC plots were calculated using a 1000 iteration bootstrapping of the scores. To compare between ROCs, we used a non-parametric test based on the AUC of the curves[25].

Olfactory perceptual fingerprints

Individual olfactory perception is typically characterized using performance-based measures, such as olfactory detection, discrimination and identification[26,27]. An alternative is not to characterize performance, but rather characterize how the world smells to an individual. Such characterizations have been termed olfactory perceptual fingerprints (OPFs), and their typical derivation relies on the perceptual distance matrix for a set of odorants[15,16]. One version of the OPF is the descriptor-based OPF. Here, an individual is characterized by how he/she applies a set of descriptors to a set of odorants. Given M odorants and N descriptors, for each participant m, for each odorant we calculate the difference between their rating along a descriptor, versus the group mean for that same odorant and same descriptor. Thus, each participant is initially described as a matrix where each entry is the difference between their perceptual rating of an odorant i along a descriptor j and the group mean of the same odorant and descriptor.

This yields M relative scores along each of N descriptors. We then average M relative scores for each of the N descriptors. This in a N dimensional representation of the individual , where each entry in is by Eq. (1):

In the current study this is simplified, as we have five odorants (self-selected) and two descriptors, namely intensity and pleasantness. This yields five relative intensity and five relative pleasantness scores. We then average the five intensity differences and five pleasantness differences for each participant, retaining two numbers that represent that participant in a two-dimensional space. The advantage of this descriptor-based approach is that it allows us to directly compare individuals, who selected different odorants. The only perquisite for generating the calculation is that a sufficient number of individuals (although not necessarily the individuals under comparison) rated a given odorant-descriptor pair, so that we have a valid mean entry for that pair. This combination of conditions renders this method ideal for the current data. We acknowledge that this measure may be weakened by cultural/geographical variability in olfactory perception[28]. We note however, that pleasantness reflects the primary physical dimension in odorant structure[21], and it is the primary dimension of olfactory perception[29]. Therefore, cross-cultural variability in odorant pleasantness is far lower than commonly thought[21,30,31], and typically overestimated because of a few canonical outlying odorants. Finally on this front, we will stress that to the extent that this is a shortcoming, it is one that can only weaken our result, not strengthen it or generate an artifactual outcome.