Acoustic surveillance of cough for detecting respiratory disease using artificial intelligence.

Research question: Can smartphones be used to detect individual and population-level changes in cough frequency that correlate with the incidence of coronavirus disease 2019 (COVID-19) and other respiratory infections?
Methods: This was a prospective cohort study carried out in Pamplona (Spain) between 2020 and 2021 using artificial intelligence cough detection software. Changes in cough frequency around the time of medical consultation were evaluated using a randomisation routine; significance was tested by comparing the distribution of cough frequencies to that obtained from a model of no difference. The correlation between changes of cough frequency and COVID-19 incidence was studied using an autoregressive moving average analysis, and its strength determined by calculating its autocorrelation function (ACF). Predictors for the regular use of the system were studied using a linear regression. Overall user experience was evaluated using a satisfaction questionnaire and through focused group discussions.
Results: We followed-up 616 participants and collected >62 000 coughs. Coughs per hour surged around the time cohort subjects sought medical care (difference +0.77 coughs·h-1; p=0.00001). There was a weak temporal correlation between aggregated coughs and the incidence of COVID-19 in the local population (ACF 0.43). Technical issues affected uptake and regular use of the system. Interpretation: Artificial intelligence systems can detect changes in cough frequency that temporarily correlate with the onset of clinical disease at the individual level. A clearer correlation with population-level COVID-19 incidence, or other respiratory conditions, could be achieved with better penetration and compliance with cough monitoring.

Introduction

Syndromic surveillance relies on the recognition of symptomatic patients by healthcare systems and proves challenging for identification of emerging, rapidly transmissible pathogens [1].

Cough is a key symptom of most respiratory diseases, including infections of public health interest, such as coronavirus disease 2019 (COVID-19) or influenza. Approximately 57% of all COVID-19 patients will develop cough during the early stages of infection and its presence is probably correlated with contagiousness [2, 3]. The ubiquitous presence of cough in respiratory infections and its low frequency in healthy individuals make it an attractive marker for syndromic surveillance [4].

The limited portability and automation of existing cough monitoring systems limit their usefulness for protracted use. Furthermore, none has so far been used to monitor community-level cough [5]. In addition, incorporation of cough monitoring into epidemiological surveillance programmes is hampered by a limited understanding of the epidemiology of cough.

Artificial intelligence systems can be trained to recognise the well-described acoustic characteristics of cough [6-8]. These patterns can be identified by a wide range of machine learning techniques, such as convolutional neural networks [6, 9]. The widespread use of smartphones provides an opportunity to deploy these systems at the scale needed to construct a community-based surveillance network [10, 11].

We hypothesised that these systems could detect changes in an individual's clinical condition, and that aggregated data could help estimate the incidence of respiratory infections, such as COVID-19. Although cough detection systems have been used in clinical research for years, this is the first attempt to apply such a tool on a community scale [12-15].

Methods

Study subjects

Participants were recruited between November 2020 and June 2021 at the University of Navarra in Pamplona (Spain), and the neighbouring communities of Zizur Mayor and the Cendea de Cizur. Recruitment strategies included direct solicitation, advertisements through municipal authorities, and the university's communication platform and social networks. Through these activities, we expected to reach up to 30 000 people.

Eligible participants needed to 1) be aged ⩾13 years; 2) own an Android or iOS smartphone able to run Hyfe; 3) be willing to install and use Hyfe as instructed; 4) accept and comply with Hyfe's privacy policy and terms of use; 5) grant access to their medical records during their participation in the study; 6) visit the University of Navarra regularly, or be a current resident of Navarra. All participants provided informed consent. This project was approved by the ethics committee for medical research of Navarra (Pamplona, Spain; reference number: PI2020/107) and the Comité d’Éthique à la Recherche du Centre Hospitalier de l'Université de Montréal (Montréal, Canada; reference number: 2021-9247).

Study design

This was a prospective observational study. The primary objective was to assess the value of digital cough surveillance as a proxy for community incidence of COVID-19. Secondary objectives were to 1) determine whether changes in cough frequency were associated with the moment of medical consultation; and 2) quantitatively and qualitatively assess the barriers and facilitators to participation in smartphone-based acoustic surveillance programmes. A full protocol describing sample size estimations and enrolment strategies was published previously [11].

Methods

Participants were asked to monitor their cough using Hyfe Cough Tracker (henceforth referred to as Hyfe; https://www.hyfe.ai), a free, automated, cough detection application, downloaded on their personal phones. Hyfe monitors ambient sounds without continuously recording. It uses a convolutional neural network to analyse explosive sounds. A cough prediction score is assigned to each sound by the machine learning model, if this score lies above a pre-determined threshold (in this study 0.7 out of a maximum score of 1), the sound is classified as a cough, stored in the participant's smartphone, and relayed to a cloud-based central dataset. Individual coughs detected ⩾0.5 s from each other are counted independently. Preliminary data indicate that Hyfe has a sensitivity of 96.34% and a specificity of 96.54% in differentiating coughs from other detected sounds [11]. Further validation to determine its performance when undetected explosive sounds are accounted for is underway (clinicaltrials.gov identifier NCT05042063).

Participants were instructed to keep Hyfe active for ⩾6 h·day−1 during the night-time, in order to minimise interference with normal daytime routines, but encouraged to use it continuously, if possible. They were instructed to monitor their cough for a 30-day period, with the possibility of prolonging their participation. Daily push notifications and periodic emails were sent to participants to maintain retention.

Study personnel reviewed medical records of participants every 2 weeks at the Clínica Universidad de Navarra and the regional public health system (Osasunbidea), looking for consultations associated with a diagnosis of respiratory disease. During each round of reviews, the national identification numbers of participants were searched in a centralised dataset, and registered consultations associated with respiratory symptoms (including COVID-19 screening tests), or a confirmed diagnosis of respiratory disease (COVID-19, influenza, respiratory syncytial virus, pneumonia, asthma, bronchitis, pharyngitis, chronic cough, COPD, gastro-oesophageal reflux disease or other nonspecific respiratory tract infections) was recorded. Daily incidence of COVID-19 in the study area was obtained from public sources [16].

Upon withdrawal, participants were instructed to rate the digital cough monitoring application on a 0–5 scale (0=very unsatisfactory to 5=very satisfactory). Participants were divided into two groups: high- (≥100 h of monitoring) and low-participation (<100 h of monitoring) users.

Participants were invited to join virtual focus group discussions to evaluate the importance given to cough and their experience using the digital cough monitoring application.

Analysis

The main outcome measure of this study was cough frequency (measured in coughs per person-hours). Secondary outcome measures included daily COVID-19 incidence in the study area, average daily usage of the application (measured in minutes per day), and the mean application's appreciation score (on a 0–5 scale).

Cough frequency and changes in clinical status

A medical consultation period was defined as the 10 days centred on the date of consultation (days −5 to +4, with day 0 being the date of consultation). All data outside the consultation period were defined as the user's cough frequency history and further divided into a pre- (before day −5) and post-consultation history (after day +4). Participants who attended at least one medical consultation during the enrolment period and for which ⩾24 h of cough monitoring was achieved within and outside the consultation period were included in the analysis.

Comparison tests were carried out using a randomisation routine, which protects results against bias from individual-level effects (large differences in user activity and/or cough rates) while preserving uncertainties inherent to low sample sizes. Cough rates during the consultation period and the participant's whole history were calculated for each user, their differences (consultation − history) were determined. The mean of these differences is treated as the average effect size in the sampled population.

To determine the significance of this observed effect size, it was compared to a distribution of effect sizes that would be expected under a null model of no difference between the two levels. This was determined using 1000 iterations of a randomisation routine in which the user records were shuffled (specifically, the field indicating days since consultation), and the average of the users’ cough rate difference was re-calculated. This routine produced a null distribution of simulated effect sizes. The proportion of null values greater than the observed value is treated as a p-value.

Acoustic surveillance and COVID-19 incidence

Participants with ⩾1 h of cough monitoring were included in this analysis. Cough was aggregated in time to create a cough frequency curve. An epidemic curve including all cases of COVID-19 diagnosed in the study area was superposed to cough data. An autoregressive moving average (ARIMA) analysis was carried out to compare confirmed cases of COVID-19 with cough frequency in the cohort, excluding participants with less than an hour of data on any specific day. The strength of the association between both variables was expressed with the auto-correlation function (ACF). This parameter ranges from −1 to +1, with values closer to 1 representing a stronger association. This analysis was only carried out for COVID-19, due to the low circulation of other respiratory pathogens during the study period.

Usage and perception of the acoustic syndromic surveillance system

Predictors of regular use were evaluated using a linear regression model to compare differences in the average daily monitoring period by age, gender, operating system (Android or iOS), number of medical consultations during the study period and number of email reminders sent to each participant.

Mean appreciation scores for the application from participants who completed the end-of-study questionnaire were disaggregated by operating systems and compared with a two-tailed unpaired t-test. Barriers and facilitators for uptake and use were qualitatively assessed in focus group discussions.

Data was organised and analysed using R Studio version 1.3 (www.rstudio.com) and SPSS Statistics version 28 (IBM Corp, Armonk, NY, USA).

Results

Characteristics of the cohort

A total of 930 participants were enrolled. However, only 616 used the application for ⩾1 h (figure 1). Although this is the largest cough cohort continually monitored to date, it represents just 1.7% of the 35 000 people estimated to have been reached in the recruitment campaign. Participants were aged 14–76 years (median 21 years, interquartile range (IQR) 20–25 years), mostly female (64.9%) and iOS users (56%). In total, 178 (28.9%) participants registered >100 h of monitoring, and 21 (3.4%) registered ⩾240 h, equivalent to 10 days of continuous monitoring. The latter groups were older (median age for those with ⩾100 h: 25 years, IQR 21–50 years; median age for those with ⩾240 h: 50 years, IQR 39–56 years), as presented in table 1.

FIGURE 1

Flow chart of participants analysed in the study. Approximately 35 000 people were estimated to have been reached via social network information campaigns. Of these, 930 were enrolled in the study. Only 616 participants recorded data for ⩾1 h, and were therefore included in the autoregressive moving average (ARIMA) and usage analyses. Similarly, only 272 participants consulted medical services during the study period. Of these, only 33 recorded ⩾24 h of data both in and outside the consultation period, and were included in the analysis of cough frequency changes as a function of consultation dates.

TABLE 1

Cohort characteristics and cough monitoring periods

	Recruited	Monitoring periods
		≥1 h	≥100 h	≥240 h
Age, years	21 (20–24)	21 (20–25)	25 (21–50)	50 (39–56)
Operating system
Android	366 (39.4)	269 (43.7)	127 (71.3)	18 (85.7)
iOS	559 (60.1)	345 (56.0)	51 (28.7)	3 (14.3)
Not specified	5 (0.5)	2 (0.3)	0	0
Participants	930	616	178	21

Data are presented as median (interquartile range), n (%) or n.

Cough frequency is higher around the time of medical consultation

272 participants attended at least one medical consultation during the study period (425 consultations in total; supplementary material), 33 of whom had ⩾24 h of monitoring both during and outside the consultation period (figure 1). For these 33 patients, hourly cough rates were higher during the consultation period, than during the rest of the monitoring history, with a difference of 0.77 coughs·h−1 (p<0.00001), equivalent to ∼19 extra coughs per day. This effect was driven by lower cough rates during the post-consultation history (after day +4, difference: 1.08 coughs·h−1; p<0.00001). When exclusively compared to the pre-consultation history (before day −5), cough rates were not significantly different (p=0.855) (figure 2, table 2). Similar results were observed when comparing subdivisions of the consultation period to the rest of the monitoring history (supplementary material).

FIGURE 2

Difference between cough rates in the consultation period compared to the participants’ monitoring history. Cough frequency during the consultation period is compared to the entire monitoring history (n=33), and the parsed pre- (n=23) and post-consultation history (n=29). Shaded areas represent the distribution of effect sizes predicted under a null model of no difference. The black line represents the actual observed difference between the consultation period and compared periods. Cough frequency during the consultation period significantly increased when compared to the entire history and the post-consultation history (p<0.00001 in both cases), but not when compared to the pre-consultation history (p=0.855).

TABLE 2

Changes in cough rates during the consultation period and participant monitoring history

	Participants	Observed difference in cough rate (coughs·h −1 )	p-value
Consultation period (days −5 to +4)#
Full user history	33	+0.77±2.62	<0.00001
Pre-consultation history (before day −5)	23	−0.25±1.89	0.855
Post-consultation history (after day +4)	29	+1.08±2.92	<0.00001

Data are presented as n or mean±sd, unless otherwise stated. #: day 0=date of consultation.

Syndromic surveillance and COVID-19 incidence

>79 000 aggregated hours of monitoring, equivalent to 3316 person-days (or 9.08 person-years) and 62 401 coughs were registered between November 2020 and August 2021 (n=616 participants). Of these, 79% were recorded between 20:00 and 08:00. Peaks of cough rates were registered in February, April–May and August 2021 (figure 3b).

FIGURE 3

Cough and usage trends compared to coronavirus disease 2019 (COVID-19) incidence. Incidence of COVID-19 in a) the entire study area compared to b) the evolution of cough trends in the monitored cohort; c) after the exclusion of the participant with chronic cough; and d) compared to the number of active users. The continuous line represents the 7-day rolling average.

In total, 14 051 cases of COVID-19 were diagnosed in the study area, in three clear waves: in January and February, late March and April, and between July and August 2021 (figure 3a). Only 72 new cases were diagnosed among study participants (51 of which occurred among participants who recorded ⩾1 h).

Cough frequency in the cohort and COVID-19 incidence in the population correlated, notably during the first and last waves (figure 3a and b). The ARIMA model analysis confirmed this association, which reached maximum strength (ACF=0.43) when lagging the cough time series by 17 days compared to the COVID-19 incidence. This indicates that cough increased, on average, 17 days after peaks in COVID-19 cases.

The number of users registering coughs changed throughout the study period, such that changes in cough frequency were based on a diminishing number of participants (figure 3d). Specifically, during the February COVID-19 wave, ∼40 participants were contributing a combined total of >555 h of monitoring time per day. At the time of the third peak there were only ∼12 participants contributing 111 h of monitoring time per day. Upon closer inspection, it was noted that almost half of all coughs registered during this wave came from a single chronic cougher who did not have COVID-19. Further investigation revealed that during this period, her cough increased to ∼400 per day, following the discontinuation of antitussive medication. The smaller peak observed in February coincides with this participant's recruitment into the study.

While the peak in coughs during the summer remains evident after excluding this participant from the ARIMA model (figure 3c), the strength of the correlation markedly reduced (ACF 0.28), indicating that given the low number of participants, trends were influenced by one person with severe chronic cough.

Usage and perception of the application

Participants with ⩾1 h of monitoring (n=616) registered an average of 130 h of total data (range 1–5000 h) and used the application for a mean±sd 336±188 min per day), equivalent to 5.6 h. Average daily usage was discretely increased in participants who received more email reminders (β=5 min per reminder; p<0.001), and those who were older (β=4.5 min per year; p<0.001). Using iOS rather than Android was associated with significantly reduced daily usage time (β= −103 min; p<0.001). Results for other predictors can be found in the supplementary material.

The mean±sd appreciation score was 3.6±1.02 (n=217). Scores were higher among Android users (3.82, 95% CI 3.64–4.00), compared to iOS users (3.38, 95% CI 3.18–3.58; p=0.001). The most common reason for withdrawal was technical problems while running the application (n=95, 43.8%), mostly among iOS users (60 out of 95; supplementary material).

Nine participants took part in focus group discussions. Participants were aged 21–65 years (median 48 years). Two (22.2%) were male and seven (77.8%) female. Seven belonged to the high-usage group (registered ⩾100 h), while two belonged to the low-usage group. In the former, cough was perceived as important only if it affected daily routines, either because it was associated with a known respiratory disease or with certain lifestyle characteristics, such as smoking. The main motivator behind constant usage was interest in helping the study team, with little importance given to its perceived health benefits. However, two participants with a history of chronic respiratory disease indicated that seeing changes in longitudinal cough trends, and their link to certain behaviours, were important motivators. Notifications were not well received in the low-participation group. Repeated bugs in the iOS version were noted in both groups, confirming results from the quantitative analysis. Summary tables with common answers provided by participants can be found in the supplementary material.

Discussion

This is the first population-based syndromic surveillance study using passive digital cough monitoring. Over the course of the study, we monitored >9 years of person-time and detected 62 000 cough sounds. We showed that cough monitoring can detect changes in cough frequencies at individual and community levels.

Our observation that cough frequency is higher around the time that individuals seek medical care suggests that passively detected changes in cough patterns could be noticed by participants, and partially influence health-seeking behaviours. While it is true that other symptoms apart from cough are likely to contribute to this process, these findings suggest that upon further refinement, smartphone-based tools could be used to detect changes of clinical relevance in cough patterns, in the context of patient monitoring or evaluation of response to treatment. These changes were driven by reductions after the 5 days following consultations. The lack of a significant difference with the pre-consultation history is likely a result of the fact that many of these participants were recruited during COVID-19 testing sessions, when many were already symptomatic, making their pre-consultation history short or nonexistent.

In addition, we demonstrated that longitudinal changes in aggregate cough data from this cohort were temporally associated with the incidence of COVID-19 in the community. However, the causal nature of this association is challenged by the fact that the cough frequency peaked, on average, 17 days after COVID-19 incidence, long after the period within which most symptoms of mild infection resolve [17].

Further confounding in this association came from a large proportion of coughs originated by a single individual with chronic cough. As discussed later, the lack of a stronger temporal association is probably caused by a low population coverage, and the low incidence of COVID-19 in the cohort. It may also be due to a low percentage of patients experiencing cough, confounding by other infectious and noninfectious causes of cough within the cohort, or exposure to other environmental tussive stimuli.

Nonetheless, the fact that the association remains visible in more than one COVID-19 wave, and after removing the chronic cougher, might indicate that some subjects in the cohort were infected in the last part of a community wave and remained undiagnosed.

Limitations

This study failed to reach an adequate uptake to produce representative data of the population (only 1.7% of the estimated reached individuals ended up using the application). Similarly, only two-thirds of these participants provided useful data, and only 21 used the application regularly throughout the study period. Addressing these issues seems complicated, considering low uptakes reported for similar contact tracing software in the past [18]. Larger multicentric projects, ideally supported by public health authorities, might help increase the number of participants in future studies.

Another important limitation is the fact that participants were instructed to preferentially monitor night-time cough. While this probably facilitated compliance by reducing interference with routine cell phone use, sleep is an inhibitor of cough [19], and we observed a higher cough frequency from 10:00 to 20:00, with coughs reducing sharply after midnight [20]. This means that the value of longer monitoring periods will rely on longer daytime monitoring. The fact that technical problems were the leading cause for discontinuation is encouraging, as these are solvable engineering challenges, many of which have already been implemented, indicating that future studies with protracted daytime monitoring time are feasible. Human factors of maintaining interest and ensuring privacy may be harder to address.

The fact that only a few participants were monitoring their cough regularly also made the results susceptible to the effect of outliers, as was observed, with large changes in the cohort's aggregated cough rate being driven by a single participant with chronic cough. Despite this, the fact that a modest correlation remained visible after excluding this participant, and the system's capacity to detect changes in cough frequency associated to medical consultations are encouraging.

It is also possible that the monitoring system detected background coughs, not produced by study participants, particularly if they used it while visiting crowded areas. The fact that participants were mostly instructed to use the application at night and in their homes, as well as the fact that louder sounds are more likely to trigger the classification algorithm, are expected to limit this confounding effect, but this should be explored in future studies.

Acoustic surveillance systems are technically capable of detecting changes in cough frequency associated with the onset and evolution of respiratory disease in individuals and populations. However, the discussed limitations prevented us from reliably responding to whether they can be used to infer the incidence of respiratory disease.

Please note: supplementary material is not edited by the Editorial Office, and is uploaded as it has been supplied by the author.

Supplementary material 00053-2022.SUPPLEMENT