Public health effectiveness of digital contact tracing in the COVID-19 pandemic: A systematic review of available data.

Background and aim Contact tracing is a key element of epidemiologic investigation and active surveillance during infectious diseases outbreaks. Digital contact tracing (DCT) are new technologies that have been increasingly adopted in different countries to support conventional contact tracing efforts to control the COVID-19 pandemic. However, scant evidence is available on its public health effectiveness. We applied the Indicator Framework issued in 2021 jointly by the World Health Organization (WHO) and the European Centre for Disease Prevention and Control (ECDC) to assess the available evidence on DCT adoption and impact in the context of the COVID-19 pandemic. Methods We carried out a systematic review following the PRISMA guidelines (Prospero registration number: CRD42021253662) to retrieve, pool, and critically appraise studies published in English from November 2019 to April 2021. We excluded mathematical models of effectiveness. Only studies representative of the general population or specific populations were included . In line with the WHO-ECDC indicator framework, outcomes of interest were grouped in indicators of: i) DCT use, ii) DCT success, and iii) DCT performance. Results We identified 1.201 citations searching the PubMed, Embase, Web of Science and The Cochrane Library. After screening, 10 studies were included. All included studies reported measures of DCT use, varying widely by study population and setting (percentage of DCT apps download from 0.01% to 58.3% in included studies). Almost no data quantified an association between DCT adoption rate and infection transmission at the community level.  Only one reported measures of DCT success (ratio of exposure notifications received to positive test results entered), while no studies were retrieved reporting measures of DCT performance. Conclusions DCT has large potential to control epidemics. Its adoption is hindered by several normative, technical and acceptance barriers in different regions and countries. Our review shows that while some evidence is available on its adoption and use in selected settings, very scant data is available on its effectiveness in the fight against COVID-19. As digitalization provides new tools for infection control at the population level, solid research is needed to quantify the public health effects of their application.

Introduction

When it comes to COVID-19, governments worldwide have applied strict physical distancing measures to control transmission. These policies have reduced case numbers, but have significant social and economic implications leading to the research of alternative integrate control strategies (1,2). In this respect, contact tracing represents an important part of epidemiologic investigation and active surveillance. Its ability to reduce disease transmission is premised on timely detection and prompt isolation of cases (3).

In particular contact tracing is a public health measure composed by three basic elements, namely, contact identification, contact listing and contact follow-up (4). The first step of this process is meant to identify people, or contacts, who have been exposed to a person, or index case, infected with a pathogen or another hazard. This stage is followed by assessing and managing the identified contacts to prevent onward transmission, referring them to therapeutic or preventative treatment, which may include isolation, quarantine, or other behavioral interventions (4,5).

Traditionally, contacts are traced by asking people identified as index cases to recall their close contacts during a recent interval associated with a high probability of infection (1,5). However, contact tracing effectiveness in breaking the chain of transmission depends on the proportion of contacts who are actually traced. In this regard, the main limit of manual contact tracing is determined by the quality of information provided, often incomplete or incorrect; other limitations include the time spent to notify contacts manually, which can delay quarantine, and the large amount of human resources engaged (1,5).

This being the case, contact tracing becomes quickly complicated and time-consuming, posing a public health issue particularly in the context of COVID-19, as the virus can be transmitted between people without even manifesting symptoms (6). Technology can be used to address some of these limitations, by automating the processing of test results or symptom reports and by use of DCT (1).

Before COVID-19 pandemics, DCT has already been used to manage Ebola outbreaks in Africa, as an example with the help of Go.Data, that has also been used in 55 projects worldwide (6).

At present, considering the penetration of digital tools and the current global pandemic, it is worth collecting data in order to estimate use and effectiveness of DCT.

In this respect the World Health Organization (WHO), in collaboration with the European Centre for Disease Prevention and Control (ECDC), have realized an evaluation framework for DCT technologies in order to establish the effectiveness of DCT applications created worldwide by national governments (7).

Despite strong government advocacy, DCT adoption rates have generally fallen short of what public health officials would hope. Indeed, while studies have shown that a high percentage of people accept and intend to download these apps, some evidence indicates that actual download rates are probably lower than predicted by positive attitudes (8).

In this context the aim of this systematic review is to understand the global employment of DCT tools among the general population and subsequently to define the impact of DCT on the prevention and control of the COVID-19 pandemic.

The WHO and ECDC framework has been used to assess and comment on the available data.

Methods

The review’s methods were defined in advance following the Prepared Items for Systematic Reviews and Meta-Analysis (PRISMA 2020) guidelines.

Search methods for identification of studies and inclusion criteria

Studies were identified by searching the electronic databases PubMed, Embase, Web of Science and The Cochrane Library. The search strategy was first developed in PubMed using a combination of free text and Mesh terms identifying: i) the concept of DCT, and ii) its involvement in the COVID-19 pandemic, and then adapted for use in the other databases. Complete search strategies are available in the Appendix. Studies published in English were included. Studies from November 2019 and April 2021 were included. We only considered for inclusion studies reporting original data from quantitative analysis. Only representative studies both of the general population or of specific populations were included. Both observational and experimental study designs were considered. We excluded opinion papers, (i.e. editorials, commentaries and letters to the Editor) not providing original data. Systematic reviews and other reviews were excluded. Mathematical models of effectiveness were excluded.

Outcomes of interest

We aim to assess the effectiveness of DCT mobile applications, using the ECDC and WHO framework indicators as keys of interpretation. The indicators available are listed in Box 1. In line with the ECDC-WHO framework, we categorized indicators of interest under three main categories: DCT “Use”, “Success” and “Performance” (Box 1).

Use

A.1: Proportion of total population who have downloaded the app.

A.2: Proportion of total population that actively uses the app.

A.3: Proportion of all positive tests that occur among app users.

A4: Proportion of positive tests among app users that are entered into the app (positive tests uploaded).

A.5: Rate of positive tests among app users relative to the rate of positive tests reported in the general population.

Success

B.1: Ratio of exposure notifications received to positive test results entered.

B.2: Proportion of diagnosed cases among app users who have previously received an exposure notification through the app.

B.3: Proportion of diagnosed cases previously notified only through the app (but not through conventional contact tracing) among all diagnosed cases.

B.4: Proportion testing positive among app users who present to testing services after receiving an exposure notification through the app.

Performance

C.1: Median (IQR) time between exposure and receipt of exposure notification through the app versus median (IQR) time between exposure and notification of contacts by conventional contact tracing services.

C.2: Median (IQR) time between symptom onset of index case and time of entering positive test result in the app versus median (IQR) time between symptom onset of index case and notification of contacts by conventional contact tracing services.

C.3. Median difference in notification speed between app and conventional contact tracing.

C.4 Proportion of new positive test results entered into the app within 24 hours of activation code issuance.

Data collection and analysis

All the retrieved studies were independently reviewed by two researchers (CM and DG), a first screening was performed based on title and abstract while full texts were retrieved for the second screening. At both stages disagreements by reviewers were resolved by consensus and consultation with senior authors.

Data from selected articles were extracted and tabulated independently by two researchers (CM and DG). Any disagreements have been resolved through discussion with a third researcher.

A standardized, pre-piloted form has been used to extract data from the included studies for assessment of study quality and evidence synthesis.

Extracted data were analysed and summarized in Excel (Microsoft Corporation, Redmond, California) spreadsheets.

The following data were extracted from included studies: country of study implementation, study design, study period, study population, sample size, response rate (if applicable) and outcomes of interest (DCT use, success and performance indicators).

Quality appraisal of included studies was carried out using The National Institutes of Health (NIH) Study Quality Assessment Tools.

Results

We identified 1201 records by searching the selected databases. After removing duplicates, 789 abstracts were retrieved. Studies were screened and selected as illustrated in Figure 1, resulting in 316 full text articles assessed for eligibility and 10 studies that were included in the systematic review.

Figure 1.

PRISMA flowchart of papers selected

Characteristics of included studies

The characteristics of the included studies are reported in Table 1. Half of the studies were conducted in Australia (30%) or Switzerland (20%); 1 study was conducted in the United Kingdom, 1 in France, 1 in India and 1 in Japan. One study was conducted transversally through different countries (USA, Italy, Norway, Singapore, South Korea, Pakistan, Australia, New Zealand, Switzerland Georgia). The majority of the studies were cross-sectional, 2 were prospective cohort studies, 1 was a custom original study and 1 was a content analysis. Study periods ranged from 4 days (9) to 78 days (16) and the most recent data were collected between August and October 2020 (40% of the papers included), whereas the oldest study started on April 2020 (16). The majority of the studies were conducted on the general adult population, only two studies were conducted on specific population settings: one on University students (13), the other on full time employees (17). The largest sample sizes were reported in the Swiss study as it considered the whole Swiss population (8.6 million people) (11), and in the Content analysis by Elkhodr M et al. (15), as it considered a major amount of countries. With reference to cross – sectional studies, the largest sample size was reported in the Indian study, in which in two different moments were interviewed 13292 and 14954 people (14). Included surveys’ response rate was reported in all but three studies and ranged from 53.9% to 90.6%. Quality appraisal of included studies is reported in Table 1.

Table 1.

Characteristics of included studies

Reference	Country of study implementation	Study design	Study period	Study Population	Sample size (N.)	Response rate	Outcomes‡			Quality Appraisal
							Use	Success	Performance
Thomas R et al., 2020, (9)	Australia	Cross-sectional, Survey	8 - 11 May 2020	General adult population †	1500	83.2%	A.1	NR	NR	FAIR
von Wyl V et al., 2021, (10)	Switzerland	Cross-sectional, Survey	28 Sep - 8 Oct 2020	General adult population	1511	NR	A.1; A.2	NR	NR	FAIR
Salathé M et al., 2020, (11)	Switzerland	Cohort Study	23 Jul - 10 Sep 2020	General adult population	8.6 million	NR	A.1; A.2; A.3; A.4	B.1	NR	GOOD
				Subpopulation of COVID-19 confirmed cases	12456
			7 Aug - 11 Sep 2020	Zurich subpopulation of RT-PCR-confirmed cases and their close contacts	420
Kendall M et al., 2020, (12)	United Kingdom	Custom	6 - 28 May 2020	General population on the Isle of Wight	141536	NA	A.1	NR	NR	NA
Montagni I et al., 2021, (13)	France	Cross-sectional, Survey	25 Sep - 16 Oct 2020	University students, health sciences at the University of Bordeaux	318	53.9%	A.1; A.2	NR	NR	FAIR
Lockey S et al., 2021, (8)	Australia	Cross-sectional	24 Jun - 21 Jul 2020	General adult population	2575	NR	A.1	NR	NR	FAIR
Sharma N et al., 2021, (14)	India	Cross-sectional, Survey	August and September 2020	General adult population (> 16 yr)	13292 and 14954	NR	A.1; A.2	NR	NR	FAIR
Elkhodr M et al., 2021, (15)	USA, Italy, Norway, Singapore, South Korea, Pakistan, Australia, New Zealand, Switzerland Georgia	Content analysis	Early July 2020	General population	Total population of included Countries	NA	A.1	NR	NR	NA
Garrett PM et al., 2021, (16)	Australia	Cross-sectional, Survey	6 Apr - 23 Jun 2020	General adult population	878	NR	A.1	NR	NR	FAIR
Kawakami N et al., 2021, (17)	Japan	Cohort and Survey	27 May - 12 Aug 2020	Full-time Japanese employees aged 20-59 years	902	90.6%	A.1	NR	NR	FAIR

† Excluding healthcare workers and people who had or thought they had COVID-19; ‡ Outcomes: A.1, A.2, A.3 A.4, A.5, B.1, B.2, B.3 B.4, C.1, C.2, C.3 C.4 (see Box 1); NA: Not applicable; NR: Not reported

Overall, 5 out of 13 (38.5%) of the indicators of interest were measured in included studies. In particular, all studies measured DCT ‘use’ indicators, only one ‘success’ indicators (11) and none reported on ‘performance’ indicators. In details, measured DCT “use indicators” were 4. ‘Proportion of total population who have downloaded the app’ (A.1) was reported in all the 10 included studies (8–17); ‘Proportion of total population that actively uses the app’ (A.2) was reported in 40% of the articles (10,11,13,14); ‘Proportion of all positive tests that occur among app users’ (A.3) and ‘Proportion of positive tests among app users that are entered into the app (positive tests uploaded)’ (A.4) were reported in 1 article out of 10 (11).

Table 2 reports quantitative data on DCT indicators of interest extracted from included studies. The proportion of population that have downloaded the app (A.1 indicator) is over the 50% (58.3% in August 2020 and 55.6% in September 2020) in one study (14). In the majority of the studies (70%) it has been assessed to be between the 20.4% (17) and 46.5% (10). Under this standard there is the data from French study by Montagni et al (11.3%) (13). In the study by Elkhodr et al (15) are showed the data available by local news at the time of July 2020 about ten applications in 9 different countries: the download proportion is lower than in the other papers. It appears to be over the 20% for three apps (TraceTogether in Singapore 35.89%, Smittestopp in Norway 26.32% and COVIDSafe in Australia 24.03%) and under 5% in the half of the apps investigated.

Table 2.

Data on DCT Use, Success and Performance indicators of included studies.

Reference	A.1 Proportion of total population who have downloaded the app (%)	A.2 Proportion of total population that actively uses the app (%)	A.3 Proportion of all positive tests that occur among app users (%)	A.4 Proportion of positive tests among app users that are entered into the app (positive tests uploaded) (%)	B.1 Ratio of exposure notifications received to positive test results entered. (%)
Thomas R, et al., 2020, (9)	37.3	NR	NR	NR	NR
von Wyl V, et al., 2021, (10)	46.5	38.8	NR	NR	NR
Salathé M, et al., 2020, (11)	27.4‡	18.9	19.6	67.2	94.8‡
Kendall M, et al., 2020, (12)	38	NR	NR	NR	NR
Montagni I, et al., 2021, (13)	11.3	4.7	NR	NR	NR
Lockey S, et al., 2021, (8)	43.1 ‡, §	NR	NR	NR	NR
Sharma N, et al., 2021, (14)	58.3 in August55.6 in September	54.2 ‡ in August51.4 ‡ in September	NR	NR	NR
Elkhodr M, et al., 2021, (15)	0.01 (USA – Care19)4.47 (Italy – Immuni)0.09 (Italy - SM-COVID-19)26.32 (Norway – Smittestopp)35.89 (Singapore – TraceTogether)1.95 (South Korea - Corona 100m)24.03 (Australia – COVIDSafe)11.88 (New Zealand - NZ COVID Tracer) 18.48 (Switzerland – SwissCovid) 2.51 Georgia - Stop Covid †	NR	NR	NR	NR
Garrett PM, et al., 2021, (16)	44	NR	NR	NR	NR
Kawakami N, et al., 2021, (17)	20.4	NR	NR	NR	NR

† Data from Local News and grey literature; NR: Not reported; ‡ Data computed by the authors; § Weighted average of available data in the paper. Download rate of the different profiles studied: Profile 1 59.80%; Profile 2 55.60%; Profile 3 45.30%; Profile 4 39.70%; Profile 5 36.80%; Profile 6 35.90%; Profile 7 32.80%.

Considering the proportion of total population that actively uses the app (A.2 indicator) there are four available data 4.7% (13), 18.9% (11), 38.8% (10) and 51.4% in September and 54.2% in August in the Indian study (14). It can be noticed that in all the four studies the A.2 indicator shows a lower percentage value in comparison with A.1 indicator. No comparison is possible between the articles about A.3, A.4 and B.1 indicators as they are only reported by one paper (11). In particular A.3 value is reported as 19.6%, the A.4 as 67.2% and B.1 as 94.8 (this data has been computed by the authors).

Discussion

Rapid identification and notification of all exposed contacts is the cornerstone of an effective contact tracing strategy. COVID-19 promoted new digital solutions to mitigate the pandemic impact on individuals and health systems. DCT is an emerging new technology, but we report scientific evidence on its effectiveness is supported by insufficient data.

The indicators proposed by the WHO and ECDC framework are suggested to be viewed as a list of alternatives from which public health authorities can choose, while considering the local implementation environment (7).

From our data analysis the “DCT use indicators” and, more in particular, rate of DCT app download and rate of DCT app use, are those most frequently measured: we observed a variable but low level of use percentage (A.1 indicator), ranging from 0.01% to 58.3%. This is more evident in some Countries such as Italy and Switzerland.

The correlation between rate of DCT use and absolute infection case reduction is hard to be assessed, but several studies on automated contact tracing for COVID-19 found out that a very high uptake is required to substantially suppress transmission. A systematic review by Grekousis and Liu (18) reports that, according to mathematical modelling, to reduce effective reproductive number (Reff) to less than 1, the population uptake of DCT apps needs to be around 90%. However, some studies are more optimistic: Abueg et al. estimate a high reduction in infection rate (73-79%) and deaths (69-78%) also with a lower DCT uptake, around 75% (19); Currie et al. estimate that the number of new infections could be reduced by more than a half with an adoption rate of 61% (20).

According to the available literature it is clear that the app uptake emerged by the studies analysed in our review is not sufficient to obtain a substantial reduction of Reff.

Our study has to be interpreted in light of both strengths and limitations. The strength of this review is that it is among the first, to our knowledge, assessing the effectiveness of the DCT applications currently in use worldwide. More importantly, we pooled, analysed and assessed available data on DCT effectiveness applying the conceptual framework produced by international health authorities (7).

Study limitations reflect the innovative nature of this systematic review. As a matter of fact the Indicator framework to evaluate the public health effectiveness of digital proximity tracing solutions edited by WHO and ECDC has been published on 28 June 2021 (7), whereas the oldest paper reviewed has been published on April 2021.

The direct consequence of this fact is that of the three dimensions analysed (use, success and performance), only “use” has a satisfactory amount of reported outcomes. “Success” has only one paper by Salathé M, et al. (11), with clear results and the “Performance” dimension has not been appraised by any of the papers. Other limitations are linked to the characteristics and quality of included studies. All data came from observational studies, which are more vulnerable to biases. One of the most cited bias is the so-called “Digital divide” reported by the cross-sectional studies that included online questionnaires. It consisted in the overestimation of app users, as the populations recruited had access to the internet and so were more likely to have a smartphone (8,10,14,16). However, the study population was representative in all the papers included. Finally, we limited our research to articles written in English.

This systematic review is conducted in an era where the debate on the usefulness of DCT is still ongoing and can lead researchers to the fill the gaps of literature.

As also evidenced by the framework authors, one of the most preeminent obstacles in the global impact assessment of DCT on health, social and economic aspects, consists of the organizational discrepancies for the epidemiological surveillance due to a different characterization of regional health systems local arrangements.

Many countries acknowledge the benefit of location data and centralized management, as it allows public health authorities to pursue more extensive investigations. Unfortunately, such features do so at the expense of privacy protection (7). Even if this approach could be helpful, it must be discouraged for ethical implications and to promote general population acceptance. Actually privacy concerns have emerged to be a relevant driver of acceptance of DCT solutions (10,21).

The results of this systematic review show that COVID-19 pandemic has induced a process of implementation of already existing digital tools through the definition of varied and heterogeneous apps that allow user data to be shared using GPS or Bluetooth technologies (22–25); however any implementation of DCT should include national or international legislation to protect personal data and privacy of users also in order to increase perceived security. Hopefully this would boost the download percentages, as well as effectiveness, according with WHO definitions (7).

In conclusion DCT could be one of the keys for countering the spread of SARS-CoV-2 but more specific evaluations are requested.

The WHO-ECDC Indicator framework to evaluate the public health effectiveness of digital proximity tracing solutions should guide the planning and conduction of future research, which is needed to evaluate and quantify the effectiveness of different DCT solutions to control epidemics and inform successful infection control strategies.