A Systematic Review of Smartphone Applications Available for Corona Virus Disease 2019 (COVID19) and the Assessment of their Quality Using the Mobile Application Rating Scale (MARS).

The global impact of COVID-19 pandemic has led to a rapid development and utilization of mobile health applications. These are addressing the unmet needs of healthcare and public health system including contact tracing, health information dissemination, symptom checking and providing tools for training healthcare providers. Here we provide an overview of mobile applications being currently utilized for COVID-19 and their assessment using the Mobile Application Rating Scale. We performed a systematic review of the literature and mobile platforms to assess mobile applications currently utilized for COVID-19, and a quality assessment of these applications using the Mobile Application Rating Scale (MARS) for overall quality, Engagement, Functionality, Aesthetics, and Information. Finally, we provide an overview of the key salient features that should be included in mobile applications being developed for future use. Our search identified 63 apps that are currently being used for COVID-19. Of these, 25 were selected from the Google play store and Apple App store in India, and 19 each from the UK and US. 18 apps were developed for sharing up to date information on COVID-19, and 8 were used for contact tracing while 9 apps showed features of both. On MARS Scale, overall scores ranged from 2.4 to 4.8 with apps scoring high in areas of functionality and lower in Engagement. Future steps should involve developing and testing of mobile applications using assessment tools like the MARS scale and the study of their impact on health behaviours and outcomes.

Background

As the Covid-19 pandemic unfolds, it becomes imperative to expand public health activities to estimate the epidemiology for this virus and assess its short term and long term impacts. [1] This medical disaster necessitates innovative solutions to address unmet needs as the disease continues to evolve and infect people globally. [2] Collection of adequate exposure information, characterizing disease burden and dissemination of information that helps with prevention, containment and contact tracing are vital to supporting public health efforts. In such situations, the wider use of social media for education and telehealth services creates ground for greater use of mobile health applications for self-guided collection of population-level data, delivery of information, addresses knowledge gaps, prevents the spread of misinformation, navigate individuals to accurate and validated health resources. [3-5]

Mobile applications provide a unique opportunity in terms of its ability to allow distant connectivity with flexibility of function, design and accessibility. [6] Current estimates suggest that the global usage of smartphones stands at 3.5 billion as of June 2020 that provides a large userbase to help coordinate and implement mobile interventions to disseminate and help with pandemic response. [7, 8] For example, the Singaporean government released a mobile phone app, TraceTogether, that is designed to assist health officials in tracking down exposures after an infected individual is identified. [9] Further, Israel passed a legislation to allow the Government to track people with suspected infection through access of their mobile phone data. [10] In the U.S. recent legislations have expanded scope of telemedicine to involve use of mobile applications for healthcare and preventive reasons. [11] Numerous applications have already been developed globally and are available (https://mhealth-hub.org/mhealth-solutions-against-covid-19) . In the U. S and UK, the COVID Symptom tracker app developed with the Coronavirus Pandemic Epidemiology Consortium (COPE) is being widely used to track COVID-related exposure and infections. [12] However, the downside of this involves access to a large number of applications, some of which can be sources of misinformation or information overload thereby creating barriers to containment and mitigation efforts. [13] Moreover, no universal guidelines exist to ensure quality of these mobile applications to a gold standard.

Our current project aims to provide a systematic review of the literature on mobile applications with multiple function including: information dissemination (ID), ID with symptom checking (SSC), contact tracing alone, SSC, training health care workers, disinfection checklist, and service Apps around COVID-19. Further, we provide an overall rating of the quality of the apps using The Mobile App Rating Scale (MARS) [14], a standardized metric, after providing an overview of their functions.

Methods

Our systematic review was registered with PROSPERO (CRD 42020180878), consisted of a mixed methods study where we provide an overview of smartphone applications designed for COVID-19 through both a literature search as well as a search of mobile platforms for applications. We describe the app features and assessed their quality in terms of engagement, functionality, aesthetics, and information provided using the MARS instrument.. [14]

Search strategy

Mobile applications were searched on the Apple AppStore (AAS), Google Play Store (GPS), National Health Services (NHS) Apps library and Myhealthapps.net. using the keywords- Covid, Corona, Pandemic, Covid-19, SARS-COV2, Symptom, Health, Tracing, Novel coronavirus, 2019-nCoV, and Tracking. We limited our search to applications released or updated for COVID-19 till May 5th, 2020 and screened these applications for their title, description of the application on mobile platforms, its functions around COVID-19 and then completed a comprehensive assessment of these mobile applications (Fig. 1a). Due to rapidly evolving nature of the COVID-19 pandemic and language restrictions, we limited our search to the United States, United Kingdom and India. Only those smartphone applications with the main subject matter as Covid-19 and in the languages English or Hindi were included in the study. Those applications that were duplicates and pay to download were excluded. Two reviewers each, based across the United Kingdom, United States and India and with access to these applications performed a comprehensive assessment of these mobile applications. Discrepancies regarding selection of apps were resolved by a third reviewer in order to reach a consensus. (Supplementary Table 1).

Fig. 1

a PRISMA- Mobile app search. b PRISMA- Literature search

Subsequently, data was extracted from each app (Supplementary Table 2) and each app was assessed using the MARS rating scale. [14] The MARS evaluation tool is divided into three broad sections- App overall quality, App subjective quality, and App specific quality. It consists of 23 questions that are designed to help analyse engagement, functionality, aesthetics, information, and subjective quality of the mobile applications. In addition, there are six final app specific questions that can be tailored to represent the target health behaviour/ function of the application/study (Fig. 2). The mean scores for each of the four subscales (Engagement, Functionality, Aesthetics, and Information) was calculated, and the mean scores of those subsections, used to rate the total quality score of the app ranged from zero to five. Hence the overall score on these subscales would range from 0 to 5 and we would develop an average score by dividing the total sum score by 4 i.e. 4 domains. The App subjective quality section and App specific section was calculated similarly. These, however, were regarded separate from the app quality score. All reviewers underwent training in the use of MARS using a 37-min video available on YouTube [15] which was followed by an independent review exercise of a common app and discussion of scores to assess consensus. Disagreements were discussed with a third reviewer, and ambiguous MARS items were clarified to ensure full comprehension of the scale. [14]

Fig. 2

MARS structure

Statistical analysis

Unpaired students t-test was performed to analyse the difference in the means obtained in each section by Asian, European, and North American apps with a p value of <0.05 considered as significant. Pearson’s correlations were used to analyse correlations between app downloads and MARS ratings. Microsoft excel was used for data segregation and analysis.

Literature search

A systematic search was conducted on PubMed (Medline), Cochrane central register for controlled trials, Google Scholar and Turning Research Into Practice medical database from 9-10th of May 2020 to identify peer-reviewed publications and grey literature on the utility of smartphone applications in the Covid-19 pandemic and help aid our ongoing assessment of mobile platforms. [16]

The screening process was conducted by two review team members using the keywords: “Covid”, “Corona”, “Pandemic”, “Covid-19”, “SARS-COV2”, “Novel coronavirus”, “2019-nCoV”, and “smartphone”. The articles were screened for relevance by title, abstract and eventually full-text review was performed for included abstracts (Fig. 1b). Articles in languages other than English were excluded. Those studies in which the subject matter was unrelated to the use of smartphone apps in Covid-19 were excluded.

Results

Our search identified 63 apps that were reviewed and performed a quality assessment on using the MARS scale. Of these, 21 apps originated from Asia, 17 from Europe, 22 from North America, 2 from Australia, and 1 from Russia (Fig. 3). Apps were classified based on their key functions- ID (n = 18), ID with tracking of individuals (n = 9), or ID with SSC (n = 15), contact tracing alone (n = 8), SSC (n = 7), training of healthcare workers (n = 3), disinfection checklist (n = 1), and service apps [Delivery or fund-raising] (n = 2).

Fig. 3

Map of country of origin

An overview of characteristics of apps

Of all the apps, 27 (39%) were developed through federal efforts, 14 (21%) by commercial units, 5 (7%) by a University, 1 (1%) by an NGO and 16 (24%) by unknown organisations/ entities (Supplementary Table 2). Data extraction for the number of downloads of the app was only possible on the android apps as that information is not available on the Apple app store (AAS). The average star rating of apps on both the stores was 4.2 (±0.07).

The number of app downloads ranged from 10+ to 50,000,000+. The most downloaded and reviewed app on the Google play store (GPS) was Aarogya Setu developed by the Government of India (50,000,000+ downloads and 343,842 reviews), with a government mandate for download by all Indians especially those who travel or are diagnosed with COVID-19, with a GPS star rating of 4.6 out of 5 (Supplementary Table 2). In addition to providing self-screening technology, it displays alerts of COVID-19 patients diagnosed within a 500 m, 1, 2, 5, and 10-km radius of the user which is a useful tool to understand the disease load in the proximity of an individual. Average star rating on play store was 4.2 (±0.5).

Mobile applications overview

App quality assessments and ratings using MARS

The mean overall MARS score obtained was 3.7 of 5 (SD = ±0.58). Of 63 applications, 23 apps had a score above and equal to 4, most of which were from the information with symptom checker (8), information with tracking (6), and purely information dissemination (5) categories. Eleven apps had a score of less than 3 (Table 1). The highest rated app was Covid-19 UAE (4.8) closely followed by First Responder Covid-19 (4.8) and the lowest score was for Corona 360 (2.4).

Table 1

App quality ratings

			App quality ratings=					App subjective ratings=	App specific ratings=
AppName:	Categories	Countr-y	A: Engagement Mean Score =	B: Functionality Mean Score =	C: Aesthetics Mean Score =	D: Information Mean Score =	App quality mean Score =	Subjective=	Awarenes-s	Knowledg-e	Attitude-s	Intentio-n to change	Help seeking behaviour-r	Behaviou-r change
Jaano	Information D + Tracking	Asia	3.4	5	3	3.4	3.63	3	4	4	4	3	3	3
Covid Locator	Information D + Tracking	Asia	3.6	5	4.3	3.4	4.03	3.5	3	3	3	3	4	3
COVA Punjab	Information D + Tracking	Asia	3.6	5	4.3	3.8	4.14	3.5	3	2	2	3	2	3
Corona Watch	Information D + Tracking	Asia	3	5	3.6	4	3.83	3.2	3	1	3	2	4	3
Aarogya Setu	Information D + Tracking	Asia	3.6	5	4.3	4.1	4.22	3.75	1	4	2	2	2	2
MapmyIndia Move: Maps, Navigation & Tracking	Information D + Tracking	Asia	3.4	5	4.6	1.8	3.44	1.2	3	2	1	1	2	1
T COVID’19	Information D + Tracking	Asia	4.2	5	5	3.8	4.47	3.5	3	4	3	3	3	3
Odisha Covid Dashboard	Information D + Tracking	Asia	3.8	5	5	3.4	4.24	3.5	3	3	3	3	3	3
Kavach	Information D + Tracking	Asia	4	5	4.6	3.8	4.32	3.5	3	4	3	3	4	4
Niramaya	Information d + check symptoms	Asia	3.2	5	4.3	2.7	3.69	2.5	3	3	3	3	3	3
Canada COVID-19	Information d + check symptoms	North America	3.6	3.75	4.33	4.6	4.05	3.25	4	5	4	4	5	4
BC COVID-19 Support	Information d + check symptoms	North America	3.4	3.75	3.33	4.4	3.70	3.5	4	4	5	4	4	4
Ask NHS	Information d + check symptoms	Europe	3.8	4.5	4.33	4.5	4.27	4.75	4	5	4	4	5	4
COVID-19 Gov PK	Information d + check symptoms	Asia	3.2	4	3.33	3.8	3.57	3.5	4	4	3	4	4	2
NHS24:Covid-19	Information d + check symptoms	Europe	3	5	4.67	4.75	4.27	4	5	5	4	3	5	4
CoVive: your COVID-19 app	Information d + check symptoms	Europe	3.4	5	4.6	4.5	4.33	3	3	5	4	2	4	2
Apollo COVID-19	Information d + check symptoms	North America	2.4	4.75	4	3.5	3.55	2	4	4	2	2	4	2
First Responder COVID-19 guide	Information d + check symptoms	North America	5	4.75	5	4.4	4.78	4.48	3	4	3	4	4	3
Coronavirus Australia	Information d + check symptoms	Australia	4.2	4.25	4.7	4	4.28	3.86	3	4	4	3	4	3
Covid-UAE	Information d + check symptoms	Asia	5	5	5	4.2	4.79	4.54	5	5	4	5	3	4
Apple COVID-19	Information d + check symptoms	North America	4.6	5	5	4.5	4.77	4.57	5	5	4	4	3	3
Your.MD: Health Journal & AI Self-Care Assistant	Information d + check symptoms	Europe	2.6	4.67	4	3.5	3.61	2.75	2	3	2	2	4	2
Symptom Tracker & Medicine, Health Symptoms Diary	Information d + check symptoms	North America	3.2	3.5	4	2.67	3.31	3.25	1	1	2	1	4	3
WebMD: Check Symptoms, Find Doctors, & Rx Savings	Information d + check symptoms	North America	2.8	4	4.33	4	3.73	3	3	4	4	2	N/A	3
Babylon- Healthcare service	Information dissemination	Europe	3.6	5	4.66	3.6	4.17	3.25	4	3	3	4	5	3
MyGov	Information dissemination	Asia	4.2	4	3	4.33	3.84	3.25	4	4	3	4	3	2
HEALTHLYNKED COVID-19 Tracker	Information dissemination	North America	3	4.5	3.33	3.6	3.57	2	2	3	3	2	3	3
Trackmetic	Information dissemination	Asia	2.2	5	3	2	2.85	1.2	1	1	1	1	1	1
Raj Covid Info	Information dissemination	Asia	3.6	5	3.6	3.4	3.85	2.7	4	3	3	3	4	3
CoronaReport - COVID-19 reports for Social Science	Information dissemination	Europe	2	2.25	2.67	3.5	2.55	1	1	1	2	2	N/A	1
Corona360	Information dissemination	Europe	1.6	3.25	3.33	2	2.43	1	2	1	1	1	N/A	2
COVID19 Care for Media	Information dissemination	Asia	3.2	4.5	3.67	4.25	3.87	3.75	4	5	4	4	N/A	3
Pakistan’s National Action Plan for COVID-19	Information dissemination	Asia	1.4	5	3	3.5	2.93	2	3	2	3	3	N/A	4
ECDC Threat Reports	Information dissemination	Europe	2	3.75	2.67	4	2.99	1.75	3	4	3	4	N/A	2
COVID-19! - The current spread of disease	Information dissemination	Europe	3.4	4.5	4.33	4.4	4.13	3.75	4	4	3	3	3	2
Relief Central | COVID-19	Information dissemination	North America	3.4	4.5	3.67	4.4	3.96	3.75	4	5	3	3	2	2
Coronavirus COVID Tracker	Information dissemination	Europe	3	4.5	4	4.4	3.93	2.75	2	5	3	3	3	2
CoronaFACTS	Information dissemination	North America	2.2	3.25	2.6	4	2.94	2	3	4	2	3	2	2
Central Coast COVID-19	Information dissemination	North America	4	4	4.7	3.5	4.03	3.79	3	3	3	3	N/A	2
Coronavirus--Covid19	Information dissemination	Russia	4.8	5	5	4.3	4.77	3.56	4	3	2	3	3	2
NJ COVID 19	Information dissemination	North America	4.4	4.75	4.7	3.2	4.21	3.96	4	4	4	4	4	3
CDC	Information dissemination	North America	2.6	4.25	4.33	4.7	4.00	3.25	5	5	5	3	5	4
COVID-19 West Bengal Government	Contact tracing	Asia	2.8	5	2.6	1.5	2.72	1.2	1	1	1	1	2	1
UP Self Quarantine App	Contact tracing	Asia	2.2	5	2.3	2.4	2.79	1.5	1	1	1	1	2	1
COVID Symptom Tracker	Contact tracing	Europe	2.4	4.5	3.67	4.33	3.62	3.5	2	1	2	2	N/A	4
Stop Covid	Contact tracing	Europe	2.4	4.75	3.67	4	3.60	2	2	1	1	2	3	4
Contact Tracing	Contact tracing	North America	2.2	4.75	2.3	2	2.63	1.25	5	5	5	5	5	4
PatientSphere for COVID19	Contact tracing	North America	3.8	4	4.3	3.8	3.97	3.68	3	3	2	3	3	2
1-Check COVID	Contact tracing	North America	4	5	4.7	4.8	4.61	4.45	3	5	4	5	5	4
Care19	Contact tracing	North America	3.2	4	4.3	3.2	3.64	3.34	2	1	3	1	N/A	4
Hustle-Serve Your Community	Service apps (Deivery) + Raise funds	North America	3.4	5	4.7	4	4.23	3.92	2	1	4	4	2	4
WE DEFEAT CORONA	Service apps (Deivery) + Raise funds	Europe	3.6	5	5	3.75	4.29	3.82	5	2	2	2	N/A	1
Disinfection Checklist COVID19	Disinfection check	North America	3.8	2.5	4.3	3.3	3.41	2.87	1	2	4	2	N/A	3
Osler COVID Learning Centre	HCW training	Australia	2.8	3.75	4	4.2	3.64	2.5	4	4	3	3	N/A	4
iCU Notes - a free Critical Care Medicine resource	HCW training	Europe	2.4	5	2.3	2.8	2.96	3	3	4	3	2	2	1
OpenWHO	HCW training	Europe	3	5	3	2.8	3.35	2	4	4	2	2	2	2
Test Yourself Puducherry	Self symptom checker	Asia	2.6	5	3.6	1.8	3.03	1.5	1	1	1	1	2	1
test yourself goa	Self symptom checker	Asia	2.4	4.75	2.6	2.2	2.84	1.5	1	1	1	1	2	1
Symptom Tracker: Health Mate	Self symptom checker	North America	3	4	3.66	4	3.64	1.5	1	1	3	3	4	2
Sentinel Monitor	Self symptom checker	North America	3.2	4.75	3.33	3.75	3.71	1.75	1	2	4	3	5	3
Mediktor- Symptom Checker	Self symptom checker	Europe	4	4.5	3.3	3	3.65	3	1	1	3	3	5	4
Corona Checker	Self symptom checker	North America	3.2	5	4.7	4.4	4.26	4.16	5	3	3	5	5	4
SymplCheck	Self symptom checker	Europe	3.4	3.75	3.33	4	3.61	2.25	2	1	1	3	4	1

All the apps were initially assessed using MARS for Engagement, Functionality, Aesthetics and Information mean scores for each subscale (out of 5) calculated (details in Fig. 2). Overall, apps scored highest in the functionality (4.5, SD = 0.6) domain, followed by aesthetics (3.8, SD = 0.8), information (3.5, SD = 0.8) and engagement (3.1, SD = ±0.8).

Covid-19 UAE scored a 5 in the Engagement, Functionality, and Aesthetics and 4.2 in the information sub-scale. In comparison, Corona360 scored 1.6 in engagement, 3.3 in functionality, 3.3 in aesthetics and 2 in information. Corona360 is an app developed by a private company, which aimed to create a map based visual database of individuals infected with COVID-19 by allowing anonymous sharing of the users’ status of infection along with their approximate location. This app lacked methods of customisation or user engagement. The singular graphic used in the app for a map with locations of infected individuals made it difficult to use and slow to load, which might have contributed to low engagement.

Apps from Asia scored higher in functionality (mean difference 0.54 (± 0.13) 95% CI = 0.3 to 0.8, p = 0.0001), while the UK (8 of 17 from Europe) and North American apps together scored higher in the information subscale (mean difference 0.6 (± 0.2) 95% CI = −1.0 to −0.1, p = 0.01). Aesthetics, engagement and total scores did not differ between the western and Asian Apps. Thus, the apps based in the Indian subcontinent were sufficiently interactive and functioned smoothly, but lacked in either quality, or quantity of information provided. In comparison, those apps developed in North America of the United Kingdom had credible, necessary information, but lacked in creative methods of presentation of this information which could be understood by the masses. A detailed analysis into each sub-scale was performed using the components of the MARS scale [14] as previously described:

Engagement

Two apps scored a 5 in this section- First Responder and Covid-19 UAE while Pakistan’s National Action Plan scored the lowest, with a score of 1.4. First Responder was developed by Stanford University and designed especially for healthcare workers to screen their symptoms and, if needed, schedule a testing appointment at Stanford Health Care. The app has a well-chosen colour scheme. Key features of this app included the use of pictorial representations of healthcare workers admixed with color-coded sections that make it easy for the users to navigate through different sections. It increased its level of interaction with the user by the self-screening feature of the app, wherein the user can assess their health themselves and be given further guidance as to whether they should self-quarantine or go to a health care provider. Enough customisation was present in the form of an edit function in self-screening history and the notification settings.

The Covid-19 UAE developed by the Ministry of Health UAE’s official mobile app on Coronavirus management provides users with real-time Coronavirus case information and keeps the app users engaged by sharing news up to date with regional news and events. Its content presentation includes eye catching graphics such as use of cartoon graphics representing viruses on top of each section. The app gives a color-coded description of current status of patients: including newly diagnosed cases, deaths and those that recovered. Moreover, it allows customisation to three different languages, including English, and a self-assessment section for Covid-19.

Pakistan’s National Action Plan for Covid-19 was created by a private company and aimed to provide information on the steps the country will take to tackle Covid-19. This app scored low in this section as it lacked any form of graphics altogether. The information was present as a PDF with the only interaction being the ability to change the pages and did not engage its users overall.

Functionality

Twenty-eight apps scored a 5 in the functionality sub-scale. Of these, Jaano and COVA Punjab worked with appropriate speed and switching between sections of the app was smooth. The menu labels and icons were interactive and clear instructions and navigations made apps like Babylon- Healthcare service and NHS24:Covid-19 easy to use and easily comprehendible.

As for those apps which scored the lowest (<2.5), the main operation of the app did not work appropriately. For example, if the purpose of the app was to provide a map of infected individuals in the locality, the map itself would not open.

Aesthetics

T COVID’19 and Odisha Covid Dashboard both available on the GPS and AAS, scored a 5 in this section. T COVID’19 developed by Government of Telangana, India from provides updates regarding the cases and deaths due to Covid-19, information about the nearest health centres, essential services, and Government announcements, and allows self-symptom checks and tracking of infected individuals. The salient feature of this application involves use of interactive icons used to depict different sub-sections of the app; for example, the icon for ‘telemedicine’ included a doctor with a chat box next to him. Odisha COVID Dashboard utilises colour codes to depict the main sections of the app, which increases its visual appeal. Each graph is explained carefully and clearly with the colours to depict the case burden i.e. red, orange, green and grey depicting confirmed, active, recovered and deceased cases respectively. Further the map provides a heatmap of the State to represent Covid-19 infection burden across state districts. The graphics used were overall of a high quality. Further, the app posts important announcements including health advisories in forms of health posters.

The apps scoring the lowest in this section (2.3) intended to track the user’s location and alert of a possibility of exposure. Poor aesthetics, including low quality graphics, and small buttons and icons made navigation difficult, and hampered the ability to perform its main function.

Information

The highest rated app in the information sub-scale was 1-Check COVID (4.8) followed closely by NHS24: Covid-19 (4.75). 1-Check COVID was developed by health experts from the University of Nebraska Medical Centre. It has an accurate and precise description on the app store, which clearly depicts the goals, i.e. providing a real-time situational awareness of Covid-19 and public health risk assessment survey. The app asked for details about the users’ symptoms, travel, medical conditions, and exposure with clarity, and provided credible health information links (e.g. Centre of Disease Control) to help the user learn more about Covid-19 features. Further, the NHS:24 app provides appropriate advice based on a simple symptom assessment. It may advise its users to call an NHS number (111) or advise self-isolation for a period, based on NHS guidelines. This information is easy to comprehend and written in simple and easy language. Besides, numerous sections covered a range of topics, and many of these contained further links to other reliable websites (e.g. NHS Inform). The Information was presented under a clear heading, categorised into sections e.g. General Advice, Testing and Physical Distancing.

App subjective quality items and app specific items (Fig. 2)

The median app subjective quality score was 3.25 (±1.01,). Top and lowest scoring apps in the specific items are summarized in Table 1.

Correlation between MARS app quality score, and the number of downloads/ app star rating

The MARS scores and number of downloads did not correlate [r(2) = 0.082, p = 0.65]. No correlation was found between the MARS score and app star ratings either [r(2) = 0.064, p = 0.64]. However, a significant correlation was found between app star ratings and MARS scores for those apps used in North America [r(2) = 0.6, p = 0.006].

Literature search

We identified 144 articles based on our search strategy (Fig. 1b). After reviewing our inclusion and exclusion criteria, 134 research studies were excluded, majority of which were due to their content being unrelated to our study. For example, those articles which focused on specialty practice during COVID-19 and telemedicine, smartphone enabled technology that did not involve an application, or studies that utilized apps to assess for mental health outcomes among COVID-19 patients.

A total of ten studies were found on this subject and have been detailed in Table 2. Majority of ongoing research focused towards development of smartphone app technology for contact tracing with the preservation of user data, apps for information dissemination to healthcare workers in a region (n = 1), and smartphone app enabled technology for diagnostics. The more descriptive studies reviewed the evidence of the utility of smartphone apps for COVID-19 based on countries’ experiences (Table 2). Only one application included in our study, The COVID-19 Symptom Tracker mobile application, conducted an observational study to provide scientific evidence of the app’s utility. Through the application, they were able to acquire important information regarding the population dynamics of the disease amongst the users. Comparisons between symptoms of users and test results allowed generation of hypotheses for further research. Important results in their study demonstrated that the app could predict in advance the COVID-19 incidence of a region. [17]

Table 2

Literature search results

Author and year of publication	Title	Time of data collection	Intervention or exposure	Sample size	Age distribution	Study design	Main outcome
Udugama B, March 30, 2020	Diagnosing COVID-19: The Disease and Tools for Detection	N/A	N/A	N/A	N/A	Review article	Review of the diagnostic and surveillance technologies available for COVID-19
Abeler J,	COVID-19 Contact Tracing and Data Protection Can Go Together	N/A	N/A	N/A	N/A	Descriptive	Described a contact tracing technology that processes minimum user data in order to protect privacy.
Yasaka TM,	Peer-to-Peer Contact Tracing: Development of a Privacy-Preserving Smartphone App	N/A	Transmission graph data structure to represent the interactions between individuals.	N/A	N/A	Descriptive	Developed a contact tracing smartphone app that respects user privacy by not collecting location information or other personal data.
Zamberg I,	A Mobile Health Platform to Disseminate Validated Institutional Measurements During the COVID-19 Outbreak: Utilization-Focused Evaluation Study	February 25th to March 13th, 2020	Mobile app named “Head to Toe” for dissemination of institutional knowledge.	125 healthcare professionals	N/A	Observational study	The use of an mHealth application was an effective tool with benefits of quick communication and daily updates during the COVID-19 outbreak in their hospital setup.
David Siu, Hong kong University.	Using Biovitals® Sentinel to Monitor Disease Progression in Subjects Quarantined for Suspected COVID-19	April 1, 2020 to December 31, 2021.	BIOVITALS platform- Armband+ remote monitoring + analytic platform	200 (estimated)	N/A	Randomised trial	Primary outcomes: Time to diagnosis using RT-PCR, and time from quarantine to diagnosis. This technology can be used to remotely monitor COVID-19 patients
Maghdid H.	A Smartphone enabled Approach to Manage COVID-19 Lockdown and Economic Crisis	N/A	Grey literature	N/A	N/A	Descriptive	Developed a smartphone-based approach to trace the contacts for confirmed COVID-19 cases. Also created a dashboard to provide a plan for government officials on how lockdown/mass quarantine can be safely lifted.
Lu N.	Weathering COVID-19 Storm: Successful Control Measures of Five Asian Countries	N/A	N/A	5 countries considered- China, Singapore, Japan, South Korea, Taiwan.	N/A	Descriptive	Described the successful control measures employed by China, Singapore, South Korea, Taiwan and Japan. Smartphone apps were used successfully by Singapore and South Korea mainly for contact tracing, alerts and self-symptom checks.
Whiting P	Accuracy of self-monitoring heart rate, respiratory rate and oxygen saturation in patients with symptoms suggestive of COVID infection.	April 1, 2020	N/A	One systematic review (n = 381) and three studies (n = 108, 101, 30)	N/A	Systematic review	Identifies the best apps for HR or BP measurement in COVID-19 patients.
Drew D.	Rapid implementation of mobile technology for real-time epidemiology of COVID-19	N/A	COVID-19 Symptom Tracker mobile application	1,234,918	Mean age = 41 (Range = 18–90 years)	Observational study	Developed a COVID-19 Symptom Tracker mobile application that was launched in the UK and the USA. This mobile application offers data on risk factors, herald symptoms, clinical outcomes, and geographical hot spots. Provided proof- of- concept for this approach.
Yang T.	Point-of-Care RNA-Based Diagnostic Device for COVID-19	N/A	Diagnostic device for COVID-19 based on RNA combining paper based and LAMP assay technology that can be performed at home.	N/A	N/A	Descriptive	Proposed a combination of a paper-based POC diagnostic device and LAMP assay technology to allow home-based self-testing. A smartphone application will be used to facilitate test results recording/sharing with health professionals.

Discussion

To our knowledge, our study is one of the first to perform a comprehensive assessment of mobile applications being used during the COVID-19 pandemic. Smartphone applications have an immense potential to control the spread of misinformation associated with the COVID-pandemic, raise awareness, allow contact tracing and finally, help improve provision of the preventive and clinical care of patients. We identified 63 mobile applications from different regions which scored across the MARS continuum and differed in terms of their functionality, aesthetics, information sharing and overall purpose. The app downloads did not correlate with quality, suggesting the need to revisit the app usage strategies among individuals and advisories by federal or health authorities.

Our results identified 63 apps that are currently being utilised across different arenas of the pandemic. Overall, the apps analysed by us demonstrated above average quality. Most (53) of the apps acquired 5/5 in functionality of which only one fifth (12) apps scored similarly in Engagement. This shows that most apps focus on their functioning without considering features that make the app equally engaging and important to an increased userbase. Moreover, the large number of apps delegated by various countries and organisations could potentially cause confusion amongst users regarding the utility and preference. The fact that the app quality did not correlate with downloads supports this possibility. Moreover, extending the usage of quality apps designed in one region to others may be a cost-efficient and relevant administrative exercise in the face of a global economic collapse. We identified Covid-19 UAE, NHS24:Covid-19, Odisha Covid Dashboard, and 1-Check COVID as exemplary apps in the engagement, functionality, aesthetics, and information domains respectively.

Interestingly, the apps developed in Asia were user-friendly and designed to ensure that most of the users understood the application and used it effectively. With over 500,000,000 smartphone users in this country [18] and considerable variations in socioeconomic status including differences in education attainment these characteristics are extremely necessary to be considered in developing a user-friendly app. On the other hand, the apps accessed from Europe/ North America remained focused on information. Though high quality and from credible sources, they often lacked creative or interactive methods to portray this information. This may potentially decrease the assimilation of the knowledge amongst the users, defeating the purpose.

Besides, apps developed in the west tended to focus on information dissemination, with very few contact tracing or symptom checking apps. On the contrary, most apps developed in India have been created with the main purpose of contact tracing, or symptom checking. Emerging events of data leaks and technological failure have raised heightened concerns over data-safety laws which range from very stringent (e.g. Germany) to blurry, and almost non-existent in some countries. In a time with abundant anxiety and chaos alongside over uncertainty about the future, this could be a recipe for disaster. Hence, future applications should take into consideration data safety and privacy norms while implementation and dissemination. [19]

‘Aarogya Setu’ a contact tracing app developed by the Government of India [20] has 50,000,000+ downloads. It’s ability to function adequately and provide credible information make it a suitable app to reach out to the large population of India. However, the requirement of constant blue-tooth access and location tracking, has been perceived as a violation of privacy, more so as it lacks the mandatory layers of system privacy expected by an App promoted at such a scale. [21, 22] The use of centralized digital contact tracing is debatable. Governments like France, UK, and Italy favour centralization wherein their public health authorities receive information immediately, about contacts of infection. Others including companies like Google and Apple prefer a decentralized approach. Here, the contacts are notified of proximity to an infected individual and can choose to share this information with health authorities. Recently a mandate to download the Aarogya Setu app for screening in public spaces received harsh criticism from proponents of ethics and cyber-laws. [23] Although the use of smartphone applications comes with numerous concerns regarding data privacy, studies have shown that it may be possible to use apps to fight Covid-19 while preserving an individual’s right to privacy. [24, 25]

Repurposing can deliver useful and quality apps in a timely fashion, with the added advantage of accessing active users who have already downloaded the app. Apps such as Babylon-Healthcare service, CDC, WebMD, and MyGov have been repurposed for COVID-19. Updating these to perform functions like contact tracing could save time and effort. Moreover, merely 24 of the 63 apps assessed by us had overlapping functions (Information dissemination with self-symptom checker/ contact tracing). The presence of most necessary functions in one place is essential to facilitate a wide usage of the app.

Research into smartphone applications for Covid-19 is sparse. Despite the plethora of Apps in the market, our literature search identified only ten studies pertaining to this subject. Most of them demonstrated that the development of a privacy preserving contact tracing technology enabled by a smartphone app is possible. Two studies focused on the utility of smartphone apps in diagnostics for Covid-19 patients, advocating its use due to its ease and self-diagnostic nature. Patients in need of heart rate monitoring, or those requiring Covid-19 testing could possibly use this novel technology themselves and report it to a healthcare provider. Future Randomised Controlled Trials (RCTs) using mobile applications for Covid-19 and its impact on public awareness, disease spread, and healthcare delivery are warranted. Health apps advocated for the protection of users during a global pandemic such as this must be backed by scientific evidence, without which a strong hold on the quality and utility of the app is bound to be missing.

The psychometric properties of the MARS scale are proven to be reliable and valid, thus the use of this tool provides great strength to our study. [14] With a large range of keywords and an additional literature search, our study has a very wide reach by using a mixed methods approach. Further, our study provides a comprehensive assessment of all mobile applications across three regions which are currently experiencing rapidly rising burden of COVID-19 related cases and deaths and hence, identifying the key scientific features of mobile applications that should be used in design and engagement remains very crucial to sustained public health efforts aimed at mitigating this disaster. However, apps in languages other than English and Hindi could not be assessed, which limits the generalizability of our results to other regions. Despite a high interrater reliability [14] the reviewer’s subjectivity might have influenced the ratings awarded and caution must be exercised while interpreting the results portrayed in this review. In addition, there is a likelihood that the features of the apps portrayed by us are different from the updated versions of the app and might have been addressed in apps developed after this review. This possibility is inevitable considering the rapidity with which apps are developed and reformed. Further, given the scope of the work, we did not take into account end-user reviews of these applications which might differ as compared to our assessment of apps. Finally, the number of downloads for an individual application might differ across regions and be based on specific laws and policies and hence this should not be considered as a sole metric of quality of apps.

Conclusion

In the recent Covid-times, the market has seen the appearance of numerous mobile-applications for tracking and managing the pandemic. The app-market remains disorganized and unregulated in several countries. The present review provides an overview of mobile applications available in United Kingdom, USA and India; summarize their strengths and limitations through a qualitative assessment; and delineates key functions and features needed for future applications. Rapid population-based longitudinal studies and randomized trials would characterize the use and efficacy of mobile apps on health knowledge, behaviours and use to limit the spread of COVID-19 and help reduce its burden on the public health and clinical systems.

Discrepancies resolved while eliminating apps by name and abstract review (DOCX 25 kb)

Characteristics of apps assessed using the MARS (DOCX 25 kb)