The CLAIRE COVID-19 initiative: approach, experiences and recommendations.

A volunteer effort by Artificial Intelligence (AI) researchers has shown it can deliver significant research outcomes rapidly to help tackle COVID-19. Within two months, CLAIRE's self-organising volunteers delivered the World's first comprehensive curated repository of COVID-19-related datasets useful for drug-repurposing, drafted review papers on the role CT/X-ray scan analysis and robotics could play, and progressed research in other areas. Given the pace required and nature of voluntary efforts, the teams faced a number of challenges. These offer insights in how better to prepare for future volunteer scientific efforts and large scale, data-dependent AI collaborations in general. We offer seven recommendations on how to best leverage such efforts and collaborations in the context of managing future crises. © Springer Nature B.V. 2021.

Introduction

Inspired by successful early use of AI by China, Taiwan, Singapore and South Korea to support the management of the COVID-19 pandemic, on 20 March 2020 CLAIRE, the Confederation of Laboratories for AI Research in Europe (CLAIRE) launched a volunteer effort to help tackle the pandemic. As the World's largest, non-profit network of AI researchers, CLAIRE was quickly able to recruit 150 volunteer AI researchers. This report describes the major activities and achievements of these volunteers, and shares experiences, lessons learnt and recommendations.

The starting point for CLAIRE's COVID-19 initiative was the insight that the AI community has much to offer in support of efforts to handle the pandemic, its societal and economic consequences, and many AI researchers and practitioners stood ready to help public institutions in the front line of the crisis (Luengo-Oroz et al. 2020). Our conviction was that AI could be successfully used across a broad spectrum of areas directly related to managing the COVID-19 crisis, such as:

Analysis of existing drugs to test their efficacy against COVID-19

Analysis of data from patients in intensive care, to support prioritisation in triage and therapy

Analysis of epidemiologic and mobility data, with the goal of better modelling and predicting the spread of the virus, and of facilitating the assessment of impact of containment actions

Use of advanced 3D printing approaches, with the goal of alleviating the scarcity of equipment for protection and intensive therapy

Use of automated scheduling and resource management approaches, with the goal of efficiently managing scarce resources in the medical sector (ICU beds, ventilators, specialists) and other key elements of public infrastructure (personnel, warehouses).

These and many other examples suggest that AI techniques can play a key role in assisting human experts with managing the pandemic and its economic aftermath. We note that, as evident even from the small set of examples given above, it is clear that a broad spectrum of AI techniques and approaches can be brought to bear; for this reason CLAIRE, whose research network spans all areas of AI, across all of Europe, saw itself as particularly well-positioned to mobilise bottom-up support for the use of AI techniques and expertise in fighting the pandemic and in managing its impact on societies across Europe and the world.

Setup phase

Directly after CLAIRE’s COVID-19 initiative was launched in late March 2020, a task force was put into place to coordinate the effort and the volunteer experts supporting it.

This task force collected information on the various initiatives on leveraging AI techniques in the context of COVID-19 and supported the development of new projects, connecting the European network of AI experts together with health institutions and governments. By the end of March, the task force had enrolled 150 volunteers, covering the full spectrum of AI methods, tools and technologies. Volunteers indicated their willingness to work on one or more of 11 research topics. Of these, a significant number of volunteers and topic team leaders were found for 7 topics:

Epidemiological data analysis—10 volunteers

Mobility and monitoring data analysis—36 volunteers

Bioinformatics (protein and molecular data analysis)—25 volunteers

Image analysis (CT scans, X-ray)—47 volunteers

Social dynamics and networks monitoring—8 volunteers

Robotics—5 volunteers

Scheduling and resource management—30 volunteers

AI & COVID-19 resource database

An initial challenge was to ensure volunteers were aware of other initiatives already underway globally. By crowdsourcing volunteers’ efforts, a catalogue of AI & COVID-19 related resources was assembled. Although not designed to be exhaustive, this catalogue now lists 129 resources, covering:

Funding opportunities (21 calls still open at time of drafting)

Datasets (30 resources)

Hackathons, challenges and webinars (7 listed)

Other initiatives (71 listed)

To enlarge the list of initiatives and provide additional information to our volunteers we linked our database to the following catalogues of funding and initiatives:

GovLab repository

COVID-19 research funding monitor

Coronavirus Funding Opportunities

NIH Open-Access Data and Computational Resources to Address COVID-19

Overview of research activities

The 7 groups of volunteers, led by the topic coordinators and with the support of the task force team, are working on several outcomes summarised below.

Epidemiological modeling and decision support

Topic coordinator: Ann Nowé, Vrije Universiteit Brussel, Belgium.

No. of volunteers: 10.

This research group works on different types of models for epidemics (Pernice et al. 2020; Report 9: Impact of non-pharmaceutical interventions (NPIs) Response Team xxxx; Data Science Institute and UHasselt xxxx), ranging from high level compartment models to agent based models, and how they can be used to study the dynamical aspects to improve complex decision taking on the effectiveness of prevention strategies. On the one hand, this involves model fitting and optimisation, on the other hand, learning and optimisation of prevention strategies, using epidemiological models as simulation environments (Libin and Guiding . xxxx).

Work is underway to identify collaboration mechanisms and structures, considering the support AI can offer in decision-making. This recognises the multi-criteria nature of the problem, balancing the needs of different stakeholders all of whom should be involved.

Mobility and monitoring data analysis

Topic coordinator: Jose Sousa, Faculty of Medicine, Health and Life Sciences, Queen’s University Belfast, Northern Ireland.

No. of volunteers: 36.

This work sets out to understand the symptoms progression through self-reported data and its integration with mobility to forecast healthcare decision making. The goal is the development of an AI multilayer learning approach capable of creating evidence based knowledge, using complex networks for self-supervised learning (LeCun et al. 2015), spatial temporal analysis and deep learning.

Work is underway to understand the data collected under the several self-reporting systems (Sun et al. 2020) and test how useful the self-reported data is to forecast events (Real-time tracking of self-reported symptoms to predict potential COVID-19 2020). The initial models will be produced using different methodologies and compared with the officially reported statistics.

Bioinformatics (protein and molecular data analysis)

Topic coordinator: Davide Bacciu, Computational Intelligence and Machine Learning Group, Universita’ di Pisa, Italy.

No. of volunteers: 35.

Work on this topic aims to (1) support the community in characterising the disease from its related structural information, including prediction of viral protein folding; (2) study the interactions between the virus and human hosts, including analysing protein–protein interaction data; (3) design and validate methodologies for filtering, retrieval, and generation of targeted drugs leveraging molecular and well as proteomic information; (4) deliver predictive insights onto the genetic features of the virus.

As a first contribution to the community, the workgroup has created a curated collection of COVID-19-related datasets useful for drug-repurposing tasks, integrating data from multiple studies (Cheng et al. 2019; Ashburner et al. 2000; Janet Piñero et al. 2019; Rose Oughtred et al. 2019) and releasing it as a network comprising protein interactions (Cheng et al. 2019), viral-host interactions (Rose Oughtred et al. 2019), genomic information (Ashburner et al. 2000) and drug interactions (Cheng et al. 2019). This resource has already been released to the community. The group will use the resource to provide a methodology for fast retrieval of drugs whose action can be correlated to target proteins, by leveraging deep learning for graphs (Bacciu et al. 2020).

Image analysis (CT scans)

Topic coordinator: Marco Aldinucci, Computer Science Dept, University of Torino, Italy.

No. of volunteers: 48.

Research in this area aims to (1) distil the current state of the art of methodologies and data sets for AI-assisted diagnosis of COVID-19 by way of imaging (TC Scan, X-ray, etc.), with the goal of making diagnosis faster, cheaper and more manageable in the hospital processes (e.g. using low-resolution images); and (2) to contribute to the improvement of multidisciplinary knowledge by cross-breeding knowledge in computer science and radiology aiming at creating better, more informative reference datasets, together with data-gathering strategies, beyond the current outbreak (Tartaglione et al. 2004; Shi et al. 2020).

The team is developing a review paper and contributes to already active projects, including EU H2020 DeepHealth and EU ERDF HPC4AI. Two further projects are motivated by the strongly perceived need to distil science from the hype COVID-19 induced in different aspects of everyday life, including scientific works (Deephealth project: EU ICT-2018; EU ICT-2018).

The first addresses a reproducibility and benchmarking task: the main publically available deep neural networks and datasets will be collected and cross-validated to compare them across a common baseline. This task will need a substantial human and compute effort. For this, the group is finalising an agreement with the Italian National Supercomputing Center CINECA that will actively support the group activity, which will require both training and inference of the cartesian product of networks, datasets and network parameters. A non-trivial but enabling aspect of the work will be designing and experimenting tools making it possible to bring AI workload to supercomputers and make AI experts efficiently use large scale platforms (Aldinucci et al. 2018; Colonnelli et al. 2002).

The second seeks to consolidate AI performance metrics for both datasets and networks, which will be needed to assess both quality and compute efficiency aspects.

Social dynamics and networks monitoring

Topic coordinator: Manlio De Domenico, Head of Complex Multilayer Networks Lab—Fondazione Bruno Kessler (FBK), Italy.

No. of volunteers: 8

This work uses AI models to analyse social media data together with social, behavioral and economic data for two main purposes: (1) monitor social dynamics to analyse the COVID-19 “infodemic” – “an over-abundance of information – some accurate and some not – that makes it hard for people to find trustworthy sources and reliable guidance when they need it” (WHO - Novel Coronavirus(2019)–with the goal of identifying, monitoring and analysing the overload of unreliable information; of collaborating with data providers to obtain free access to relevant data; and of creating an interdisciplinary hub of experts to fight the “infodemic”; and (2) develop early-warning signals to support policy, informed by spatio-temporal analysis of emotions and sentiments; quantifying and modelling the socio-behavioural response. Social media are playing a crucial role for spreading information, both reliable and unreliable, during the COVID-19 pandemic. Efforts are devoted to unravel the role played by both humans and software-assisted (i.e., social bots) in disseminating false or inflammatory content for social manipulation, a phenomenon recently discovered during political events (Stella et al. 2018), with the ultimate goal of attracting or driving collective attention (Domenico and Altmann 2020) towards a specific information.
Products of the individual team members, such as the infodemic observatory model (Gallotti et al. 2004) developed by the topic coordinator within the Complex Multilayer Networks Lab at FBK, allow to monitor the current infodemic globally, in each country, or at sub-regional resolution in real time. Information, complemented with the analysis of cognitive content, based on natural language processing and computational psycholinguistics, might help to shed light on mass psychology and socio-behavioral response to the pandemic. Results can be used to support policy and decision makers with adequate and zone-specific actions.

Such tools can be disseminated and further developed with the support of the entire research team.

Robotics

Topic coordinator: Alessandro Saffiotti, AASS Cognitive Robotic Systems Lab, School of Science and Technology, Orebro University, Sweden.

No. of volunteers: 5

Work in this area investigates possible uses of robotic systems and robotic technologies in response to the current COVID-19 emergency and to its aftermaths, as well as strategies to improve technological preparedness to possible future crises Specifically, this team has studied: the use of mobile robots for disinfection of environments; specialized laboratory robots for biological tests and drug development; telepresence robots for social and medical assistance; manufacturing robots for flexible production. These uses of robotic technologies are in line with a recent editorial in Science Robotics (Yang 2020).

The group maintains a catalogue of robotic offers and demands relevant to the COVID-19 emergency, and it is liasing active research laboratories across Europe. We have found that the liaison aspect is especially important during a crisis, when access to laboratory resources and material may be seriously limited. It is also supporting euRobotics (the association of European robotic stakeholders) in writing a white paper on the potential usage of robotic technology in the COVID-19 emergency.

Scheduling and resource management

Topic coordinator: Marco Maratea, Dipartimento di Informatica, Bioingegneria, Robotica e Ingegneria dei Sistemi. University Genova, Italy.

No. of volunteers: 30.

The group working on this topic has focused on automated planning and scheduling, and resource management in healthcare systems leveraging AI (deductive) methodologies and tools. An initial assessment of relevant resources has been completed, and a review of relevant publications, data and projects is underway. In addition, collaboration with the Galliera hospital in Genova, Italy, is underway to assist with workforce scheduling and automated planning of the utilisation of operating rooms with scarce resources and equipment (Alviano et al. 2018; Dodaro and Galatà 2019).

Recommendations for future efforts in a crisis

Unfortunately, it is more than likely that our societies will be confronted in the not-so-far future with other crises of similar scale. The results of our efforts thus far demonstrate that rapidly assembled volunteer efforts including large teams of experts, although complex to initiate and coordinate, can make valuable contributions in this context. Important initiatives may result from bottom-up efforts, which may consolidate in white papers, joint project applications and dissemination of annotated datasets. However, preparedness for such future events can be improved in a number of ways, and there are important lessons to be learned. Here, we outline some of our experiences and cautiously formulate some recommendations based on these.

Involving domain experts and public authorities is difficult in a crisis

Many domain experts in public and medical authorities were already preoccupied with tackling the pandemic, limiting their scope to assist volunteer teams. As a result, teams had to develop their own analysis of the problems to tackle, seeking to engage with experts later in the development process as the crisis began to be controlled. The teams who most rapidly developed research outputs were those where the topics being worked on were close to their existing expertise.

Recommendation 1

Effectively interfacing with domain experts and public authorities in a crisis situation is challenging, but this should not discourage qualified volunteers.

Recommendation 2

The contribution that voluntary expert teams can make should be taken into account in planning for future crises.

See also Recommendation 6 below.

The need for open licenses and standards for data

The ethical implications of processing medical and other sensitive data, and the strategic and policy impacts of research during a crisis pose major challenges. While researchers were fully committed to respecting European citizens’ privacy, in accordance with European values, fundamental rights and regulations such as GDPR, the weak standardisation of the data collected on COVID-19 and embryonic state of open data access in the medical and epidemiological field made it difficult to compile bigger data sets needed for the data-driven approaches. Thus ethical, data management and standardisation efforts should be carefully considered from the outset of future volunteering efforts.

Open licenses designed by Creative Commons have been used for several of the products from this effort to encourage reuse. CLAIRE has more broadly analysed the Creative Commons Open COVID Licence, and welcomes this open approach to sharing research products. Accelerating the development of open licenses and standards for medical data and models, such as epidemiological models, and applying them consistently would reduce these challenges in future.

Recommendation 3

Address and coordinate ethical issues, standardisation and data management at the beginning of the research effort. Consider using open licences to support and accelerate data availability.

The need for large open datasets and infrastructures

Many AI techniques, notably from the area of machine learning, depend on access to large-scale data. Work with platforms like Twitter and Facebook, and with mobile telecommunication service providers, should become more routine in the future to help speed up the large-scale analyses required to inform policy based on quantitative measures of human behavioural responses to the pandemic. This also demonstrates the need for a European data space, such as that proposed in the European Data Strategy and the European Open Science Cloud, which should include such datasets. Of course, it is very important to not only ensure the quality of such data sets, but also to protect the rights of citizens, in particular their privacy.

Recommendation 4

Support the development of a European data space and an open data approach to medical and sensitive data for scientific purposes, while protecting individuals anonymity, dignity and human rights.

The role of large scale research infrastructures

The ability of large-scale research and development infrastructures, such as the Robert Koch and Francis Crick Institutes (or, indeed, Apple and Google), to redeploy expertise to work effectively on the pandemic is notable. They have offered public authorities single points of contact for key expertise and helped rally efforts of related communities.

Many scientists in AI have shown they are eager to dedicate significant time and effort to voluntary activities which might not be necessarily sustainable in a short-time horizon or according to more conventional funding channels. The CLAIRE initiative purposefully decided to build on this to go beyond a sterile communication exercise, to bootstrap a number of concrete scientific collaborations on a voluntary basis. While this effort has demonstrated that volunteer efforts can be effective, this observation supports the case for a large-scale investment into an AI hub (or lighthouse centre) in Europe, acting as the reference point of European nations and institutions for all AI research and development.

Since the largest part of the research community kept on using conventional means and strategies to deal with the COVID-19 crisis—notably: competitive research funding, publication (though accelerated by a largest use of arXiv distribution services) and networking—a European centre for AI could promote an innovative approach to research collaboration, funding and dissemination.

Fast international uptake was only possible thanks to existing research networks and network organisations, or built upon personal international academic networks or ongoing project consortia. Outreach and links to initiatives across the EU network were difficult to deploy or to set up, given the grass-roots nature of most initiatives.

A European network would establish permanent relations with all the relevant AI institutions and initiatives globally.

Recommendation 5

Establish a European hub (or lighthouse centre) for AI of very substantial scale.

Bridging communication between medical and AI expertise

The software platforms and hardware with suitable computational power needed to make use of advanced AI techniques (as recommended by AI experts) are lacking in many hospital environments, whether for e-Health or other solutions. While the function of hospitals is first-and-foremost to deliver medical care, encouraging the future development collaborations between local hospitals and AI researchers and investment into infrastructure that enables these collaborations during normal times would help reduce these barriers in future crises. The approach can be extended to other areas using, for example, national risk registers that identify topics of concern to build networks with those who have to manage crises.

Recommendation 6

Set up stable collaboration between hospitals and AI researchers, and other areas of work where future crises can be expected.

Organising large teams remotely

Remotely organising teams of as many as 47 volunteers to quickly decide on research priorities and means of delivering that research presents its own challenges. Preventing fragmentation that dilutes effort, documenting research plans and the work underway, communicating effectively within the group and disseminating results become significant overheads that are not easy to resolve using slow, traditional methods. There is a rich selection of tools available to address many of these issues, used especially in the software industry, but familiarity with these tools within the scientific community varies greatly. This pushes teams towards the simplest, lowest-common-denominator, legacy solutions as well as towards pre-existing networks and project teams. While even simple videoconferencing and document sharing tools have enabled substantial work to be progressed and completed, scientific researchers should build their familiarity with complex and feature-rich collaboration tools that now exist. This will improve inter-institutional research, assisting both building of teams, internal collaboration, and dissemination of results.

Recommendation 7

Scientific researchers should become fluent with the collaboration tools and techniques routinely used, for example, in the software industry.

Conclusions

As the COVID-19 pandemic and its wider ramifications have yet to play out, it is far too early to draw any definitive conclusions. But the outcomes of CLAIRE's COVID-19 initiative suggest that bottom-up, expert-driven, non-profit endeavours can play an important role. It also offers further evidence supporting the basic premise that AI can and should play a key role in handling crises such as this one, both in the health and medical aspects of the COVID-19 pandemic, and in the societal and economic recovery to come. The highly interdisciplinary nature of AI makes it an ideal discipline to create bridges with other scientific domains to attack important societal problems and crises.

In crises resources are limited, time is of the essence and the consequences of action or inaction are severe but difficult to predict. We are convinced that AI, with its potential to support human analysis, planning and decision making has much to offer not just in the context of the current pandemic, but also for handling future crises. We have shown that many experts are willing to work quickly together on novel solutions for the benefits of society. Such collaboration depends on quick access to large amounts of data and information, computation and, most importantly, to each other even when social distancing measures severely restrict physical interaction.

We remain aware that when used, developed and deployed under the pressure of exceptional circumstances, AI technology can be a double-edge sword. It is technologically quite easy to put in place systems that might be difficult to dial back once the crisis is over, eroding privacy and other fundamental rights aided by advanced AI tools and techniques. We must not allow this to happen and must develop standards and frameworks that permit rapid progress without eroding human dignity.

Especially in times of crisis, we need to keep our eyes and resources firmly on AI that enhances human intelligence, helps us recognise and avoid our biases and limitations, and that is designed and used to protect and further our interests as individuals and societies, managing potential risks and reinforcing our European values and the goal of developing human-centred AI.

Despite its limited scale, the experience of the CLAIRE COVID-19 initiative has not only made concrete progress on COVID-19 problems, but offered insights on the potential and limitations of a non-conventional, voluntary and bottom up approach based on the good will of AI experts, fully aware of the societal role of their knowledge and of the importance of an open dissemination of science and contributions to open scientific data collaborations with all stakeholders of our innovation ecosystem.

Additional information on the CLAIRE COVID-19 initiative can be found in the website https://covid19.claire-ai.org/