Comparative-effectiveness research of COVID-19 treatment: a rapid scoping review.

OBJECTIVES: The COVID-19 pandemic has stimulated growing research on treatment options. We aim to provide an overview of the characteristics of studies evaluating COVID-19 treatment.
DESIGN: Rapid scoping review DATA SOURCES: Medline, Embase and biorxiv/medrxiv from inception to 15 May 2021.
SETTING: Hospital and community care. PARTICIPANTS: COVID-19 patients of all ages.
INTERVENTIONS: COVID-19 treatment.
RESULTS: The literature search identified 616 relevant primary studies of which 188 were randomised controlled trials and 299 relevant evidence syntheses. The studies and evidence syntheses were conducted in 51 and 39 countries, respectively.Most studies enrolled patients admitted to acute care hospitals (84%), included on average 169 participants, with an average age of 60 years, study duration of 28 days, number of effect outcomes of four and number of harm outcomes of one. The most common primary outcome was death (32%).The included studies evaluated 214 treatment options. The most common treatments were tocilizumab (11%), hydroxychloroquine (9%) and convalescent plasma (7%). The most common therapeutic categories were non-steroidal immunosuppressants (18%), steroids (15%) and antivirals (14%). The most common therapeutic categories involving multiple drugs were antimalarials/antibiotics (16%), steroids/non-steroidal immunosuppressants (9%) and antimalarials/antivirals/antivirals (7%). The most common treatments evaluated in systematic reviews were hydroxychloroquine (11%), remdesivir (8%), tocilizumab (7%) and steroids (7%).The evaluated treatment was in favour 50% and 36% of the evaluations, according to the conclusion of the authors of primary studies and evidence syntheses, respectively.
CONCLUSIONS: This rapid scoping review characterised a growing body of comparative-effectiveness primary studies and evidence syntheses. The results suggest future studies should focus on children, elderly ≥65 years of age, patients with mild symptoms, outpatient treatment, multimechanism therapies, harms and active comparators. The results also suggest that future living evidence synthesis and network meta-analysis would provide additional information for decision-makers on managing COVID-19. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Broad literature search and study selection yielded 915 study reports, including 616 relevant studies (188 randomised controlled trials) and 299 evidence syntheses.

Detailed charting of study populations, interventions and outcomes of included studies and reviews were conducted to analyse characteristics and trends in the included literature and to elucidate lessons for future research.

Practical implications for future research with respect to study design, populations, interventions, comparators, outcomes and methodological approaches were identified.

Semiautomation approach to study selection, allowing for a very broad literature search and screening approximately 290 000 titles/abstracts in about 40 person-hours over 2.3 weeks.

This is a scoping review and as such, we did not assess the risk of bias of the included studies and evidence syntheses.

Introduction

The current global pandemic of COVID-19 has resulted in a high burden of disease and mortality worldwide.1 2 The lack of effective treatments for COVID-19 has resulted in the almost constant production of studies and evidence syntheses evaluating potential treatment options, as illustrated by thousands of study protocols in clinical trial registries and hundreds of review protocols in systematic review registries.3 4 Attempts to synthesise this evidence thus far have resulted in various scoping reviews focusing on single drugs or isolated drug classes.5–9 Better understanding of the characteristics of study populations, treatments and outcomes of this research is a prerequisite to the design and conduct of future comparative-effectiveness research.

The objective of this rapid scoping review was to provide an overview of the characteristics of studies examining COVID-19 treatment.

Methods

The conduct of the rapid scoping review was guided by the JBI (formerly Joanna Briggs Institute) guide for scoping reviews, alongside the World Health Organization (WHO) guide to rapid reviews.10 11 Compared with a scoping review, we used streamlined methods in this rapid scoping review (eg, single reviewers conducted study selection). An integrated knowledge translation approach was used to engage with the knowledge users from Health Canada (MK) and Public Health Agency of Canada (MP) throughout the conduct of the rapid scoping review, including during: research question development, literature search, study inclusion, interpretation of results and draft report. The protocol for the review was registered using the Open Science Framework (https://osf.io/ypz7x). The discussion section includes minor amendments that occurred to the conduct of the review from the original protocol. Reporting of results was guided using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension to Scoping Reviews statement.12 Our research question was ‘What evidence exists on the treatments for COVID-19 in primary studies and reviews’, which is appropriate for the scoping review methodology.13

Patient and public involvement

Since this work was carried out as part of a rapid response to the COVID-19 pandemic project, timelines did not allow for participation of any patients or members of the public in this rapid scoping review.

Literature search

Comprehensive literature searches and citation screening were used in combination to gather relevant evidence from MEDLINE, EMBASE and preprint servers (biorxiv/medrxiv).14 The literature was initially searched from inception to 21 May 2020 and subsequently updated to 15 May 2021. Titles/abstracts were identified for screening using the Continuous Active Learning (CAL) tool, which uses supervised machine learning (see online supplemental appendix 1 for the description and performance of the tool).14 For archives that could be retrieved in their entirety (eg, MEDLINE, preprint servers), the CAL tool applied broad relevant search terms (online supplemental appendix 1). This search was supplemented by a literature search conducted by an experienced librarian in EMBASE (online supplemental appendix 2). The literature search was not restrict by language or publication status.

Eligibility criteria

The eligibility criteria followed the PICOS framework and consisted of:

Population: Individuals of any age who were clinically and/or laboratory diagnosed with COVID-19.

Intervention: Any compounds under investigation in human clinical trials as potential COVID-19 therapies (online supplemental appendix 3). Chinese medicine and complementary and alternative medicine—either alone or in combination with these medications—were excluded.

Comparator: Any of the interventions listed above, no intervention or placebo.

Outcomes: Any reported outcome.

Study designs: Primary studies of any design with a comparator group. Evidence syntheses of such studies were included, including systematic reviews, scoping reviews, rapid reviews, meta-analysis and overviews of reviews.

Study selection

A streamlined approach to study selection was used for the rapid scoping review. In combination with manual screening by reviewers, the CAL tool was used to identify and rank the titles and abstracts most likely to meet the inclusion criteria. This process continued iteratively until none of the identified articles met the inclusion criteria. For manual screening, a screening form based on the eligibility criteria was prepared for reviewers to aid in making consistent judgements on article relevance. A pilot-test was conducted using a random sample of 10 titles/abstracts until reviewers reached at least 75% agreement. Subsequently, screening was completed by single reviewers.

Data charting and coding

A charting form was developed and calibrated among the entire review team using two randomly selected full-text articles to ensure a standard approach to data collection. Following successful completion of the pilot-test, included studies were charted by single reviewers and verified by a second reviewer to ensure accuracy. Methodological quality or risk of bias appraisal of included studies was not conducted since this is a scoping review.10

The items collected included study characteristics (eg, study duration, study design, country of conduct), patient characteristics (eg, type of diagnosis, mean age), intervention and comparator details (eg, type of intervention, dose, frequency, duration) and outcome measures details (eg, mortality, viral clearance and hospital admission).

Pharmacological agents were grouped by their therapeutic category.15 Study primary outcomes were grouped together to reflect the clinical, virology, respiratory, inflammatory, cardiology and olfactory status and measures of COVID-19.16 17 The numbers of effect and harm measures were derived by counting the outcomes from the description of study outcomes. Authors’ conclusions were coded into the following categories: favour treatment, favour control, indeterminate and other.18 Pairs of reviewers conducted the data coding independently, with discrepancies reviewed and resolved through discussion by a pair of reviewers.

Synthesis

The charted and coded data were summarised descriptively for all patient population, interventions, comparators, outcomes and conclusion statements. The data were stratified by study design (randomised controlled trials vs non-RCT) and review type (review conducted according to a review protocol or otherwise).

Data repository

All material related to this review, including EndNote databases, extracted data in MS Excel, coding categories and analysis procedures written in the statistical software R are available at https://knowledgetranslation.net/comparative-effectiveness-research-of-covid-19-treatment-a-rapid-scoping-review-data-repository/.

Results

Figure 1 displays the literature search results. The semi-automation process with CAL and human reviewers allowed for the screening of approximately 290 000 titles/abstracts in about 40 person-hours over 2.3 weeks. Specifically, CAL identified 289 844 COVID-19 records and 4183 potentially relevant titles/abstracts. Title/abstract screening by reviewers resulted in 1542 potentially relevant reports. Report screening by reviewers resulted in 915 relevant reports, including 616 studies and 299 evidence syntheses. The list of included primary studies and evidence syntheses is in online supplemental appendix 4 and 5, respectively.

Figure 1

Flow diagram of included studies. Notes: *Estimated number of unique titles/abstracts based on: Medline (Ovid) includes preprints on COVID-19 from Medrxiv and Biorxiv, and large overlapping records between Medline and Embase. The flow chart was modified from the PRISMA 2020 statement.25

Characteristics of included studies

Figure 2 displays the timing when the studies were available online; on average 48 primary studies per month were published from July 2020 to April 2021. Table 1 displays the characteristics of the 616 included studies of varying design, including randomised controlled trials (188 studies (31%)), retrospective cohort studies (304 (49%)) and prospective cohort studies (70 (11%)), among others. The median study duration was 28 days and the median sample size was 169 participants. Public sources provided funding for about one-third of the studies; RCTs were funded often by private funding sources (27% relative to 3% for non-RCT). The primary studies were conducted in 51 countries, including the USA (26%), China (17%), Italy (8%), Spain (7%), France (6%), India (4%), Iran (3%), UK (3%) and Brazil (3%), among others (online supplemental table A1, online supplemental appendix 6).

Figure 2

Timing of available online of included studies*. The numbers of primary studies and systematic reviews for May 21 are higher because the literature search ended at 15 May 2021.

Table 1

Study, participant and outcome characteristics

Study characteristics	Total (N=616)	RCT (n=188)	Non-RCT (n=428)
Study design
Randomized controlled trial	188 (31%)	188
Retrospective cohort	304 (49%)		304 (71%)
Prospective cohort	70 (11%)		70 (16%)
Case–control	27 (4%)		27 (6%)
Controlled clinical trial	23 (4%)		23 (5%)
Controlled before-and-after	4 (1%)		4 (1%)
Study setting
Acute care hospital	515 (84%)	145 (77%)	370 (86%)
Intensive care unit	44 (7%)	4 (2%)	40 (9%)
Community	42 (7%)	34 (18%)	8 (2%)
Community and hospital	6 (1%)	3 (2%)	3 (1%)
Nursing home	3 (0%)	0 (0%)	3 (1%)
Not reported	6 (1%)	2 (1%)	4 (1%)
Country
United States of America	161 (26)	37 (20)	124 (29)
China	107 (17)	27 (14)	80 (19)
Italy	47 (8)	2 (1)	45 (11)
Spain	41 (7)	3 (2)	38 (9)
France	39 (6)	5 (3)	34 (8)
India	23 (4)	15 (8)	8 (2)
Iran	21 (3)	15 (8)	6 (1)
United Kingdom	21 (3)	19 (10)	2 (0)
Brazil	17 (3)	13 (7)	4 (1)
Turkey	12 (2)	1 (1)	11 (3)
Mexico	11 (2)	6 (3)	5 (1)
Argentina	10 (2)	7 (4)	3 (1)
Study duration
Median duration in days (IQR)	28 (14–30)	21.5 (14–28)	28 (20–35)
Sample size
Median # participants (IQR)	169 (74–475)	120 (60–394)	194 (82–592)
Study sponsor
Public	206 (33%)	78 (41%)	128 (30%)
No funding	165 (27%)	21 (11%)	144 (34%)
Private	63 (10%)	50 (27%)	13 (3%)
Public and private	18 (3%)	13 (7%)	5 (1%)
Not reported	164 (27%)	26 (14%)	138 (33%)
Participant characteristics
Average age (years)
Median (range)	60 (6–88)	56 (27–77)	62 (6–88)
Average percent of male participants
Median (IQR)	61 (53–69)	59 (50–66)	62 (54–70)
SARS‐CoV‐2 diagnosis
PCR test	436 (71%)	146 (78%)	290 (68%)
PCR and other*	105 (17%)	33 (18%)	72 (17%)
Not specified	75 (12%)	9 (5%)	66 (15%)
Case severity†
Severe	163 (26%)	39 (21%)	124 (29%)
Mild or moderate	46 (7%)	25 (13%)	21 (5%)
Moderate or severe	33 (6%)	17 (9%)	16 (4%)
Severe or critical	30 (5%)	7 (4%)	23 (5%)
Moderate	24 (4%)	14 (8%)	10 (2%)
Mild	22 (3%)	16 (9%)	6 (1%)
Mild, moderate or severe	14 (2%)	6 (3%)	8 (2%)
Mild, moderate, severe or critical	8 (1%)	2 (1%)	6 (1%)
Moderate, severe or critical	4 (1%)	1 (1%)	3 (1%)
Not specified	117 (19%)	34 (19%)	83 (19%)
Special age group‡
Elderly (eg, ≥65 years of age)	11 (2%)	2 (1%)	9 (2%)
Children (eg, <16 years of age)	2 (0%)	1 (1%)	1 (0%)
Type of primary outcome
Death/survival§	198 (32%)	20 (11%)	178 (42%)
Clinical status/measures¶	119 (19%)	71 (38%)	48 (11%)
SARS‐CoV‐two virology status/measures**	61 (10%)	29 (15%)	32 (7%)
Respiratory status/measures††	53 (9%)	19 (10%)	34 (8%)
Safety/adverse events‡‡	43 (7%)	9 (5%)	34 (8%)
Composite outcome involving death§§	39 (6%)	10 (5%)	29 (7%)
Resources measures¶¶	20 (3%)	6 (3%)	14 (3%)
Invasive mechanical ventilation	15 (2%)	4 (2%)	11 (3%)
Admission to intensive care unit	11 (2%)	1 (1%)	10 (2%)
Admission to acute care hospital	9 (1%)	3 (2%)	6 (1%)
Inflammatory status/measures***	9 (1%)	4 (2%)	5 (1%)
Emergency room visit	4 (1%)	2 (1%)	2 (0%)
Cardiology status/measures†††	3 (1%)	1 (1%)	2 (1%)
Olfactory status/measures‡‡‡	3 (0%)	2 (1%)	1 (0%)
Hospital discharge	2 (0%)	1 (0%)	1 (0%)
Other status/measures§§§	9 (1%)	2 (1%)	7 (2%)
Not reported	18 (3%)	4 (2%)	14 (3%)
No of effect outcomes
Median # of outcomes (IQR)	4 (2–7)	6 (4–9)	3 (2–6)
No of harm outcomes
Median # of outcomes (IQR)	1 (0–3)	2 (1–5)	0 (0–2)

*Other diagnostic modality such as lung imaging or suspected COVID-19 cases.

†Case severity according to the clinical spectrum of SARS-CoV-2 infection by the National Institute of Health25

‡Age group as reported in the included studies.

§Death/survival or time to death.

¶Clinical status/measures such as improvement/deterioration or time to such events.

**SARS‐CoV‐2 virology status/measures such as viral load or duration to PCR negative.

††Respiratory status/measures such as whole lung lesion volumes or blood oxygen saturation.

‡‡Safety/adverse events such as other infections than SARS-CoV-2, acute kidney injury or drug tolerance.

§§Composite endpoints involving death such as death and invasive mechanical ventilation or death and admission to intensive care unit.

¶¶Resources measures such as length of hospital stay.

***Inflammatory status/measures such as plasma levels of C reactive protein, or changes in ratio of oxygen saturation index, the ratio of pulse oximetry (SpO2)/fraction of inspired oxygen (FiO2).

†††Cardiology status/measures such as cardia endpoints with max high-sensitivity cardiac troponin level and stroke.

‡‡‡Olfactory status/measures such as loss of smell and taste.

§§§Other primary outcome such as time from COVID-19 symptoms onset to treatment or organ support-free days.

RCT, randomised controlled trial.

Most studies were conducted with participants admitted to acute care hospital (84%). Participants were on average 60 years of age, including 61% male, and mostly with confirmed COVID-19 via PCR test (table 1). About one-third of the included studies enrolled participants with severe or critical COVID-19 conditions. Few studies (0.3%) enrolled children (eg,<16 years of age) or the elderly (eg,≥65 years of age, 2%). Figure A1 displays the cloud of words often used to describe the participants (online supplemental appendix 6). Typical words used were COVID-19, COVID-19 patients, hospitalised, severe, pneumonia, ICU, outpatient, respiratory distress, invasive mechanical ventilation, critically ill and supplemental oxygen, among others.

The median number of effect outcomes was four, and the corresponding number of harm outcomes was one (table 1). Common primary outcomes included death/survival (32% of the included studies), clinical status/measures (19%), virology status/measures (10%), respiratory status/measures (9%), safety/adverse events excluding death (7%) and composite outcomes involving death (6%, for example, intubation and death, or intensive care admission and death), among others.

The included studies evaluated 827 treatment arms (711 single-drug and 116 multiple-drug treatment arms) against 616 control arms, of which 106 (17%) control arms involved active comparators (table 2). The treatment arms consisted of 215 unique treatment options (online supplemental table A2, online supplemental appendix 6). The most common treatments were tocilizumab (11%), hydroxychloroquine (9%), convalescent plasma (7%), steroid (4%), lopinavir combined with ritonavir (4%), methylprednisolone (3%), remdesivir (3%), enoxaparin (2%), hydroxychloroquine combine with azithromycin (2%) and anakinra (2%), among others.

Table 2

Treatment options frequently evaluated in included studies

All individual treatments	Total	RCT	Non-RCT
Total	827	231	596
1. Tocilizumab	87 (11%)	12 (5%)	75 (13%)
2. Hydroxychloroquine	78 (9%)	22 (10%)	56 (9%)
3. Convalescent Plasma	55 (7%)	15 (6%)	40 (7%)
4. Steroid	37 (4%)	1 (0%)	36 (6%)
5. Lopinavir/ritonavir	29 (4%)	5 (2%)	24 (4%)
6. Methylprednisolone	26 (3%)	3 (1%)	23 (4%)
7. Remdesivir	25 (3%)	16 (7%)	9 (2%)
8. Enoxaparin	18 (2%)	1 (0%)	17 (3%)
9. Hydroxychloroquine/azithromycin	18 (2%)	2 (1%)	16 (3%)
10. Anakinra	16 (2%)	2 (1%)	14 (2%)
Treatment type—common single treatment	Total	RCT	Non-RCT
All single treatments	711	202	509
1. NS-immunosuppressant	126 (18%)	27 (13%)	99 (19%)
2. Steroid	110 (15%)	15 (7%)	95 (19%)
3. Antiviral	97 (14%)	40 (20%)	57 (11%)
4. Antimalarial	87 (12%)	25 (12%)	62 (12%)
5. Anticoagulant	66 (5%)	5 (3%)	61 (12%)
Anticoagulant-therapeutic	17 (2%)	2 (1%)	15 (3%)
Anticoagulant-prophylactic	14 (2%)	0 (0%)	14 (3%)
6. Convalescent plasma	56 (8%)	16 (8%)	40 (8%)
7. Antibiotic	29 (4%)	7 (3%)	22 (4%)
8. Anti‐Inflammatory	20 (3%)	8 (4%)	12 (2%)
9. Interferon therapy	16 (2%)	7 (3%)	9 (2%)
10. Antiparasitic	14 (2%)	12 (6%)	2 (0%)
10. Immunomodulatory	14 (2%)	4 (2%)	10 (2%)
Treatment type—common combined treatment
All combined treatment option	116	29	87
1. Antimalarial/antibiotic	19 (16%)	2 (7%)	17 (20%)
2. Steroid/NS-immunosuppressant	10 (9%)	0 (0%)	10 (11%)
3. Antimalarial/antiviral/antiviral	8 (7%)	1 (3%)	7 (8%)
4. Antiviral/antiviral	5 (4%)	3 (10%)	2 (2%)
4. Antiviral/interferon	5 (4%)	0 (0%)	5 (6%)
5. Antimalarial/antiviral	4 (3%)	0 (0%)	4 (5%)
5. Antimalarial/antiviral/antibiotic	4 (3%)	4 (14%)	0 (0%)
5. Antiparasitic/antibiotic	4 (3%)	3 (10%)	1 (1%)
5. Antiviral/antiviral/antiviral	4 (3%)	0 (0%)	4 (5%)
5. Antiviral/antiviral/antiviral/interferon	4 (3%)	0 (0%)	4 (5%)
5. Antiviral/NS-immunosuppressant	4 (3%)	3 (10%)	1 (1%)
5. NS-immunosuppressant/steroid	4 (3%)	0 (0%)	4 (5%)

NS-immunosuppressant, non-steroidal immunosuppressant; RCT, randomised controlled trial.

Table 2 also displays the common therapeutic categories of the evaluated treatment. The most common therapeutic categories were non-steroidal immunosuppressant (18%), steroid (15%), antiviral (14%), antimalarial (12%), anticoagulant (5%), convalescent plasma (8%), antibiotic (4%), anti-inflammatory (3%), interferon therapy (2%), antiparasitic (2%) and immunomodulatory (2%), among others (details in online supplemental table A3, online supplemental appendix 6). Common therapeutic categories involving multiple drugs were the combination of antimalarial/antibiotic (16%), steroid/non-steroidal immunosuppressant (9%), antimalarial/antiviral/antiviral (7%), 2-antivirals (4%) and antiviral/interferon (4%), among others (online supplemental table A4, online supplemental appendix 6).

Characteristics of included evidence syntheses

Figure 2 displays the timing when the evidence syntheses were available online, on average 22 reviews appeared each month from May 2020 to April 2021. Table 3 displays characteristics of the 299 included evidence syntheses, including 88 (29%) evidence syntheses and 211 (71%) evidence syntheses conducted with and without a review protocol, respectively. Commonly conducted evidence syntheses included systematic review with meta-analysis (63%), systematic review (24%), meta-analysis (4%, none mentioned the use of a review protocol), scoping review (3%) and rapid review (3%), among others. Most reviews (83%) included RCT and non-RCT studies. The median number of data sources was 5 and the median number of included studies was 14. The evidence syntheses were conducted in 39 countries, including the USA (19%), China (14%), India (11%), Iran (6%) and the UK (6%), among others (online supplemental table A5, online supplemental appendix 6).

Table 3

Evidence synthesis characteristics

	All (n=299)	With protocol (n=88)	Without protocol (n=211)
Review type
Systematic review with meta-analysis	189 (63%)	66 (75%)	123 (58%)
Systematic review	73 (24%)	15 (17%)	58 (27%)
Meta-analysis	12 (4%)	0 (0%)	12 (6%)
Scoping review	10 (3%)	3 (3%)	7 (3%)
Rapid review	8 (3%)	1 (1%)	7 (3%)
Network meta-analysis	2 (1%)	1 (1%)	1 (0%)
Rapid review with meta-analysis	2 (1%)	1 (1%)	1 (0%)
Systematic review with network meta-analysis	2 (1%)	0 (0%)	2 (1%)
Overview of systematic reviews	1 (0%)	1 (1%)	0 (0%)
Review abstract
Structured abstract	159 (53%)	47 (53%)	112 (53%)
Abstract with no structure	140 (47%)	41 (47%)	99 (47%)
Eligibility criteria
Report eligibility criteria	259 (87%)	86 (98%)	173 (82%)
Eligibility criteria are unclear	40 (13%)	2 (2%)	38 (18%)
Include randomised controlled trials
Include RCTs only	51 (17%)	19 (22%)	32 (15%)
Include different study designs	248 (83%)	69 (78%)	179 (85%)
No of data sources
Median (IQR)	5 (3–6)	6 (4–7)	4 (3–6)
No of included studies
Median (IQR)	14 (7–28)	17 (7–38)	14 (7–25)
Common country
1. United States of America	57 (19%)	13 (15%)	44 (21%)
2. China	40 (14%)	13 (15%)	27 (13%)
3. India	34 (11%)	12 (13%)	22 (10%)
4. Iran	18 (6%)	3 (3%)	15 (7%)
4. United Kingdom	18 (6%)	3 (3%)	15 (7%)
5. Saudi Arabia	13 (4%)	1 (1%)	12 (6%)
6. Canada	12 (4%)	5 (6%)	7 (3%)
7. Italy	12 (4%)	8 (9%)	4 (2%)
8. Indonesia	9 (3%)	2 (2%)	7 (3%)
9. Malaysia	7 (2%)	0 (0%)	7 (3%)
10. Egypt	5 (2%)	2 (2%)	3 (1%)
10. France	5 (2%)	3 (3%)	2 (1%)
10. Peru	5 (2%)	1 (1%)	4 (2%)
10. Taiwan	5 (2%)	1 (1%)	4 (2%)

RCT, randomised controlled trial.

The evidence syntheses evaluated 518 treatment arms against 299 control arms (table 4). The treatment arms consisted of 115 unique treatment options (online supplemental table A6, online supplemental appendix 6). The most common treatment options were hydroxychloroquine (11%), remdesivir (8%), tocilizumab (7%), steroids (7%), convalescent plasma (6%) and lopinavir/ritonavir (5%), among others (table 4 and online supplemental table A6, online supplemental appendix 6).

Table 4

Treatment options evaluated in systematic reviews

Treatment option	Total (n=518)	With protocol (n=152)	Without protocol (n=366)
Hydroxychloroquine	58 (11%)	15 (10%)	43 (12%)
Remdesivir	39 (8%)	11 (7%)	28 (8%)
Tocilizumab	35 (7%)	10 (7%)	25 (7%)
Corticosteroid	35 (7%)	10 (7%)	25 (7%)
Convalescent plasma	33 (6%)	10 (7%)	23 (6%)
Lopinavir-ritonair	24 (5%)	8 (5%)	16 (4%)
Chloroquine	19 (4%)	6 (4%)	13 (4%)
Hydroxychloroquine/azithromycin	14 (3%)	1 (1%)	13 (4%)
Antivirals	12 (2%)	4 (3%)	8 (2%)
Anticoagulant	11 (2%)	2 (1%)	9 (2%)
Azithromycin	11 (2%)	3 (2%)	8 (2%)
Favipiravir	10 (2%)	1 (1%)	9 (2%)
Hydroxychloroquine/chloroquine	10 (2%)	4 (3%)	6 (2%)
Colchicine	9 (2%)	2 (1%)	7 (2%)
Dexamethasone	9 (2%)	1 (1%)	8 (2%)
Arbidol	7 (1%)	1 (1%)	6 (2%)
Invermectin	7 (1%)	3 (2%)	4 (1%)
Glucocorticoid	7 (1%)	3 (2%)	4 (1%)
ACEI/ARB	6 (1%)	4 (3%)	2 (1%)
Therapeutic anticoagulant	5 (1%)	2 (1%)	3 (1%)
Prophylactic anticoagulant	4 (1%)	3 (2%)	1 (0%)
Anakinra	4 (1%)	3 (2%)	1 (0%)
Famotidine	4 (1%)	1 (1%)	3 (1%)
JAK-inhibitors	4 (1%)	2 (1%)	2 (1%)
Sarilumab	4 (1%)	4 (3%)	0 (0%)

ACEI/ARB, Angiotensin Converting Enzyme Inhibitors and Angiotensin-Receptor Blockers; HCQ, Hydroxychloroquine; JAK-inhibitors, Janus kinase inhibitors.

Treatment evaluation according to authors’ conclusion

Table 5 displays the results of the treatment evaluation according to authors’ conclusion. Among the included primary studies and evidence syntheses, the conclusion was in favour of treatment in 50% and 36% of the evaluated treatment arms, respectively.

Table 5

Treatment evaluation according to authors’ conclusion

Studies evaluating treatment benefits/harms	All studies	RCT	Non-RCT
# of evaluated treatment arms	827	231	596
Favour evaluated treatment	413 (50%)	120 (52%)	293 (49%)
Favour control	63 (8%)	15 (7%)	48 (8%)
Indeterminate /neutral	258 (31%)	90 (39%)	168 (28%)

RCT, randomised controlled trial.

Discussion

We completed a rapid scoping review for Health Canada and Public Health Agency of Canada to identify pharmacologic treatments for COVID-19. A comprehensive search of electronic databases, trial registries and other grey literature sources from inception to May 2020 identified 9 controlled trials and 19 cohort studies with approximately 8000 participants. Updated to 15 May 2021, the search of electronic databases identified 915 relevant reports, including 616 studies with approximately 15.4 million participants and 299 evidence syntheses.

With respect to study population, existing studies put much emphasis on adult patients admitted to acute care hospitals. Future studies need to focus on children, older adults aged ≥65 years and patients with mild symptoms in community settings. Future study populations will need to reflect a broader range of age groups as the current pandemic evolves to affect younger age groups.19 20

With respect to treatment, many studies and reviews evaluated antimalarial agents. Existing studies emphasised preventing and treating cytokine surge with steroids and non-steroidal immunosuppressants, including interleukin‐6 inhibitors (eg, tocilizumab, sarilumab), interleukin‐1 antagonist (eg, anakinra), anti‐IL‐1β monoclonal antibody (eg, canakinumab), TNF‐alpha inhibitor (eg, adalimumab) and Janus kinase inhibitors (eg, baricitinib, ruxolitinib). Future studies may need to explore treatment for patients not responding to these agents, such as immunomodulators (eg, thymosin-α1). Existing studies put much emphasis on monotherapy; future studies need to evaluate combination therapy that addresses the multiple aspects of COVID-19, such as virology, respiratory, inflammatory and cardiology. Future studies may also need to explore outpatient treatment for patients with mild symptoms, and treatment options not frequently evaluated in existing studies, such as therapeutic anticoagulants.

With respect to comparators, most existing randomised controlled trials used placebo comparators while most observational studies used standard of care as comparator; future studies may consider active treatment as comparators, especially when evaluating treatments aiming to produce incremental improvement against effective treatments. Methodological issues related to the selection and delineation of comparators in studies evaluating combination therapies deserve attention. For example, a study evaluated multimechanism approach with medications targeting early immunomodulation, anticoagulation, and viral suppression to prevent catastrophic cytokine release syndrome encountered large variation in clinical characteristics of study participants and standard-of-care comparators in the five participant hospitals in two countries, including differences in disease severity and different doses of colchicine and types of steroids used across comparative groups.17

With respect to outcomes, about one-third of the included studies used mortality as the primary outcome. Tracking this outcome may require sufficiently long study duration, perhaps longer than the median duration of less than a month observed among existing studies, especially in patients with prolonged respiratory problems, suggesting longer follow-up duration for future studies. Of note, few existing studies used composite endpoints involving death, including endpoints such as intubation and intensive care admission. This use seems to be particularly suitable to capture the respiratory, immunology and cardiovascular aspects of COVID-19, as well as mortality. Few existing studies focused on harms due to treatment and among those that evaluated benefits and harms, the median number of reported harms was only one; future studies need to put more emphasis on harm evaluation. Existing RCTs put much emphasis on the use of clinical status/measures as primary outcome measures. Future trials may consider other primary outcomes that are relevant to patients, such as pneumonia, acute respiratory distress syndrome, multiorgan failure and septic shock, among others.

With respect to study design, our results showed a breakdown of 30% and 70% for RCTs and observational studies, respectively. Future trials are needed for evaluating combination therapies. Observational studies will remain pertinent in the evaluation of combination therapies, especially when rich data becomes available with their use in practice. Our review excluded qualitative studies, but we wish to emphasise the importance of these studies in elucidating the experience of COVID-19 patients.

With respect to evidence synthesis, we identified a small number of meta-analyses conducted without the associated systematic review and review protocol (n=13). This practice needs to be scrutinised because of the associated high risk of bias in the results, which could be wrong, but appeared to be convincingly precise.21 Existing evidence syntheses mostly evaluated monotherapy; future evidence syntheses will need to include data from the evaluation of combination therapy. The number of existing network meta-analyses was low (n=4); future network meta-analyses are needed to identify effective treatment given a plethora of treatment options, as well as to identify effective component treatment options addressing multiple aspects of COVID-19.22 Given the growing literature, there is a definitive need for living evidence synthesis, in which the synthesis is updated regularly as new studies become available.23 The results suggest that monthly updates may become necessary.

With respect to the growing literature, the use of automation tools like CAL for study selection will become essential to ensure a highly sensitive yield of relevant studies, responsive timelines for decision-making and reduced workload for reviewers. In this rapid scoping review, we used a continuous active learning approach that integrates machine learning with feedback instructions from reviewers. This approach allowed the screening of approximately 290 000 titles/abstracts in about 40 person-hours over 2.3 weeks. We believe this approach is indispensable for future reviews involving large body of literature. This approach called for slight changes in our review conduct and reporting, of note the reported number of the titles/abstracts excluded by the automation tool in the flow chart (see figure 1).

There are several limitations of this review. This is a scoping review, and as such, we did not assess the risk of bias in the included studies and reviews. Initially, the review protocol called for a borrowing strength of evidence approach, including studies evaluating treatment for SARS and MERS. The initial literature search in May 2020 included electronic databases, trial registries, Cochrane Library and other grey literature sources. Given the growing literature on COVID-19 by May 2021, the current review was focused only on COVID-19 treatment, with relevant studies identified from MEDLINE, EMBASE and preprint servers.

In this scoping review, the evaluated treatment options appeared to attain a reasonable chance of being more effective than their comparators, approximately 50% and 30% according to the authors’ conclusions from the included studies and reviews, respectively. However, we did not extract outcome data or combine them to verify the authors’ conclusions. To provide a broad overview of the comparative effectiveness research on COVID-19 treatment, we included reports from preprint servers, but these reports had not gone through peer review. Despite these limitations, the methods used in this review were carefully selected to address the needs of our knowledge users from Health Canada and Public Health Agency of Canada. In addition, we made the material from this rapid scoping review available in an online data repository as the data may be useful for conducting systematic reviews of specific therapies or for updating the current review.24

Conclusions

This rapid scoping review characterised a growing body of comparative-effectiveness studies and evidence syntheses evaluating hundreds of monotherapy and combination therapy options addressing the multiple sequelae of COVID-19. The results suggest future studies in children, elderly (eg, ≥65 years of age) and patients with mild symptoms, with additional data on outpatient treatment, multimechanism therapy, harms and active comparators. The results also suggest that future living evidence synthesis and network meta-analysis would provide additional information for decision-makers on managing COVID-19.