Critical Review of the Scientific Evidence and Recommendations in COVID-19 Management Guidelines.

BACKGROUND: Little is known about the quality and potential impacts of the guidelines for coronavirus disease 2019 (COVID-19) management.
METHODS: We systematically searched PubMed, Web of Science, Cochrane Library, guideline databases, and specialty society websites to evaluate the quality of the retrieved guidelines using the Appraisal of Guidelines for Research and Evaluation II.
RESULTS: A total of 66 guidelines were identified. Only 24% were categorized as "recommended" for clinical practice. The 211 identified recommendations for COVID-19 management were classified into 4 topics: respiratory support (27), acute respiratory distress syndrome management (31), antiviral or immunomodulatory therapy (95), or other medicines (58). Only 63% and 56% of recommendations were supported by, respectively, assessment of the strength of the recommendations or level of evidence. There were notable discrepancies between the different guidelines regarding the recommendations on COVID-19 management.
CONCLUSIONS: The quality of the guidelines for COVID-19 management is heterogeneous, and the recommendations are rarely supported by evidence.

INTRODUCTION

The coronavirus disease 2019 (COVID-19) pandemic has become a global public health crisis. As of June 17, 2021, COVID-19 has affected >176 million people in >200 countries or regions and resulted in >3.8 million deaths [1]. The economic burden and health threat of COVID-19 are extremely dreadful and have become more severe as the number of global infections and deaths increases [2,3].

The management of the disease relies largely on symptomatic and supportive treatments. For severe or critically ill patients with acute respiratory distress syndrome (ARDS) and sepsis, in addition to supplemental oxygen, mechanical ventilation, and ARDS-specific therapies, antiviral and antibiotic treatments must also be considered. To face the rapid global spreading of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and the difficulty for overburdened front-line workers and policy-makers of staying up to date on the emerging literature, many national and international organizations have issued rapid advice or interim guidelines for COVID-19 management. These guidelines have integrated the best possible information in response to health and social care emergencies to help frontline health care professionals improve clinical outcomes [4]. However, little is known about the quality and variability of the recommendations among different guidelines. Additionally, substantial differences exist in clinical practices across countries and hospitals. To provide the best care possible, clinicians need to understand the discrepancy of recommendations among guidelines and choose evidence-based recommendations developed by trustworthy methodologies [5].

Many issues related to the clinical management of acute COVID-19 remain to be clarified, including the role of noninvasive ventilation (NIV), high-flow nasal cannula (HFNC), usage of corticosteroids or other supportive therapies, and various antiviral medications. In this study, we aimed to systematically review and evaluate currently available guidelines for acute COVID-19 management and specifically compare the available recommendations and the quality of supporting evidence.

METHODS

We registered this study protocol and reported the results according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines (CRD42020180074) [6].

Data Sources and Searches

We searched PubMed, Web of Science, and Cochrane Library for COVID-19 guidelines published through June 17, 2021, as well as websites of international organizations, government health institutions, relevant specialty societies, guideline-specific databases, and Google Scholar. The bibliographies of included studies were further screened for additional potentially relevant articles. The detailed search strategies and results are presented in the Supplementary Data (Supplementary Methods 1 and 2). Our search was restricted to guidelines developed by international or national health care organizations and medical societies published in English.

Guideline Selection

To be included, guidelines had to make specific recommendations for the management of acute COVID-19 regarding NIV, HFNC, use of corticosteroids or other supportive therapies, and use of various antiviral medications. When several versions of the same document were available, only the latest version was retained. For each eligible guideline, we thoroughly searched for supplementary supporting documents to better inform our assessments. The following types of document were excluded: (i) guidelines regarding diagnosis, home care, or prevention and control of infection; (ii) guidelines for special populations such as newborns, children, or pregnant women; (iii) guidelines developed by autonomous medical institutions or nonprofessional societies; (iv) documents such as systematic reviews, clinical trials, commentaries, case series, letters, or chapters in books or booklets; (v) documents that were not published in English or not available in full-text format; (vi) specialty guidelines for triage, tracheotomy, complications, palliative treatment, and rehabilitation.

Data Extraction and Quality Assessment

Two reviewers (Z.W. and J.X.) independently screened and extracted all relevant information from included guidelines using predesigned forms. Whenever discrepancies arose, resolution was achieved by consensus or by consulting the third expert adjudicator (M.J.). The quality of eligible guidelines was independently evaluated by 4 appraisers who had been trained in clinical practice guidelines appraisal using the Appraisal of Guidelines for Research and Evaluation II (AGREE II) instrument [7-9]. AGREE II contains 23 items within 6 domains. Each item was scored on 7-point Likert scale that varies from 1 (strongly disagree) to 7 (strongly agree). A standardized score was calculated as the percentage of the maximal possible score for each domain using the formula provided in the AGREE II user’s manual: (actual score – minimal possible score)/(maximal possible score – minimal possible score) [9]. The standardized scores ranged from 0% to 100%. Guidelines with an overall score >60% were classified as “recommended,” between 30% and 60% as “recommended with modifications,” and <30% as “not recommended” [10, 11].

Data Synthesis and Analysis

Eight clinicians independently extracted the general characteristics of the eligible guidelines and the recommendations concerning respiratory support, ARDS management, antiviral and immunomodulatory therapies, and other pharmacologic treatments (corticosteroids, antibiotics, antipyretics, and neuraminidase inhibitors). We developed recommendation matrices to assist with systematic comparison, categorization, and summarization, including comparison of the strength of the recommendation and quality of evidence.

The general characteristics, standardized score in each domain, distribution of level of evidence, and strength of recommendation of each eligible guideline were depicted using either median and range, mean and SD, or frequency and percentage. Subgroup analyses were performed according to country, target population, type of guideline, and development method. Agreement among the 4 reviewers was measured by intraclass correlation coefficient (ICC) with a 95% CI. All analyses were conducted using R software, version 3.6.1 (http://CRAN.R-project.org; R Foundation, Vienna, Austria).

RESULTS

Characteristics of the Included Guidelines

A total of 66 guidelines met the inclusion criteria (Figure 1). The general characteristics of these eligible guidelines are shown in Figure 2 and Supplementary Table 1. Eleven (17%) were developed by international organizations [4, 12–21], and 13 (20%) originated from North America [22-34], 1 (2%) from South America [35], 31 (47%) from Europe [36-66], and 10 (15%) from the Asia-Pacific area region [67-76]. Only 25 (38%) guidelines provide an approximate update interval, with an average of 2.2 months (min: 0.25 months; max: 6 months). Twenty-four guidelines (36%) were developed using evidence-based methods, and 20 (30%) graded the strength of recommendations, of which only 9 (14%) appraised the quality of evidence (Supplementary Table 2). In order to compare the strength of the recommendations and quality of evidence obtained by different grading systems, a composite grading system applicable to all recommendations was generated (Supplementary Table 3).

Figure 1.

Search and selection flowchart for the retrieval of COVID-19 management guidelines. aGuidelines for pregnant women, children, newborns, diagnosis, quarantine, home care, protection, radiology, ultrasound, triage, tracheotomy, complications, palliative treatment, rehabilitation. Abbreviation: COVID-19, coronavirus disease 2019.

Figure 2.

Proportion of the COVID-19 management guidelines according to general characteristics. aProtocol, interim guidance, recommendation, consensus.

Abbreviations: COVID-19, coronavirus disease 2019; EB, evidence-based; GRADE, Grading of Recommendations Assessment Development and Evaluation.

Quality Assessment of the Guidelines

The standardized AGREE II scores obtained by all guidelines for each domain and the overall assessment are shown in Supplementary Table 4. The guidelines scored moderately in the domains scope and purpose (mean, 68%; range, 31%–89%) and clarity and presentation (mean, 69%; range, 21%–85%) but scored highly variably in the other 4 domains (Figure 3). Most guidelines (n = 38, 70%) were categorized as “recommended with modifications” for use during the COVID-19 pandemic, 12 (22%) were “recommended,” and 4 (7%) were “not recommended” (Supplementary Table 4). The overall agreement of the 4 appraisers was considered good (ICC, 0.849; 95% CI, 0.833–0.864). Supplementary Table 5 shows the quality scores of the guidelines across the different subgroups. The quality of the guidelines issued during 2021, originating from North America, developed by evidence-based methods or by >1 organization, was higher than the other guidelines.

Figure 3.

AGREE domain scores of all included guidelines. The vertical line at 60% represents the cutoff score at or above which a domain was considered “adequately addressed.” The vertical line at 30% represents the cutoff score at or below which a domain was considered “poorly addressed.” Abbreviation: AGREE, Appraisal of Guidelines for Research and Evaluation II.

Recommendations

General recommendations relevant to COVID-19 management in adults are listed in Table 1. Two hundred eleven recommendations were extracted from 66 documents. Among these, only 62.6% (132 recommendations) and 56.4% (119 recommendations) were supported by, respectively, assessment of strength or quality of evidence.

Table 1.

General Recommendations for Management of COVID-19

Topic	Type of Intervention	Guidelines That Provide Recommendations	No. of Recommendations (%)
			Extracted	Supported by an Assessment of Strength	Supported by Quality of Evidence
Respiratory support	Timing of start of oxygen therapy	WHO-toolkit [12], CCCGWG [26], WHO [13], PAHO [14], Thomas et al. [16], SITA&SIP [54], CTS&CACP [70], PCS [71], NHC&SATCM [69], Chinese experts [76], NICE (managing symptoms) [47], ASID [68], AARC [27], SSC [4], FSRD [60]	7	4 (57)	3 (43)
	Target of oxygen therapy	WHO [13], Poland [59], WHO-toolkit [12], CCCGWG [26], Thomas et al. [16], PAHO [14], PCS [71], Kluge et al. [37], SITA&SIP [54], INMI [44], CTS&CACP [70], NHS (rapid guideline) [52], ITS&IRS [41], NHS (critical care) [44], ICM [57], NHS (oxygen therapy) [51], FSRD [60], SSC [4], AARC [27]	11	4 (36)	2 (18)
	HFNC and NIV	WHO-toolkit [12], NHC&SATCM [69], SSC [4], CCCGWG [26], ITS&IRS [41], ARIR&AIFI [42], NHS (management) [50], WHO [13], CTS&CACP [70], PCS [71], AARC [27], NCCET [67], FSRD [60], ERS [19], BTS&ICS [61], SIAARTI&EAMS [45], Thomas et al. [16], SIMIT [43], ASID [68], Kluge et al. [37], NHS (critical care) [44], INMI [44], NHS (rapid guideline) [52], SIMIT [43], ICM [57], ISCCM [75], GRS [58], NIH [23], ASAIO [29]	9	PCS [71] 3 (33)	3 (33)
Management of ARDS	Endotracheal intubation	SSC [4], ICSI [38], SIAARTI&EAMS [45], CCCGWG [26], NHS (rapid guideline) [52], NHS (critical care), NHS (management) [50], WHO [13], PAHO [14], Kluge et al. [37], NIH [23], PCS [71], NCCET [67], WHO-toolkit [12], NHC&SATCM [69], Thomas et al. [16], ITS&IRS [41], ARIR&AIFI [42], CTS&CACP [70], SAS&ANZICS [15], ISCCM [75]	4	4 (100)	3 (75)
	ARDS ventilation	NIH [23], CCCGWG [26], NHS (critical care), SSC [4], WHO [13], CTS&CACP [70], PAHO [14], PCS [71], WHO-toolkit [12], NHC&SATCM [69], AARC [27], NCCET [67], NHS (oxygen therapy) [51], SAS&ANZICS [15], Kluge et al. [37], Chinese experts [76], Thomas et al. [16], SIAARTI&EAMS [45], ASAIO [29], ICSI [38], ASAIO (ECMO) [30], INMI [44], GRS [58]	15	13 (87)	11 (73)
	Hemodynamics	WHO-toolkit [12], NHC&SATCM [69], WHO [13], PCS [71], NHS (oxygen therapy) [51], Kluge et al. [37], SSC [4], CCCGWG [26], NHS (critical care), NHS (management) [50], PAHO [14], NIH [23], AARC [27], Chinese experts [76]	8	8 (100)	8 (100)
	ECMO	NIH [23], ATS [24], SSC [4], WHO [13], NCCET [67], WHO-toolkit [12], NHC&SATCM [69], ICSI [38], CCCGWG [26], NICE (critical care) [48], PCS [71], NHS (critical care), NHS (oxygen therapy) [51], CTS&CACP [70], PAHO [14], Kluge et al. [37], AARC [27], ASAIO (ECMO) [30], NHS (ECMO) [49]	4	3 (75)	2 (50)
Antiviral or immunomodulatory therapy	General recommendations	SIMIT [43], Korea [73], NHC&SATCM [69], HSE.ie (antiviral therapy), ASID [68], PAHO [14], ATS [24], NIH [23], WHO-toolkit [12], INMI [44], ITS&IRS [41], NHS (rapid guideline), Kluge et al. [37], Chinese experts [76]	8	4 (50)	5 (63)
	Chloroquine or hydroxychloroquine	NHC&SATCM [69], HSE.ie (antiviral therapy), INMI [44], SIMIT [43], ICMR [74], SIAARTI&EAMS [45], ASAIO [29], SITA&SIP [54], ACOEM [28], Korea [73], AST&ERS [20], NIH [23], ASAIO (ECMO) [30], IDSA [25], Brazil [35], CMAJ [72], ACP [31], WHO [13], PAHO [14], NCCET [67], ERS [19], WHO (therapeutics) [21], SSC [4]	11	5 (45)	5 (45)
	Hydroxychloroquine/chloroquine plus azithromycin	IDSA [25], HSE.ie (antiviral therapy), ACP [31], PCS [71], Brazil [35], PAHO [14], NIH [23], NCCET [67], ERS [19]	3	2 (67)	1 (33)
	Lopinavir/ritonavir	NHC&SATCM [69], SIAARTI&EAMS [45], INMI [44], Korea [73], SIMIT [43], HSE.ie (antiviral therapy), SITA&SIP [54], Poland [59], CCCGWG [26], ASAIO (ECMO) [30], WHO [13], PAHO [14], NCCET [67], Brazil [35], NIH [23], ERS [19], IDSA [25], SSC [4], CMAJ [72]	6	3 (50)	3 (50)
	Remdesivir	INMI [44], IDSA [25], PCS [71], ACOEM [28], BMJ GDG panel [56], NICN (critical care), NHS (tocilizumab) [66], CCCGWG [26], ASAIO [29], NHS (remdesivir) [64], ACP (remdesivir) [32], Belgium Task Force [36], Poland [59], NCCET [67], AST&ERS [20], NIH [23], ACP (remdesivir), SSC [4], HSE.ie (antiviral therapy), SIMIT [43], Kluge et al. [37], Korea [73], SITA&SIP [54], NHS (rapid guideline), ATS [24], ASAIO (ECMO) [30], ERS [19], WHO [13], PAHO [14], WHO (therapeutics) [13]	10	9 (90)	7 (70)
	Interferon	NHC&SATCM [69], CMAJ [72], NCCET [67], ACOEM [28], Korea [73], NIH [23], ERS [19]	11	8 (73)	7 (64)
	IL-6 inhibitors	PCS [71], HSE.ie (tocilizumab) [39], NHC&SATCM [69], INMI [44], SIMIT [43], ERS [19], Belgium Task Force [36], IDSA [25], NIH [23], ASAIO (ECMO) [30], SSC [4], ACOEM [28]	5	3 (60)	2 (40)
	IL-1 inhibitors	NIH [23]	1	0	0
	Convalescent plasma	NHC&SATCM [69], PCS [71], Chinese experts [76], SIMTI&SIdEM [53], CTS&CACP [70], Korea [73], ASAIO (ECMO) [30], ACOEM [28], PAHO [14], NCCET [67], NIH [23], IDSA [25], SSC [4], CMAJ [72]	5	2 (40)	3 (60)
	IVIG	NHC&SATCM [69], NIH [23], Korea [73], NCCET [67], Chinese experts [76]	5	2 (40)	3 (60)
	Ribavirin	NHC&SATCM [69], ACOEM [28], Korea [73], CMAJ [72]	3	1 (33)	2 (67)
	Favipiravir	NCCET [67], SITA&SIP [54], Korea [73], ACOEM [28]	2	1 (50)	2 (100)
	HIV protease inhibitors	NIH [23]	2	2 (100)	2 (100)
	JAK inhibitors	ACOEM [28], NCCET [67], NIH [23]	3	3 (100)	3 (100)
	Umifenovir	SITA&SIP [54], NCCET [67]	1	1 (100)	1 (100)
	Oseltamivir	Chinese experts [76], Brazil [35], SITA&SIP [54]	2	1 (50)	1 (50)
	Sarilumab	NHS (sarilumab) [65], NCCET [67], NICE (critical care) [48], NIH [23]	2	1 (50)	1 (50)
	Tocilizumab	NHS (tocilizumab) [66], IDSA [25], Belgium Task Force [36], NHS (rapid guideline) [52], NCCET [67], CTS&CACP [70], PAHO [14], NICE (critical care) [48], SITA&SIP [54], ASAIO (ECMO) [30], NIH [23], Brazil [35]	3	2 (67)	2 (67)
	Ivermectin	WHO (therapeutics) [21], ACOEM [28], NIH [23], NCCET [67], IDSA [25]	3	2 (67)	3 (100)
	Bamlanivimab or/plus etesevimab	NIH [23], IDSA [25], NCCE	2	2 (100)	2 (100)
	Casirivimab or/plus imdevimab	ACOEM [28], IDSA [25], NCCET [67]	2	2 (100)	2 (100)
	Other	NCCET [67], NIH [23], PAHO [14], Chinese experts [76]	5	5 (100)	4 (80)
Other pharmacologic treatments	Corticosteroids	CTS&CACP [70], NHC&SATCM [69], SSC [4], CMAJ [72], NIH [23], PAHO [14], Kluge et al. [37], SIMIT [43], CCCGWG [26], IDSA [25],WHO (corticosteroids) [17],WHO (therapeutics), ERS [19], NHS (management) [50], ICSI [38], CDC [22], NHS (ECMO) [49], ASID [68], ATS [24], NCCET [67], AST&ERS [20], NHS (rapid guideline) [52], Brazil [35], ICM [57], Korea [73], ACOEM [28], WHO [13], NHS (tocilizumab) [66], Poland [59], SITA&SIP [54]	24	13 (54)	12 (50)
	Antibiotics	CTS&CACP [70], ASID [68], WHO-toolkit [12], ICSI [38], Brazil [35], NICE (managing pneumonia), NHS (management) [50], SSC [4], Belgium Task Force [36], NIH [23], NHS (rapid guideline) [52], NHC&SATCM [69], ACOEM [28], NICE (antibiotics) [55], SIMIT [43], CCCGWG [26], WHO [13], INMI [44] NICE (antibiotics) [55], PAHO [14], SITA&SIP [54], Korea [73], NHS (critical care), NHS (management) [50], Kluge et al. [37], CTS&CACP [70]	17	10 (59)	7 (41)
	Antipyretic	WHO-toolkit [12], PCS [71], PAHO [14], WHO [13], NIH [23], SSC [4], NICE (managing symptoms) [47]	9	5 (56)	4 (44)
	Neuraminidase inhibitor	WHO-toolkit [12], CCCGWG [26], NHS (critical care), ASID [68], Brazil [35], Korea [73], SITA&SIP [54], WHO [13]	8	4 (50)	3 (38)

The acronyms of guidelines and organizations are defined in Supplementary Table 1.

Other abbreviations: ARDS, acute respiratory distress syndrome; ECMO, extracorporeal membrane oxygenation; HFNC, high-flow nasal cannula; IL, interleukin; IVIG, intravenous immunoglobulin, NIV, noninvasive ventilation.

Recommendations Related to Respiratory Support

Oxygen Therapy

Ten documents [12–14, 16, 26, 29, 47, 54, 69, 76] recommended timely supplementation of oxygen for patients with COVID-19 in the circumstance of severe acute respiratory infection (SARI), respiratory distress, hypoxemia, shock, or other severe symptoms (Supplementary Table 6). Seven documents recommended the use of SpO2 as an indicator and initiation of oxygen therapy under specific SpO2 threshold levels as follows: <90% (Surviving Sepsis Campaign [SSC] [4]: strong, moderate; American Association for Respiratory Care [AARC] [27] and World Health Organization [WHO] [13]: strong, ungraded); <92% (SSC [4]: strong, low; French Society of Respiratory Diseases [FSRD] [60] and Australasian Society for Infectious Diseases [ASID] [68]: ungraded; AARC [27]: weak, ungraded); or <93% (Chinese Thoracic Society [CTS] & Chinese Association of Chest Physicians [CACP] [70] and Pakistan Chest Society [PCS] [71]: ungraded); or significantly below baseline [68]. Moreover, it was recommended that the SpO2 target level not be maintained above 96% [4, 14, 16, 27, 54] (SSC [4] and Pan American Health Organization [PAHO] [14]: strong, moderate; AARC [27]: strong, ungraded; Thomas et al. [16] and Italian Society of Anti-infective Therapy [SITA] & Italian Society of Pulmonology [SIP] [54]: ungraded).

NIV/HFNC

Multiple guidelines [4, 12, 13, 19, 23, 26, 41, 42, 50, 61, 67, 69–71] recommended that NIV/HFNC be used (SSC [4] and National COVID-19 Clinical Evidence Taskforce [NCCET] [67]: weak, low; European Respiratory Society [ERS] [19]: weak, very low; WHO [13], COVID-19 Clinical Care Guidance Working Group [CCCGWG] [26], and AARC [27]: weak, ungraded) under close monitoring (within 1–4 hours) in cases of worsening respiratory status and early intubation in a controlled setting. However, in some of the guidelines [16, 37, 43, 45, 68, 75], NIV was not recommended (ungraded). Some guidelines recommended against HFNC for adult patients with COVID-19 (ungraded) [37, 43–45, 52].

Recommendations for ARDS Management

Endotracheal Intubation

It was consistently recommended that endotracheal intubation be performed by the most trained and experienced operators in most documents [4, 13, 14, 26, 37, 38, 45, 50, 52] (National Institutes of Health [NIH] [23]: strong, very low; SSC [4], CCCGWG [26], and WHO [13]: strong, ungraded) (Supplementary Table 7). Upon worsening of the patient condition, early endotracheal intubation in a controlled setting was recommended by most guidelines [4, 12–16, 23, 26, 37, 41, 42, 50–52, 69] (SSC [4], CCCGWG [26], WHO [13], and NIH [23]: strong).

ARDS Ventilation

Most documents consistently recommended using low tidal volume (Vt), airway platform pressure <30 cmH2O, prone ventilation, and neuromuscular blockade agents in ARDS. Most guidelines [4, 13–15, 23, 26, 27, 44, 45, 67, 69] suggested using a higher positive end-expiratory pressure (PEEP) strategy (WHO [13]: strong, ungraded; SSC [4], PAHO [14], and NIH [23]: weak, low; CCCGWG [26], WHO [13], and AARC [27]: weak, ungraded), whereas several documents (AARC [27]: strong, ungraded; Intensive Care Society of Ireland [ICSI] [38], CTS & CACP [70] and PCS [71]: ungraded) recommended setting the PEEP appropriately, with PCS [71] and CTS & CACP [70] considering PEEP titration or low PEEP (ungraded). Inhaled pulmonary vasodilator as rescue therapy for ARDS was recommended in 6 documents (SSC [4]: weak, very low; AARC [27] and American Society for Artificial Internal Organs [ASAIO; extracorporeal membrane oxygenation {ECMO}] [30]: weak, ungraded; NIH [23]: ungraded, very low; ICSI [38] and Kluge et al. [37]: ungraded). However, some guidelines (SSC [4] and PAHO [14]: strong, low; NIH [23]: strong, moderate; AARC [27]: strong, ungraded; PCS [71]: ungraded) recommended against the routine use of inhaled nitric oxide.

Hemodynamics

Most documents [4, 12–14, 23, 26, 27, 37, 50–52, 69, 71, 76] recommended a conservative fluid strategy and crystalloid fluid [4, 12–14, 23, 26] for patients in shock (CCCGWG [26] and WHO [13]: strong, ungraded; PAHO [14]: strong, low; SSC [4] and NIH [23]: weak, low; AARC [27]: weak, ungraded). Regarding the type of vasopressor, recommendations [4, 13, 14, 23, 26, 52] were generally consistent and indicated norepinephrine as the first-line vasoactive agent (NIH [23]: strong, low; SSC [4] and PAHO [14]: weak, low; CCCGWG [26] and WHO [13]: weak, ungraded; and Chinese experts [76]: weak, ungraded).

ECMO

In mechanically ventilated adults with refractory hypoxemia despite optimizing ventilation, the use of rescue therapies, proning, and ECMO was suggested by most documents [4, 12–14, 23, 24, 26, 27, 37, 38, 48, 51, 52, 67–71] (ASAIO [ECMO] [30]: strong, ungraded; SSC [4], PAHO [14], and NIH [23]: weak, low; CCCGWG [26], WHO [13], and AARC [27]: weak, ungraded). However, ASAIO (ECMO) [30] recommended against the initiation of ECMO before maximizing conventional therapies for ARDS, in particular prone positioning (strong, ungraded).

Recommendations for Antiviral Therapy

Four guidelines (Lombardy Section of the Italian Society of Infectious and Tropical Diseases [SIMIT] [43], Korea [73], National Health Commission [NHC] & State Administration of Traditional Chinese Medicine [SATCM] [69], Ireland’s Health Services [HSE.ie; antiviral therapy] [40]) were in favor of trying antiviral therapy (Supplementary Table 8). However, currently, most guidelines [12, 23, 24, 37, 41, 44, 52, 68, 76] do not recommend any special drugs or therapies for COVID-19.

Chloroquine/Hydroxychloroquine

Ten guidelines [24, 29, 40, 43–45, 54, 69, 73, 74] recommended chloroquine/hydroxychloroquine as antiviral therapy (American College of Occupational and Environmental Medicine [ACOEM] [28]: weak, moderate; Korea [73]: ungraded, very low), while 11 guidelines [4, 13, 14, 19, 21, 23, 25, 31, 35, 67, 72] recommended against its use (Infectious Diseases Society of America [IDSA] [25], ERS [19], PAHO [14], and SSC [4]: strong, moderate; WHO [13], WHO (therapeutics) [21], NCCET [67], and NIH [23]: strong, high; Brazil [35]: weak, low; Canadian Medical Association Journal [CMAJ] [72]: weak, ungraded).

Azithromycin Combined With Chloroquine/Hydroxychloroquine

This combination was no longer recommended in the updated guidelines from the PCS [71], IDSA [25], American College of Physicians (ACP) [31], Brazil [35], PAHO [14], NIH [23], NCCET [67], and ERS [19] (NIH [23]: strong, high; PAHO [14]: strong, moderate; IDSA [25]: strong, low; Brazil [35]: weak, very low; NCCET [67]: strong, low; ERS [19]: weak, moderate).

Lopinavir/Ritonavir

Eight guidelines [40, 43–45, 54, 59, 69, 73] recommended the use of lopinavir/ritonavir (Korea [73]: ungraded, very low). However, SSC [4], Brazil [35], CMAJ [72], WHO [13], PAHO [14], NCCET [67], NIH [23], ERS [19], and IDSA [25] recommended against the routine use of this combination for severe patients (NIH [23]: strong, very low; WHO [13]: strong, ungraded; ERS [19]: strong, low; SSC [4]: weak, low; Brazil [35]: weak, very low; CMAJ [72]: weak, ungraded; NCCET [67], PAHO [14], and IDSA [25]: strong, moderate).

Remdesivir

Most guidelines [4, 20, 23–26, 29, 32, 36, 44, 56, 59, 64, 67, 71] recommended the use of remdesivir as antiviral therapy (IDSA [25]: weak, moderate; ACOEM [28]: ungraded, low). The NIH [23] (ungraded, very low) recommended that patients who have not shown clinical improvement after 5 days of therapy have a treatment extension for up to 10 days. However, Korea [73], SITA & SIP [54], and the National Health Service (NHS; rapid guideline) [52] recommended the use of remdesivir only in clinical trials (Korea [73]: ungraded, very low; NHS (rapid guideline) [52]: weak, ungraded).

Interferons

Only NHC & SATCM [69] recommended that interferon-ɑ be tried in hospitals (ungraded). CMAJ [72] recommended the use of interferon-ɑ only in the context of clinical trials (weak, ungraded). Interferon β-1a (NCCET [67]: weak, very low) and interferon gamma (NCCET [67]: weak, very low) were only recommended in the context of clinical trials. However, CMAJ [72] (weak, ungraded) and ERS [19] (weak, very low) advised against the use of interferon-β.

Interleukin-6 Inhibitors

HSE.ie (tocilizumab) [39], NHC & SATCM [69], IRCCS National Institute for Infectious Diseases (INMI) [44], SIMIT [43], ERS [19], and the Belgium Task Force [36] recommended the use of interleukin-6 inhibitors, while PCS [71] recommended this treatment only in severe cases. Only ACOEM [28] (weak, low) advised avoiding its routine use.

Interleukin-1 Inhibitors

The NIH [23] acknowledged insufficient evidence to recommend either for or against the use of interleukin (IL)-1 inhibitors.

Convalescent Plasma

NHC & SATCM [69], PCS [71], and Chinese experts [76] recommended that convalescent plasma could be used for patients with rapid disease progression or in severe or critical states (ungraded). Additionally, ASAIO (ECMO) [30] and ACOEM [28] mentioned this therapy as a treatment option. However, SSC [4] (weak, low) and CMAJ [72] (weak, ungraded) suggested avoiding the routine use of convalescent plasma in critical cases.

Intravenous Immunoglobulin

Only NHC & SATCM [69] suggested the use of intravenous immunoglobulin (IVIG) in severe and critical cases (ungraded), while Chinese experts [76] suggested using caution with IVIG.

Ribavirin

Recent guidelines from Korea [73] and CMAJ [72] did not recommend the use of this drug (Korea [73]: weak, very low; CMAJ [72]: weak, ungraded).

Recommendations of Other Pharmacologic Treatments

Antibiotics

Indications for antibiotic use were different across guidelines (Supplementary Table 9). Seven documents (NHS [rapid guideline] [52], NHS [management] [50], CTS & CACP [70], National Institute for Health and Care Excellence [NICE; antibiotics] [55]: ungraded; Korea [73]: strong, moderate; WHO [13]: strong, ungraded; and SIMIT [43]) recommended using antibiotics in patients who presented with reasonable evidence of bacterial infections. Empirical use of antimicrobials/antibacterial agents was suggested for patients with sepsis [12, 13, 26, 54, 70] (WHO [13]: strong, ungraded), mechanical ventilation, respiratory failure [4, 14] (SSC [4] and PAHO [14]: weak, low), or possible secondary bacterial infection and in a critical state [35, 37, 38, 44, 52, 68] (ACOEM [28]: ungraded, low). Six documents [13, 35, 52, 55, 69] recommended against blind or inappropriate use of antibiotic drugs (WHO [13]: strong, ungraded; Brazil [35]: weak, very low; Korea [73]: strong, moderate; NICE [antibiotics] [55]: strong, ungraded).

Neuraminidase Inhibitor

Five documents [12, 13, 26, 35, 52] (Brazil [35]: weak, very low) recommended empirical neuraminidase therapy for suspected cases of influenza. However, Korea [73] (weak, moderate) and Brazil [35] (strong, very low) recommended against its use.

Antipyretic

Antipyretics were recommended for fever in mild [12, 13], moderate [71], severe [12, 71], and critically ill adults [4]. PAHO [14] (weak, low) suggested that antipyretics should be used for temperature control and the choice of drug should be adapted to the patient’s comorbidities. Regarding drug selection, paracetamol was recommended rather than nonsteroidal anti-inflammatory drugs (NSAIDs) [47, 71]. The NIH [23] (strong, very low) recommended that clinicians use acetaminophen or NSAIDs.

Corticosteroids

A total of 30 guidelines [4, 13, 14, 17, 22–26, 28, 37, 38, 43, 49, 50, 54, 59, 67–70, 72] provided recommendations regarding corticosteroids. Three documents (CMAJ [72]: weak, ungraded; and SIMIT [43]: ungraded) recommended using corticosteroids in patients with ARDS, and low-dose corticosteroid therapy was preferred over no corticosteroid therapy in patients with refractory shock (SSC [4], PAHO [14], NIH [23]: weak, low). Moreover, the use of glucocorticoids was recommended for patients with COVID-19 without hypoxemia requiring supplemental oxygen (IDSA [25]: weak, low) and patients with severe COVID-19 (WHO [corticosteroids] [17], WHO [therapeutics] [21]: strong, moderate), but not for patients with nonsevere COVID-19 (WHO [corticosteroids] [17]: weak, low; WHO [13]: strong ungraded; NHS [tocilizumab] [66]: ungraded) or with severe COVID-19 without ARDS (CMAJ [72]: weak, ungraded). Six documents (ICSI [38], CDC [22], NHS [rapid guideline] [52], NHS [management] [50], ASID [68], and Korea [73]: ungraded, very low) recommended against the routine use of corticosteroids.

DISCUSSION

This study critically reviewed the scientific evidence and recommendations from guidelines on acute COVID-19 management. Generally, the quality of the existing guidelines was low and highly variable. The recommendations across guidelines had considerable discrepancies and lacked clear links between recommendations and underlying evidence.

Some of our results were similar to those of a previous study [5] that reviewed guidelines produced early during the pandemic. However, with the progress of the pandemic, our understanding of COVID-19 has deepened gradually. As more and more new evidence emerges, the recommendations also need to be updated in a timely manner. The RECOVERY trial showed that the use of dexamethasone lowered the 28-day mortality rate among patients who were receiving either invasive mechanical ventilation or oxygen alone at randomization [77]. In another trial involving patients with ARDS who were undergoing mechanical ventilation, the 60-day mortality rate was 15 percentage points lower among patients receiving dexamethasone than among those receiving standard of care [78]. Consequently, the recommendations regarding dexamethasone in NIH and NCCET guidelines have changed. It is crucial to evaluate temporal changes in guidelines’ quality and help clinicians become aware of discrepancies and adjustments of the recommendations to guide clinical decision-making and improve patient outcomes.

As COVID-19 was an emerging infectious disease, there was no direct evidence available to develop evidence-based guidelines on short notice. Most existing guidelines were interim guidance or rapid guidelines that largely relied on experts’ experiences. The developers may not have had enough time to compose the guidelines according to the standard methods and procedures such as conducting systematic reviews of the evidence and literature. Yet, compared with the guidelines issued at early stages (in 2020), the quality of those issued in 2021 clearly improved, owing to a deeper understanding of the disease and new emerging evidence. Currently, clinical information related to the optimal management of acute COVID-19 is evolving quickly, and many clinical trials are ongoing, which will provide evidence of higher quality. Most guidelines are living documents that are updated frequently as newly published data and other authoritative information become available. We believe that it is reasonable to adopt current recommendations into practice to improve management while waiting for new evidence to surface.

Guideline documents are generally consistent regarding the timing to start oxygen therapy in COVID-19 patients. Considering the harmful potential of hyperoxia and depletion of oxygen resources, it is not appropriate to maintain SpO2 at a high level (>96%). Guidelines differ on NIV/HFNC recommendations, mainly based on previous clinical experience in the respective countries; NIV/HFNC for COVID-19 has been associated with a high failure rate, worse outcomes, and a possible increase in the risks of aerosolization and delayed intubation, especially for the use of NIV. Based on an unblinded clinical trial [79] and a meta-analysis [80] performed before the COVID-19 pandemic, some guidelines have suggested that HFNC is preferable over NIV in adults with COVID-19 and acute respiratory failure. It remains uncertain whether NIV/HFNC should be used in adults with COVID-19. Therefore, any patients receiving HFNC or NIPPV should be monitored closely and vigilantly to facilitate intubation in case of rapid deterioration. Early intubation may be particularly appropriate when patients have additional acute organ dysfunction or chronic comorbidities, or when HFNC and NIPPV are not available [81, 82].

For critical COVID-19 with ARDS, the recommendations were generally consistent. For septic shock, all documents consistently recommended resuscitation and vasopressors, measures derived from the Surviving Sepsis Campaign “International Guidelines for Management of Sepsis and Septic Shock” [83]. The overall opinion on indication for ECMO was relatively consistent. Based on a preliminary report [84], HNS [49] proposes revised and strict ECMO inclusion criteria in response to the COVID-19 pandemic. In addition to the above conditions, it was also necessary to score and evaluate the potential benefits of the recommendations. Given the limited clinical resources, ASAIO (ECMO) [30] recommended against the use of ECMO before conventional therapies. However, NIH [19] recommended either for or against the routine use of ECMO for patients with refractory hypoxemia because of the lack of available conclusive evidence [85, 86].

During the initial phase of the COVID-19 pandemic, there were no drugs or therapy proven to be effective. As China and Italy were the first countries to suffer from COVID-19 outbreaks, most guidelines in these 2 countries recommended various antiviral treatments. With the publication of additional clinical research results related to the use of more remdesivir, the guidelines published after April 2020 gave more detailed suggestions in terms of subgroups of patients, dosage, and duration. A number of clinical studies on COVID-19 treatment with remdesivir, favipiravir, and tocilizumab have shown beneficial outcomes. Therefore, recent guidelines have recommended the use of antiviral drugs for patients with severe COVID-19.

Clinical trials testing baloxavir marboxil, darunavir-cobicistat, HIV protease inhibitors, mesenchymal stem cells (MSCs), and umifenovir have been published. However, the use of these treatments was only recommended for clinical trials, and these treatments should be cautiously used due to various potential adverse reactions. More clinical trials are ongoing [87], and candidate drugs are under development [88].

The guidelines generally agree on the scope of application of antibiotics and recommend their empirical use in severe or critical cases with sepsis, but not for mild or uncomplicated cases [4, 14, 37, 44, 71]. Some documents [13, 23, 26] recommended constant reassessment in order to de-escalate early or stop antibiotic treatments, while others recommended against blind or inappropriate use of antibiotic drugs [13, 55, 73]. Disease severity and suspected co-infection are important indications for antibiotic use. Fever is one of the most common symptoms of COVID-19, and most guidelines agree that the administration of antipyretics should be based on fever symptoms, disease severity, and comorbidities [4, 12–14, 71]. Regarding drug selection, because NSAIDs like ibuprofen have been reported to potentially increase ACE2 expression [89] and inhibit antibody production [90], paracetamol was recommended rather than NSAIDs in the NICE (managing symptoms) [47] and the PCS guidelines [71]. However, the Food and Drug Administration rapidly stated that there was no evidence linking the use of NSAIDs to worsening COVID-19 [91].

Whether corticosteroids could be used in COVID-19 patients remained controversial until August 2020. Most guidelines recommended the use of corticosteroids for certain conditions [4, 14, 23, 37, 69], whereas other guidelines [59, 72] recommended a total treatment duration of 7–10 days, with progressive dose reduction. The Korean guidelines [73] recommended against routine use of corticosteroids based on adverse effects such as prolonged viral replication, which may result in mechanical ventilation and higher mortality [22] or increased exposure to fungal pathogens [52]. Recent guidelines have consistently recommended the use of corticosteroids at early stages of severe illness.

Our study has several strengths. We had a broad inclusion of guidelines located in a broad range of geographical locations. Furthermore, we evaluated the quality of the guidelines using AGREE II, which is an internationally validated instrument for guidelines assessment. Our team was composed of experts from different backgrounds, gathering front-line clinical experts and methodologists. A significant degree of agreement among the 4 reviewers was achieved, which improved the reliability of our findings. However, some limitations could bias our study and limit generalizability. First, the guidelines were limited to publications in the English language. Second, it was difficult to identify recommendations that were not developed by evidence-based methods. This limitation may be mitigated by the involvement of reviewers with clinical experience to capture recommendations. Third, we could not always identify all the supplemental materials necessary for quality assessments and might have underestimated guideline quality. Finally, the AGREE II instrument focuses on methods of guideline development and transparency of reporting, but does not assess the potential impacts of the recommendations on patient outcomes [92].

CONCLUSIONS

The quality of existing guidelines for COVID-19 management was generally low and highly variable. There were considerable recommendation discrepancies between guidelines and a general lack of evidence, especially for the recommendations related to respiratory support and antiviral therapy.

Click here for additional data file.