Renin-Angiotensin Aldosterone System Inhibitors and COVID-19: A Systematic Review and Meta-Analysis Revealing Critical Bias Across a Body of Observational Research.

Background Renin-angiotensin aldosterone system (RAAS) inhibitor-COVID-19 studies, observational in design, appear to use biased methods that can distort the interaction between RAAS inhibitor use and COVID-19 risk. This study assessed the extent of bias in that research and reevaluated RAAS inhibitor-COVID-19 associations in studies without critical risk of bias. Methods and Results Searches were performed in MEDLINE, EMBASE, and CINAHL databases (December 1, 2019 to October 21, 2021) identifying studies that compared the risk of infection and/or severe COVID-19 outcomes between those using or not using RAAS inhibitors (ie, angiotensin-converting enzyme inhibitors or angiotensin II type-I receptor blockers). Weighted hazard ratios (HR) and 95% CIs were extracted and pooled in fixed-effects meta-analyses, only from studies without critical risk of bias that assessed severe COVID-19 outcomes. Of 169 relevant studies, 164 had critical risks of bias and were excluded. Ultimately, only two studies presented data relevant to the meta-analysis. In 1 351 633 people with uncomplicated hypertension using a RAAS inhibitor, calcium channel blocker, or thiazide diuretic in monotherapy, the risk of hospitalization (angiotensin-converting enzyme inhibitor: HR, 0.76; 95% CI, 0.66-0.87; P<0.001; angiotensin II type-I receptor blockers: HR, 0.86; 95% CI, 0.77-0.97; P=0.015) and intubation or death (angiotensin-converting enzyme inhibitor: HR, 0.64; 95% CI, 0.48-0.85; P=0.002; angiotensin II type-I receptor blockers: HR, 0.74; 95% CI, 0.58-0.95; P=0.019) with COVID-19 was lower in those using a RAAS inhibitor. However, these protective effects are probably not clinically relevant. Conclusions This study reveals the critical risk of bias that exists across almost an entire body of COVID-19 research, raising an important question: Were research methods and/or peer-review processes temporarily weakened during the surge of COVID-19 research or is this lack of rigor a systemic problem that also exists outside pandemic-based research? Registration URL: www.crd.york.ac.uk/prospero/; Unique identifier: CRD42021237859.

renin‐angiotensin aldosterone system

Risk of Bias in Non‐Randomized Studies—of Interventions

thiazide diuretic

Clinical Perspective

What Is New?

This review identified critical biases in 164 of the 169 studies that, until October 21, 2021, evaluated the associations between renin‐angiotensin aldosterone system (RAAS) inhibitor use and COVID‐19 outcomes, biases that could have distorted the evidence upon which health policy was based.

As of December 2021, at least 52 meta‐analyses were conducted against guidelines, pooling studies with critical risk of bias.

Synthesizing two nationwide studies, which used methods that, as best as possible, isolated RAAS inhibitor—COVID‐19 interactions, this meta‐analysis provided further evidence that RAAS inhibitor use does not increase the risk of severe COVID‐19 outcomes.

What Are the Clinical Implications?

The findings of this study support directives from health authorities, early in the pandemic, that users of RAAS inhibitors could continue their treatment safely.

Although bias‐minimizing methods needed to be applied to isolate the interaction between RAAS inhibitor use and COVID‐19, that methodological approach limits the generalizability of these findings; and, thus, it is not known if these findings are applicable to those with underlying comorbidities and those in combination therapy; the proportion of the population who are most at risk of severe COVID‐19 outcomes and who represent the majority of people using RAAS inhibitors.

Since it was found that SARS‐CoV‐2 gains entry into human cells via angiotensin‐converting enzyme (ACE) 2, there has been an inundation of research evaluating if the use of renin‐angiotensin aldosterone system (RAAS) inhibitors increases the risk of a SARS‐CoV‐2 infection and/or severe COVID‐19 outcomes. Those studies, predominantly observational in design, were driven by the notion that RAAS inhibitors, such as ACE inhibitors and or angiotensin II type‐I receptor blockers (ARBs), may upregulate the expression of ACE2. , , As the pandemic progressed, however, it became evident that, like a lot of COVID‐19 research, many RAAS inhibitor—COVID‐19 studies have suffered from the speed at which they were conducted ; often using nonrepresentative samples where the risk of selection (ie, collider) bias is increased. At least 52 meta‐analyses have assessed associations between RAAS inhibitor use and COVID‐19 risk by collating parts of that body of research. , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , However, little attention has been paid to selection bias, or other biases for that matter, which may distort the interaction between RAAS inhibitor use and COVID‐19.

Given the sheer volume of previous research, it is understandable if the appeal of additional RAAS inhibitor—COVID‐19 studies may be subsiding. However, in the interest of improving the methods used in observational research and, subsequently, enhancing its value, there is an urgent need to review how the scientific community has attempted to address this issue; to what extent does bias exist across that body of RAAS inhibitor—COVID‐19 research and does that bias significantly distort the interaction between RAAS inhibitor use and COVID‐19 risk? Accordingly, this study retested the hypothesis that RAAS inhibitor use is associated with important, severe COVID‐19 outcomes, using only data from observational studies without critical risk of bias.

METHODS

The data that support the findings of this study are available within the article. The protocol of this systematic review and meta‐analysis was registered on PROSPERO (Registration number: CRD42021237859) before the study commenced and is available in full on the National Institute for Health Research International Prospective Register of Systematic Reviews (https://www.crd.york.ac.uk/PROSPERO). This study was reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta‐Analyses checklist (Table S1).

Data Sources and Searches

Systematic searches were performed in MEDLINE, EMBASE, and CINAHL databases from December 1, 2019, near to when the first case of SARS‐CoV‐2 was identified, until October 21, 2021. A combination of subject headings for COVID‐19 and RAAS inhibitors was used (Table S2). Searches were limited to “human” studies only. A manual search of the citations included in identified reviews and articles selected for full‐text retrieval was also performed.

Study Selection, Inclusion and Exclusion Criteria

Two investigators (J. L. and F. C. T.) independently performed study selection using Covidence, an online, Cochrane approved, software designed for conducting systematic reviews. Discrepancies in inclusion or exclusion were solved through consultation with a third investigator (J. S.). To address the risk of bias in all observational studies of RAAS inhibitor—COVID‐19 associations, the systematic review initially identified studies that compared the risk of infection with COVID‐19 and/or severe COVID‐19 outcomes (eg, hospitalization, admission to an intensive care unit, intubation and/or death with COVID‐19) between those using an ACE inhibitor or an ARB and those not using a RAAS inhibitor. Only data from studies on people aged ≥18 years were included. Review articles were excluded, but as stated previously, their reference lists were screened. Studies indexed to preprint servers only, which are not certified by peer review, were excluded. Conference abstracts were also excluded. Finally, inclusion was limited to studies originally published in the English language or where translated copies had been made available.

Risk of Bias Assessment

Given that this review focussed on observational studies, the risk of bias assessment for each study was performed independently by J. L. and F. C. T. using the Risk of Bias in Non‐Randomized Studies—of Interventions (ROBINS‐I), developed by the Cochrane Bias Methods Group and the Cochrane Non‐Randomized Studies Methods Group. The ROBINS‐I tool includes seven domains of potential bias: (1) bias due to confounding, (2) bias in selection of participants into the study, (3) bias in the classification of interventions, (4) bias due to deviations from intended interventions, (5) bias due to missing data, (6) bias in the measurements of outcomes, and (7) bias in selection of the reported result.

A study was deemed to have a low risk of bias overall if it was judged to have a low risk of bias across all domains; a moderate risk of bias overall if it was judged to have a low or moderate risk of bias across all domains; a serious risk of bias overall if it was judged to have a serious risk of bias in at least one domain, but not at a critical risk of bias in any other domain; and a critical risk of bias overall if it was judged to have a critical risk of bias in at least one domain. If there was nothing indicating a serious or critical risk of bias, but there was a lack of information to make a judgment in any of the domains, a study was deemed to have “no information” regarding its risk of bias. Discrepancies in bias classification were resolved by discussion. As recommended, any study with a critical risk of bias was excluded from the subsequent synthesis and analysis.

Outcomes

Although risk of bias was assessed in studies of the association between RAAS inhibitor use and the risk of infection, it was planned from the outset that only severe COVID‐19 outcomes (eg, hospitalization, admission to an intensive care unit, intubation, death) would be evaluated by the meta‐analysis. Indeed, severe outcomes are most relevant considering that they are the burden to health care systems, whereas risk of infection data have an unknown potential for bias because of changes in COVID‐19 testing strategies in outpatient care and, likely, a high rate of missed cases. Severe outcomes that could be pooled from the eligible studies included hospitalization and a combination of intubation and death with COVID‐19.

Data Extraction

The main characteristics of each study eligible for inclusion in the meta‐analysis were summarized in duplicate by J. L. and F. C. T. into a preformatted spreadsheet. These characteristics included (1) first author names and year of publication, (2) country, (3) study design, (4) sample size, (5) comorbidities, (6) number of relevant severe COVID‐19 events, and (7) the weighted estimates for each outcome of interest.

Statistical Analysis

The weighted hazard ratios (HR) and the corresponding 95% CI detailing the association between using an ACE inhibitor or an ARB in monotherapy and hospitalization, intubation, or death with COVID‐19 were pooled in fixed‐effects meta‐analyses. Those using a RAAS inhibitor in monotherapy were compared with those using a non‐RAAS inhibitor in monotherapy (ie, a calcium channel blocker or thiazide diuretic [TZD]). A fixed‐effects meta‐analysis was chosen as the primary analysis given the low number of studies included in the meta‐analyses. Indeed, a random‐effects meta‐analysis may not adequately estimate the heterogeneity and weights when so few studies are available. Recognizing that there are contrasting opinions as to when it is appropriate to use either a fixed‐ or random‐effects model, a random‐effects meta‐analysis was conducted as a secondary (sensitivity) analysis. Heterogeneity was quantified using the I² statistic (I²>50%) and tested using Cochran’s Q statistic (P<0.10). Publication bias was not assessed as there were fewer than 10 studies available for the analysis. A P value of <0.05 was considered statistically significant. Analyses were performed using R, version 4.0.0 (R‐Core Team, Vienna, Austria).

Certainty of the Evidence Assessment

The Grading of Recommendations Assessment, Development and Evaluation (GRADE) was performed in duplicate by J. L. and F. C. T. to provide an assessment of the quality of (ie, certainty of) the evidence produced by the meta‐analyses. The GRADE’s official software package, GRADEpro Guideline Development Tool (McMaster University and Evidence Prime Inc.), was used to summarize the findings of the meta‐analyses and rate the certainty of the evidence for each outcome as high, moderate, low, or very low. More information about the GRADE can be found in Data S1.

RESULTS

Study Selection and Risk of Bias Analysis

The systematic search identified 169 observational studies that assessed the associations between RAAS inhibitor use and the risk of contracting a SARS‐CoV‐2 infection and/or experiencing severe COVID‐19 outcomes (Figure 1). , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Of those 169 studies, 164 had critical risks of bias and were excluded from the meta‐analysis (Figure 2). Among these 164 studies, critical biases were most commonly due to confounding bias (n=157), selection bias (n=146), and bias due to deviations from the intended interventions (n=67).

Figure 1

Flow diagram of the study selection process.

 

Figure 2

Risk of bias assessment using the Risk of Bias in Non‐Randomized Studies—of Interventions (ROBINS‐I).

The risk of bias is indicated as an overall risk of bias (ie, across all domains), as well as for each separate domain of potential bias, in each study that met the inclusion criteria of this systematic review and meta‐analysis. In total, 164 out of 169 were judged to have a critical risk of bias and were excluded from the meta‐analysis. Green indicates a low risk of bias, yellow indicates a moderate risk of bias, orange indicates a serious risk of bias, red indicates a critical risk of bias, and grey indicates that there was a lack of information in the study to make a judgment on the risk of bias for that specific domain.

The risk of bias in the classification of the interventions, the measurement of the outcomes, and the selection of the reported result was predominantly low across the entire body of research. Notably, a substantial number of studies failed to provide information about how missing data were handled (n=90) or if important cointerventions (ie, other antihypertensive therapies) were addressed (n=60).

Of the five studies without critical risks of bias, two studies assessed only the association between RAAS inhibitor use and risk of infection (a nonsevere outcome that does not burden health care systems) and, as planned, were not included in the subsequent data synthesis. , A third study without critical bias had eligible data in its secondary analysis, but the event rates for the relevant outcomes were not available ; and, thus, it could not be pooled in the meta‐analysis.

Study Characteristics

Two nationwide studies, one each from France and Sweden, could be included in the meta‐analysis (Table 1). , Although the study from France also completed analyses on those using a combination of antihypertensive therapies, the meta‐analysis considered only outcomes commonly assessed in each study and, thus, included data from all French (n=1 186 987) and Swedish (n=164 655) residents with uncomplicated hypertension who used either an ACE inhibitor, ARB, calcium channel blocker, or TZD in monotherapy (Note: data for French citizens using a TZD in monotherapy were not available). Given that as‐treated data were available only in the study from Sweden, only intention‐to‐treat data were used in the meta‐analysis. Both studies excluded those with known cardiovascular and/or kidney diseases in order to limit confounding bias. The follow‐up period was similar in both the French and Swedish studies, running from February 15, 2020, to June 7, 2020, and from January 1, 2020, to June 23, 2020, respectively; covering the first wave of the COVID‐19 pandemic in each country.

Table 1

Main Characteristics of the Studies Included in the Meta‐Analysis

Study	Country	Study period	Population*	Comorbidities	Hospitalization with COVID‐19	Intubation or death with COVID‐19 †
Loader et al, 163 2021	Sweden	January 1, 2020–June 23, 2020	All residents in Sweden (n=164 655) with uncomplicated hypertension using, in monotherapy, an ACE inhibitor (n=47 998) ARB (n=68 239) CCB or TZD (n=48 418)	Those with preexisting cardiovascular disease and kidney diseases were excluded	ACE inhibitor (n=94) ARB (n=135) CCB or TZD (n=107)	ACE inhibitor (n=16) ARB (n=19) CCB or TZD (n=26)
Semenzato et al, 201 2021	France	February 15, 2020–June 7, 2020	All residents in France (n=1 186 987) with uncomplicated hypertension using, in monotherapy, an ACE inhibitor (n=353 236) ARB (n=582 031) CCB (n=251 720)	Those with diabetes, cardiovascular disease, chronic respiratory disease, and/or chronic renal failure in the 5 years before the study were excluded.	ACE inhibitor (n=340) ARB (n=690) CCB (n=384)	ACE inhibitor (n=72) ARB (n=148) CCB (n=99)

ACE indicates angiotensin‐converting enzyme; ARB, angiotensin II type‐I receptor blocker; CCB, calcium channel blocker; and TZD, thiazide diuretic.

Although only data for treatment by monotherapy are presented in the table, it should be noted that Semenzato et al. (2021) also conducted analyses on those in combination therapy.

Intubation was only an outcome in the study by Semenzato et al. (2021), meaning only deaths were recorded in Loader et al. (2021).

Meta‐Analyses

The fixed‐effects meta‐analyses indicate that in those using an ACE inhibitor or an ARB in monotherapy, there is a lower risk of hospitalization (ACE inhibitor: HR, 0.76; 95% CI, 0.66–0.87; P<0.001; ARB: HR, 0.86; 95% CI, 0.77–0.97; P=0.015) and intubation or death (ACE inhibitor: HR, 0.64; 95% CI, 0.48–0.85; P=0.002; ARB: HR, 0.74; 95% CI, 0.58–0.95; P=0.019) with COVID‐19 (Figure 3), when compared with those using a calcium channel blocker or TZD in monotherapy. The heterogeneity in each mortality analysis was most likely driven by the low number of events in the study from Sweden. The direction of the estimates varied little when a random‐effects meta‐analysis was used (Figure S1).

Figure 3

Forest plots for each outcome assessed in the fixed‐effects meta‐analyses.

Presented are the associations between (A) the use of an ACE inhibitor in monotherapy and hospitalization, (B) the use of an ACE inhibitor in monotherapy and intubation or death, (C) the use of an ARB in monotherapy and hospitalization and (D) the use of an ARB in monotherapy and intubation or death. ACEi indicates angiotensin‐converting enzyme inhibitor; ARB, angiotensin II type‐I receptor blocker; CCB, calcium channel blocker; HR, hazard ratio; and TZD, thiazide diuretic.

Certainty of the Evidence

The GRADE indicated that the protective effects detected by the meta‐analyses are probably not clinically relevant, that the risk of hospitalization, intubation, or death varies little between those using a RAAS inhibitor and those using a calcium channel blocker or TZD (Table 2). Although the meta‐analyses were based on observational studies, certainty in the evidence for each outcome began with a high rating because of the use of the ROBINS‐I. There were no reasons to downgrade the certainty of the evidence for each of the hospitalization outcomes. However, given that only one of two studies provided intubation data in the intubation or death outcomes, certainty of the evidence for these outcomes was downgraded by one level to a moderate rating owing to serious indirectness in the outcomes between studies.

Table 2

Summary of Findings Including the Certainty of the Evidence

Outcomes	Anticipated absolute effects* (95% CI)		Relative effect (95% CI)	№ of participants (studies)	Certainty of the evidence (GRADE)	Comments
	Risk with a CCB or TZD	Risk with an ACE inhibitor or ARB
ACE inhibitor use and the risk of hospitalization with COVID‐19	164 per 100 000	124 per 100 000 (108–142)	HR 0.76 (0.66–0.87)	701 372 (2 observational studies)	⨁⨁⨁⨁ HIGH	The risk of hospitalization with COVID‐19 differs little between those using an ACE inhibitor and those using a CCB or TZD in monotherapy
ACE inhibitor use and the risk of intubation or death with COVID‐19	42 per 100 000	27 per 100 000 (20–35)	HR 0.64 (0.48–0.85)	701 372 (2 observational studies)	⨁⨁⨁◯ MODERATE†,‡,§	The risk of intubation or death with COVID‐19 differs little between those using an ACE inhibitor and those using a CCB or TZD in monotherapy
ARB use and the risk of hospitalization with COVID‐19	164 per 100 000	141 per 100 000 (126–159)	HR 0.86 (0.77–0.97)	950 408 (2 observational studies)	⨁⨁⨁⨁ HIGH	The risk of hospitalization with COVID‐19 differs little between those using an ARB and those using a CCB or TZD in monotherapy
ARB use and the risk of intubation or death with COVID‐19	42 per 100 000	31 per 100 000 (24–40)	HR 0.74 (0.58–0.95)	950 408 (2 observational studies)	⨁⨁⨁◯ MODERATE†,‡,§	The risk of intubation or death with COVID‐19 differs little between those using an ARB and those using a CCB or TZD in monotherapy

GRADE Working Group grades of evidence—High certainty: We are very confident that the true effect lies close to that of the estimate of the effect. Moderate certainty: We are moderately confident in the effect estimate: The true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different. Low certainty: Our confidence in the effect estimate is limited: The true effect may be substantially different from the estimate of the effect. Very low certainty: We have very little confidence in the effect estimate: The true effect is likely to be substantially different from the estimate of effect. ACE indicates angiotensin‐converting enzyme; ARB, angiotensin II type‐I receptor blocker; CCB, calcium channel blocker; GRADE, Grading of Recommendations Assessment, Development and Evaluation; HR, hazard ratio; and TZD, thiazide diuretic.

*The risk in users of an ACE inhibitor or an ARB (and its 95% CI) is based on the assumed risk in the users of a CCB or TZD and therelative effect of the intervention (and its 95% CI).

Explanations

†A composite outcome of intubation and death was used in the meta‐analysis, but intubation data were not available in Loader et al. (2021), resulting in the certainty of the evidence being downgraded by one level due to serious indirectness.

‡Point estimates vary greatly between each study, but the importance of this is questionable when considering the weighting of each study and does not decrease certainty in the evidence.

§Heterogeneity is most likely explained by the low number of deaths in Loader et al. (2021).

⨁⨁⨁⨁

⨁⨁⨁◯

⨁⨁⨁⨁

⨁⨁⨁◯

DISCUSSION

This study reveals that current policy regarding the safety of using RAAS inhibitors during the ongoing COVID‐19 pandemic is based almost entirely on a body of research that has severe limitations due to critical risk of bias. However, when only studies without critical risk of bias were pooled, results were consistent with the majority of that body of research; indicating that RAAS inhibitor use does not increase the risk of severe COVID‐19 outcomes.

Critical risk of bias was most often attributed to confounding bias and/or selection bias, found in 164 of 169 observational studies relevant to this meta‐analysis. Confounding bias was introduced mainly by the inclusion of people with preexisting cardiovascular and kidney diseases, confounding factors that are inherently uncontrollable when aiming to isolate the effect of RAAS inhibitor use on COVID‐19 outcomes. Selection bias was introduced by the inclusion of samples restricted to people who were tested for/tested positive to a SARS‐CoV‐2 infection and/or who had been hospitalized due to COVID‐19; sampling strategies that form cohorts unrepresentative of the general population.

These biases have the potential to create a spurious within‐sample association between two variables, so called collider bias (eg, in the context of RAAS inhibitor—COVID‐19 studies: frailty due to cardiovascular disease, with a high likelihood of being prescribed a RAAS inhibitor; and frailty due to an adverse COVID‐19 course), that affects the probability of being included in the sample (eg, being hospitalized). To minimize such biases in the context of RAAS inhibitor—COVID‐19 observational research, a primary prevention sample of yet uninfected people using RAAS inhibitors or a relevant comparator drug class needs to be studied in order to isolate the interaction between RAAS inhibitor use and COVID‐19, not in those already affected by the virus.

Ultimately, only five observational studies used bias‐minimized study designs, two of which could be included in the meta‐analysis. The conclusion from this research, that RAAS inhibitor use does not increase the risk of severe COVID‐19 outcomes, is consistent with the eight randomized controlled trials that have evaluated RAAS inhibitor—COVID‐19 associations thus far. , , , , , , , Although the 164 studies with critical biases cannot isolate the interaction between RAAS inhibitor use and COVID‐19, the value of that research should not be discounted entirely. Indeed, those studies combined contributed to a detailed characterization of the population who experienced a severe COVID‐19 disease course, identifying the people (eg, elderly people, people with comorbidities) who need more attention or greater protection (eg, prioritized vaccination) during the ongoing pandemic.

The exclusion of people with preexisting cardiovascular and kidney diseases from the studies pooled in the meta‐analyses is likely to prompt questions about the limitations of this study: Do these findings apply to those with such comorbidities? Would these findings not be more relevant if the analyses include those with comorbidities, those who are more at risk of severe a COVID‐19 disease course? Indeed, it is unknown whether the findings of this meta‐analysis extend to those with underlying comorbidities, the proportion of the population who are most at risk of severe COVID‐19 outcomes and who represent the majority of people using RAAS inhibitors. Although it is important that this limitation is acknowledged, the fact remains that people with preexisting cardiovascular and kidney diseases could introduce confounding so intractable that statistical methods will not be able to control for this bias, necessitating the exclusion of those people to isolate any interaction between RAAS inhibitors and COVID‐19. This methodological consideration represents an effort to produce observational research that, as best as possible, emulates a randomized controlled trial, a methodological principle that seems to have been overlooked in many drug‐safety and effectiveness studies during the COVID‐19 pandemic, not only those pertaining to RAAS inhibitors.

Other potential limitations must also be addressed. This meta‐analysis did not conduct an analysis including people on combination therapy with antihypertensive drugs. However, given the results from the nationwide study in France, it would appear that these findings based on monotherapy data could be extended to those on combination therapy. Owing to data availability, only intention‐to‐treat data were used in the analysis and variations in the prescribed RAAS inhibitor dose were not addressed by the meta‐analysis. Thus, it is unclear how an as‐treated analysis would affect the findings. However, the nationwide study from Sweden indicated that there is not much variation in the estimates between an intention‐to‐treat and an as‐treated analysis. Finally, both studies in the meta‐analysis (like most RAAS inhibitor—COVID‐19 research) covered only the first wave of the pandemic, where event rates were relatively low compared with subsequent waves, and when early variants of the virus existed, possibly limiting the data in terms of their relevance to the evolving COVID‐19 situation.

This study was not the first systematic review and meta‐analysis to assess the associations between RAAS inhibitor use and COVID‐19 outcomes. As of December 2021, at least 52 meta‐analyses had been conducted, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , each of which pooled studies that have a critical risk of bias. Critical biases limit confidence in the evidence produced by these meta‐analyses as much as they do in each of the studies that they were based on. Current guidelines from the Cochrane Collaboration instruct that these meta‐analyses should not have been conducted, that studies with critical risk of bias should not be synthesized. However, only six of the 52 previous meta‐analyses followed current guidelines and used the ROBINS‐I to assess risk of bias. , , , , , It is unclear as to why the authors of five of those six studies did not identify any critical risks of bias and proceeded with their meta‐analyses. One meta‐analysis, however, deviated from the guidelines, establishing their own unstandardized scoring system for the ROBINS‐I.

Of the other 46 meta‐analyses, only 18 referred to having completed a risk of bias assessment. Whether they recognized it or not, 34 studies used the Newcastle‐Ottawa Scale for assessing the risk of bias. The Newcastle‐Ottawa Scale, which has not been recommended by the Cochrane Collaboration since well before the COVID‐19 pandemic (in a now archived version of the guidelines), provides a quality score where a higher value is interpreted as a lower risk of bias. Although this scale might detect a critical risk of bias in one domain, a study may still be deemed to be of high quality overall and, thus, have a low risk of bias overall because of strengths in other domains; essentially allowing for an important, potentially association‐distorting bias(es) to go overlooked and explaining why some studies proceeded with their meta‐analyses. In contrast, the ROBINS‐I recognises that a critical risk of bias needs to exist in only one domain to limit confidence in an entire study and to provide reason for it to be excluded from a data synthesis. The Cochrane Collaboration’s transition from the Newcastle‐Ottawa Scale to the ROBINS‐I further reflects the need for more rigor in the design of observational research.

Three meta‐analyses acknowledged use of a risk of bias assessment tool that is integrated into the GRADE. This tool, by admission of its creators, was not as comprehensive as other methods available at the time (Note: created long before the development for the ROBINS‐I) ; and like the Newcastle‐Ottawa Scale, it also overlooks important sources of bias. Only eight out of 52 meta‐analyses used the GRADE to summarize their findings and determine the certainty of their evidence; a process that is crucial in being able to properly interpret the clinical importance of findings from a meta‐analysis (eg, as demonstrated in this study: the difference between concluding a protective effect and no effect). Collectively, this poor methodology demonstrates a lack of knowledge around the current guidelines for systematic reviews and meta‐analyses, among authors, reviewers, and journal editors alike.

CONCLUSIONS

Despite wavering interest, additional studies related to COVID‐19 risk are inevitable, emphasizing the urgent need for this present systematic review and meta‐analysis. Indeed, to improve future research, the inadequate methodologies used by almost an entire body of observational research, as well as by the meta‐analyses that collated those studies, needed to be highlighted. If further RAAS—COVID‐19 observational studies are to be conducted, it would be interesting if they extended follow‐up to include subsequent waves of the pandemic, where event rates were substantially higher and when new variants of COVID‐19 exist(ed); evaluating if associations change over time with variations in infection rates and in the virus. That research must address the methodological issues (eg, confounding bias and selection bias) identified in most previous studies in order to, as best as possible, isolate the interaction between RAAS inhibitor use and COVID‐19. Researchers should also recognize that the methodological principles needed to improve RAAS—COVID‐19 studies extend to other drug safety and effectiveness studies; methodological approaches that are crucial to enhancing the scientific value of observational research.

In summary, this study presents data that, fortunately, support directives from health authorities early in the pandemic, that antihypertensive therapies could be continued safely as per normal. However, it is not certain whether the effect estimates would remain consistent throughout subsequent waves of the pandemic or whether the findings are applicable to those with underlying comorbidities. Most significantly, this study reveals the extent of bias in RAAS inhibitor—COVID‐19 studies, showing how wrong things can go when there is carelessness in research; that poorly designed studies continue, with little question, to direct subsequent research and health policies alike. This raises an important question: Have research methods and/or peer review processes been temporarily weakened during the surge of COVID‐19 research, potentially because of a desire to be the first published or to be the first to publish, or is this lack of rigor a systemic problem that also exists outside pandemic‐based research?

Sources of Funding

This project (Dr Jordan Loader) has received funding from the European Union’s Horizon 2020 Research and Innovation Programme under the Marie Skłodowska‐Curie grant agreement no. 898829. This research was also supported by funding from the Swedish Heart‐Lung Foundation and Anders Wiklöf.

Disclosures

Dr Jordan Loader has completed medical writing services, separate of this study, for AstraZeneca; Professor Johan Sundström reports stock ownership in companies providing services to Itrim, Amgen, Janssen, Novo Nordisk, Eli Lilly, Boehringer, Bayer, Pfizer, Takeda, and AstraZeneca. The remaining authors have no disclosures to report.

Data S1

Tables S1–S2

Figure S1

Click here for additional data file.