Systematic reviews of antihypertensive drugs: A review of publication trends, characteristics, and quality.

This review presents publication trends, characteristics, and quality of systematic reviews (SRs) of randomized controlled trials (RCTs) of antihypertensive drugs (AHTDs). Between 1985 and 2017, 1,173 SRs were published, and in the last 20 years, 10, 35, and 116 were published in the year 1996, 2006, and 2016, respectively. Angiotensin-converting enzyme inhibitors/angiotensin II receptor blockers were the most common class of drugs studied. Fourteen percent of the SRs were prospectively registered/published protocol. Three-fourth of the SRs did not report a full search strategy, and 45% did not report a PRISMA or similar diagram. Of the 34 SRs published in the five high impact factor journals in the last 10 years, 15%, 21%, and 65% have unclear, low, and high risk of bias, respectively. There has been a steady increase in the publication of SRs of RCTs of AHTDs. However, adherence to standard methods of conduct and reporting continues to be low.

INTRODUCTION

Cardiovascular diseases (CVDs) are the leading cause of death globally, and hypertension is the leading risk factor for CVD. , Antihypertensive drugs (AHTDs) are among the most commonly used prescription drugs worldwide. Drug regulatory agencies have approved many AHTDs primarily based on evidence of efficacy and safety from randomized controlled trials (RCTs). Although RCTs are considered the gold standard for generating evidence of effects of interventions, health care decisions based on only some of all the available RCTs are not considered credible. Systematic reviews (SRs) aim to identify all relevant literature on a topic, critically appraise, and summarize evidence to answer well‐defined questions. Decision‐makers, guideline developers, and health care providers use SRs to inform decisions to improve health care. Mapping of SRs can be a useful one‐stop resource for the consumers of evidence synthesis to enable evidence‐informed research and health care decisions. Mapping reviews can provide information about the current state, research trends, and future research needs in a particular area of interest. Numerous SRs of effects of AHTDs have been published so far. Mapping reviews have been produced in several disciplines of medicine, , , but not of SRs of AHTDs. We, therefore, undertook this study with the objective of identifying SRs of RCTs of AHTDs and sought to assess their publication trends, characteristics, and quality.

METHODS

Literature search

Systematic literature searches were conducted in Ovid MEDLINE (1946 to September Week 1 2017) and Epistemonikos (inception to September 21, 2017). Epistemonikos is a database of SRs that is regularly updated with the articles published in PubMed and 18 other sources. , The search strategy included relevant Medical Subject Heading (MeSH) and keywords. The full search strategy is reported in Table S1.

Eligibility criteria for including studies

Eligible studies were SRs that included one or more RCT(s) of AHTD(s) irrespective of the participants’ condition at baseline and outcomes assessed. We excluded conference abstracts, SRs published in non‐English languages, pooled analyses, meta‐analyses, and individual patient data analyses with no description of systematic processes for the identification and inclusion of RCTs. A SR was defined as a study that searched at least one bibliometric database, provided at least one eligibility criterion for the inclusion of studies, and synthesized primary studies. We also included studies if they were reported by the authors as a SR, even if they did not satisfy the aforementioned definition of a SR. In the case of updated SRs, the most recent SR was included. Two reviewers independently, in duplicate, screened the title and abstract of all retrieved records initially and then full text of potentially eligible studies against the eligibility criteria, using Rayyan web application. Conflicts between the reviewers in the inclusion of studies were resolved by discussion or by involving a third reviewer.

Data collection and analysis

Using a standardized, piloted data extraction form, two reviewers independently, in duplicate, extracted the following data from each included SR: publication date, number of authors, country of the corresponding author, information on participants, interventions, and outcomes. To assess the methodological and reporting quality, we extracted data on SR protocol registration/publication, Preferred Reporting Items for Systematic Reviews and Meta‐Analyses (PRISMA) or similar flow diagram, search strategy, language restrictions, number of reviewers involved in the process of study selection, data collection and risk of bias assessment, assessment of heterogeneity, subgroup/sensitivity/meta‐regression analyses, and assessment of publication bias. Conflicts between the reviewers in data extraction were resolved by discussion or by involving a third reviewer. Data were summarized and reported descriptively and graphically. Continuous variables were reported as mean or median and discrete variables as numbers along with percentages.

Risk of bias assessment of SRs published in high impact factor journals and utilization in hypertension guidelines

We performed risk of bias assessment using the ROBIS tool in SRs published between October 2007 to September 2017 (10 years) in the leading five high impact factor journals identified from the 2016 Journal Citation Reports by Thomson Reuters. These journals included the Lancet, the Journal of the American Medical Association, Lancet Oncology, the British Medical Journal, and Journal of American College of Cardiology. We used the ROBIS tool as its validity relaibility and applicability properties are similar to that of the AMSTAR tool, and it takes less time for scoring. The risk of bias was assessed in duplicate by two reviewers independently. We also assessed the proportion of these SRs cited for recommendations in the two major hypertension guidelines: 2017 American College of Cardiology [ACC]/American Heart Association [AHA] and 2018 European Society of Hypertension [ESH]/European Society of Cardiology [ESC]).

RESULTS

The initial search identified 11,119 records. After removing duplicates, the titles and abstracts of 9,556 records were screened, and of these, 6,761 irrelevant records were excluded. Remaining 2,795 records were assessed in full, and of these, 1,173 eligible SRs were included in the review. PRISMA flow diagram reporting the process of selection of studies is shown in Figure 1.

FIGURE 1

PRISMA flowchart of selection of studies

Trends in the publication of systematic reviews

The first two SRs of RCTs of AHTDs were published in 1985 by Collins et al. and Yusuf et al. and colleagues, , and since then, there has been a steady increase in the rate of publication, reaching over 100 SRs per year in 2013. The number of publications of SRs were 10, 35, and 116 in the year 1996, 2006, and 2016, respectively. Seventy‐seven percent of all the SRs were published over the decade between 2007 and 2017, with the highest number (n = 118) published in 2015.

Trends in publications of SRs, including the top 5 countries with the most number of SRs published based on the country of the corresponding author, are reported in Figure 2. Most SRs originated from the United States of America (USA; 301 [26%]). However, in the last five years (2012–2016), China (n = 137 [12%]) has published more SRs than the United States (n = 102 [9%]), and SRs from China accounted for 26% of the SRs produced globally.

FIGURE 2

Trends in publication of systematic reviews of RCTs of AHTDs‐all countries and top 5 countries with the highest number of publications

Included SRs were published in 381 journals. The top ten journals with most SRs published are reported in Figure S1. The Cochrane Database of Systematic Reviews published 119 SRs, accounting for 10% of all the included SRs.

Characteristics of the systematic reviews

A summary of the characteristics of included SRs is reported in Table 1. Of the 1,173 SRs included, 206 (18%) involved qualitative synthesis only (without meta‐analysis) and 967 (82%) involved qualitative and quantitative synthesis (SRs with meta‐analysis). Among SRs that involved quantitative synthesis, IPD meta‐analyses and network meta‐analyses were uncommon (n = 34, 3.5%, n = 50, 5%, respectively). On an average, there were five authors per SR. Thirty‐two (3%) SRs were published by a single author, and the highest number of authors was 24.

TABLE 1

Summary of characteristics of included systematic reviews (n = 1,173)

Parameter	Count
Review authors, median (IQR)	4 (3 to 6)
Databases searched, median (IQR)	3 (2 to 3)
RCTs included, median (IQR)	10 (6 to 20)
Registered or published protocol	168 (13.5%)
Included a PRISMA or a similar flow diagram	643 (54.8%)
Included English‐language articles only	386 (32.9%)
Reported full literature search strategy	324 (27.6)
Involved at least two reviewers in selecting studies	542 (46.1%)
Involved at least two reviewers in data extraction	614 (52%)
Performed risk of bias assessment	704 (60.1%)
Involved at least two reviewers in risk of bias assessment	355 (50.4%)
Included RCTs and observational studies	132 (11.25%)
Reported method for quantitative synthesis	873 (90.3%)a
Assessed heterogeneity (applicable for meta‐analysis only)	861 (82.4%) a
No or minimal heterogeneity, or addressed it using subgroup, sensitivity, or meta‐regression analysis	635 (63.6%) b
Assessed publication bias (applicable for meta‐analysis only)	393(40.7%) a
Reported method for handling missing data	139 (14.4%)a
Performed sex‐wise analysis	26 (2.7%) a
Performed race‐wise analysis	21 (1.8%) a

Values are number (%) unless reported otherwise.

Abbreviations: IQR, interquartile range; RCTs, randomized controlled trials.

Percentage calculated for SRs performing quantitative synthesis.

Percentage calculated for SRs performing quantitative synthesis, and reported heterogeneity.

MEDLINE/PubMed and/or EMBASE were the most commonly searched databases accounting for 95% (n = 1,120) of the SRs. Forty percent of SRs searched Cochrane Central Register of Controlled Trials (CENTRAL; n = 469), and 20% (n = 240) searched a single database. Of the studies that searched single database, MEDLINE/PubMed and/or EMBASE were the most commonly searched databases (92%; n = 223).

Included SRs had 22,966 citations of RCTs of AHTDs. When all these citations were imported to Google Scholar or PubMed, about 70% were identified as duplicates. However, in the remaining 30% there could still be multiple publications from the same trial. A median of 10 RCTs (interquartile range [IQR] 6 to 20) were included per SR, with the study by Fischer et al including the highest number of RCTs (n = 1,372).

Participant characteristics

Ninety percent (n = 1,053) of SRs specified participants’ baseline health condition. Among these, majority (32%) involved patients with hypertension, followed by heart failure (11%) and renal diseases (9%; Figure S2).

Antihypertensive drug classes and clinical outcomes studied

Approximately one‐quarter of the SRs studied AHTDs in general without specific focus on any class (n = 305 [26%]). Among the remaining, more SRs focused on angiotensin‐converting enzyme inhibitors (ACEIs) or angiotensin II receptor blockers (ARBs) compared to other classes (n = 277, 23%; Figure S3). Seventy (6%) SRs focused on combination of AHTDs. Two hundred and fifty (25%) SRs included placebo as the only comparator, 175 (15%) had active only comparator, and 501 (43%) had both placebo and active comparators. There were also SRs that compared AHTDs with non‐AHTDs (n = 129, [11%]) and non‐pharmacological therapies (n = 61, [5.2%]). In thirty three percent of the SRs (n = 388) cardiovascular disease events or mortality was the primary or main outcome of interest. (Figure S4), and in thirty percent of the SRs (n = 353) BP outcomes were either primary and/or secondary outcomes.

Source of funding

Funding information was not reported for 52% (n = 609) of the SRs. Of those that reported, 23% (n = 266) were not funded, and 10% and 6% were funded by government and non‐profit/charity, respectively (Figure S5).

Methodological and reporting quality of the systematic reviews

Fourteen percent (n = 168) of the SRs were prospectively registered or had a corresponding protocol published, and there were significantly higher rates of prospective registration after 2011 (47 [9.5%] vs 121 [17.7%]) when PROSPERO started registering protocols. Of the studies (n = 553 [47.2%]) that were reported as “systematic review” either in the title or in the abstract, 1.4% did not meet the definition of a SR. Twenty percent of the included SRs searched a single database, and 60% did not search CENTRAL. Seventy‐two percent of the SRs did not report a full search strategy, and 55% of the SRs reported a PRISMA or a similar diagram. There was significant improvement (26 [7%] vs 617 [76%]) in the reporting of PRISMA or a similar diagram after the publication of the PRISMA statement in 2009. In nearly half of the SRs, study selection (54%) and data collection (48%) were not performed in duplicate. Eleven percent of the SRs synthesized evidence from both randomized controlled trials and observational studies. Fifty‐seven percent of the SRs reported performing risk of bias assessment; however, only half of them reported performing it in duplicate. For quantitative synthesis of the data, majority (42%) of the SRs used both random‐ and fixed‐effect model, followed by random effects only (37.9%), fixed effect only (9.8%), and Bayesian method (0.01%), and 64.6% provided justification for the choice of the model. Ten percent of the SRs did not report any information on the model used. Information on dealing with the missing data was reported in 14.4% of the SRs, and 74% of these used imputation methods. Publication bias was assessed in 41% of the SRs. Significant improvement in the methodological aspects of the conduct of SRs was observed over the years, expect for the publication language restriction with most SRs still including English‐language publications only (Figure 3).

FIGURE 3

Systematic reviews of RCTs of AHTDs:publication trends based on aspects related to methdological quality

ROBIS assessment

The risk of bias assessment using the ROBIS tool was performed for 34 SRs that were published in the leading five high impact factor journals. Of these, five (15%) had unclear risk of bias, seven (21%) had low risk of bias, and 22 (65%) had high risk of bias. Risk of bias by individual SRs is reported in Table S2, and across the four domains of the SRs is presented in Figure 4.

FIGURE 4

Risk of bias in systematic reviews (n = 34) published in high impact factor journals, by domains of ROBIS tool

Domain 1: Eligibility criteria

Six [17.6%] SRs were rated as having high risk of bias for this domain. Eligibility criteria were clearly stated in all the SRs. RCTs with less than 300 participants were excluded in one SR without providing a justification. Only English‐language publications were included in five SRs.

Domain 2: Identification and selection of studies

Thirteen (38%) SRs had risk high risk of bias, and 10 (29%) had an unclear risk of bias for this domain. Eight SRs did not search the relevant databases (either searched a single database or did not search other relevant major databases). Two SRs did not report the name of the databases searched. All SRs performed secondary searches in addition to databases search to identify additional relevant RCTs. Fourteen (41.2%) SRs did not report a full strategy (including three that reported no information at all), and therefore, it was not possible to assess whether the search strategies were comprehensive enough to retriev all the eligible studies. Five (14.7%) SRs restricted searches to English‐language publication, and in 15 (44.1%) SRs, there was no information reported on the number of reviewers involved in study selection.

Domain 3: Data collection and study appraisal

For this domain, 13 (38%) and 8 (24%) SRs had unclear and high risk of bias, respectively. In 12 SRs, the number of reviewers involved in data extracted was not reported. In all included SRs, study characteristics and results were appropriately collected for use in the synthesis. In 7 (21%) SRs, the risk of bias assessment in included studies was not performed. One SR assessed quality using the Jadad scale, which does not assess allocation concealment. Among 25 SRs that assessed the risk of bias in the included studies, 12 (35%) did not report if the risk of bias assessment was performed in duplicate.

Domain 4: Synthesis and findings

Thirteen SRs (38%) were rated as high risk of bias for this domain. In all SRs, included primary studies were appropriately considered in the synthesis, statistical analysis plan was clearly stated, and meta‐analyses were appropriately conducted based on the type of data. Statistical heterogeneity was not adequately addressed in three (9%) SRs. In one SR, the robustness of the findings was neither demonstrated through publication bias assessment nor sensitivity analysis. In 11 (32%) SRs, the risk of bias in the included RCTs was high and was not addressed while synthesizing the findings.

Systematic review utilization in clinical practice guidelines

Utilization of SRs in the two recent major hypertension guidelines (2017 ACC/AHA‐2017, 2018 ESH/ESC) is reported in Table S3. Of the 34 SRs published in five leading high impact factor journals in the last 10 years, 8 (23%) SRs were cited in guidelines, of which all eight were cited in the 2017 ACC/AHA guidelines, and five of these (15%) were cited in the 2018 ESH‐ESC guidelines. One SR utilized by guidelines had low risk of bias, five had high risk of bias, and the remaining two had an unclear risk of bias. The characteristics that were common among these SRs were all eight studied AHTDs without focusing on any particular class and assessed either CVD or mortality outcomes. Corresponding authors of these SRs were from Australia (4), UK (3), and Sweden (1).

DISCUSSION

Summary of the main findings

This review found that over the last three decades, there has been a steady increase in the publication of SRs of RCTs of effects of AHTDs. Most SRs were published over the last decade, the current rate of publication is more than 100 per year, and 10% of all SRs were published in the Cochrane Database of Systematic Reviews.

Overall, most SRs originated from the United States followed by China; however, in the last five years, China has published more SRs than the United States. Most SRs involved adults with hypertension, also many SRs involved patients with non‐cardiovascular conditions. ACEIs/ARBs were the most commonly studied AHTD classes, and cardiovascular disease events were the commonly assessed outcomes.

There seems to be an improvement in the methodological and reporting quality of SRs over time. However, many SRs still have low methodological and/or reporting quality because of several reasons: lack of reporting of full search strategy, searching a single database and/or not searching CENTRAL, lack of reporting of PRISMA or a similar flow diagram, lack of information about the process of including studies, data extraction, lack of risk of bias assessment in included RCTs, lack of analysis to address heterogeneity, and lack of assessment of publication bias.

Despite the availability of SR protocol registration platforms (eg, PROSPERO since 2011) and journals (eg, Lancet, British Medical Journal, and PLOS) making registration of SRs a requirement for publishing SR results papers, only 1 in 5 SR protocols was registered or published in the last 5 years. Despite its importance in finding all relevant studies relevant to the SR objectives, less than half of the SRs searched CENTRAL, and 1 in 5 SRs searched only one database. Reporting of PRISMA or a similar flow diagram appears to have improved since the publication of PRISMA statement in 2009, yet more than one‐quarter of SRs still did not report such a flow diagram. For the accuracy of the data collected, it is important to have at least two authors involved in screening, data extraction, and risk of bias assessment. However, about half of the SRs did not report how these tasks were performed. Only fourteen percent of the meta‐analyses reported method of handling missing data. Evaluation of the risk of bias using ROBIS in 34 SRs published in the high impact factor journals revealed that there were methodological issues in 80% SRs.

The increase in the rate of publication of SRs in recent years may be attributed to the emergence of many consultancies and specialized organizations to conduct SRs, and the availability of free meta‐analyses software, like Review Manager, MetaXL, and R. Moreover, there has also been an increased demand for SRs as a result of increased recognition of their importance in informing clinical practice health care policies. Recently, Fontelo et al , reported that most publications of SRs originated from the United States, and Futamura and colleagues reported that 10% of SRs in the area of atopic eczema are Cochrane SRs. Both findings are consistent with the findings of the current study.

Strengths and limitations

To our knowledge, this is the first mapping review of SRs of RCTs of effects of AHTDs. We performed an extensive search for relevant literature and performed review steps following the Cochrane handbook of systematic reviews of interventions. We reviewed and screened supplementary files to obtain all relevant information. Despite these strengths, a few limitations should be noted. First, we did not include non–English‐language SRs because of a lack of resources to process translations, this may have underestimated the number of SRs published from China. However, China was still the leading country publishing highest number of SRs in the last five years. Second, we performed a formal assessment of the risk of bias, using the ROBIS tool, only in a sample of SRs, as it was not feasible to do so for all the SRs given the limited resources available to us. , Lastly, our assessment of SR protocol registration or publication was based on information available in the SRs results manuscript (PROSPERO registration number, citation of the published protocol); however, to confirm this, we did not check the PROSPERO register nor we made attempts to identify published protocols.

Implications for practice and research

Although journals are increasingly asking for registration or publication of the SR protocol for accepting SR results manuscripts, robust mechanisms to improve adherence to methodological and reporting standards are needed to improve the production and dissemination of SRs. Users of evidence synthesis should exercise caution while using evidence from SRs that did not adhere to methodological and reporting quality standards.

CONCLUSION

There has been increasing trend in the production of SRs of RCTs of AHTDs over the last few decades. Although there has been an improvement over the years, both methodological and reporting quality of most SRs continue to remain poor, even for those that are published in high impact factor journals. Frameworks that improve adherence to methodological and reporting standardards and that reduce redundancy in conducting SRs by building upon previous work to create “living systematic reviews” are likely be an efficient and effective approach to evidence synthesis to inform practice and policy.

CONFLICTS OF INTEREST

None.

AUTHORS CONTRIBUTION

Hariprasad Esam ‐ Literature search, screening, data collection, analysis, interpretation of data and writing & revising the manuscript. Raju Kanukula ‐ Literature search, screening, data collection, and revising the manuscript. Rupasvi Dhurjati, Aerram Rupa, and Sindhujareddy Chevireddy ‐ Screening, data collection, and revising the manuscript. Soumyadeep Bhaumik, Emily Atkins & Mark D. Huffman ‐ Analysis, and revising the manuscript. Anthony Rodgers & Abdul Salam ‐ Conception or design of the work, analyses, interpretation of data, writing & revising the manuscript.

Supplementary Material

Click here for additional data file.