Profile and quality of published reviews on COVID-19.

Dear Editor,

COVID‐19 has created the necessity to rapidly generate evidence to enlighten many blind spots encompassing the pandemic, from pathophysiology to management. Scientific journals have timely responded to this challenge by prioritizing COVID‐19 research, with proactive editorial efforts favouring open access to articles, launching calls for papers, implementing specific sections and special issues on COVID‐19, among others. However, the impact of all these measures on the overall quality and adequacy of research are largely unknown and several authors have shown concern in this regard. , ,

Reviews of the literature should provide comprehensive and summarized accounts of the evidence on a particular research topic, and therefore, studying their contents and quality might provide a good indication of the overall rigour of research on COVID‐19.

We searched PubMed for reviews addressing COVID‐19 published from 1 January to 8 April 2020 and screened them individually. The search strategy for initial identification of articles included the following expression: [“coronavirus” and “review”]. Exclusion criteria from the analysis were as follows: articles that did not address COVID‐19, articles that did not include a review of the literature, articles written in languages other than English, protocols, guidelines, consensus statements and expert opinion. Reporting of the study conforms to broad EQUATOR guidelines. The search flow diagram according to the PRISMA Statement is shown in Figure 1. Descriptive statistical analysis was carried out using SPSS software v.23.

FIGURE 1

PRISMA flow diagram. The initial search retrieved 382 articles, of which 143 were excluded due to the following reasons: 58 (40.6%) were not reviews, 41 (28.6%) were not written in English, 23 (16.1%) did not address COVID‐19, and 21 (14.7%) were guidelines, consensus statements or expert opinion on guidance

The characteristics of the 239 reviews analysed are shown in Table 1. These findings reflect the abundant scientific production triggered by the COVID‐19 pandemic. In the context of a rapidly spreading pandemic with dismal consequences, large amounts of observational studies were carried out in a very short period and this evidence was synthetized into reviews providing pathophysiological insight and guidance for diagnosis and management of COVID‐19. Within thirteen weeks, the international scientific community produced and published hundreds of freely accessible reviews of different approaches to rapidly cope with the theoretical and practical challenges posed by COVID‐19. Such a herculean effort at the global level, likely without precedent, is to be complemented.

TABLE 1

Characteristics of 239 review articles on COVID‐19 included in the analysis

Type of review, n (%) a
Systematic review with meta‐analysis	8 (3.3)
Systematic review without meta‐analysis	17 (7.1)
Narrative reviews	194 (81.2)
Scoping reviews	3 (1.3)
Other type of reviews	17 (7.1)
Search strategy described
Total	55 (23.0)
Narrative reviews (N = 194)	17 (8.8)
Systematic reviews (N = 25)
PRISMA Statement 5 followed	13 (52)
Protocol registered in PROSPERO 8	1 (6.8)
Cochrane Database Systematic Reviews	1 (6.8)
GRADE methodology 6 used in reviews addressing recommendations
N = 64	3 (1.3)
Open access, and Editorial and publication periods
Open access	233 (97.5)
Time elapsed between submission and acceptance (N = 164), median days (IQR)	6 (2‐13.8)
Time elapsed between acceptance and online publication (N = 168), median days (IQR)	5 (2.2‐9.8)
Explicit mention to International Committee of Medical Journal Editors criteria 4 fulfilment
Authorship criteria	80 (33.5)
Financial and nonfinancial disclosures	200 (83.7)
Corrections published online	3 (1.3)
Authors, nationality
Multinational	62 (25.9)
Country of corresponding author
China	82 (34.3)
United States	49 (20.5)
United Kingdom	15 (6.3)
Iran	13 (5.4)
Italy	12 (5.0)
India	11 (4.6)
Canada	7 (2.9)
Singapore	6 (2.5)
Germany	5 (2.1)
Other	39 (16.3)
Main topic of the review
Overview/General aspects	74 (31.0)
Epidemiology/infection control	24 (10.0)
Governance/policy/health services	11 (4.6)
Virology/Immunology/Pathophysiology	26 (10.9)
Diagnosis	19 (7.9)
Management	58 (24.3)
Other b	27 (11.3)
Review tailored for a particular specialty/subspecialty	55 (23.0)
Anaesthesiology	7 (12.7)
Gastroenterology and liver diseases	7 (12.7)
Paediatrics	4 (7.3)
Gynaecology and obstetrics	6 (10.9)
Cardiology	5 (9.1)
Critical care medicine	3 (5.5)
Emergency medicine	3 (5.5)
Other	20 (36.4)

In 11 (4.6%) studies the words “rapid” or “fast” appeared in the title.

Five studies (18.5%) addressed the efficacy a/or potential role of the following in the management of COVID‐19: herbal medicine (1), natural products (1), traditional Chinese (2) and Indian medicine (1).

On the other hand, our findings raise cause for concern. Firstly, the speed of acceptance (25% of articles were accepted within 2 days or less of submission) as well as the low correction rate makes it unlikely that scientific rigour was uniformly and thoroughly maintained during this period. Secondly, a generally poor methodological appraisal of reviews was found, as shown by the low proportion of studies providing the search strategy, the low number of systematic reviews and the low percentage of these following PRISMA and GRADE principles. These caveats reveal the difficulties to reach an adequate balance between urgent knowledge needs and scientific rigour, globally. Thirdly, publication speed seemingly entailed a very low rate of fulfilment of ICMJE criteria on authorship and few systematic studies including meta‐analysis and PROSPERO registration. Currently, over 500 hundred ongoing reviews have been registered, showing that a second wave of highest quality evidence is being generated after the first three months of “emergence publication mode.” It is worth noting that approximately quarter of publications were international enterprises. This is shocking provided that the COVID‐19 pandemic is a global threat requiring joint international initiatives.

In conclusion, during the early period of the COVID‐19 pandemic the scientific community, including journals, has rapidly generated a large amount of reviews on the increasing and necessary evidence produced to understand and tackle with the COVID‐19 pandemic. However, this has probably been achieved with the cost of lowering the quality threshold in many instances. Scholars and journal editors are called on to make a joint effort to transition to a high‐quality research‐reporting period regarding COVID‐19 at the global level.