Reporting and data sharing level for COVID-19 vaccine trials: A cross-sectional study.

BACKGROUND: The results and data availability of vaccine trials directly affect the decisions of healthcare providers, the public, and policymakers as to whether the vaccine should be applied. However, the reporting and data sharing level of COVID-19 vaccine studies are not clear.
METHODS: A cross-sectional study was conducted. A systematic search up to 9 May 2021 in 12 databases and an updated search to 6 July 2021 were conducted in the Cochrane Living Systematic Review and Network Meta-Analysis database to identify COVID-19 vaccine trials. The basic characteristics of included trials were summarized. The reporting level was assessed according to the CONSORT checklist. The data sharing level was assessed by open science practices. Types of incomplete reporting including protocol deviation, lack of primary outcomes clarity, and the omission of harms were analyzed.
FINDINGS: Finally, thirty-six COVID-19 vaccine articles reporting on 40 randomized controlled trials were included in this analysis. Based on the CONSORT checklist, the mean reporting score was 29.7 [95% confidence interval 28.7, 30.7]. Thirty-one articles (31/36, 86.1%) had data sharing statements, twenty-five articles (25/36, 69.4%) provided access to the source data. Twenty-seven articles (27/36, 75.0%) had protocol deviation, lack of primary outcomes clarity, or the omission of harms.
INTERPRETATION: The reporting and data sharing level of COVID-19 vaccine trials were not optimal. We hope that the reporting and data sharing of future trials will be improved. We recommend establishing a comprehensive, accurate data sharing system for future vaccine trials. FUNDING: This work was supported by the National Key R&D Program of China (2019YFC1710400; 2019YFC1710403).

Evidence before this study

The reporting and data sharing level of COVID-19 vaccine studies are not clear.

Added value of this study

We assessed the reporting level according to the CONSORT 2010 checklist and the data sharing level by open science practices. We found five problems in the reporting of COVID-19 randomized controlled trials (RCTs). First, the reporting of randomization items for 97.2% articles was not sufficient, which could lead to bias in the results. Second, for the inclusion and exclusion criteria, only 21.4% trials reported the infection risk of participants in trials that included vaccine efficacy as outcomes. Third, the definition of immunogenic outcomes was unclear in 21.4% trials. Fourth, the reporting of sample size calculation was deficient in 36.1% trials. Fifth, the outcomes and estimation were insufficiently reported. In 86.1% COVID-19 vaccine trials, the authors provided data sharing statements regarding their data management plans. Unfortunately, few trials made their data and associated analytical code available. 75% of articles had protocol deviations such as a change of time point, or the addition or ignoring of a primary outcome.

Implications of all the available evidence

The authors reporting COVID-19 vaccine trials should pay more attention to the shortcomings identified above to prevent the omission of critical points. The value of real data sharing for COVID-19 vaccine trials should be highlighted. The authors should provide access to their protocols and the raw data with analytical codes. We recommend establishing a real-time data sharing system for COVID-19 vaccine trials. We hope our study can help to improve the reporting and data sharing level of ongoing COVID-19 vaccine trials. Even more importantly, we hope our study can serve as the groundwork for establishing a comprehensive and accurate data sharing system for future vaccine trials.

Alt-text: Unlabelled box

Introduction

Globally, as of 7 February 2022, there have been 394,757,639 confirmed cases of Coronavirus Disease 2019 (COVID-19), including 5,738,898 deaths. For any infectious disease, a vaccine is the best means to safeguard public health, especially where no effective treatment is available. Due to the global pandemic, vaccines with long-term protection, high efficacy, acceptable safety profile, large-scale manufacturing, and widespread distribution are urgently needed. According to different targets and technologies, COVID-19 vaccines can be divided into the following types: inactivated vaccines, recombinant spike protein vaccines, viral vector vaccines, RNA vaccines, live attenuated vaccines, and virus-like particles vaccines.2, 3, 4 As of 7 February 2022, 115 COVID-19 vaccine trials were registered in the WHO International Clinical Trials Registry Platform. 335 vaccine candidates were under development, and 141 of them were in different phases of clinical trials.

A series of systematic reviews7, 8, 9, 10, 11 has been published summarizing the evidence of all published clinical trials that investigated the safety, tolerability, immunogenicity, and efficacy of vaccine candidates against COVID-19. The results of systematic reviews rely on the quality of the original trials, how these trials are reported, and how trial data are shared. The data from the original trials is the key information, necessary for the implementation of trial results in public policy. Because of the urgency of the pandemic, it is unlikely that different researchers will be able to conduct more trials with the same vaccine, thus providing comparative, systematic data. This makes available trial data even more important for all stakeholders, and will influence decisions, from policymakers to the public. Therefore, the quality of reporting and the transparent sharing of data are important.

To evaluate the reporting and data sharing status of COVID-19 vaccine trials, we assessed trial reporting level according to the CONSORT 2010 checklist and the data sharing level by open science practices.

Methods

A cross-sectional study was conducted. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) for cross-sectional checklist was followed.

Eligibility criteria

All RCTs that assessed the safety and/or efficacy of a COVID-19 vaccine were included with no restriction to study type (i.e, superiority, non-inferiority, or equivalence trials). Interim analyses, pooled analyses, and post-hoc analyses of RCTs were excluded. Non-human studies and studies not published as full papers (such as conference abstracts, protocols) were excluded. Preprint articles were excluded in the formal analysis, but the key results of these articles are provided in Supplementary Tables 7 and 8.

Search strategy

A systematic search was conducted to identify RCTs testing a COVID-19 vaccine in the following 7 databases: MEDLINE, Excerpta Medica Database (EMBASE), Cochrane Central Register of Controlled Trials (CENTRAL), Chinese Biomedical Literature Service System (CBM), China National Knowledge Infrastructure (CNKI), Wanfang Data and VIP Chinese Medical Journal Database. Five preprint databases, MedRixv, BioRxiv, Social Science Research Network (SSRN), Research Square, and Open Science Framework (OSF), were also searched. Duplicates were removed. The RCTs included in systematic reviews and meta-analyses which assessed the efficacy of COVID-19 RCTs were traced back; this was considered as a complementary search. There was no restriction to the language of publication. The search time was up to 9 May 2021. An updated search, to 6 July 2021, was conducted in the Cochrane Living Systematic Review and Network Meta-Analysis (COVID-NMA) database. The detailed search strategy is shown in Supplementary Table 1.

Data management and selection

Endnote X7.8 was used to screen the records. First, the titles and abstracts were screened by researchers (YD, LZ) independently based on inclusion and exclusion criteria. Then, the full texts were screened by researchers (YD, LZ) independently based on inclusion and exclusion criteria. Discrepancies were resolved through discussion or by asking senior researchers (ZB, DM).

Data collection and statistics

An information extraction table using Microsoft Excel 2010 was made. Before formal data collection, three researchers (YD, JL, and LZ) were trained. Two included studies were assessed using the extraction table as a pretest for consistency validation. Data was collected and coded by two researchers (JL, LZ), while a third researcher (YD) conducted a second check. Discrepancies were resolved through discussion or by asking two senior researchers (ZB, DM). The overall reporting scores were analyzed using descriptive statistics (mean and 95% CI). The category data was presented as a number (n) and percent (%). The detailed items were as follows:

1. Basic characteristics: title, authors, language, publication (year, month and the journal), the country and region where the study was carried out, recruitment dates, participants (targeted population, infection risk, sample size, sex and/or gender, race and/or ethnic, age), vaccine (platform, type, developer, number of doses, dose schedule), outcomes in terms of efficacy, immunogenicity, safety, and tolerability with time points (observation and follow up).

2. CONSORT checklist reporting scores

The reporting level of the included RCTs was assessed using the CONSORT 2010 checklist. Each item was scored in terms of two possibilities: “1” for “reported” or “not applicable”, “0” for “insufficiently reported” or “unreported”. A CONSORT total score for each trial was calculated. Each item score was summed to provide a total score. The specific rules for scoring each item/subitem were made according to CONSORT Explanation and Elaboration (E&E) document. The reporting of the eligibility criteria of participants and outcomes including efficacy, immunogenicity, safety, and tolerability were the specific focuses during the assessment process.

3. Open science practices

The data sharing level of COVID-19 vaccine RCTs was assessed by open science practices.

3.1 Whether there is registration information, including registration number, with accessible protocol? Whether there is a completion date in the registration?

3.2 Whether there is a data sharing statement, and whether the statement fulfils the ICMJE requirements?

3.2.1 Will individual participant data be available (including data dictionaries)?

3.2.2 What data will be shared?

3.2.3 What other documents will be available?

3.2.4 When will data be available (start and end dates)?

3.2.5 Whether there is a contact person for the data?

3.2.6 For what types of analyses?

3.2.7 By what mechanism will data and code/algorithm/software be made available?

3.2.8 Whether there is a direct link to the data (such as a data citation)?

4. Other incomplete reporting types

Other types of incomplete reporting including protocol deviation, lack of clarity in reporting primary outcomes, and the omission of harms were assessed by the following items. Each article was given a score of “1” if any incomplete reporting type was found. If not, the score was “0” for these items.

4.1 Was there inconsistency of primary outcomes between the protocol (including registration information) and the published article?

4.2 Were the primary outcomes specified?

4.3 Were safety outcomes omitted?

Results

A total of 779 articles were retrieved and screened according to the eligibility criteria. Of these, 36 COVID-19 vaccine articles16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51 and 40 RCTs in total were included. The flowchart of screening COVID-19 vaccine RCTs is shown in Figure 1.

Figure 1

Flow chart.

Basic characteristics

The earliest two COVID-19 vaccine RCTs were published on 20 July 2020, from the United Kingdom (UK) and China. 70% of the RCTs were conducted in three countries: China (14/40, 35.0%), the US (9/40, 22.5%), and the UK (5/40, 12.5%). The 36 articles were published in three high impact journals, namely New England Journal of Medicine (9/36, 25.0%; 2020 JIF = 91.245), The Lancet (8/36, 22.2%; 2020 JIF = 79.321), and The Lancet Infectious Diseases (8/36, 22.2%; 2020 JIF = 25.071). There were 9 RCTs in phase 1, 14 RCTs in phase 1/2, 7 RCTs in phase 2, 4 RCTs in phase 2/3, and 6 RCTs in phase 3.

The phase 3 data of the following types of COVID-19 vaccines were published: WIV04, HBO2, CoronaVac, mRNA-1273, BNT162b2, GAM-COVID-VAC, ChAdOx1 and Ad26.COV2.S. The basic characteristics of COVID-19 vaccine trials, including general information, vaccine-related information, and outcomes (efficacy, immunogenicity, safety, and tolerability) are shown in Supplementary Tables 2–4.

Reporting level and features

The mean CONSORT reporting score of COVID-19 vaccine articles was 29.7 [95%CI 28.7, 30.7]. However, the reporting of some items in Methods and Results showed the most deficiencies. Because these two sections are key to the reliability of vaccine trials’ results, we elaborated on both. For the reporting scores of other parts, please refer to Table 1.

Table 1

Reporting scores based on CONSORT checklist.

Section/Topic	Item No	Checklist item	Item Reporting n (%)
Title and abstract
	1a	Identification as a randomised trial in the title	26 (72.2)
	1b	Structured summary of trial design, methods, results, and conclusions (for specific guidance see CONSORT for abstracts)	30 (83.3)
Introduction
Background and objectives	2a	Scientific background and explanation of rationale	36 (100)
	2b	Specific objectives or hypotheses	36 (100)
Methods
Trial design	3a	Description of trial design (such as parallel, factorial) including allocation ratio	35 (97.2)
	3b	Important changes to methods after trial commencement (such as eligibility criteria), with reasons	35 (97.2)
Participants	4a	Eligibility criteria for participants	35 (97.2)
	4b	Settings and locations where the data were collected	24 (66.7)
Interventions	5	The interventions for each group with sufficient details to allow replication, including how and when they were actually administered	36 (100)
Outcomes	6a	Completely defined pre-specified primary and secondary outcome measures, including how and when they were assessed	22 (61.1)
	6b	Any changes to trial outcomes after the trial commenced, with reasons	15 (41.7)
Sample size	7a	How sample size was determined	23 (63.9)
	7b	When applicable, explanation of any interim analyses and stopping guidelines	9 (25.0)
Randomisation:
Sequence generation	8a	Method used to generate the random allocation sequence	26 (72.2)
	8b	Type of randomisation; details of any restriction (such as blocking and block size)	22 (61.1)
Allocation	9	Mechanism used to implement the random allocation sequence (such as sequentially numbered containers), describing any steps taken to conceal the sequence until interventions were assigned	15 (41.7)
Implementation	10	Who generated the random allocation sequence, who enrolled participants, and who assigned participants to interventions	1 (2.8)
Blinding	11a	If done, who was blinded after assignment to interventions (for example, participants, care providers, those assessing outcomes) and how	30 (83.3)
	11b	If relevant, description of the similarity of interventions	31 (86.1)
Statistical methods	12a	Statistical methods used to compare groups for primary and secondary outcomes	34 (94.4)
	12b	Methods for additional analyses, such as subgroup analyses and adjusted analyses	33 (91.7)
Results
Participant flow (a diagram is strongly recommended)	13a	For each group, the numbers of participants who were randomly assigned, received intended treatment, and were analysed for the primary outcome	34 (94.4)
	13b	For each group, losses and exclusions after randomisation, together with reasons	24 (66.7)
Recruitment	14a	Dates defining the periods of recruitment and follow-up	35 (97.2)
	14b	Why the trial ended or was stopped	36 (100)
Baseline data	15	A table showing baseline demographic and clinical characteristics for each group	36 (100)
Numbers analysed	16	For each group, number of participants (denominator) included in each analysis and whether the analysis was by original assigned groups	36 (100)
Outcomes and estimation	17a	For each primary and secondary outcome, results for each group, and the estimated effect size and its precision (such as 95% confidence interval)	14 (38.9)
	17b	For binary outcomes, presentation of both absolute and relative effect sizes is recommended	36 (100)
Ancillary analyses	18	Results of any other analyses performed, including subgroup analyses and adjusted analyses, distinguishing pre-specified from exploratory	36 (100)
Harms	19	All important harms or unintended effects in each group (for specific guidance see CONSORT for harms)	34 (94.4)
Discussion
Limitations	20	Trial limitations, addressing sources of potential bias, imprecision, and, if relevant, multiplicity of analyses	35 (97.2)
Generalisability	21	Generalisability (external validity, applicability) of the trial findings	34 (94.4)
Interpretation	22	Interpretation consistent with results, balancing benefits and harms, and considering other relevant evidence	36 (100)
Other information
Registration	23	Registration number and name of trial registry	36 (100)
Protocol	24	Where the full trial protocol can be accessed, if available	26 (72.2)
Funding	25	Sources of funding and other support (such as supply of drugs), role of funders	26 (72.2)

In the Methods, 35 articles (35/36, 97.2%) provided the eligibility criteria of participants. Twenty-seven trials (27/40, 67.5%) included a diverse population (3 or more races and/or ethnic groups). Most trials included participants between 18–65 years old; however, 16 trials (16/40, 40.0%) included elderly people above 65 years old, and 3 trials (3/40, 7.5%) included young people below 18 years old. Fourteen articles (14/36, 38.9%) measured vaccine efficacy; 3 articles (3/14, 21.4%) reported the infection risk of SARS-CoV-2 among the participants (item 4 participants).

For item 5, Interventions, all the articles sufficiently reported vaccine-related information, including the platform, type, developer, number of doses, route of administration, and dose schedule. There were 4 vaccine platforms, namely inactivated virus (13/40, 32.5% trials), viral vector (non-replicating, 12/40, 30.0% trials), RNA-based (7/40, 17.5%), and protein subunit (8/40, 20.0%) in the 40 trials. These vaccines were injected intramuscularly with 1-3 doses, over intervals ranging from 14 to 56 days.

For item 6, Outcomes, 16 articles (16/36, 44.4%) provided insufficient information on the primary outcomes. How the primary outcomes were assessed was not explained in ten articles (10/36, 27.8%). The timepoint of primary outcome assessment was not given in three articles (3/36, 8.3%). One article (1/36, 2.5%) did not indicate what the primary outcomes were. Among the 36 articles, 26 (26/36, 65.0%) inconsistencies were found between outcomes in the protocol and in the article. The time points were different for primary outcomes in the article and protocol for 9 (9/36, 25.0%) included articles; eight (8/36, 22.2%) lacked primary outcomes without explanation (not mentioned in the article); primary outcomes were added in the article without explanation (not mentioned in the protocol) in four articles (4/36, 11.1%); two (2/36, 5.6%) changed the primary and secondary endpoints. Among the articles whose outcomes were immunogenic (28/36, 77.8%), six (6/28, 21.4%) failed in defining the seroconversion/seropositivity.

For item 7, Sample size, 23 articles (23/36, 63.9%) reported the calculation basis of the sample size, and nine articles (9/36, 25.0%) reported the principle of interim analysis/abortion of the trial. Five trials’ sample size calculation was based on the statistical power. The sample sizes for other trials were not calculated based on statistical power but determined according to WHO and the clinical trial recommendation in various countries for COVID-19 or the number of available vaccines.

For items 8–10 concerning randomization, the reporting scores of item 8, Random sequence generation, reached 26.7 and 22.6 for items 8a and 8b, respectively. 15 articles (15/36, 41.7%) reported the mechanism and steps of allocation concealment (item 9). Only one article (1/36, 2.8%) reported personnel who generated the random allocation sequences, recruitment of subjects, and assigned interventions (item 10, Implementation).

For item 11, Blinding, 30 (30/36, 83.3%) articles reported who was blinded and how; 31 (31/36, 86.1%) reported the similarity of interventions among groups.

In the Results, all articles (14/14, 100.0%) which included efficacy outcomes reported the estimated effect size and its precision (95% CI) of vaccine efficacy. All articles that included immunogenic outcomes (36/36, 100%) established the hypothesis test for immunogenic outcomes and provided the p value for the comparison among groups; however, no article reported the estimated effect size and its precision (95% confidence interval). One article (1/36, 2.8%) established the hypothesis test and reported the estimated effect size and its precision (95% CI) of adverse events (AEs)/adverse reactions (ARs).

For item 19, Harms, 34 (34/36, 94.4%) articles reported AEs/ARs. Among these trials, local/systematic AEs/ARs and serious AEs were used as safety outcomes. Eight articles (8/34, 23.5%) did not specify whether the AEs/ARs were unsolicited or solicited. Two articles (2/36, 5.6%) did not mention harm.

Open science practices

The data sharing information for each trial is shown in Supplementary Table 5. All articles had registration information (including the registration number and place). Twenty-six articles (26/36, 72.2%) had an accessible protocol. Twenty-six articles (26/36, 72.2%) did not have the completion date in the registration. Thirty-one articles (31/36, 86.1%) had data sharing statements. Twenty-five articles (25/36, 69.4%) were available for individual participant data; four articles (4/36, 11.1%) did not provide individual participant data. Concerning what data would be shared, nine articles (9/36, 25.0%) did not provide the details; one article (1/36, 2.8%) indicated data would not be shared. Regarding document availability, 26 articles (26/36, 72.2%) provided the protocol, and/or statistical analysis plan, and/or informed consent form. For the start and end dates of data sharing, 24 articles (24/36, 66.7%) gave the date (one year/5 years/infinite). As to who can access the data, nine articles (9/36, 25.0%) did not provide details; 15 (15/36, 41.7%) did not specify the requesters; 11 (11/36, 30.6%) required researchers to provide a scientifically sound proposal; and two (2/36, 5.6%) specified that only qualified researchers could have access. For specifying the types of analyses, 21 articles (21/36, 58.3%) required the analyses should fulfill the research proposal/purpose/academic research. Twenty-five articles (25/36, 69.4%) described the specific access mechanism. Nineteen articles (19/36, 52.8%) indicated that an email contact was necessary, 15 (15/36, 41.7%) mentioned the need to sign a data access agreement, eight (8/36, 22.2%) mentioned a secure online platform, five (5/36, 13.9%) specified that the request should be sent through a specific website. For the availability of the codes, only two articles (2/36, 5.6%), published in NatureMedicine had the intention to share by sending request to emails. None of the articles provided a direct link, such as a dataset citation. Examples to show the types of sharing of data and codes are shown in Table 2.

Table 2

Examples of data availability.

Types of sharing the data	Examples of data availability
Actual sharing the data with associated analytical codes	The study protocol, de-identified individual participants data, statistical analysis plan will be shared. The researchers who submit a sound proposal will be reviewed after signing a data access agreement. The data was analyzed by software ***. The data with all analytical codes are available at https://doi.org/10.25504/FAIRsharing***.
Intent to share the data with a possible access mechanism	The data is available by sending request to the corresponding author's email. The request will be reviewed by the sponsor and principal investigator. The de-identified individual participants data will be shared.
Intent to share the data without an access mechanism	The data is available after the trial is complete.
Refuse the share the data	The data collected for their study will not be made available to others.

Other incomplete reporting types

Twenty-seven articles (27/36, 75.0%) had selective reporting, lacked primary outcome clarity, or failed to report harms. The results are summarized in the “reporting level and features” part (items 6b, 6a, and 19 respectively). The details of these incomplete reporting types are shown in Supplementary Table 6.

Discussion

Our study explored the reporting level, data sharing, and other incomplete reporting types in COVID-19 vaccine trials. Based on our study, we found that the reporting and data sharing levels of COVID-19 vaccine trials were relatively better than the previous assessment of other studies,52, 53, 54 which promoted the quick publication of COVID-19 vaccine trials. However, even during such a pandemic, the level of reporting and data sharing was not satisfactory.

The reporting of randomization items for 97.2% included RCTs was not sufficient. For example, how the allocation concealment was implemented, and by whom, was not reported in 58.3% and 97.2% included RCTs, respectively. Therefore, bias in the results of these trials cannot be ruled out. The authors of vaccine trials should strictly follow the reporting of randomization, especially for the implementation of randomization. The efficacy, immunogenicity, safety, and tolerability of a vaccine are four major outcomes. Evaluation of vaccine efficacy must be based on analysing the infection risk of participants. However, 78.6% trials did not report the infection risk of participants Furthermore, it is difficult to directly promote vaccine applications in countries and regions of different infection risks. As for immunogenicity, seroconversion rate is a key endpoint that can reflect how many subjects will have sufficient antibody response after vaccination. But only some of the vaccine trials defined the seroconversion or seropositivity. Without that definition, the immunogenicity of a vaccine cannot be estimated. Safety and tolerability are critical endpoints related to public health. However, 8 (8/36, 22.2%) trials did not indicate whether the safety outcomes such as AEs and SAEs were solicited. Without sufficient reporting, safety risks cannot be fully assessed. Moreover, a sufficient follow-up time is necessary to evaluate the efficacy, immunogenicity, and safety of a vaccine.

Publishing a study protocol is needed for knowing the transparency and assessing reporting biases of a trial. The lack of protocol availability prevents full assessment of outcome reporting bias and other reporting biases. However, 72.2% trials did not have completion dates in the registration, which can lead to publication bias. In 86.1% trials, the authors provided data sharing statements regarding their data management plans. Unfortunately, none of these trials made their data and associated analytical code publicly available. In 66.7% trial reports, the authors indicated that interested persons could email them, and the authors would consider the merits of the request. However, this is not data sharing as practiced in the open science ecosystem. Ideally, authors provide an access link to their protocols and the raw data with analytical codes. It is recommended that datasets be deposited in repositories, and relevant information included as formal citations in the article reference list. This includes datasets generated during the study as well as existing datasets analyzed during the study. At a minimum, citations of datasets should include author(s), title, publisher (repository name), and identifier.

Seventy-five percent of trials deviated from their protocols; for example, time points of a primary outcome changed, or there was the addition or ignoring of a primary outcome. It can lead to substantial bias in the results of these trials if the deviation is not fully reported. Extenuating circumstances like COVID-19 can trigger unplanned changes to RCTs and introduce methodological, ethical, feasibility, and analytical challenges that can potentially compromise the validity of findings. The CONSERVE (CONSORT and SPIRIT Extension for RCTs Revised in Extenuating Circumstances) statement provides a guideline for reporting these unplanned changes.

Among 16 articles59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74 in the preprint stage at the time of analysis, seven68, 69, 70, 71, 72, 73, 74 have been published.,,,,,, It was found that the reporting and data sharing level of these vaccine trials were improved and that some incomplete reporting practices were corrected after formal publication (Supplementary Table 7). This phenomenon affirms the principle that peer review can improve the quality of manuscripts and make conclusions more reliable. Therefore, the results of preprints that have not yet been peer-reviewed should be used with caution, and the journal should open a green channel for COVID-19 (or other pandemic related) vaccine manuscripts to speed up the peer-review process.

Our study had limitations. Some items in the three evaluation tools have different but overlapping meanings, which can be integrated into a consolidated evaluation system. For instance, (i) the registration information in “open science practices” is also assessed in the CONSORT item 23 registration and the item 24 protocol; (ii) protocol deviation for primary outcomes, whether the primary outcomes are specified in “other incomplete reporting types” are also assessed in the CONSORT item 6 outcomes; (iii) omission of harms is also assessed in the CONSORT item 19 harms.

To overcome the shortcomings in reporting of existing COVID-19 trials, we believe that the CONSORT statement and the CONSERVE guide should be used to disclose information about the process and results while giving due attention to the shortcomings identified above to prevent omissions. In addition, the value of data sharing for COVID-19 vaccine trials should be highlighted. We recommend establishing a real-time data sharing system for COVID-19 vaccine trials. In this system, we envision that the principal investigator of each vaccine trial would be required to release the results of the vaccine trials and upload them to the system upon completion. Researchers from different countries would be able to replicate, analyze, and evaluate their results. When any citizen in the world wants to know the efficacy and safety of the vaccine, he or she could directly access the latest clinical trials and clinical trial evaluation results related to the vaccine by simply registering, and the information could be read in any language and easily understood by the public.

Conclusion

The reporting and data sharing level were not optimal. We hope our study can help to improve the reporting and data sharing level of ongoing COVID-19 vaccine trials. We also hope that this study can serve as the groundwork for establishing a comprehensive and accurate data sharing system for future vaccine trials.

Ethics

Not applicable.

Role of funding source

This work was supported by the National Key R&D Program of China (2019YFC1710400; 2019YFC1710403). None of the sponsors had a role in any aspect of this study, including design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, approval of the manuscript, or decision to publish.

Contributors

ZB and DM led the supervision of the study and contributed equally to this work. ZB and DM are the guarantors. ZB and DM conceptualised the study. ZB, DM and YD developed the search strategies and made the assessment standards. YD, JL and LZ screened, extracted, and analysed data. ZB, DM, YD, and JL drafted the manuscript. XZ and JM provided critical comments to the manuscript. All authors provided detailed comments on earlier drafts and approved the final manuscript. All authors had full access to all the data in the study and accept responsibility to submit for publication. ZB, DM, and YD verified the underlying data. The corresponding authors attest that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

Data sharing statement

The raw data has been submitted to the OSF platform and can be accessed at https://osf.io/cvmzx/. The lead authors (the manuscript's guarantors) affirm that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as originally planned (and, if relevant, registered) have been explained.

Declaration of interests

All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare: DM is chair of EQUATOR Network and led the initial development of CONSORT reporting guideline. ZB is Director of the Chinese EQUATOR Centre. YD, JL, LZ and XZ are the academic staff in the Chinese EQUATOR Centre. JM is a clinical specialist working at The Chinese University of Hong Kong. All authors consider incomplete research reporting to be a common and important problem and believe reporting guidelines and checklists to be useful tools.