A scientificity and feasibility evaluation of COVID-19 clinical studies registered in China.

BACKGROUND: An outbreak of novel coronavirus infection in Wuhan, China, in early 2020 has now developed into a worldwide pandemic. Researchers in China and around the world have conducted many clinical studies on the scientific response to infectious diseases. Here, we review and summarize the registration protocols for clinical research of the novel coronavirus disease (COVID-19).
METHODS: We searched all the registered studies in all platforms under the International Clinical Trials Registry Platform (ICTRP) of the World Health Organization (WHO) before March 11, 2020, screened the registration scheme for novel coronavirus, extracted the basic research information, research quality, feasibility information, and described the scientificity and feasibility of interventional research.
RESULTS: From January 23, 2020 to March 11, 2020, 379 clinical studies were registered, 260 were therapeutic studies among them, and 96.8% studies were conducted in China by Chinese researchers. Some studies were registered 2 weeks just after the virus was identified, and up to 22 studies registered for one day. The number of interventional studies was greater than observational studies (263 vs. 116). Among the interventional studies, nearly 50% studies were funded, 87.7% were randomized, and 79.4% of the primary outcome indicators were objective. However, the sample size of the studies ranged from 60 to 200, with the total sample size accounting for 40% of the confirmed infected population. More than 60% studies might face the problem of insufficient sample size.
CONCLUSIONS: COVID-19 clinical study registration has been considerable, rapid, and high quality in study design. However, the feasibility of these studies may still be problematic, especially in such as insufficient sample size, poor coordination among multidisciplinary teams, and weak quality control in the research process. 2020 Annals of Translational Medicine. All rights reserved.

Introduction

On December 29, 2019, a severe pneumonia cluster case with unknown cause was reported in a hospital in Wuhan, and the Chinese government notified the World Health Organization (WHO), after verification (1). On January 7, 2020, Chinese researchers isolated the virus from the patient’s lower respiratory tract, the pathogen of the outbreak was confirmed as a novel coronavirus (2019-nCoV) and named coronavirus disease 2019 (COVID-19) by the WHO on February 12, 2020 (2). The epidemic started in China and spread into a large-scale outbreak across other parts of the world. On March 11, 2020, the WHO declared the outbreak to be a global pandemic (3). To date, 638,146 COVID-19 cases and 30,105 deaths have been confirmed globally (4).

As an emerging disease, COVID-19 is still in the early stages being understood, and there are a series of questions to be answered in terms of diagnosis, treatment, and prognosis assessment. The Chinese government has published seven versions of the treatment plan to fight the epidemic, which has become an essential part of the Chinese initiative against COVID-19 (5). However, the methods to improve the treatment plan, further improve efficacy, and reduce the case fatality rate require more scientific evidence. Scientists from all over the world have conducted or are conducting in-depth scientific research on epidemic prevention and control. On March 13, 2020, the WHO announced a total of 1,584 academic papers had been published globally on COVID-19, including narrative reviews or opinions (814 articles), case studies/series (213 articles), and epidemiological studies (124 articles) (6). As the virus first outbreak in China, Chinese scientists have been able to carry out many clinical studies due to having access sooner to a higher number of cases. Some registered clinical studies have reported on drugs and other treatments concerning COVID-19 (7,8). However, whether the research protocols of these studies are scientific and feasible has not yet been examined. Therefore, our research group used the registration data of the clinical research registration platform to collate and analyze the registered research protocols, evaluate these research protocols concerning interventional research, and make suggestions for the design and smooth implementation of clinical research.

Methods

Data sources and retrieval

On March 11, 2020, we performed a search on the International Clinical Trials Registry Platform (ICTRP) of the WHO and clinicaltrials.gov database, with keywords including “Covid-19”, “COVID,” “2019-novel Corona Virus (2019-ncov)”, “Novel Coronavirus Pneumonia (NCP)”, “Severe Acute Respiratory Infection (SARI)”, “Severe Acute Respiratory Syndrome-Corona Virus-2 (sars-cov-2)”, “nCoV”, “Coronavirus”, and “Severe Acute Respiratory Syndrome (SARS)”.

Inclusion and exclusion criteria

To filter the articles retrieved, we set the inclusion criteria to be all registered studies of the novel Corona Virus for 2019, while the exclusion criteria included duplicated studies, canceled studies, non-population-based studies, and studies on other viruses.

Research screening and data extraction

Research screening and data extraction were performed by four researchers in the public health profession. All the researchers involved in the research, screening and data extraction were trained and began extracting the data after completing the training. The extracted data included general information (research type, registration time, primary sponsor, centers of research implementation, fund support, ethical status) and study design (grouping, sample size, randomization, blind, main efficacy indicators, etc.). If there were questions concerning the data extraction, these would be discussed with senior researchers.

Statistical analysis

SPSS 24.0 (IBM Corp., NY, USA) was used for all statistical analyses. Categorical variables were summarized as the counts (percentages). Continuous variables were expressed as medians (25th percentile, 75th percentile) with the abnormal distribution. Since there were no statistical inferences in this paper, no statistical tests were performed.

Results

Basic registration information

As of March 11, 2020, 330 records were obtained from the ICTRP, and 145 records were obtained from the ClinicalTrials.gov website. After merging the two datasets, 19 duplicate records were excluded according to their registration ID, 56 records of non-novel coronavirus researches were excluded according to the title and research purpose, 4 records were excluded according to non-population-based studies, and 17 (4.3%) records were excluded as canceled studies (). A total of 379 records were included in this study (). Therapeutic research accounted for 68.6% of patients, and 367 (96.8%) registered studies being conducted by research leaders in China. As of March 11, 2020, only 2 (0.5%) studies have been completed.

Figure 1

The flow chart of registered studies retrieval and screening.

Table 1

Basic information for all registered studies

Factors	Count	Percent (%)
Interventional study
Yes	263	69.4
No	116	30.6
Type of study
Preventive investigation	58	15.3
Etiologic study	9	2.4
Therapeutic study	260	68.6
Diagnostic study	33	8.7
Prognoses study	19	5.0
Nation
China	367	96.8
Other	12	3.2
Status
Recruiting	203	53.6
Not recruiting	174	45.9
Completed	2	0.5

Distribution of registration number over time

The first clinical study was registered online on January 23 with a randomized controlled trial (RCT) design. This was only 2 weeks after the coronavirus was identified, and 554 cases were confirmed in China that day. Chinese research focused on clinical research fairly earlier and moved rapidly, considering the need or registration review, research program formulation, and ethical issues. The days with the highest number of studies registered were February 18 and 19, 2020 (both with 22 studies registered). The specific registration date distribution is shown in . The trend of interventional and observational studies is the same, but the number of interventional studies is greater than observational studies (263 vs. 116).

Figure 2

The number of registered COVID-19 studies in the International Clinical Trials Registry.

Basic information on interventional research in China

Since interventional studies need to be registered while observational studies do not, the following analysis only selected the interventional studies conducted in China, with a total of 257 studies included ().

Table 2

Basic information for interventional study registered in China

Factors	Count	Percent (%)
Type of study
Preventive investigation	14	5.4
Therapeutic study	3	1.2
Diagnostic study	234	91.1
Prognoses study	6	2.3
Ethical review
Yes	82	79.6
No	21	20.4
Fund
Longitudinal fund	87	33.9
Pharmaceutical company	27	10.5
Self-finance	89	34.6
Unspecified	54	21.0
Type of subject
Not patient	17	6.6
All patient	149	58.0
Common patient	42	16.3
Severe patient	49	19.1
Group number
1	28	11.0
2	179	70.2
3	33	12.9
4 or more	15	5.9
Random allocation
Yes	199	87.7
No	28	12.3
Blinding
Yes	35	13.7
No	220	86.3
Number of primary outcome indicators
1	126	49.4
2	40	15.7
3 or more	89	34.9
Objectivity of primary outcome indicators
Yes	201	79.4
No	26	10.3
Unable to determine	26	10.3
Number of implementation centers
Single	163	65.7
Multi-center	85	34.3
Institution of primary sponsor admit patient
Yes	110	76.9
No	33	23.1
Sample size
Median [p25, p75]	100 [60, 200]
Min-Max	9–2,000
Total	32,333

Missing values were not included in the analysis, the missing values were ethical review [154], group number [2], random allocation [30], blinding [2], number of primary outcome indicators [2], objectivity of primary outcome indicators [4], number of implementation centers [9], institution of primary sponsor admit patient [114].

Therapeutic research accounted for 91.1% of cases. Among the available registered research, 79.6% of the studies had been given ethical approval, and national or local longitudinal funds supported. Pharmaceutical companies have also provided greater support in the research. Nearly 60% of the studies included all types of patients, only 16% included common cases, and nearly 20% included severe cases only. The subjects were divided into two groups in 70% of the studies, while there were 15 studies with more than 4 groups.

Scientificity of the study protocol

In nearly 90% of studies with two groups or more, the subjects were randomly assigned. Nearly 15% of all interventional studies used blinding in the study design. For the outcome indicators, 1–2 primary outcomes indicators are generally recommended, but 35% of the studies have more than 2 indicators. The study with the highest number of indicators included 37 primary outcomes indicators, and 11 studies had more than 10 primary outcomes indicators. The primary indicators were objective indicators in 80% of the studies.

Feasibility of the study protocol

The institution of primary sponsor does not admit cases in nearly 25% of the studies, so there is a question to be answered whether a principal investigator can manage and control the unit of data collection well. Concerning the study sample size, half of the research was conducted with sample sizes of between 60 and 200 individuals. We could not determine the calculation method of the sample size, but sample sizes in RCT studies deserve careful consideration as the results may not meet statistical requirements. Based on the existing literature data, it is assumed that a total of at least 422 subjects would be needed for the outcome of mortality, and a total of at least 128 subjects would be needed for the outcome of the severe cases proportions (). Only 14 (5.4%) and 92 (35.8%) studies met the sample size requirement on mortality and severe case, respectively, and thus most studies did not meet sample size requirements. We calculated cases needed for the studies, and the total was 32,333 cases, which accounts for about 40% of all confirmed cases in China.

Table 3

Simulated sample size calculation based on different outcomes

Outcomes	Control group	Experimental group	Ratio of sample size in two groups	Sample size of two groups
Mortality	5%	1%	1:1	422
	5%	2%	1:1	804
Proportion of severe cases	13%	2%	1:1	128
	13%	5%	1:1	278
	13%	8%	1:1	754

Assuming alpha =0.05 and beta =0.2 with double side Two Independent Proportions Power Analysis (Fisher’s exact test) test. At the same time, based on the existing researches, the outcomes of the control group and the experiment group were assumed, and the sample size estimation was simulated.

Discussion

When an unknown disease occurs, how our researchers respond is an important question. In response to the COVID-19 epidemic, scientists worldwide are working diligently to explore and help prevent and control the epidemic scientifically. At present, more than 2,800 academic papers have been published worldwide (6). We have conducted a comprehensive review and summary of the research registration scheme for this disease, laying the foundation for a scientific response. Our researchers responded quickly to this disease, starting the research at an early stage of the disease, and registered studies have also been vast, which shows the positive scientific attitude of the researchers in dealing with the disease. Simultaneously, administrative and commercial companies have also deployed various funds to encourage researchers to carry out scientific exploration, with half of the research conducted having been supported by these funds.

What research should be prioritized when a new disease occurs? Some researchers have pointed out that most important research questions should concern virus transmission, asymptomatic and presymptomatic virus shedding, diagnosis, treatment, vaccine development, the origin of the virus, and viral pathogenesis (9). Among the types of research registered, therapeutic research, preventive investigation, and diagnostic research are the three most common types. Therapeutic research accounts for nearly 70% of these studies, indicating that our greatest need is the treatment of infectious diseases. In the case of a surge in the number of infected people, it is imperative to find the most appropriate treatment for prevention and control. Other preventive investigation studies can demonstrate the infectious characteristics of diseases utilizing infectious disease investigation and provide a basis for formulating prevention and control strategies to cut off transmission routes. As nucleic acid detection and diagnosis methods had been developed before this outbreak (10), there are not many diagnostic studies, but even with the rapid diagnosis and accurate classification of patients, many problems still need to be solved.

To date, 89% of the research has been designed with a parallel comparison. Among these parallel comparison study designs, 88% adopted randomization, 14% adopted blinding, and 80% used objective indicators. Thus, it can be stated the clinical studies have been conducted scientifically, and have used methods and technical means to ensure reproducible results and minimization of bias. Meanwhile, some RCTs have obtained important results (11,12). Cao and colleagues’ trial showed no benefit was observed with lopinavir-ritonavir treatment beyond standard care in severe COVID-19 patients (11).

In terms of feasibility, the protocol of these studies shows that there may be major deficiencies. Most researchers from Beijing and Shanghai led some studies, but they did not have enough patients to study in their cities. If the main research units do not participate in the collection of cases, brings significant challenges to the management and control of the research implementation process (13). Second, the sample size estimation method of these studies has not been made available, but, for the case fatality rate of the disease and the proportion of severe to mild cases, at least 70% of the studies may not meet sample size requirements. The total sample size has reached 40% of the cases confirmed in China. Most of the studies have not been completed, or the collection of data has not been started, which may result in subsequent studies being incomplete. Therefore, we suggest that in the face of major emerging infectious diseases, our research management department should actively manage studies, select scientific and feasible programs, and focus on quicker research, to avoid duplication and wasting research efforts on unnecessary interventions. Also, we can select research that is of adequate quality and which has a satisfactory preliminary result, to carry out international multicenter studies, and continue to optimize the efficiency of research to avoid the waste of experimental resources.

It is also important to note all new clinical trials in which enrollment began on or after January 1, 2019, which must include a data sharing statement in the trial’s publicly posted registration (14). In response to the COVID-19 epidemic, rapid data sharing is the basis for more effective public health action. Efforts for expedited data and results reporting should not be limited to clinical trials but should include observational studies and operational research (15). Indeed, the timely dissemination of scientific information is an effective way to reduce public panic about COVID-19 (16).

Some limitations of this study should also be addressed. Firstly, although clinical research registration has been implemented for many years, some clinical investigators may not register all studies on the Internet, and the collection of our data might thus have been biased to some extent. Secondly, certain registration information, especially related to the sample size calculations and randomization methods, is incomplete or not required in the registry; therefore, our research quality and feasibility evaluation might have been subject to certain restrictions. This also suggests we should strengthen the supervision and quality control of future clinical trial registration, avoid considering low-quality studies with insufficient information for registration, and improve the quality of clinical research registration platforms.

Conclusions

Registrations for clinical studies on COVID-19 has been substantial, the response of researchers has been fast, and the study design quality has been high. However, there are also shortcomings in general, such as insufficient sample size, poor coordination among multidisciplinary teams, and weak quality control in the research process.

The article’s supplementary files as