Nonsignificant trends in COVID-19 trials: Is there a significance?

To date, hundreds of clinical trials have been conducted on dozens of already available drugs or natural/herbal substances to repurpose for the treatment of COVID‐19, but with no consistent statistically significant positive results.

Even, drugs such as hydroxychloroquine and Ivermectin that proved in some studies to be beneficial and thus included in many national guidelines for therapy, failed to keep their position after the release of negative results of further larger randomized comparative studies. This situation is still going on and a lot of studies with conflicting results are being published causing a lot of confusion.

Remdesvir, after granting the FDA approval as the first drug approved for COVID‐19, is facing such confusing situation, after the two major large multinational studies SOLIDARITY and DisCoVeRy concluded that there is no significant benefits.

In my viewpoint, I can argue that the inconsistent time to initiate antiviral drug therapy during the course of the disease is the major contributing factor for this dilemma.

Back to review basic science; it is already well‐defined that the natural history of symptomatic COVID‐19 starts by early rapid viral replication that reaches peak levels during the first week of symptoms. Then the virus levels steadily decline, independent of whether patients will recover spontaneously or progress to a severe or critical stage of infection. , Clinically, stage 1 of COVID‐19 usually spans the first 7 to 10 days of the illness. It is usually manifested as a febrile illness with upper respiratory and/or gastrointestinal symptoms that is mostly categorized in severity as mild (febrile symptoms with no evidence of pneumonia in computerized tomography scan [CT scan]). Then, a host inflammatory response phase may progress in up to 20% of infected individuals to be manifested by the appearance of pneumonia in CT scan and respiratory symptoms of varying severity from moderate to severe (pulmonary/stage 2). The host inflammatory response might become unpredictably exaggerated in a subset of these patients progressing to hyper‐inflammatory stage (stage 3) with cytokine storm and systemic inflammatory response syndrome. This hyper‐inflammatory stage is heralded with severe/critical clinical manifestations including the picture of acute respiratory distress syndrome (ARDS), multiorgan dysfunctions, and shock with a high fatality rate.

Hence, starting antiviral therapy during the early phase of rapid viral replication (first week of symptoms or stage 1 of disease) is the anticipated appropriate time to evaluate the antiviral efficacy. , However, most of the published clinical studies included patients to start the antiviral therapy during the second or even third stage of the disease (when the viral replication becomes no more important and the patient is under attack of his/her immune system).

Consequently, the hard endpoints especially the all‐cause mortality or the case‐fatality rate in most of the reported COVID19 comparative studies did not reach statistical significance.

From basic statistics and epidemiologic views the reason is that: to detect a difference in case fatality rate between groups included in the early phase of infection (or with mild disease severity) a very large sample size is needed (given the already low fatality rate in such population). Hence we often see nonsignificant trends.

On the other hand, in studies on late advanced or critical cases (when case fatality rate is high), antivirals become of low biological plausibility and no more lifesaving. Other heterogeneous uncontrollable factors such as the behavior of the genetically determined host inflammatory response, the appropriate use of anti‐inflammatory drugs, O2, ventilators, nursing, and other critical care interventions become more vital independent lifesaving variables at such late stages.

So, late advanced or critically severe cases (who are already under mechanical ventilation at baseline) would better be separated from inclusion with other subsets of disease severity when testing efficacy of antiviral drugs, as they can be regarded as a biologically different population than the other subsets of COVID‐19 patients. This could reduce the heterogeneity and statistical confusion.

In addition, to my mind, if all the studied efficacy endpoints in a single, or multiple similar studies with low heterogeneity, show concordant non‐statistically significant trends in favor of one direction (study intervention), concordance of these trends could be considered at least a signal against the mere chance as a causal factor. Significance of concordant trends could be even more indicative, when any of these endpoints reaches statistical significance, in‐addition to the sound biological reasoning.

To substantially demonstrate this viewpoint, I conducted a review from literature for the results of clinical studies on some of the antiviral drugs including remdesivir and some other antiviral drugs (in separate reports) with special focusing on the inclusion criteria, disease severity and the duration of symptoms before initiation of the antiviral drugs, all in relation to the outcome.

I used the data from COVID‐NMA website : https://covid-nma.com/metacovid/ as a starting point to select the relevant studies. COVID‐NMA is an international initiative working in conjunction with the World Health Organization (WHO), led by a team of researchers from Cochrane and other institutions.

I reviewed all the seven studies on remdesivir included in the meta‐analysis at COVID‐NMA website, by October 2021, for all‐cause mortality, then I downloaded the full‐texts and revised the data for errors or heterogeneity and conducted a simple meta‐analysis for the results of all‐cause mortality at Day 28.

Table 1 is downloaded from COVID‐NMA website. It shows that all the five studies that included patients in the early phase (or with mild disease) together with other levels of severity (or balanced subsets of severity) showed trends for lower all‐cause mortality at Day 28 by remdesivir. , , , , Whereas the other two studies that excluded this subset (early/mild cases) did not show these trends but even a reversed trend affecting the overall result of the meta‐analysis. ,

Table 1

All‐cause mortality analysis downloaded from COVID‐NMA website

Away from complicated statistical procedures, we simply excluded these two unbalanced studies from meta‐analysis in‐order to reduce heterogeneity and for their high risk of bias. We conducted a simple meta‐analysis for the risk of all‐cause mortality at Day 28 including all patients of the other five studies after excluding all data of the subset of patients who were under mechanical ventilation at baseline in the WHO SOLIDARITY trial from analysis as shown in Table 2.

Table 2

Meta‐analysis of all‐cause mortality excluding patients from SOLIDARITY study who were under mechanical ventilation at baseline

Studies	Remdesivir			Control			RR	95% CI	p
	Deaths (k1)	Total (n1)	%	Deaths (k2)	total (n2)	%
Spinner CD. 4	5	396	1.3	4	200	2.0	0.63	0.17–2.33	0.49
Ader F (DISCOVERY) 5	34	414	8.2	38	418	9.1	0.90	0.58−1.4	0.65
Pan H (SOLIDARITY) 6	203	2489	8.2	232	2475	9.4	0.87	0.73–1.04	0.13
Beigel JH (ACTT‐1) 7	59	541	10.9	77	521	14.8	0.74	0.537–1.013	0.06
Barratt D 8	1	43	2.3	3	58	5.2	0.45	0.048–4.175	0.48
Total	302	3883	7.8	354	3672	9.6	0.81	0.71–0.93	0.004

Abbreviation: CI, confidence interval.

This meta‐analysis showed that the relative risk of all‐cause mortality at Day 28 is 0.81 (95% confidence interval [CI]: 0.71–0.93). This means that remdesivir reduced the relative risk for all‐cause mortality at Day 28 by 19% with a high statistical significance p = 0.004.

Furthermore, after we removed the data of all patients who were under mechanical ventilation at baseline from the three major studies (SOLIDARITY, DISCOVERY, and ACTT‐1) and repeated the meta‐analysis, the relative risk of all‐cause mortality at Day 28 further decreased to 0.79 (95% CI: 0.678–0.929), with a highly significant relative risk reduction of 21% (p = 0.0039) (Table 3 and Figure 1).

Table 3

Meta‐analysis of all‐cause mortality excluding patients from the three major studies who were under mechanical ventilation at baseline

Studies	Remdesivir			Control			RR	95% CI	p
	Deaths (k1)	Total (n1)	%	Deaths (k2)	total (n2)	%
Spinner CD	5	396	1.3	4	200	2.0	0.63	0.17–2.33	0.49
Ader F (DISCOVERY)	24	339	7.1	25	344	7.3	0.97	0.57–1.67	0.92
Pan H (SOLIDARITY)	203	2489	8.2	232	2475	9.4	0.87	0.73–1.04	0.13
Beigel JH (ACTT‐1)	31	402	7.7	48	364	13.2	0.58	0.38–0.9	0.01
Barratt D	1	43	2.3	3	58	5.2	0.45	0.05–4.17	0.48
Total	264	3669	7.2	312	3441	9.1	0.79	0.68–0.93	0.0039

Abbreviation: CI, confidence interval; RR, Relative Risk.

Figure 1

Forest‐plot for all‐cause mortality at Day 28, excluding patients on mechanical ventilation at baseline

From this simple meta‐analysis together with data from all the seven studies collectively we can conclude that in patients admitted to hospital later than 10 days with severe/critical COVID‐19, remdesivir was not associated with statistically significant clinical benefits. However, there has been concordant numerical trends for reduction in all‐cause mortality and in time to clinical improvement in those treated earlier. Excluding patients on mechanical ventilation from meta‐analysis turned these trends to be statistically significant.

So, late advanced or critically severe cases who are already under mechanical ventilation are better considered a biologically separate population not likely to get benefit from remdesivir. They are better excluded from studies for antiviral efficacy or further studied as a separate population not a subgroup. However, I can argue that remdesivir could be reasonably prescribed for the treatment of other subsets of patients with COVID‐19, as it could reduce mortality and shorten the time to clinical recovery specially when given early after the start of illness.

CONFLICT OF INTERESTS

The authors declare that there are no conflict of interests.