Evaluating Clinical Efficacy of Antiviral Therapy for COVID-19: A Surrogate Endpoint Approach.

INTRODUCTION: Efficient evaluation with an early surrogate endpoint, taking into account the process of disease evolution, may not only clarify inconsistent or underpowered results but also provide a new insight into the exploration of a new antiviral therapy for treating COVID-19 patients.
METHODS: We assessed the dynamics of COVID-19 disease spectrum, commencing from low-risk (no or low oxygen supplement), medium-risk (non-invasive ventilator or high oxygen supplement), and high-risk (extracorporeal membrane oxygenation or invasive ventilator) risk state on enrollment, and then the subsequent progression and regression of risk states until discharge or death. The efficacy of antiviral therapy in altering the dynamics was assessed by using the high-risk state as a surrogate endpoint based on the data retrieved from the two-arm Adaptive Covid-19 Treatment Trial.
RESULTS: Using the high-risk state as a surrogate endpoint, remdesivir treatment led to a decrease in the high-risk COVID-19 state by 34.8% (95% CI 26.7-42.0%) for a 14-day period and 29.3% (95% CI 28.8-29.8%) up to 28 days, which were consistent with a statistically significant reduction of death by 30.5% (95% CI 6.6, 50.9%) up to a 28-day period. The estimates of numbers needed to be treated were 100.9 (95% CI 88.1, 115.7) for using the high-risk COVID-19 state as a surrogate endpoint for a 14-day period and 133.3 (95% CI 112.5, 158.0) were required for averting one death from COVID-19 up to 28 days.
CONCLUSIONS: We demonstrate the expedient use of the high-risk COVID-19 disease status as a surrogate endpoint for evaluating the primary outcome of the earliest death.

Key Summary Points

Digital Features

This article is published with digital features, including a summary slide, to facilitate understanding of the article. To view digital features for this article go to https://doi.org/10.6084/m9.figshare.14170898.

Introduction

As of the end of December 2020, the coronavirus disease 2019 (COVID-19) pandemic has claimed up to 1.7 million lives among more than 78 million confirmed cases worldwide [1]. The soaring numbers of COVID-19 patients encountered in a series of outbreaks and resurgences has challenged the capacity of medical care systems and hampered the provision of optimal critical care services [2-4]. The prolonged length of stay for COVID-19 patients requiring hospitalization and intensive care before recovery and discharge further aggravates this situation [5].

While non-pharmaceutical interventions remain the mainstay for current containment strategies for COVID-19 [6], effective antiviral agents play another crucial role in coping with this stringent situation [7, 8], due to their potential for not only reducing the risk of disease progression to the critical status and death but also accelerating the rate of recovery and discharge [9]. Among the proposed antiviral therapies, a series of case reports and observational studies have demonstrated remdesivir as a promising compound [10-13]. To guide the clinical use of remdesivir following the principle of evidence-based medicine, four randomized controlled trials (RCTs) have been conducted [14-17]. However, the evidence provided by the RCTs remains inconclusive. The RCT conducted in China reported promising but non-significant results due to the early termination and underpowered study [15]. Another RCT evaluated the clinical outcomes of COVID-19 patients by comparing the remdesivir regimen of 5 and 10 days [16]. A lack of a control arm results in limited implication for the efficacy based on this clinical trial. A study focusing on treating moderate COVID-19 cases, including a standard care group as a comparator, to assess the clinical efficacy for 5- and 10-day regimens [17]. However, the inconsistent results for the outcomes of death, clinical improvement, and recovery between the two regimens have confused the clinical interpretations [18]. In a recent final report of the Adaptive Covid-19 Treatment Trial (ACTT-1), a significant benefit in increasing the odds of recovery and discharge by 29% [95% confidence interval (CI) 12–49%] was reported [19]. However, a 27% reduction of mortality with a marginally statistically significant result [hazard ratio (HR) = 0.73, 95% CI 0.52–1.03] was also presented by the trial [19].

The current clinical trials of antiviral therapies have focused on the evaluation of clinical efficacy by using the outcomes of either discharge or mortality. Although the two outcomes are the goal of main interest in assessing the clinical benefit of antiviral therapy, the uncertainty involved with the dynamics of COVID-19 evolution during the treatment course renders the evaluation inefficient and requires sample sizes more than expected to derive the results with statistical significance. The disease state on enrollment when the antiviral therapy was provided for patients further incurred variation across treatment groups. Although the second point can be tackled by using a stratified randomization, as in the ACTT-1 study, this design-based approach can be challenging in both the enrollment of patients and the implementation of the study. The use of a surrogate endpoint can be an efficient alternative to cope with the heterogeneity involved in evaluating the clinical efficacy of antiviral therapy [20, 21]. In terms of COVID-19 mortality, in the high-risk state, the use of invasive support for ventilation and circulation can be deemed as an early surrogate endpoint.

While the clinical efficacy of antiviral therapy such as remdeisvir has been studied by using these RCTs to prove a significant result of discharge and borderline risk reduction of death, efficient evaluation with an early surrogate endpoint may provide auxiliary evidence for enhancing the statistical power and providing a new insight into clinical use when a new antiviral therapy is introduced. With the understanding of how the administration of antiviral therapy affects the dynamics mechanism in relation to the evolution of a disease spectrum model from low- to a high-risk state, two primary outcomes, discharge and death, can further facilitate an evidence-based clinical decision-making for timely and efficient administration of antiviral therapy.

We first assessed the clinical efficacy of remdesivir for a 14-day period in altering the dynamics of COVID-19 with standard care using the high-risk COVID-19 state as a surrogate endpoint. We then compared the results of this surrogate endpoint with those based on the primary endpoint of death from COVID-19 up to 28 days in order to examine whether a surrogate endpoint can be predictive of the primary endpoint of death from COVID-19. Elucidating the dynamics of COVID-19 evolution from a low-risk to a high-risk state would make a contribution to deciphering the clinical efficacy of remdesivir.

Methods

Disease Spectrum Model for the Evolution of COVID-19

Following the recommendation from the WHO R&D Blueprint Group [22], the COVID-19 disease severity can be categorized into three risk levels: low-risk (hospitalized with or without supplemental oxygen, corresponding to categories 4 and 5 in the ACTT-1 trial), medium-risk (hospitalized requiring high-flow oxygen therapy or non-invasive mechanical ventilator, corresponding to category 6 in the ACTT-1 trial), and high-risk (hospitalized receiving invasive mechanical ventilation or extracorporeal membrane oxygenation (ECMO), corresponding to category 7 in the ACTT-1 trial). Two primary outcomes, discharge and death, are of great interest. The disease spectrum model depicting the progression and regression of COVID-19 patients and the two final destinations of discharge and death are shown in Fig. 1.

Fig. 1

Disease evolution model for COVID-19

The proposed disease spectrum of the COVID-19 model allows for both the progression and regression of the disease course due to clinical deterioration and improvement in response to treatments and therapies, respectively, for COVID-19 patients classified into the three risk states as indicated in Fig. 1. The administration of an effective antiviral agent would be expected not only to facilitate the rate of regression, yielding a higher chance of discharge, but also to reduce the rate of deterioration toward the high-risk state, leading to a decreased risk of death during follow-up. These postulates have prompted us to model the disease evolution of COVID-19 with and without the administration of remdesivir treatment in order to assess whether and why clinical efficacy of antiviral therapy would be able to facilitate discharge and reduce the risk for death.

Study Design and Data Sources

Data used for demonstrating the mechanism accounting for the clinical efficacy of antiviral therapy were derived from the two-arm ACTT-1 trial. The details of the study design and the results of the final report have been described in full elsewhere [14]. In brief, the ACTT-1 trial employed a stratified, randomized, controlled trial design to assess the clinical efficacy of remdesivir treatment for COVID-19 patients across the spectrum of disease severities [14, 19]. The status of the disease severity including death and discharge were evaluated 14 days and 28 days after enrollment, which constitute the structure of panel data for the estimation of the daily transition rate of the evolution of COVID-19. Table 1 summarizes the distribution of three risk levels (low, medium, and high) on enrollment (baseline risk state) and that on day 14 given the risk state on enrollment reported by the ACTT-1 trial [14].

Table 1

Summary of empirical data on COVID-19 disease status for enrolled subjects by treatment groups

Baseline risk state	Risk state on 15th day after enrollment	Remdesivir		Placebo		P value
		Frequency	%	Frequency	%
Enrollment
Low	–	289	(56.4)	259	(51.3)	0.185
Medium	–	98	(19.1)	99	(19.6)
High	–	125	(24.4)	147	(29.1)
Total		512	(100)	505	(100)
Day 14
Low	Discharge	207	(80.9)	153	(72.2)	0.003
	Low	30	(11.7)	21	(9.9)
	Medium	2	(0.8)	3	(1.4)
	High	12	(4.7)	14	(6.6)
	Death	5	(2.0)	21	(9.9)
	Total	256	(100.0)	212	(100.0)
Medium	Discharge	41	(57.7)	34	(44.2)	0.245
	Low	6	(8.5)	11	(14.3)
	Medium	6	(8.5)	6	(7.8)
	High	5	(7.0)	13	(16.9)
	Death	13	(18.3)	13	(16.9)
	Total	71	(100.0)	77	(100.0)
High	Discharge	14	(13.9)	18	(15.7)	0.852
	Low	21	(20.8)	28	(24.3)
	Medium	7	(6.9)	5	(4.3)
	High	43	(42.6)	45	(39.1)
	Death	16	(15.8)	19	(16.5)
	Total	101	(100.0)	115	(100.0)

The frequencies of each category of the disease severity by the two arms of the ACTT-1 trial were abstracted from Table 2 of original article [14]. The low-risk group includes the category of 4 (hospitalized, not requiring supplemental oxygen but requiring on going medical care) and 5 (hospitalized, requiring supplementary oxygen), the medium-risk group corresponds to category 6 (hospitalized, requiring noninvasive ventilation or use of high-flow oxygen devices), and the high-risk group corresponds to category 7 (hospitalized, receiving invasive mechanical ventilation or ECMO).

Table 2

Estimated results on the efficacy of remdesivir treatment in reducing the high-risk COVID-19 state

	Estimate(95% CI)
14-day observation
Low-risk	0.652 (0.580, 0.733)
Medium-risk	0.878 (0.677, 1.140)
28-day observation
Low-risk	0.707 (0.702, 0.712)
Medium-risk	0.938 (0.818, 1.076)

Data used in the current study were abstracted from a published article without any private and identifiable information. Ethical approval is thus not required for this study.

Statistical Analysis

The empirical data on the frequencies by disease severity on enrollment (initial risk status), and the transition from each of the initial risk statuses to the observed risk status, and on the discharge and death on the assessment, were used for estimating the transition rates and the corresponding probabilities of disease progression for COVID-19 patients by using a continuous time Markov process [23], in conjunction with a Bayesian Markov Chain Monde Carlo method.

Supported by the information on the dynamics of COVID-19 from the low- to the high-risk state, the impact of antiviral therapy in predicting the reduction of death can be assessed by using the high-risk state as a surrogate endpoint. The benefit of remdesivir treatment in altering the natural evolution of COVID-19 can thus be assessed by comparing the probability of progression to the high-risk COVID-19 state for the remdesivir-treated group in contrast to the standard care group.

As far as the evaluation of the primary endpoint with death up to 28 days, the benefit of remdesivir treatment in reducing the risk of death was estimated by comparing the overall probability of death for the three risk states on enrollment between the remdesivir-treated and the placebo arm. A ratio of the risk of death for the two study arms (risk ratio; RR) was used to quantify the clinical efficacy. Regarding the efficacy of remdesivir treatment for discharge, a relative risk approach, taking into account the dynamic aspect of disease evolution, was applied to fully capture the benefit of antiviral treatment on COVID-19 patients in improving the regression on the spectrum of disease severity until discharge. Specifically, the odds of two transition probabilities on regression (from medium- to low- risk; from high- to medium- and low-risk, and discharge) to progression (from low- to medium- and high-risk; from medium- to high-risk) was first estimated for the remdesivir-treated arm and the placebo arm, respectively. The odds ratio was then derived from the two odds to quantify the clinical efficacy of remdesivir treatment for the improvement of discharge. These results predicted by using the dynamics of COVID-19 were further compared with the corresponding findings reported by the original ACTT-1 trial.

The absolute benefits of antiviral therapy in terms of COVID-19 death and discharge were assessed by comparing the difference in the probabilities of two groups with the observation period of 28 days. The number needed to treat (NNT) required for averting one COVID-19 death and gaining one discharge were assessed by using the inverse of the absolute differences in event rates for the two study arms.

The statistical significance of all the estimated results was assessed by using the two-tailed test with 5% type I error, and also the values of the lower and upper limits of 95% CI.

Results

Empirical Data on the Evolution of COVID-19

Table 1 summarizes the empirical data on the evolution of COVID-19 in light of the risk state on enrollment retrieved from the two-arm ACTT-1 trial. Combining two treatment arms, participants at low-risk status on enrollment had a remarkably higher discharge rate (76.9%) compared with those at the state of medium- (50.7%) and high-risk (15.7%) during a 14-day period (P < 0.001). The higher discharge rate was noted for the remdesivir-treated arm compared with the placebo arm among participants with low- (80.9% vs. 72.2%, P = 0.003) and medium- (57.7% vs. 44.2%, P = 0.245) risk on enrollment, although the latter did not reach the level of statistical significance due to the small number of enrolled patients. A lower death rate for the remdesivir-treated arm compared with the placebo arm was observed for subjects in the low-risk state (2.0% vs. 9.9%, P = 0.003) on enrollment.

For participants in the low-risk state on enrollment, the remedesivir-treated arm had a lower chance of subsequent progression to medium- (0.8% vs. 1.4%) and high- (4.7% vs. 6.6%) risk state on day 14 compared with the placebo arm (P = 0.003). For those at medium-risk on enrollment, a lower but not statistically significant rate of progression to the high-risk state for the remdesivir-treated arm compared with the placebo arm (7.0% vs. 16.9%, P = 0.245) was noted.

Supplementary S-Table 1 summarizes the numbers and rates of death by the risk states on enrollment. As indicated by these empirical results of the ACTT-1 interim report [14], while a statistically significant efficacy of remdesivir treatment (death rate = 2.0% (5/256), P = 0.001) compared with placebo [death rate = 9.9% (21/212)] was demonstrated for patients in the low-risk state, the corresponding differences for the medium- (P = 0.858) and high-risk groups (P = 0.493) did not reach the level of statistical significance (S-Table 1). These results suggest that the high-risk disease state can be a surrogate endpoint to efficiently assess the efficacy of the antiviral treatment for early disease.

Risk of COVID-19 Evolution Through the Disease Spectrum

Supplementary S-Table 2 shows the estimated results of transition probabilities at day 14 and 28, respectively, based on the disease evolution model depicted in Fig. 1. In the remdesivir-treated arm, the discharge rate increased with time from 78.6%, 58.2%, 23.2% at day 15 to 89.2%, 74.8%, and 45.3% at day 28 for low-, medium-, and high-risk state, respectively, in parallel with an increase in death rate from 2.0%, 7.0%, and 21.3% at day 14 to 3.9%, 11.3%, and 29.6% at day 28. The similar trends were also noted in the placebo arm.

Evaluation Using High-Risk Sate as a Surrogate Endpoint

Table 2 shows the results of evaluating the efficacy of remdeisvir treatment using high-risk state as the surrogate endpoint for the primary endpoint of mortality. The administration of remdesivir to the low-risk group led to a statistically significant reduction in the progression to high-state by 34.8% (95% CI 26.7, 42.0%) and 29.3% (95% CI 28.8, 29.8%) on the 14- and 28-day observation, respectively. For medium-risk patients, the corresponding benefits were attenuated to 12.2% (95% CI − 14.0, 32.3%) and 6.2% (95% CI − 7.6, 18.2%) without reaching the level of statistical significance.

Table 3 shows the estimated results on the clinical efficacy of remdesivir treatment by using discharge and death as outcomes. Consistent with the approach based on the surrogate endpoint, the administration of remdesivir resulted in a statistically significant reduction in COVID-19 death by 32.5% (95% CI 21.0, 42.3%, RR = 0.675, 95% CI 0.577, 0.790) evaluated at day 14. The corresponding figure was estimated as 30.5% (95% CI 6.6, 50.9%, RR = 0.695, 95% CI 0.491, 0.934) at day 28. Regarding the clinical efficacy of remdesivir treatment in accelerating discharge for COVID-19 patients, remdesivir treatment resulted in a statistically significant improvement by 34.3% (95% CI 0.7, 79.2%, RR = 1.343, 95% CI 1.007, 1.792). It should be noted that the effect sizes of point estimates were close to those reported by the final results of the ACTT-1 trial including the 27% reduction in mortality (HR = 0.73, 95% CI 0.52–1.03) and 29% (95% CI 12–49%) increase in discharge but our results demonstrated the statistical significance of remdesivir treatment in terms of accelerating discharge and also reducing the risk of death rather than a non-significant finding of the risk for the reduction of death.

Table 3

Estimated results on the efficacy of remdesivir treatment

Outcomes	Estimate(95% CI)
COVID-19 death
Relative risk ratio	0.695 (0.491, 0.934)
Risk difference	21.0% (17.2, 25.6%)
NNT	133.3 (112.5, 158.0)
Recovery and discharge
Relative risk ratio	1.343 (1.007, 1.792)
Risk difference	22.0% (8.9, 54.7%)
NNT	127.3 (116.5, 139.0)

Absolute Risk Reduction and Number Needed to Treat

Based on the estimated results at day 14, the NNT for remdesivir to avert one high-risk COVID-19 subject was estimated as 100.9 patients (95% CI 88.1, 115.7). Table 3 shows the absolute differences between the two arms, being 21.0% (95% CI 17.2, 25.6%) for reducing the risk of death and 22.0% (95% CI 8.9, 54.7%) for discharge at day 28. Translated from this absolute risk difference, the NNT of remdesivir treatment required to avert one death was estimated as 133.3 (95% CI 112.5, 158.0), and the corresponding NNT for facilitating one discharge was 127.3 (95% CI 116.5, 139.0).

We further examined the efficacy of remdesivir treatment by assessing the clinical efficacy when the antiviral therapy was provided to patients in the low-, medium-, and high-risk states with the results listed in S-Table 3. For the outcome of death, remdesivir treatment shows an attenuated trend in reducing the risk of COVID-19 death by the increase in risk levels, ranging from 0.418 (95% CI 0.415–0.420, low-risk patients) to 0.775 (95% CI 0.725–0.828, high-risk patients). A similar trend can be observed for the outcome of discharge. For the low-risk patients, remdesivir treatment results in a statistically significant increase in the odds of discharge by 61.0% (95% CI 23.2, 110.2%). Regarding medium- (20.9%, 95% CI − 14.7, 71.4%) and high-risk patients (− 17.6%, 95% CI − 44.6, 22.4%), the estimated results on the clinical efficacy are reduced, and fail to reach statistical significance.

Discussion

We demonstrate here using the high-risk state of COVID-19 as a surrogate endpoint is predictive of the primary endpoint of death at day 28, as 35% efficacy in reducing the high-risk state of COVID-19 at day 14 was consistent with 31% efficacy in reducing death at day 28.

Regarding the clinical efficacy of remdesivir in accelerating recovery and discharge, a 34% (95% CI 1–79%) benefit was estimated. Translated from the efficacy for the surrogate endpoint at day 14, the NNT of remdesivir treatment to avert high-risk COVID-19 patients was estimated as 100.9 (95% CI 88.1–115.7). The corresponding NNTs required for the primary endpoints of death and discharge were 133.3 (95% CI 112.5–158.0) and 127.3 (95% CI 116.5–139.0), respectively, which can be assessed earlier by using the high-risk COVID-19 case.

Although remdesivir was considered as a promising antiviral agent with the potential for treating COVID-19 patients [9, 10], there is still a lack of evidence-based clinical guidance. The results from the two RCTs with compromised quality provide no remedy. While around 1300 participants of the two RCTs were enrolled in a recently published guideline for the clinical use of remdesivir, the recommendation remained elusive with weakly supported evidence [25, 26]. Due to uncertainty given the current evidence, the authors suggested the continued recruitment for RCTs [25]. The impact of this uncertainty observed in the current randomized controlled trials can be seen from the recent recommendations against the use of remdesivir in COVID-19 patients announced by WHO [27]. In contrast, as the first compound approved by the US Food and Drug Administration in treating COVID-19 patients, remdesivir remains an antiviral therapy approved in the United States [28]. This conflict in the suggested treatment for COVID-19 patients could hamper the efficient implementation for newly developed therapies, which is urgently needed instead of relying only on strictly non-pharmaceutical interventions [8].

We thus applied a dynamic model to decipher how the clinical efficacy of antiviral therapy like remdesivir affects the mechanism of COVID-19 evolution through the spectrum of disease severities, as this approach also facilitates the use of the high-risk state, which is pivotal to further progression to death, as a surrogate endpoint for evaluating the clinical efficacy of the antiviral therapy. While the controversial results between ACTT-1 and the RCT in China confuse evidence-based decisions on the provision of remdesivir treatment for COVID-19 patients, our algorithm provides a framework for elucidating not only the mechanism of disease evolution but also the transparency and availability of data to strengthen the evidence for timely clinical decisions on the use of remdesivir in the face of the COVID-19 pandemic.

Compared with the results of the interim [14] and final [19] reports of the ACTT-1 trial derived from conventional time-to-event analysis, our estimated results are close to the point estimates in terms of the two outcomes, but are based on the detailed dynamics of the COVID-19 disease spectrum and show statistical significance for both outcomes, whereas there is only a marginally statistically significant result for the reduction of death reported by the ACTT-1 trial. This is because we make full use of information on the intermediate outcomes of COVID-19 before death, in contrast to information relying only on the primary outcome of death reported by the RCT design, which may reduce the uncertainty information message because elucidating the dynamics of COVID-19 evolution can reveal whether the net force goes toward the high-risk or the low-risk state. The former would be more likely to increase the risk of death, whereas the latter implies that discharge would be ready once the low-risk state is reached. This accounts for why there was a higher discharge rate in the treated group compared with the standard care group, which was very strong in the low-risk state but only modest in the medium-risk and high-risk states. Therefore, the role of remdesivir not only inhibits the progression of COVID-19 from the low-risk to the high-risk state but also facilitates the regression from the high-risk state to the low-risk state. Quantifying how antiviral therapy affects the dynamics of COVID-19 plays an important role in the prevention of COVID-19 patients deteriorating into the high-risk state, which is a critical issue in treating infected vulnerable populations.

There is one limitation of the current study. We used the empirical data derived from the published ACTT-1 trail in conjunction with a multi-state model, and with the surrogate endpoint of the high-risk disease state to prove the clinical efficacy of the antiviral therapy of remdesivir. However, our results are only based on one RCT study, so the findings using such a surrogate for primary endpoint should be validated by other RCT studies.

In conclusion, the clinical efficacy of novel antiviral therapy assessed by an early surrogate endpoint of the high-risk state, taking into account the dynamics of disease evolution, can be a solution to providing the scientific evidence for guiding its clinical use in treating COVID-19 patients. By using the high-risk disease state as the surrogate endpoint, we have demonstrated its usefulness for assessing the clinical efficacy of antiviral treatment in terms of mortality reduction. The antiviral therapy, in altering the natural evolution of COVID-19 deciphered by the underlying mechanism, further provides a new clinical insight into an efficient and adequate treatment for COVID-19 patients with the consideration of disease severity.

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 27 kb)

We demonstrate the statistically significant efficacy of remdesivir in reducing the surrogate endpoint of the high-risk COVID-19 state by 34.8% for a 14-day period.

The result of the surrogate predicts the efficacy for the primary endpoint of COVID-19 death by 30.5% up to a 28-day period.

Remdesivir treatment results in a 34.3% significantly higher odds of discharge.

The findings support the early clinical use of remdesivir in treating COVID-19 patients.