The world is waiting, use sequential analysis and get us the evidence-based treatment we need for COVID-19.

In spite of the relatively high morbidity and mortality, there is no approved medication yet for COVID-19. There are more than 200 ongoing trials on different drugs or vaccines, but new medications may take until 2021 to develop. Defining the optimal number of patients to be included in a study is a considerable challenge in these interventional researches. Ethical considerations prompt researchers to minimize the number of patients included in a trial. This gains particular importance when the disease is rare or lethal which is particularly so in the case of COVID-19. It is of paramount importance to explore some of the available tools that could help accelerate the adoption of any or some of the many proposed modalities for the treatment of diseases. These tools should be effective, yet efficient, for rapid testing of such treatments. Sequential analysis has not been frequently used in many clinical trials where it should have been used. None of the authors in published literature, as far as we know, used sequential analysis techniques to test potential drugs for COVID-19. In addition to its usefulness when the results of new forms of treatment are quickly needed, other important benefit of sequential analysis includes the ability to reach a similar conclusion about the utility of a new drug without unduly exposing more patients to the side effect of the old drug, in particularly, for the treatment of a rare disease.

Introduction

The novel coronavirus disease (COVID-19), the latest in the series of emerging coronavirus diseases, was first identified in December 2019 in the city of Wuhan. It has since spread globally resulting in one of the most challenging recorded pandemics in the history of mankind. Until now, the disease had infected more than four million and claimed the lives of more than a quarter of million people.

The World Health Organization (WHO) had declared this outbreak on 30 January 2020 as a Public Health Emergency of International Concern (PHEIC) and a Pandemic on 11 March 2020. Local transmission of the disease has been recorded in most countries across all six WHO regions.

The relatively high morbidity and mortality launched the hunt for an effective treatment modality directed either at the virus itself or at its different complications [1].

Far from being perfect, patients are given empirical antibiotics, antiviral therapy (Oseltamivir, Remdesivir, Ribavirin, Sofosbuvir, Lopinavir/Ritonavir, Favipiravir ….), fluids, immunosuppressive and systemic corticosteroids, and invasive mechanical ventilation, in addition, to support for other vital organs [2,3].

As there is no approved medication yet, there is an urgent need for specific treatment targeting COVID-19 [3]. Research into potential treatments for COVID-19 started in January 2020. Several antiviral and other drugs in various stages of clinical trials are being tested [4,5]. Currently, there are more than 200 ongoing trials on different drugs or vaccines [6]. It is widely presumed that new medications may take until 2021 to develop. The WHO recommended volunteers take part in trials of the effectiveness and safety of potential treatments [7]. Ethically we should ensure that certain statistical standards are met in the drug’s clinical trials and that the drug will not have an undue harmful adverse effect on humans.

Defining the optimal number of patients to be included in a study is a considerable challenge in intervention research. In small samples, because of a lack of statistical power, indeterminate results are expected. On the other hand, traditional trials entail the risk of still including patients at a time when enough information is already available to answer the trial question. Ethical considerations prompt researchers to minimize the number of patients included in a trial. This is, even so, when the disease is rare or lethal [8]. That is particularly so in the case of COVID-19.

Traditional comparative clinical trials are difficult to conduct when large sample sizes are required, as recruitment may be challenging and increase study duration. In addition, the power to evaluate efficacy in relevant subgroups may be limited. Costs may be so high that trials either not performed [9] or not completed [10].

As more new drugs are to be discovered, traditional designs come at their limits. It would be of paramount importance to explore some of the tools, whether new or already known, that could help accelerate the adoption of the many proposed modalities for the treatment of diseases as COVID-19. These tools should be effective, yet efficient, for rapid testing of such treatments.

One of these tools is the Sequential Analysis. It is a method of continuous periodic assessment during an experiment, where a decision can be taken early at cutoff points. It is a useful method for optimizing the sample size in clinical trials and is a promising technique for rare or urgent studies [9,11]. Sequential designs should be considered when it is ethically undesirable to continue randomizing vulnerable subjects at a time when enough information has accumulated to decide which treatment is superior. It could be used to decide about the optimal treatment strategy in the clinical setting when results should be obtained with a minimum number of patients [12,13]. Although these approaches were developed as early as the 1960 s, they are relatively unknown. The sequential probability ratio test (SPRT) is the term that is used currently for this particular form of statistical analysis where the sample is not fixed in advance and stopped as soon as significant results are observed according to pre-defined rule. Sequential analysis has not been frequently used in many clinical trials where it should have been used [14].

It is generally considered unethical to continue randomizing patients and thus exposing half of them to an inferior or least desirable intervention, when the already gathered data are considered sufficient to determine a positive treatment effect or to determine that there is no clinically relevant benefit. Patients should be entered only if the responsible clinician is uncertain about the most appropriate treatment for that particular patient [15]. The rational of use of Sequential Analysis and advantages in this condition is appealing (Table 1).

Table 1.

Rational and advantages for using sequential analysis design in clinical trials.

Overcoming the disadvantages of enrolling more than necessary patients to achieve statistically significant results thereby lowering the number of subjects needed in a trial. Elimination of unnecessary costs by achieving economy in sample size. Potentially life-saving reductions in the time needed to establish a drug’s safety. Saving participants from unknown risks. Increased efficiency by potentially detecting differences sooner than traditional sampling. Availability of information about the effect of interventions can be made as soon as enough information is assembled to end the inclusion into the trial. Ensuring ethical and moral consideration. Reduction of the costs of clinical trials lead to lower cost of treatments and consequent reduction in the overall cost of health care [16,17]. Improved patient care.

Our aim is to discuss the importance of the technique, the concept behind the methodology, its usefulness in reference to reaching a quick and an appropriate treatment for COVID-19, the methodology, the types, and possible drawbacks.

The use of sequential analysis in Clinical Trials and/or COVID 19 in the medical literature

We searched PubMed using the terms (‘COVID-19’ AND ‘Treatment,’ OR ‘COVID-19’ AND ‘Chloroquine’ OR ‘COVID-19’ AND ‘Hydroxychloroquine’ OR ‘COVID-19’ AND ‘Vaccine’). The search yielded 189 articles. Out of these, 26 were obtained when ‘COVID-19’ AND ‘vaccine’ only are used. Using ‘COVID-19’ AND ‘Clinical study’ OR ‘COVID-19’ AND ‘Clinical trials’ yielded 59 different articles.

We also searched the literature using the following keywords: (‘sequential trial’ OR ‘sequential design’ OR ‘sequential experiment’ OR ‘sequential analysis’ OR ‘sequential test’ OR ‘triangular trial’ OR ‘triangular test’ OR ‘sequential probability ratio’ OR ‘boundaries approach’ OR ‘adaptive designs’ OR sequential probability ratio test (SPRT) OR “repeated significance testing (RST).

In addition, we searched the literature using these keywords with (AND ‘Clinical Study’, OR ‘Clinical Trial’, OR ‘Controlled Clinical Trial’). Our search yielded 20,562 different manuscripts in the last decade that cited these different terms related to the tool, out of which 19,971 were in the English language. Among these, 7343 were published in the last 5 years. Only 767 of them were clinical trials and/or clinical studies, while 1921 were review articles.

Most of these manuscripts using sequential analysis were not ‘clinical trials’ or ‘clinical studies’. The number of manuscripts using clinical trials/studies increased progressively till 2018. However, there was a dramatic drop of up to 50% in 2019 (Figure 1). Possible explanations for this drop could be the need for a particular expertise and the continuous presence of an experienced statistician/epidemiologist during the experiment. It is noteworthy to consider that the reasons behind the low use of the technique and this drop should take particular consideration in separate studies.

Figure 1.

The number of sequential analysis manuscripts in PubMed using clinical trials in comparison to other sequential analysis publications during the period 2010–2019.

None of the authors used sequential analysis techniques to test potential drugs for COVID-19. Which is surprising as the necessity would dictate its use in such conditions. One of the most important benefits of sequential analysis is the ability to reach a similar conclusion about the utility of a new drug without being subjected to the side effect of both newly testing and the old drug on large number of patients. This is extremely important in the assessment of treatment of a rare disease with a small number of cases or when the results are quickly needed (Text-Box-1). This absence of the use of sequential analysis in COVID-19 up to our knowledge till now in the literature might be due to the limited knowledge of such technique, or to its complexity, and/or to the inertia to leave a known technique for a new one.

Concept and application of sequential analysis

An investigator might wish to have an up-to-date record at any stage, either for general information or because the appropriate sample size depends on quantities that (s)he can estimate only from the data itself. Alternatively, (s)he may have no intrinsic interest in the intermediate results but may be able to achieve economy in sample size by taking them into account. Simulations have shown that the average sample size needed to complete a trial using a sequential method is always smaller than that of the corresponding fixed design by a median of 77% (range: 15–90%) [17]. It is possible that by using only 25 patients we could reach the same conclusion that we would have otherwise arrived at by 135 patients in a total sample of 150 with 95% confidence [18]. This may also lead to a reduction of trials that are stopped early with indeterminate results. The methodology is advantageous for clinical trials in emergency medicine. The potential avoidance of unnecessary experimenting with vulnerable patients was clearly evident in the antenatal administration of corticosteroids for lung maturation in babies of women at risk of premature delivery [19].

The history and concept of sequential analysis date back to World War II. The focus was on the accuracy and speed of production with minimal testing, yet producing reliable results. Originally developed for use in quality control studies in the realm of manufacturing, this specific sequential hypothesis test was developed originally by Abraham Wald during the War, and is now referred to as SPRT as stated earlier [20].

The main concept that lies behind the technique is that in contrast to conventional designs of clinical trials, where the sample size is usually large and is determined a priori before the onset of the trial, subjects are recruited and data are collected in a sequential manner over time in this technique. Decision-making on sample size is performed in real-time as data are collected in sequential analysis, as opposed to retrospectively on fixed sample size, as is typically done, based on the accumulated information [21].

Sequential analysis has developed extensively since the mid-sixties, and the medical and statistical literature contains many models, some specific to particular research models, and some even developed just to service a particular research project. Peter Armitage introduced the use of sequential analysis in medical research, focusing on how the methods can be used in drug trials [22]. The focus in the work of Armitage was on the Sequential (Paired) Analysis, where comparable paired observations are made, and where the difference between the pairs is tested sequentially against the null hypothesis. Stuart Pocock’s made the tool popular in medicine [23]. In the 1990 s, the analysis methodologies for (unpaired) group sequential analysis was developed [24].

Method of performance and analysis using sequential analysis

Sequential trials are pre-planned. The expected (or minimal) clinically relevant effect size stopping rules are defined. Repeated series of comparisons are stopped as soon as a decision can be made as to whether one treatment can be regarded as superior to another, or that both are equally effective. In its simple form, successive values of X are plotted to form a ‘sample-path’, and sampling stops when the sample path crosses either of the boundaries (Figure 2). This enables fewer observations for the same degree of validity in comparison to other conventional methods where the number of observations is fixed in advance. Several sequential procedures exist, including group sequential designs, which include the well-known principle of interim analyses; boundaries designs, which include the SPRT, and repeated significance testing (RST); and the adaptive designs [21]. Some would categorize sequential analysis into Time sequential, Cohort sequential, and Cross sequential. The cohort sequential measures groups of participants as they age. It combines the best features of both longitudinal and cross-sectional designs. It studies specific age groups over time.

Figure 2.

Double triangular test showing the performance of the intervention at different power levels (α = 0.05 and α = 0.01).

The most appropriate design and method of analysis of a sequential investigation depends on the purpose of the investigation. The statistical formulation of that purpose may take one of several forms, usually either estimation of some quantity to a given degree of precision or testing a hypothesis with a given size and given power against a given alternative hypothesis. Suppose that one wishes to test a specific hypothesis, H0, in such a way that if H0 is indeed true it will usually be accepted and that if an alternative hypothesis H1 is true, H0 will usually be rejected. An example might include choosing between two drugs or a drug versus control. In our case, for example, that could be a comparison of a drug to treat COVID-19 as chloroquine versus an antiviral or a combination of these drugs versus any one of them or an immunosuppressive drug. In a paired technique, criteria that are used for the comparability of cases might include age, gender, type, and severity of complications. Again in this particular case of COVID-19, it might also include Bacillus Calmette–Guérin (BCG) as an example for comparability to test a treatment hypothesis.

Instead of defining a sample size, a pair of statistical borders are drawn, one to decide the rejection of the null hypothesis and the other to accept. Before the start of the experiment, the boundaries are calculated based on the alternative hypothesis and the desired levels of type I and II errors. According to the power desired, the boundaries would differ accordingly (Figure 2). The boundaries design relies on a graphical rule, where a V statistic, representing the amount of information (proportional to the number of subjects whose primary outcome is known) gathered in the course of a trial, is plotted on the X-axis, and a Z statistic, representing the effect size, is plotted on the Y-axis. The data is then obtained sequentially and plotted against these borders. The two statistics Z and V are calculated based on the data accumulated thus far, and plotted, creating a so-called sample-path. Analysis can either be done after each patient has reached the primary outcome (‘continuous’ sequential analysis) or after a fixed or variable number of patients have reached the primary outcome (group sequential analysis). A conclusion is reached when a boundary is crossed: the intervention at hand is deemed effective or not effective. If the broken line first leaves the region bounded by the straight lines through the upper boundary, then H1 is accepted. If the broken line leaves the region through the lower boundary, then H0 is accepted. Inclusion and randomization of subjects are continued as long as the sample path remains between the boundaries, the continuation region. An example of a (double) triangular test is illustrated in Figure 2.

Applications of sequential design

Sequential analysis is used mainly in experimental clinical trials. However, it could be applied in other conditions as to quality assurance, surveillance of vaccine safety, double-blind allergen-exposure tests, dose-finding, the optimal ventilation technique in preterm neonates [25-27], hearing testing [28,29], metabolic screening, computerized testing of human examinees as a termination criterion in a variable-length computerized classification test (CCT) [30], detection of anomalous medical outcomes [31], and monitoring the performance of doctors, surgeons and other medical practitioners in such a way as to give early warning of potentially anomalous results. Another important use that merits particular attention is a cumulative meta-analysis and trial sequential analysis of randomized trials in systematic reviews [32,33].

An extension of the SPRT method called Maximized Sequential Probability Ratio Test (MaxSPRT) is a one-sided alternative hypothesis with an upper stopping boundary and is used in several medical research studies [34].

There are several other improvements in the applications of sequential methodologies in clinical research. Examples include the development of user-friendly software packages, such as PEST, Cytel’s East6TM and SEQUENTIAL. These made sequential trials much easier to perform including when to stop and using unpaired analysis.

Conclusion

Sequential analysis is not frequently used in many clinical trials where it should have been. It was not used to test potential drugs for COVID-19 until now, in spite that it is an effective and an efficient tool for rapid testing of such treatments. We urge our colleagues to use this tool to get us an evidence-based treatment and quick answer(s) regarding the effectiveness and side effects of these proposed drugs that are desperately needed in our arsenal for fighting the COVID-19 pandemic.