Has the COVID-19 Crisis Affected the Growth of United States Food and Drug Administration Drug Approvals? The Answer is Not Yet! A Time Series (Forecasting) Study.

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-19; HCoV-19; COVID-19) has affected all daily activities. Has it also affected the number of United States (FDA) drug approvals over time? The short answer from empirical time series forecasting is not yet. Care should be taken as the crisis continues through maintaining the scientific, economic, political, and social supportive structures to sustain momentum. This conclusion is based on analyzing the results of (non-overlapping) forecasting routines (viz., complex exponential smoothing, auto-regressive fractionally integrated moving average, extreme learning machine, and multi-layer perceptron) performed on longitudinal (1939-present) FDA (CDER) drug approvals taking into regard pre- and extant-COVID-19 eras. This is an initial study and there are caveats with the approach, and as such, all data and programs are provided to support replication of the results and furthering of the investigation.

Drug Discovery and Development (DDD) and the COVID-19 Crisis

Successful drug discovery and development requires aligning a diverse set of national and international scientific, economic, political, and social structures committed to by patients, healthcare providers, institutions, and governments to maneuver an idea to discovery to development to market approval and subsequent delivery of a quality, safe, and effective medicine for a given condition or disease. It is presumed that disruptions in these support systems (e.g., a bear market) would translate into a downward pressure of some degree on the number of drug approvals over time, a widely used key performance indicator of success of this process [1] and its subprocesses [2]. A potentially obvious and significant disruption today would be that of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-19; HCoV-19; COVID-19) crisis.

Even though COVID-19 has a limited history (recognized sometime around the end of 2019), it has dramatically altered everyday activities due to ease of transmission and degree of severity [3]. As of August 30, 2020, there have been approximately 25.1 M total cases and 844,000 deaths worldwide, with the USA contributing 6 M cases and 183,000 deaths.1 On a local, national, and international level, governments have responded to this crisis in a variety of ways including restrictions on mobility, recommendations (mandates) on the use of personal protective equipment (e.g., masks), person-to-person distancing, social gathering, and so on [4]. These interventions and the (very real) fear of getting the disease has directly affected the drug development process in a variety of ways including challenging clinical trial recruitment—with some estimates suggesting that a majority (up to 80%) of trials being interrupted over some duration [5].

Our Key Question: Has the COVID-19 Crisis Affected FDA Drug Approvals: And If So But How Much?

While it may be easy to conceive that the COVID-19 crisis has disrupted a broad swath of support structures of the DDD process would translate into an observable disruption in the number of drug approvals over time, it is challenging to test this hypothesis empirically, due to several reasons (e.g., the intrinsically high attrition rate in drug development, the time lag of affect to actually observer a change to the time dynamics). Here, the author attempts to address the question: has the COVID-19 crisis affected the number of approvals over time; and if so, by how much? In what follows, the general approach, data and statistical methodology, as well as results and interpretation of the analysis, are presented.

An Attempt to Assess the Impact of COVID-19 on US Drug Approvals

This work presents a first step toward considering the impact of COVID-19 on US drug approvals. The general approach is as follows: compare the forecast of US drug approvals without COVID-19 (that is, use input data from the origin of the reporting period (Jan 1939) to March 2020) with that of a forecast built with data from COVID-19 (that is, use input data from Jan 1939 to August 2020). Thus, the approach seeks to consider if there was a measurable/meaningful change in the rate of US drug approvals attributable due to COVID-19.

Beyond the statistical nuances of the forecasting methods and data, there are at least three important assumptions with this approach: First, it treats COVID-19 as a singular and constant impulse. Second, the argument is based on notion of ‘ceteris paribus,’ wherein it is assumed that ‘all things are equal’ underlying the COVID-19 and non-COVID-19 forecasts except for the presence of the COVID-19 impulse (e.g., the scientific, regulatory evidentiary standards and patient needs) are still intact (that is, there is a stability in the underlying variation defined by history). And, third, given the depth and breadth of the COVID-19 crisis on every day processes (including FDA and/or Sponsor resource availability), it is hypothesized that a difference in approval rates may be observed during the 8-month (March–August) reporting period. Indeed, this may not be the case, and additional reporting periods should be tested to further evaluate COVID-19 effects on the rate of US approvals.

Understanding the difficulty with the conceptual design as well as the statistical challenges of forecasting complex dynamics, all data and R Project code are included in the Electronic Supplementary Materials section, with the hope that readers would use the information to replicate, challenge and/or extend the approach presented here. A discussion on the assumptions and future directions are work are discussed later in this commentary.

Data Collection and Statistical Method

In the USA, the Centers of Drug Evaluation and Research (CDER) branch of the Food and Drug Administration (FDA) “regulates over-the-counter and prescription drugs, including biological therapeutics and generic drugs.2” CDER is not the only institution that regulates the DDD process, but it is one that provides a publicly, reliable and valuable source of longitudinal metrics regarding the DDD process from the dawn of the review process (Jan 1939) to the present [1]. The monthly number of US CDER Approvals (time series) has followed an interesting dynamic from its slow inception to the dramatic rise in the 1970s to the early millennium, and subsequent drastic fall with a subsequent re-rise (Fig. 1).

Figure 1.

Time Evolution of the Total Number of FDA (CDER) Approvals (see text for details).

Generally, the idea to elucidate if the number of US Drug Approvals has been affected by the COVID-19 crisis would be to compare the forecasts assuming the COVID-19 crisis with that of the current environment. Specifically, the algorithm would take the form:

Construct non-COVID-19 forecast:

Input data: FDA CDER Approvals time series data originating from beginning of reporting period (September 1939) to December 2019

Output forecast: January 2020 to December 2022

Construct COVID-19 forecast:

Input data: FDA CDER Approvals time series data originating from beginning of reporting period (September 1939) to August 2020.

Output forecast: September 2020 to December 2022

Compare the Non-COVID-19 from the COVID-19 24-month forecasts

From a mathematical perspective, the data in Fig. 1 represents a non-stationary, non-normal, non-linear, non-seasonal, persistent [memory; long-range dependent (LRD)] time series that has structural anomalies/changes [6]. These characteristics led to selecting the forecasting routines used to support the analysis. While readers are encouraged to review the citations for mathematical rationale and implementation details, for brevity, the following R Project [7-10, 12-15] routines were used:

Complex exponential smoothing (CES): A method that constructs and leverages a latent information potential variable with that of the actual (explicit) observations into a complex variable (R Project: ‘smooth’ [7, 8]).

Auto-Regressive Fractionally Integrated Moving Average (ARFIMA): A historically useful model to accommodate time series with LRD. (Note: While the data are non-stationary, no pre-filtering was performed, as accuracy values suggest a generally robust model. See Supplementary Materials) (R Project: ‘arfima’ [9])

Extreme learning machine (ELM) and Multi-Layer Perceptron (MLP): Both are machine learning / artificial neural network routines: while the former is a generalized single-hidden layer feed forward networks, the latter applies multiple (hidden) layers of fully connected neurons (R Project: ‘nnfor’ [10]).

The results are presented in Fig. 2 and Electronic Supplementary Materials. It is noted that various accuracy measures were calculated (e.g., mean error, root mean squared error, and mean absolute error) and demonstrated similarity across all models comprising the forecasts, with MLP having the smallest error generally.

Figure 2.

Forecasts Based on the Number of FDA (CDER) Drug Approvals Consider COVID-19: a Complex Exponential Smoothing (CES; upper left), b auto regression fractional moving average (ARFIMA); upper right), c multilayer perception (MLP; lower left), and d extreme machine learning (ELM; lower right). Note: Only the mean forecast is presented (95% intervals are accessible as described in the Supplementary Materials)

A Potential Answer: No, the COVID-19 Crisis Has Not Affected FDA Drug Approvals (At Least Not Yet)

Prior to discussing the results, it is important to better understand what is presented in Fig. 2. Each graph represents the time course of the number of FDA (CDER) approvals since 2018 (a recent window of Fig. 1). In each figure, the orange and blue lines represent the non-COVID-19 (January 1939—December 2019) and COVID-19 [January 1939–Aug 2020; i.e., the non-COVID-19 time series with the additional 8 months of data (the COVID-19 era)] actual monthly observations, respectively. The red and green lines represent the forecasts based on the non-COVID and COVID actual observations, accordingly.

The key results from the analyses (as shown in Figs. 2) show that:

A qualitative comparison of the absolute number (actual) of US FDA (CDER) drug approvals observations in 2020 with that of last couple of years shows little clear impact of the crisis. In fact, the monthly values have remained relatively constant over 2020.

All (mean) forecasts extrapolated from the non-COVID-19 data were incrementally higher than that of the COVID-19 data. In fact, the difference may be considered negligible (< 75 approvals per month and well within the 95% confidence intervals of the forecasts, particularly toward the end of the forecasts (see Electronic Supplementary Material)).

Thus, we can conclude that (thus far) the US (CDER) approvals are persistent, yielding thus far little to no negative affect due to the COVID-19 crisis. Assuming the key performance indicator of drug discovery and development is indeed the number of US (CDER) drug approvals, then it seems that the various scientific, political, economic, and social buttresses supporting the DDD process have thus far prevented a moderate to serious downward trend effect brought about by the pandemic, which may have been expected due to the length and depth of the COVID-19 crisis thus far.

For the sake of patients everywhere, we should remain vigilant through supporting the various structures comprising the DDD process.

Future work may include reassessing the forecast intermittently over the coming months/quarters/years. The data used to support the forecast spans monthly values for nearly 100 years (1939–2020). The COVID-19 crisis occurred relatively recently (defined here as March 2020) and lasted 8 months (August 2020) in the approach above. It may be that the 8 months of data collection were imperceptible to the forecasts from a statistical (methodology) perspective, or that the changes to the underlying systems (e.g., company/FDA staff attending to submitting reviewing marketing applications) had not changed substantially to realize a change. Re-performing the forecasting may eventually realize a bifurcation should the underlying systems be affected. The forecasting approach may thus be used as a monitoring tool to quantify the impact of COVID-19.

An additional avenue of inquiry would be a redesign of the approach to consider COVID-19 with evolving magnitude and direction (vector) as opposed to a singular constant impulse (scalar). Within a given jurisdiction, the incidences of COVID-19 pandemic are evolving – ebbing and flowing depending on a variety of factors including the degree and type of restrictions [11]. A variable impulse may more accurately reflect reality.

Lastly, as US drug approvals is an important key performance indication (KPI), modeling is itself a valuable pursuit. It would be of significant import to thus further techniques that may be used to ‘tease’ out the impact of specific influences. Ultimately, this metric integrates scientific prowess, policy potency, and economic strength (among other influences); therefore, it would be important to better understand how these influences operate on this KPI. In the context of COVID-19, this is particularly of value as a society we may be able to then concentrate our resources in such a way as to optimize the affected variable(s) to deliver life-enhancing medications to patients, the ultimate objective of our work.

Below is the link to the electronic supplementary material.

Electronic supplementary material 1 (DOCX 154 kb)