Understanding variation in reported covid-19 deaths with a novel Shewhart chart application.

OBJECTIVE: Motivated by the coronavirus disease 2019 (covid-19) pandemic, we developed a novel Shewhart chart to visualize and learn from variation in reported deaths in an epidemic. CONTEXT: Without a method to understand if a day-to-day variation in outcomes may be attributed to meaningful signals of change-rather than variability we would expect-care providers, improvement leaders, policy-makers, and the public will struggle to recognize if epidemic conditions are improving.
METHODS: We developed a novel hybrid C-chart and I-chart to detect within a geographic area the start and end of exponential growth in reported deaths. Reported deaths were the unit of analysis owing to erratic reporting of cases from variability in local testing strategies. We used simulation and case studies to assess chart performance and define technical parameters. This approach also applies to other critical measures related to a pandemic when high-quality data are available.
CONCLUSIONS: The hybrid chart detected the start of exponential growth and identified early signals that the growth phase was ending. During a pandemic, timely reliable signals that an epidemic is waxing or waning may have mortal implications. This novel chart offers a practical tool, accessible to system leaders and frontline teams, to visualize and learn from daily reported deaths during an epidemic. Without Shewhart charts and, more broadly, a theory of variation in our epidemiological arsenal, we lack a scientific method for a real-time assessment of local conditions. Shewhart charts should become a standard method for learning from data in the context of a pandemic or epidemic.

Introduction

During the coronavirus disease 2019 (covid-19) pandemic, reports of new cases and deaths have dominated the news media. In areas hit hardest by the virus, residents and their leaders anxiously await the latest daily figure. An increase in reported deaths from the previous day can bring despair or panic; a decrease offers hope that the tide is turning. Every day brings a new and seemingly meaningful story that could influence individual behavioral choices, impact organizational planning, and/or distort decision-making at local and national levels. International media outlets are inclined to sensationalize the pandemic with headlines fixating on day-to-day differences, as exemplified by a sample of media headlines characterizing the trajectory of covid-19 mortality in the UK (Fig. 1) during the spring of 2020. This all-too-common type of reporting by the press obscures a full-system view of the epidemic landscape. Instead it provokes hyperreactive responses from policy-makers and public citizens alike.

Figure 1

Run chart of reported covid-19 deaths in the UK annotated with a sample of media headlines.

We know that the number of reported deaths each day—as with anything we measure—will fluctuate. Without a method to understand if these ‘ups and downs’ simply reflect natural variability, we will struggle to recognize signals of meaningful improvement [1] in epidemic conditions. To model the spread and impact of the covid-19 pandemic, we focus on publicly reported deaths (rather than cases) because variability in local testing strategies leads to an erratic evaluation of confirmed cases [2, 3]. The Shewhart chart outlined here could be applied to track other measures associated with an epidemic such as hospitalizations, ICU admissions, and intubations.

A fundamental tenet of the science of improvement is that understanding the sources of variation in a system is an essential aspect of any effort to bring about improvement in that system. In 1931, Walter Shewhart theorized that variation in a measure has two potential origins: common causes and special causes [4]. ‘Common causes’ are inherent in the system over time, affecting everyone in the system and all system outcomes. ‘Special causes’ are not part of the regular system but arise because of particular circumstances or some ‘special’ source of variation that can be assigned to an identifiable cause. Shewhart established the ‘control chart’ tool to operationalize this theory [5], providing subject matter experts (people with in-depth knowledge of the local system) with a formal method to identify special causes and to evaluate interventions they make to change the system of common causes. Shewhart’s theory and charts have been successfully applied to numerous problems across healthcare, industry, government, education, and human welfare [6-11], but their application to epidemics has been limited.

The bulk of previous studies using Shewhart charts in an epidemiological context has focused on infection control and hospital epidemiology [12-16]. More relevant to our approach is a relatively sparse literature based on the use of control charts to enable a timely detection of unusual patterns in public health data [17, 18]. A 1946 study tracked a poliomyelitis epidemic with X-bar charts, suggesting that ‘the industrial control chart may prove to be of general usefulness in many kinds of epidemiological work’ [19, p1510]. Shewhart U-charts have been used to track disease cases and hospital referrals as a dynamic warning system for health authorities, care providers and the general public and thus ‘reduced the possibility of precipitating adverse events by indicating appropriate reactions to normal variations’ [20, p280].

Researchers have used moving average, exponentially weighted moving average and/or cumulative sum charts for signaling the start or end of outbreaks [21-24], enabling rapid detection and ‘subsequent timely public health actions to decrease unnecessary morbidity and mortality’ [25, p3309]. Woodall’s 2006 review highlighted the opportunities for public health surveillance and encouraged further investigation of public health applications with control charts [26]. Previous research has suggested that ‘of the many tools used by continuous quality improvement, perhaps the most important for the epidemiologist to understand is the control chart’ [27, p102]. Yet its potential to support public health initiatives is still widely unrecognized.

This paper describes the application of Shewhart’s theory of variation to develop a novel control chart to track daily reported deaths in a specific geographic area during the covid-19 pandemic.

Methods

We designed a novel hybrid Shewhart chart to evaluate the trajectory of reported deaths using a typical epidemiological curve (Fig.2), but this application does not require the theoretical assumptions usually associated with this model [28].

Figure 2

Hypothetical example of a typical epidemiological curve with three phases.

Our hybrid Shewhart chart models three phases of an epidemic (pre-growth, growth and post-growth). The appropriate chart specifications for each phase are described in this section. Technical details are included in the Appendix.

Phase 1 (pre-growth)

The pre-growth phase begins on the first day of a reported death in a geographic region. The Shewhart chart for this phase is a basic C-chart [1] for count data in a period where daily reported deaths in a geographic area are low and stable. The focus of our analysis at this stage is detecting when the pattern of reported deaths suggests that the epidemic has begun to shift into an exponential growth phase.

First, we plotted daily reported deaths on a run chart until there are enough observations to meet the minimum requirements for an effective C-chart. We recommend calculating control limits only after at least eight total deaths have been reported. A standard C-chart is used to monitor the pre-growth phase and to detect a signal of special cause variation (number of deaths above the upper limit (UL) or a shift of 8 successive days above the center line (CL). Either of these signals would suggest a transition into the exponential growth phase. The C-chart for reported covid-19 deaths in Singapore (Fig. 3) was stable with no indications of exponential growth, an example of a location where the epidemic remained in the pre-growth phase (as of April 25, 2020).

Figure 3

Example of a chart for a country in a pre-growth phase.

A special cause signal on the C-chart is an indication that the exponential growth period may have started. The appropriate time to start the second phase I-Chart is indicated by:

A total of at least eight reported deaths (for a legitimate C-chart)

The presence of a special cause signal on C-chart indicating a meaningful increase in deaths

A significant non-negative regression slope after at least 5 days of observations included in regression calculations

We continued to plot daily observations on the C-chart until all of the above criteria are satisfied. For example, the C-chart from South Korea (Fig. 4) showed a special cause signal above the upper limit with the observation of nine deaths on March 20. Since then, the lack of a positive slope from the regression on log deaths gave no indication that the exponential growth phase had begun (as of April 25, 2020). So, we continued the C-chart and updated the chart center line and limits after each data point was added.

Figure 4

Example of a chart for a country that has yet to transition into an exponential growth phase.

Phase 2 (growth)

The growth phase starts when the pattern of reported epidemic deaths ceases to be stable and begins to grow exponentially (Fig. 5). An I-chart is used to monitor the growth phase and to identify a signal that exponential growth has ended. Phase 2 potentially begins on the day when the C-chart first reveals a special cause signal. However, we continue to plot data on the C-chart for the next 5 days, after which we fit a regression of the base-10 logarithm of daily reported deaths on the number of successive days starting with data reported on the day that triggered the special cause signal on the pre-growth C-chart [29]. When the lower limit of a 95% confidence interval for the regression slope is non-negative, we begin to model the growth phase by plotting observations on the I-chart and using regression coefficients for the calculation of its sloping center line and control limits [30].

Figure 5

Example of a chart for a country in an exponential growth phase.

The hybrid C-chart and I-chart for Peru (Fig. 5) show an example of a country that has started exponential growth and remains in phase 2 (as of April 25, 2020). The non-linear curved center line and control limits result from the transformation of logarithmic calculations back to the scale of reported deaths. The starting point for the Peru I-chart was the observation of 17 deaths on April 2 that initially triggered a special cause signal on the C-chart. We continued plotting new reported deaths and updated the calculations of the regression center line and control limits each day. After the exponential growth phase in Peru continued for more than 20 days (on April 22), we ‘froze’ the center line and control limits (that is, we ceased to incorporate subsequent data in fitting the regression and calculating chart limits). This process of ‘freezing’ control limits after 20 observations is a standard practice for Shewhart charts [1].

Frozen limits should be extended into the future as a guide to interpret subsequent observations plotted on the chart. This is an important distinction relative to many epidemiological models that do not use control limits to assess the stability of cases or deaths over time and therefore are not equipped to detect a point-to-point variation that reflects a meaningful change in the measures of interest. We continue plotting daily observations in the growth phase, looking for signals of special cause variation. Observations above the upper control limit or shifts of eight successive observations above the center line are indications that the growth rate is increasing over the previous days. Observations below the lower control limit or shifts of eight successive observations below the center line are indications of a decreasing rate of growth or, possibly, the end of the growth period.

Phase 3 (post-growth)

The post-growth phase begins when the trajectory of reported deaths no longer exhibits exponential growth, after which we expect the pattern on subsequent days to plateau or decline. If a special cause signal below the lower limit occurs before the growth phase has persisted for 20 days, we freeze calculation of limits and plot the number of reported deaths for the following day. If this next daily observation also falls below the lower control limit, we conclude that the growth phase has likely ended. We continue to plot subsequent counts of reported deaths on the chart but do not update the limits. Plotting reported deaths from the covid-19 epidemic in Spain on our hybrid chart (Fig. 6) indicates that the pre-growth phase ended on March 10 with a special cause signal triggered by the observation of 23 deaths. I-chart calculations were frozen on March 29 after 20 days of exponential growth in Spain. A special cause below the lower limit on April 3 signaled the end of the growth phase. After an observation below the I-chart lower limit on April 4 signaled the start of phase 3 in Spain, the pattern of reported deaths has continued (as of April 25, 2020) to trend downward.

Figure 6

Example of a chart for a country that has moved past an exponential growth phase.

Hybrid Shewhart chart for reported deaths

Unlike the linear control limits most commonly associated with control charts, the center line and limits for the growth phase on the hybrid Shewhart chart are non-linear (Fig. 6). The trajectory of an epidemic in multiple phases may be more clearly visualized on a logarithmic scale (Fig. 7) because the number of reported deaths spans different orders of magnitude (from tens of deaths to thousands). To return to the original scale with counts of reported deaths (Fig. 6), we exponentiate the logarithmic values for the center line, upper limit and lower limit.

Figure 7

Example of a log-scale chart for a country that has moved past an exponential growth phase.

Our Shewhart charts are updated with each daily report of new covid-19 deaths for international countries as well as US states and territories. Displays are available to the public via an online interactive covid-19 chart application developed by Informing Ecological Design and a comprehensive covid-19 data dashboard hosted by the Institute for Healthcare Improvement [31, 32]. There may be slight differences across these platforms with respect to data sources and the implementation of our method. Our World in Data was the data source used to create the charts seen in this paper [33].

Discussion

This application of Shewhart charts offers at least four distinct opportunities for learning during an epidemic. First, it is a useful method to engage subject matter experts on the ground and to enhance understanding about the extent to which the myriad practice and policy changes implemented by governments and executed by frontline staff are working. Second, the charts provide us with signals to identify when the number of new deaths in a locality (e.g. city, county, state, or country) has begun to grow exponentially. Third, it allows us to use initial exponential growth patterns as a basis to identify when daily deaths have stopped increasing, an indication that the trajectory of reported deaths in an area is entering the flat part of the epidemiologic curve or perhaps beginning to trend downward. Fourth, our approach minimizes the risk and psychological toll that can occur when people are primed by media and public health reports to respond to every data point as if it is a meaningful signal of conditions getting better or worse (when they may not be changing at all).

For example, newspaper headlines from the UK (Fig. 1) portray variation in covid-19 death reports as if each daily number represented a special narrative for significant changes in epidemic conditions. Visualizing these data on our hybrid Shewhart chart (Fig. 8) reveals the changes in covid-19 deaths in the UK during this time frame (March 6–April 15, 2020) to be common cause variation.

Figure 8

Novel hybrid Shewhart chart for daily reported covid-19 deaths in the UK.

Numerous methods are available to model the growth of an epidemic and to predict the peak number of deaths in specific areas, but we are unaware of other methods that detect the first day an area either enters or exits the initial exponential growth phase. Our approach provides a tool to support practitioners and subject matter experts in recognizing conditions associated with different epidemic phases and to inform containment strategies and public health policy decisions. The capability to detect in a geographic area when exponential growth is beginning and ending has significant implications for emergency planning and preparedness. For example, it would be difficult to justify loosening of social distancing procedures while the pattern of daily reported deaths has yet to reveal a meaningful signal of decline.

The next stages of our work on the application of Shewhart charts to the epidemic curve will focus on modeling the plateau around the apex, tracking the descent in the number of reported deaths and monitoring the final stages where deaths are reported at a low, stable level. For example, a C-chart could be useful to rapidly detect if reported deaths begin to increase after epidemic containment measures are relaxed or discontinued. A G-chart for the number of days between reported deaths [1] could also be used as a simple surveillance system to monitor conditions in the post-growth phase.

Our approach to these charts is limited by the validity of data reporting. The variation in reported deaths includes variation in reporting methods and operational definitions across locations and over time [34-36]. Because measurement variability is built into the calculation of control limits, Shewhart’s method is less reactive and thus advantageous in such a context. Issues with the quality and consistency of epidemic data strengthen the argument for involvement of subject matter experts (those with local knowledge or clinical expertise) to guide the interpretation of charts.

Conclusion

During a pandemic, early and reliable signals that the number of reported deaths is waxing or waning may have life and death implications. The science of improvement offers methods to represent and interpret variation that will be essential for successful organizational changes in response to covid-19 [37]. The novel hybrid control chart described in this paper is a practical tool to support learning from reported data during an epidemic or pandemic and approach that is accessible and useful to system leaders and frontline teams. Without Shewhart charts—and, more broadly, a theory of variation—in our epidemiological arsenal, we lack a scientific method to assess in real time the extent to which epidemic conditions are improving. Shewhart charts should become a standard part of how we learn from and visualize data in the context of a pandemic or epidemic.

Appendix: Calculations for Hybrid Shewhart Chart

Phase 1 (pre-growth)—the number of deaths each day is low and stable

To plot epidemic data for a location in a pre-growth phase, we use the C-chart. This chart is useful for counts of incidents within a defined region in which the overall chance of an incident is small [1]. Calculations of control limits for a C-chart are based on the standard error for a Poisson distribution. We used the following process to create a phase 1 C-chart:

Select start date as the day when the data set has first death. Plot the number of deaths each day.

After at least eight total deaths have occurred, calculate the CL as the average of daily counts and the UL, CL + 3* sqrt(CL)] using data from all days since the initial death. After the growth phase begins, this chart is best viewed on the log scale, as with the example of Italy (Fig. 9).

Figure 9

Example of linear and logarithmic hybrid Shewhart charts.

Continue plotting and updating the limits each day, looking for a special cause signal (a point above the UL or eight points in a row above or below the CL). Eight consecutive points below the CL could occur at the beginning of the series. If you reach 20 days in this phase, freeze CL and UL and extend these limits into the future. When a special cause signal occurs, end the C-chart and switch to the growth chart for the second phase. This last data point in the pre-growth phase will become the first data point in the second phase.

Example of linear and logarithmic hybrid Shewhart charts with epidemic phases.

Phase 2 (growth)—daily deaths are trending higher each day.

To plot epidemic data for a location in an exponential growth phase, we use an I-chart (also known as an X-chart or Xmr-chart) for individual measurements of volume counts [1]. The CL calculation is based on the linear regression [1, 30] of the logarithm of reported epidemic deaths (Y) on sequential day counts (X). A moving range, calculated as the difference between consecutive data regression residuals, is used to define control limits using only the common cause variation by screening out individual moving ranges that are inflated by special cause variation:

Use the data value from the last day on the C-chart from phase 1 (the point that triggered the special cause signal) as the first day on the growth I-chart.

For the first 5 days of phase 1, just plot the daily deaths. After five data points are available in phase 2, begin calculating a CL and limits each day. Extend the limits 7 days into the future.

To calculate the CL and limits, transform the counts using the log10 function. The log of 0 is not defined, so 0 values will be interpreted as missing. Adding 0.1 to each data point in the growth phase alleviates this issue.

Calculate the intercept and slope using a linear regression analysis for the log10 data series. This regression line will become the center line for the I-chart based on the regression line.

Develop the limits for the I-chart by calculating the moving range of the residuals and screening the moving ranges to then calculate a revised average moving range (MRbar). Calculate the upper and lower limits for each day as (CL ± 2.66*MRbar). Alternatively, the median of the moving ranges may be used in the calculation [1].

Transform the center line, upper limit and lower limit back to the original count scale using the inverse power transformation. Extend the CL and limits to 7 future days.

Plot on the chart the original counts and the transformed center line and limits. (This chart will approximate a logistic function for the growth period of a death curve.)

Another option with advantages for visualization is to present this chart using a log scale (Fig. 9 with examples of charts for Italy).

Continue to update this chart with new data each day until you reach 20 days in the growth phase. Then freeze the CL and limits and extend them 7 days into the future.

Either a data point below the lower limit or eight consecutive points below the center line are indications that the growth period has ended and this specific geography has reached the peak in the number of reported daily deaths. These special cause signals indicate the possibility that the location may have reached post-growth phase 3. A review of the chart is required to make this decision.

If a special cause signal occurs before the 20-day freeze, plot the point for the next day. If this point also is (or is part of) a special cause signal, there is strong evidence that the growth period has ended.

Phase 3 of the chart (post-growth)—daily reported deaths are flat or dropping.

Phase 3 occurs when an area has reached and/or passed the peak of mortality, after which reported deaths are expected to decline. Continue to plot the number of deaths without updating the limits for phase 1 or phase 2.