A hybrid Shewhart chart for visualizing and learning from epidemic data.

OBJECTIVE: As the globe endures the coronavirus disease 2019 (COVID-19) pandemic, we developed a hybrid Shewhart chart to visualize and learn from day-to-day variation in a variety of epidemic measures over time. CONTEXT: Countries and localities have reported daily data representing the progression of COVID-19 conditions and measures, with trajectories mapping along the classic epidemiological curve. Settings have experienced different patterns over time within the epidemic: pre-exponential growth, exponential growth, plateau or descent and/ or low counts after descent. Decision-makers need a reliable method for rapidly detecting transitions in epidemic measures, informing curtailment strategies and learning from actions taken.
METHODS: We designed a hybrid Shewhart chart describing four 'epochs' ((i) pre-exponential growth, (ii) exponential growth, (iii) plateau or descent and (iv) stability after descent) of the COVID-19 epidemic that emerged by incorporating a C-chart and I-chart with a log-regression slope. We developed and tested the hybrid chart using international data at the country, regional and local levels with measures including cases, hospitalizations and deaths with guidance from local subject-matter experts.
RESULTS: The hybrid chart effectively and rapidly signaled the occurrence of each of the four epochs. In the UK, a signal that COVID-19 deaths moved into exponential growth occurred on 17 September, 44 days prior to the announcement of a large-scale lockdown. In California, USA, signals detecting increases in COVID-19 cases at the county level were detected in December 2020 prior to statewide stay-at-home orders, with declines detected in the weeks following. In Ireland, in December 2020, the hybrid chart detected increases in COVID-19 cases, followed by hospitalizations, intensive care unit admissions and deaths. Following national restrictions in late December, a similar sequence of reductions in the measures was detected in January and February 2021.
CONCLUSIONS: The Shewhart hybrid chart is a valuable tool for rapidly generating learning from data in close to real time. When used by subject-matter experts, the chart can guide actionable policy and local decision-making earlier than when action is likely to be taken without it.

Introduction

During the COVID-19 pandemic, many models and data dashboards have emerged [1, 2]. Few of these models have focused on detecting early signs of trouble and improvement for epidemic measures, compromising local and national decision makers’ ability to detect meaningful changes from noise, risking poor decisions with real consequences [3, 4]. A theory and method are needed to differentiate meaningful signals—also known as ‘special causes’—from natural fluctuation associated with any measurement. This is especially important now against a backdrop of increasing public expectations and hopes that we are approaching the end of the pandemic despite the fact some geographies are experiencing a record number of cases and deaths linked to COVID-19 variants [5]. By what method will we know—as quickly as possible—if areas are improving or beginning to see a resurgence? For nearly 100 years, the quality improvement field has benefited from Shewhart’s theory of variation (Box 1). Shewhart described two types of variations in data: common causes—inherent in the system over time, similar to random day-to-day variation—and special causes that are not part of the regular system and may signal that something has changed in a meaningful way, inventing the ‘Shewhart chart’ to visualize this theory. In the first two months of the COVID-19 pandemic, we developed a novel Shewhart chart for daily deaths, focused primarily on identifying the day a ‘special cause’ signal indicated rapid growth in reported deaths [6]. As the pandemic unfolded, aligning with the classic epidemiologic curve, some locations, such as Italy and Spain, moved from exponential growth to a ‘post-growth’ time period (epoch) reporting reductions in daily deaths [7]. After a period of declining deaths, China in the spring of 2020 entered a stable period of zero daily deaths [8]. Moreover, a number of locations, including South Korea and several US states (for example, Alabama and Maine) reported deaths that remained relatively low and stable over a few weeks at a time, with no evidence of exponential growth [9].

Background to the Shewhart chart

The science of improvement is rooted in the appreciation of a system and understanding the nature of variation in that system. Walter Shewhart theorized that variation has two potential origins: common causes—inherent in the system over time, affecting everyone in the system and all system outcomes—and special causes that are not part of the regular system [23]. Special cause variation comes from an identifiable circumstances or cause; special causes may be anomalous, or they may be indicative of changes to the regular system and the need to reassess common cause variation. Shewhart operationalized this theory with the ‘control chart’ [24]. His method has been applied in diverse industries and extensively in health-care systems [25-27]. Shewhart’s method has had limited epidemiological applications, with much of previous research in this area limited to modeling the spread of hospital-acquired infections [28]. Shewhart charts have been used for public health surveillance and have recently been applied in the context of the COVID-19 pandemic [1, 29].

Our previous methodology stopped at describing how to develop a Shewhart chart across the full range of the epidemiologic curve because many geographies at the time had not yet moved through the entire range. The progression of the COVID-19 pandemic provides an opportunity to extend the methodology, equipping policy and decision makers with practical feedback signaling whether and when their efforts to contain the pandemic are succeeding or not. Specifically, to extend the methodology to provide timely signals of when rapid growth is ending, when COVID-19 events start to decline, and to detect potential new periods of rapid growth. Based on our study across multiple geographies, we defined four distinct epochs, aligning with the classic epidemiologic curve: (i) pre-exponential growth; (ii) exponential growth; (iii) plateau or descent and (iv) stability after descent (Figure 1). Our earlier description of a hybrid chart aligned with the first two of these epochs [5]. In this paper, we describe our approach to developing a hybrid Shewhart chart based on a four-epoch framework to characterize the trajectory of an epidemic (Figure 1). We outline the method and approach for our hybrid Shewhart chart, summarizing criteria for transitioning from one epoch to another within a geographic region. We describe application of the chart with publicly available COVID-19 pandemic data including daily reported cases, hospitalizations and deaths [10, 11, 12].

Figure 1

Hypothetical example of an epidemiological curve for events in four epochs.

Methods

Using the hybrid Shewhart chart we identified four epochs describing time periods associated with the trajectory of an epidemic: (i) pre-exponential growth; (ii) exponential growth; (iii) plateau or descent and (iv) stability after descent (Figure 1 and Table 1). In this section, using illustrative examples from publicly available data, we define these epochs [8, 11, 12]. We describe criteria for the start and end of each epoch, as well as specify situations when signals of special cause variation indicate that chart limits should be re-calculated within the same epoch. We use ‘event’ to represent observations from epidemic measures such as cases, deaths or hospitalizations.

Table 1

Epoch chart type and specifications and criteria for entering an epoch, changing phase within an epoch and leaving an epoch

	Shewhart chart type and specifications	When to enter the epoch?	When to change phase within an epoch?	When to leave the epoch?
Epoch 1: Pre-exponential Growth
	C-chart: Centerline and upper limits are calculated from the mean number of events up to the most recent data point. Limits are updated up to the latest data point	At the start of the epidemic when: At least eight cumulative events have been reported and At least five days with one or more events	At least one of: Two consecutive points above the upper limit Eight consecutive points above the centerline Eight consecutive points below the centerline	To Epoch 2 when: A phase change has been signaled and subsequent data indicate exponential growth
Epoch 2: Exponential Growth
	I-chart: Sloping centerline and limits are calculated from a regression of the log-transformed events over time Transform center line and limits back to the original scale Freeze the limits after 21 days	From Epoch 1, 3 or four when: A phase change has signaled and The current data indicate exponential growth	At least one of: Two consecutive points above the upper limit Two consecutive points below the lower limit Eight consecutive points above or the centerline Eight consecutive points below the centerline	To Epoch 3 when:A phase change has been signaled by at least one of Eight points below the centerline Two consecutive points below the lover limit. and at least one of: The current data indicate no exponential growth (plateau) The current data indicate a downward slope in events (descent)
Epoch 3: Plateau or descent
	I-chart: Sloping centerline and limits are calculated from a regression of the log-transformed events over time Transform center line and limits back to the original scale Freeze the limits after 21 days	From Epoch 2 when: A phase change has signaled and at least one of: The current data indicate no exponential growth (plateau) The current data indicate a downward slope in events (descent)	At least one of: Two consecutive points above the upper limit Two consecutive points below the lower limit Eight consecutive points above or the centerline Eight consecutive points below the centerline	To Epoch 2 when: A phase change has been signaled by eight points above the centerline or by two consecutive points below the upper limit. and The current data indicate exponential growth To Epoch 4 when: The lower limit <2 for reported deaths (proportional value for other events)
Epoch 4: Stability after descent
	C-chart: Centerline and upper limits are calculated from the mean number of events up to the most recent data point. Limits are updated up to the latest data point	From Epoch 3 when: The lower limit <2 for reported deaths (proportional value for other events)	At least one of: Two consecutive points above the upper limit Eight consecutive points above or below the centerline	To Epoch 2 when: A phase change has been signaled and the current data indicate exponential growth

Epoch 1: Pre-exponential growth

Epoch 1 begins with the first reported daily event. In Epoch 1, daily counts remain relatively low and stable with no evidence of exponential growth. During Epoch 1, some phases indicative of step increases or decreases in daily events can occur. Epoch 1 ends when rapid growth in events starts to occur and the chart moves into Epoch 2. As described in Table 1, daily events in Epoch 1 are displayed using a C-chart with a horizontal centerline calculated from the average number of events. Example: Applying the method described in Table 1, Figure 2a illustrates Epoch 1, displaying daily reported COVID-19 deaths in South Korea during February, when the first death was reported, to June 2020. A C-chart with a centerline of 2.9 deaths per day revealed a signal of special cause variation on March 19, the first of 2 days with reported deaths above the upper chart limit [13]. In this new phase, no exponential growth was detected. Reported deaths in South Korea continued in Epoch 1, with limits calculated around a centerline of 5.4 deaths per day. On April 22, another step change was indicated by 8 consecutive days with daily deaths below the centerline. New Epoch 1 limits were calculated around the new centerline of one death per day as of June 2020. The South Korea chart demonstrates that there can be multiple phase shifts, signals of things getting worse and better, within the same epoch.

Figure 2

COVID-19 reported deaths for each of the four epochs. (a) Daily COVID-19 reported deaths illustrating pre-exponential growth (Epoch 1). (b) Daily COVID-19 reported deaths illustrating exponential growth (Epoch 2). (c) Daily COVID-19 reported deaths illustrating a plateau or descent (Epoch 3). (d) Daily COVID-19 reported deaths illustrating stability after descent (Epoch 4).

Epoch 2: Exponential growth

Epoch 2, often alarming to the viewer and those experiencing the epidemic, represents the period where daily events begin to grow rapidly and exponentially. During Epoch 2, phases may occur when the rate of exponential growth accelerates or decelerates. Epoch 2 ends when events start to level off (plateau) or decline. Daily events in Epoch 2 are displayed using an I-chart with a centerline based on the slope of the log-transformed events over time (Table 1).

Example: Figure 2a illustrates the Epoch 2 criteria detailed in Table 1 using daily reported COVID-19 deaths from Peru during March to May 2020. Throughout March, Peru was in Epoch 1 until on April 2 a special cause was detected signaling the beginning of a new phase. A regression line fit through the subsequent days of the log of deaths found a significant positive slope, signaling the start of Epoch 2. I-chart limits were calculated using data for the subsequent 21 days, and then frozen [8]. By May 9, eight consecutive days with daily deaths below the centerline signaled a deceleration of exponential growth and a second phase in Epoch 2. The regression line for this new phase remained consistent with exponential growth; new I-chart limits were calculated based on data from the subsequent 21 days, and frozen again on May 28. In other words, since growth was still exponential, albeit at a slower rate, Peru remained in Epoch 2.

Epoch 3: Plateau or descent

Epoch 3 represents the period when daily events stop increasing exponentially and start to plateau or descend. Epoch 3 can end as daily values start to return to pre-exponential growth values. Troublingly, Epoch 3 can also end with a return to exponential growth (Epoch 2)—a sign the pandemic is taking a turn for the worse again. For Epoch 3, a plateau is shown using an I-chart with a centerline based on the average (log-transformed) daily events. Mirroring Epoch 2, a descent is plotted once a significant downward slope has been detected, using an I-chart with a centerline based on the slope of the log-transformed events over time (Table 1).

Data shown in Figure 2c, from March to May 2020 for the UK illustrates the transition into and within Epoch 3, showing both a plateau and then decline in daily events. Throughout March and early April, daily deaths in the UK increased exponentially (Epoch 2). On April 9, the first of two consecutive days with daily deaths less than the lower limit signaled a new phase. A regression line from the log of daily deaths in this new phase did not differ significantly from zero, indicating a plateau and the start of Epoch 3. A horizontal centerline and I-chart limits using the mean number of daily deaths were calculated. On May 6, eight consecutive days below the horizontal centerline signaled a new phase within Epoch 3. A regression line fitted to the log of daily deaths in this phase had a significant negative slope indicating a decline, and a downward centerline and I-chart limits were calculated. No special cause was indicated during the remainder of May 2020.

Epoch 4: Stability after descent

Epoch 4 represents a period similar to Epoch 1 pre-exponential growth, when a descent in daily events has occurred and is low and stable. Epoch 4 can end if further signs of trouble are detected and there is a return to exponential growth (Epoch 2).

Example: To illustrate Epoch 4, Figure 2d shows daily COVID-19 reported deaths in Italy during June to August 2020. Heading into July, deaths in Italy were in Epoch 3 and declining daily. On July 12, the lower limit dropped below two, signaling a return to relatively low deaths and the start of Epoch 4 (Table 1). The resulting C-chart suggests that daily deaths remained in Epoch 4 beyond the end of August. The limit of two deaths was empirically based on reviewing a number of examples. For other events, this limit should be based on the expected ratio to deaths. For example, the average number of cases per death is about 60, so the lower limit for cases should be less than 100 (rounded from 120).

Adjusting for day of week

Variation in reporting of COVID-19 measures occurs in many settings by day of week, with fewer events reported on a weekend than during the week. This variation can interfere with reliable detection of special cause variation. For the charts described in this paper, we adjusted for day-of-week variation by using the median difference between the observed and centerline values within Epoch 2 and Epoch 3. We then applied the hybrid Shewhart chart method to these weekday-adjusted event counts.

Results

Since April 2020, versions of these hybrid charts and related code have been shared on a number of different platforms [9, 14, 15]. The following examples of applications of the hybrid chart illustrate the four-epoch framework and its effectiveness in describing and predicting the epidemic in specific geographies.

UK: detecting signs of trouble in COVID-19 deaths

In the UK, in late summer, daily reported deaths were in Epoch 4, averaging 10.5 per day, until on September 17, special cause signaled trouble (Figure 3). After September 17 deaths began increasing exponentially indicating a return to Epoch 2. Moreover, another special cause signaled further trouble on October 18 as the exponential increase in adjusted daily deaths accelerated. National action within England, in the form of a 4-week lockdown was finally announced by the Prime Minister on 31 October, 44 days after the first signal in this hybrid Shewhart chart [16]. Subsequently, on November 12, a special cause signal suggested that the rapid increase in deaths was slowing, moving into Epoch 3.

Figure 3

UK adjusted daily COVID-19 reported deaths.

This example shows how the hybrid chart can be used by the public as well as policy and decision makers to provide early signals of trouble as well as more optimistic signals that action to curtail the pandemic is having an impact.

California, USA: learning from statewide and county-level COVID-19 cases

Since the first reported case on 26 January 2020, adjusted daily reported cases in the state of California have moved through multiple epochs. Figure 4 shows how cases in California entered exponential growth during February, May and October during 2020. The chart detected signals of plateau and decline in daily cases (Epoch 3), without reaching stability after descent (Epoch 4).

Figure 4

California: adjusted daily COVID-19 reported cases.

Daily cases may also be viewed at a county level. Figure 5 shows how visualizing daily reported COVID-19 cases per 1000 population is useful for learning from variation in the incidence of COVID-19 across a region. On 3 December 2020, California announced a stay-at-home order [17]. This order came after hybrid Shewhart charts detected increases in daily reported cases: 11 days after a signal in Fresno County, 16 days after a signal in Los Angeles County, 30 days after a signal in Sacramento County and 10 days after a signal in San Francisco County. Although there was variation in daily population-adjusted cases, special cause signals indicated that cases had declined or plateaued 53 days following the stay-at-home order in Fresno County, after 22 days in Los Angeles County, after 7 days in Sacramento County and after 26 days in San Francisco County.

Figure 5

COVID-19 reported cases in Fresno, Los Angeles, Sacramento and San Francisco counties, California. (a) Fresno: adjusted daily COVID-19 reported cases. (b) Los Angeles: adjusted daily COVID-19 reported cases. (c) Sacramento: adjusted daily COVID-19 reported cases. (d) San Francisco: adjusted daily COVID-19 reported cases.

This example illustrates how the hybrid charts can be used as a guide to rapid action in large geographies and smaller, local geographies, such as US counties. With COVID-19 outbreaks occurring within communities, it is important that local decision makers have a tool they can use to rapidly detect whether or not events are increasing and to know if their efforts are resulting in improvement, or not.

Ireland: learning from a dashboard of four related epidemic measures

Figure 6 showing data from December 2020 to March 2021 illustrates how public health officials in Ireland are using the hybrid charts to learn from COVID-19 cases, hospitalizations, intensive care unit (ICU) admissions and deaths [12]. The charts detected trouble with special cause signals of increasing cases on December 18, followed by increases in reported hospitalizations (December 27), ICU admissions (December 29) and deaths (January 3).

Figure 6

COVID-19 reported cases, hospitalizations, ICU admissions and deaths in Ireland. (a) Daily COVID-19 reported cases. (b) Daily COVID-19 reported deaths and hospitalizations. (c) Daily COVID-19 reported ICU admissions. (d) Daily COVID-19 reported deaths.

Following the Irish government announcing stricter restrictions on December 24 and additional restrictions on December 30, the charts detected a decline in cases (Epoch 3) from January 11, followed by a decline in hospitalizations (January 22) as well as reductions in ICU admissions (February 1) and deaths (January 31) [18]. Characterizing variation in these four epidemic measures with respect to changes across four epochs is informative to leaders and the public. The charts detected signs of trouble sequentially in measures representing different levels of severity in epidemic conditions: from cases, to more serious hospitalizations and ICU admissions, to measures of mortality [19]. Moreover, this example illustrates the lag time for potential restrictions aimed at reducing cases to first reduce hospitalizations and ICU admissions before finally impacting deaths.

Consequently, leaders and policy makers can use signals from these sequential charts to obtain earlier signals of when the pandemic may be changing. For example, they may use changes in cases or hospitalizations as an earlier signal that changes in even more serious events such as ICU admissions and deaths may occur soon after. This can facilitate earlier decisions on what actions to take, such as increasing or easing lockdowns and mask mandates. Additionally, leaders and policy makers can use these sequential charts to explore whether there are changes in the lag time from a signal in cases to a signal in deaths, where a decrease in the lag time may indicate the arrival of a more severe and transmissible COVID-19 variant, and an increase in the lag time may indicate the positive impact of population-level vaccinations.

The big picture: learning from COVID-19 deaths across the globe

The hybrid chart is designed for use within a specific geographic region. It is also insightful to display daily deaths in aggregate (Figure 7) from all world regions and countries reporting daily epidemic data. While the global chart is not actionable for local or regional leaders at this level of aggregation, it could be used to inform international decision-making and be incorporated into the World Health Organization COVID-19 dashboard [20].

Figure 7

Worldwide COVID-19 Reported Deaths.

Discussion

Statement of principal findings

This paper describes a novel hybrid Shewhart chart methodology. Application of this chart provides opportunities to support learning, action and decision-making at a local level during a pandemic. The use of four epochs to describe the trajectory of epidemic measures offers a simple framework to communicate epidemic conditions to the public and other decision makers. We have shown how the Shewhart chart methodology can be used with a variety of pandemic measures, including cases, hospitalizations, ICU admissions and deaths. These measures may be plotted as counts or as population-adjusted rates. With multiple case study examples from across areas of the world, we demonstrated that the hybrid charts detect special cause variation and identify specific dates when changes in epochs occur, as well as signal transitions of phases within an epoch. Absent from many projection models, these specific dates can portend signs of trouble in the course of the pandemic when leaders and decision makers should consider taking action. For example, the UK chart signaled trouble (an increase in epidemic measures) more than 6 weeks before the government implemented restrictions [21]. Additionally, dates associated with special cause variation may also signal situations in which public health measures and other local government actions have been followed by improvement in epidemic measures. For example, in California, a special cause signal in Los Angeles County indicated that COVID-19 cases had started to plateau 3 weeks after the state announced a stay-at-home order. The charts detected a decline in cases 1 month later.

Strengths and limitations

The strength of the approach outlined here is that it is the only model of COVID-19 events that applies a temporal theory variation that enables the detection of meaningful changes, good or bad, almost to the day. This methodology is dependent on the availability of consistent and reliable data. Variation in reporting by day of week can be problematic. However, our approach for weekday-adjusted values does not apply in all contexts. For example, some settings do not report epidemic data at all on a Saturday or a Sunday. In other locations, reporting occurs only once or twice a week and reporting days may vary across weeks. To account for systematic variation from diosyncratic data reporting, optimal application of the hybrid charts will involve local subject-matter experts with insight into how data are reported. Additionally, a number of settings have reported ‘astronomical’ high values and/or implausibly low values of epidemic measures on one or more particular dates. For example, many countries reported zero cases and deaths on Christmas Day (25 December 2020). Many US states reported zero values on the Thanksgiving holiday (23 November 2020). In other settings, unexpectedly large values were reported on a single date; for example, China reported 1290 deaths on 17 April 2020. Their previous high had been 473 deaths. Yet, on April 16 and 18, China reported zero deaths. Local subject-matter experts should be involved in such situations to provide guidance on why extreme or unusual values may be reported on a given day and the extent to which it would be appropriate to include them, or not, in calculations for the hybrid chart.

Interpretation within the context of the wider literature

A recent review of 158 public web-based COVID-19 dashboards identified seven ‘actionability’ features for adoption by dashboard developers, including reporting data over time as well as introduction of key infection control measures to facilitate an understanding of their effects [22]. Our annotated hybrid charts for reported COVID-19 data meet these recommendations and go further by applying a formal theory of variation. In the wider pandemic field, models are often constructed with the intention of providing forecasts of the future spread and consequences of the disease. Often, models featuring many parameters are constructed, based on data from settings where a pandemic has occurred and then applied to new settings in order to forecast the consequences of the pandemic. These models often change over time, producing different estimates of likely consequences on a daily basis [1-3]. They can be helpful to decision makers as a way to inform the potential impact of their choices. However, unlike the hybrid Shewhart charts described in this paper, they do not readily provide an indication of when the pandemic measure of interest has signaled a change in trajectory in real time. Moreover, such pandemic modeling may be out of the scope of many local public health decisions makers.

Implications for policy, practice and research

Shewhart charts should be a standard tool to learn from variation in data during an epidemic. Medical professionals, improvement leaders, health officials and the public could use this chart with reported epidemic measures such as cases, testing rates, hospitalizations, intubations and deaths to rapidly detect meaningful changes over time. This is especially important given the presence of COVID-19 variants that could lead to a resurgence.

Conclusion

Conventional types of Shewhart charts are not sufficient to model data series that follow a classic epidemic curve with multiple epochs. Our hybrid chart is an appropriate and flexible method to meet this challenge.

Data for the number of reported COVID-19 cases and deaths at a country level	Our World In Data: https://www.quantamagazine.org/the-hard-lessons-of-modeling-the-coronavirus-pandemic-20210128/
Data for the number of reported COVID-19 cases and cases by states within the USA	New York times COVID-19 Github: https://github.com/nytimes/covid-19-data
Data for the number of reported COVID-19 cases, hospitalizations, ICU admissions and deaths for Ireland	Covid Statistics Profile HPSC Ireland Open Data https://data.gov.ie/dataset/covidstatisticsprofilehpscirelandopendata1