Pandemics and methodological developments in epidemiology history.

The crisis spurred by the pandemic of COVID-19 has revealed weaknesses in our epidemiologic methodologic corpus, which scientists are struggling to compensate. This article explores whether this phenomenon is characteristic of pandemics or not. Since the emergence of population-based sciences in the 17th century, we can observe close temporal correlations between the plague and the discovery of population thinking, cholera and population-based group comparisons, tuberculosis and the formalization of cohort studies, the 1918 Great Influenza and the creation of an academic epidemiologic counterpart to the public health service, the HIV/AIDS epidemic, and the formalization of causal inference concepts. The COVID-19 pandemic seems to have promoted the widespread understanding of population thinking both with respect to ways of flattening an epidemic curve and the societal bases of health inequities. If the latter proves true, it will support my hypothesis that pandemics did accelerate profound changes in epidemiologic methods and concepts.

This brief retrospective of catastrophic pandemics since the 17th century suggests that pandemics did accelerate profound changes in epidemiologic methods and concept.

The possible impact of pandemics on epidemiologic methods and concept has not been explored as systematically in the past.

The public at large may have a better understanding of epidemiologic findings if the covid-19 pandemic has promoted the widespread understanding of population thinking both with respect to ways of flattening an epidemic curve as well as the societal bases of health inequities.

Is there a causal link between pandemics and methodological developments in epidemiology? This article explores whether pandemics of the past have been accompanied by major advances in epidemiologic research methods. Pandemics are particular because they threaten the established order, they challenge our way of life and of exerting power, and they appear as a failure of human societies against nature. They create a sense of urgency to find solutions to problems that older methods failed to handle properly. Thus, it is reasonable to wonder whether they also represent watersheds, discrete moments in the refinement and improvement of the methods used in a discipline which, etymologically, is the science of epidemics.

Assuming, as I have proposed elsewhere [[1], [2], [3]], that there is no epidemiologic science before the end of the 17th century, this is a brief retrospective of catastrophic pandemics since the 17th century, that is, the plague in the 17th century, cholera in the 19th century, tuberculosis and influenza in the early 20th century, and HIV/AIDS in the 1980s.

Plague

Recurrent plague outbreaks between the 14th and the 17th centuries disrupted urban life and unsettled civic powers. In London, the pandemics of plague progressively led to the systematic collection of death counts [4]. Quantitative plague reports were introduced in 1518 and expanded to other individual causes of death in 1554 and 1555. Serious episodes of plague in 1592 and 1625 were associated with the first broadside Bills and the regular, weekly handbills with parish-by-parish counts of plague deaths. At some points, these Bills of Mortality allowed for, as shown in Fig. 1 , an organized evacuation of the city when the persistent increase in plague deaths in some parishes indicated that a new outbreak was unfolding.

Fig. 1

Scenes in London during the plague of 1665. Facsimile reproduction from a pictorial broadside of 1665–1666. Some of the scenes show an ordered retreat away from London by water and by land. Wellcome in Creative Commons.

When John Graunt analyzed this continuous series, week after week, for several decades of mortality data, he saw, as expected, that plague deaths waved across time, but he also discovered a phenomenon never reported before: deaths from some causes other than the plague occurred with great constancy [5]. For instance, the number of deaths from “Consumption and Cough,” that is, mostly tuberculosis, that would occur any year in London could be predicted from the number of deaths observed during the previous year: about 2000. Graunt discovered that occurrence of events in populations is predictable and potentially comparable and can therefore be a source of scientific knowledge. The properties of events in populations could not even have been suspected before the 17th century because there were no data to observe them. A comparative health science had remained elusive because the life course of individuals, the only observation available to physicians, is unpredictable and incomparable. Thus, the surveillance system built up to restrain the disruptive consequences of the plague, the catastrophic pandemic of the time, appears to have provided the foundation to the most decisive methodological advance in the history of social and human sciences: the discovery of population thinking.

Cholera

Before the 19th century, cholera had been endemic in certain parts of Asia but remained localized until faster means of transportation and routes made it feasible for sick travelers to survive the trip from, say Bombay, India, to Marseille, France, or from Rotterdam, Netherlands, to New York or Cartagena, Colombia. Since then, cholera spread in waves or pandemics. The second pandemic (1826–1837) reached Europe, North Africa, and North America. The third (1841–59) also reached Central and South America. These pandemics were scary. During the 1831–1832 pandemic, the case fatality ratio in Vienna, Austria, was 49.02% of 4,360 cases [6]. In an official report about the 1832 cholera pandemic in Paris and its surroundings, 48.6% of 4,907 medically certified cholera deaths of all ages occurred in the first 24 hours [6] ([7], p.70). It was speculated that the stench emanating from rotten material caused cholera if inhaled by susceptible persons. Susceptibility was an individual trait, and there was no way of proving it existed. However, John Snow believed there was some kind of still unobserved living thing that could multiply in the water supply and that caused the cholera syndrome when ingested from water or food or from sleeping in infected linen. He conducted a well-known survey—different from his investigation of the outbreak related to the water pump in Broad Street—that allowed him to compare Londoners residing in the same parishes, equally exposed to stench and air pollution, but drinking water of different degrees of cleanliness. Leaden pipe belonging to private companies conveyed water from sections of the Thames, one contaminated and one not, to tanks and then from these tanks to pumps situated in the individual houses ([8], p. 16, 46–47). Thus, the pandemics of cholera, the greatest pandemic of the 19th century, stimulated the first use at a population scale of a comparative study design to explore a causal association. Group comparison has become the other defining trait of epidemiology besides population thinking.

Tuberculosis

Tuberculosis has affected human populations probably since the Neolithic, as the Bacillus is thought to have adapted to humans from domesticated cattle in dwellings shared promiscuously by humans and animals [9]. In 1900, many in public health believed that susceptibility to tuberculosis was hereditary. Persons with tuberculosis in 1900 affluent Western societies did not die consumed by their disease anymore as had been the case until the first half of the 19th century. They could have offspring. Eugenicists feared that, at some point, every human being would become susceptible to tuberculosis, leading to the extinction of the human species. The perception of tuberculosis then was analogous to a pandemic. Of all diseases, it had become the main killer. However, no methods existed to study whether persons with tuberculosis had a survival advantage: the latency between infection and diagnosis was long, and the cross-sectional surveys that had been used in the 19th century, by John Snow, for example, were biased because they comprised only selected survivors. Precisely for this reason, the German physician, Wilhelm Weinberg, invented a new design, referred to today as a cohort study, never used as neatly and at this scale before. Using population registries of Stuttgart, in Germany, he followed up 25,786 children from birth to the age of 20 years, comparing those from families in which a parent had died from tuberculosis with children from families in which the parent had not died from tuberculosis [10]. Children from tuberculous parents lived shorter lives. Tuberculosis conferred no selection advantage and did not threaten the human species with extinction. Weinberg's report is equivalent to a textbook of epidemiology, which described in detail the new design and analytic methods of his study. His review of hundreds of thousands of records, individual follow-up of dozens of children, and complex computations without a calculator was a huge endeavor unlikely to have been deemed worthwhile if it were not for the hope of an enormously important outcome. Comparative cohort studies will turn out to be a key determinant for the achievements of the subsequent chronic disease epidemiologic research.

Influenza 1918

Until 1918, public health was overseen by health officers and civil servants. In April 1918, Wade Hampton Frost was still working for the U.S. Public Health Service when Surgeon General Rupert Blue made him responsible for his Influenza Task Force. Frost conducted a massive epidemiological house-to-house survey in 18 U.S. communities and established the prevalence of infection in the population in the fall of 1918. Contemporaneously, the Johns Hopkins University School of Hygiene and Public Health (SHPH) admitted its first students on October 1, 1918. William Henry Welch, who founded the SHPH, approached Frost in the summer of 1919 and made him Chair of the newly formed epidemiology department in September 1919. This was the first such department ever. Frost later became the first “Professor of epidemiology” in the United States. I am not aware of documents establishing the connection between the pandemic and the academization of epidemiology, and, as an epidemiologist, Frost is primarily remembered for his work on tuberculosis, and not for his work on influenza. Nonetheless, both Welch and Frost had been actively involved in the response to the 1918 flu and for the years 1919–1920 all of Wade Hampton Frost's publications [[11], [12], [13], [14], [15]] are about influenza. Thus, with hindsight, we can appreciate that the 1918 pandemic may have spurred the emergence of an academic parallel to the public service dedicated to public health and epidemiology. The later development of an epidemiologic corpus of methods and concepts mostly came from these schools of public health and academic epidemiology [16].

HIV-AIDS

The HIV pandemic unsettled epidemiology at first because epidemiologists had become used in the 20th century to studying chronic diseases and were taken by surprise by this resurgence of infectious scourges. Since World War II, the epidemiologic corpus of study designs had been substantially refined but the tools to study causes of disease were restricted to fixed exposures and outcomes well defined in time (e.g., to tobacco and lung cancer) [17]. The epidemiologic usage of the causal concepts of confounding, interaction, and mediation was still in their infancy [18]. When the first cases of Kaposi's sarcoma were reported in 1981, epidemiologists focused on noninfectious exposures such as recreational drugs, which fitted the tobacco-lung cancer model [19,20]. The viral nature of the disease was established by virologists in laboratories [21]. A major question from then on was the assessment of the efficacy of HIV/AIDS time-varying treatments. Contemporaneously, a fresh approach to causality, now known as the potential outcome framework and the corresponding causal models, was occurring, inspired by work in the analysis of randomized controlled trials [[22], [23], [24]]. The first application of these developments in epidemiology was unrelated to the HIV/AIDS epidemic [25]. They looked appropriate to handle the “healthy worker effect,” an up-to-then intractable methodological issue in occupational epidemiology: when studying the effect of an occupational exposure on a disabling illness, sick workers, who terminate employment early, are at an increased risk of death although no further exposed, resulting in a biased assessment of the association among the healthier workers remaining in the study. These causal models might have remained of limited usage in epidemiology for some time if it had not been shown that they were also appropriate for estimating the effect of time-varying treatment on the time to clinical AIDS among HIV-infected subjects: the fact that sicker patients were more likely to both be treated and die, an issue referred to as treatment by indication, generated selection and confounding issues analogous to those of the healthy worker effect [26]. The watershed happened with James Robins's application of “causal models” to “the Analysis of Randomized and Nonrandomized AIDS Treatment Trials Using A New Approach to Causal Inference in Longitudinal Studies” [27]. These methods are now used routinely in health and social science research, including epidemiology, and are textbook matter [28]. Thus, the deadly HIV/AIDS pandemic appears to have accelerated the integration of a new conceptual framework that resulted in the refinement of causality-related concepts in epidemiology and more generally in human and social sciences.

Discussion

The cases of the plague, cholera, tuberculosis, influenza, and HIV/AIDS seem to indicate that these pandemics have accelerated major methodological developments in epidemiology. These changes would have occurred but at a later time and differently. However, one may argue that pandemics and development of methods are in reality two separate, independent narratives that, by chance, appear to be chronologically associated.

For example, the recollection of specific methodological developments may, in retrospect, make us associate them with specific pandemics that occurred at about the same time. However, this type of reverse causality would not be true for the 1918 influenza pandemic. The great influenza long appeared, as Major Greenwood put it, “epidemiology's crux” because of the lack of major work or influential epidemiologic work that emerged from it ([29], p.137). I am not aware of anyone connecting the creation of the school of public health at The Johns Hopkins University to the 1918 influenza [30], but today, with hindsight, the connection seems very likely, as I have tried to show previously, and its historical implications were huge: the creation of an academic arm of epidemiology established the foundations for the later development of a corpus of formal and rigorous methods for the conduct of epidemiologic research.

In favor of the causal link is the difficulty of finding counterexamples of minor epidemics that led to methodological breakthroughs of the same magnitude as those described previously for pandemics. In the 18th century, the Royal Society innovated in designing a comparative study of the mortality associated with natural smallpox infection and that provoked by variolation [31]. It involved a subtle analysis of the bills of mortality and an intercontinental physician survey. Natural smallpox was ten times more lethal than variolation, a finding leading to a wave of smallpox inoculation, at least in the United Kingdom. In his 1747 nonrandomized trial on board, a naval warship doctor James Lind allegedly compared 6 treatments, a single one of which (citrus fruits) was effective in curing scurvy [32]. But these 18th century methodological breakthroughs were minor compared with the discovery of population thinking which gave birth to all population-based sciences we know today, such as demography, statistics, sociology, evolutionary genetics, and so on. In the 19th century, the researches of Pierre-Charles-Alexandre Louis [33] and Ignaz Semmelweis [34] were innovative in their own ways, but their long-term methodological legacy is not comparable to that of Snow. In the 20th century, the refinement of the case-control study design was induced by etiologic research on cancer [35] and cardiovascular diseases [36], which were on the rise and progressively replacing acute infectious diseases and tuberculosis as public health priorities. The epidemic of pellagra also led to methodologically innovative work by Joseph Goldberger [37]. But none of this work rivals the contribution of Wilhelm Weinberg [38], which opened the way to the study of chronic disorders, including the elucidation of the health effects of smoking [39], by rigorously establishing the methodological basis of cohort studies and survival analysis. Finally, no disease or epidemic has shaken the conceptual foundations of causal inference in epidemiology as much as the HIV/AIDS epidemic.

It may be that the critical conditions of a pandemic stimulate an urgent fresh look at older methods that failed to prevent the disasters. Above all, the common trait of pandemics is the unusual, immediate, surrounding, unavoidable presence of death, in neighbors, relatives, and friends. Similar situations did not occur in other epidemics, such as smallpox in the 18th century, any of the acute infectious diseases such as diphtheria, measles, and so on in the 19th century [40], or the cancer and cardiovascular epidemics of the 20th century. These were all real epidemics, with their toll of deaths and suffering, but they did not halt movement in society and make people urgently reflect on the limits and the dead ends of the available methods.

However, my argument would be weakened if one could argue that the link with pandemics is not specific and that innovative methods and concepts in epidemiology have followed other types of happenings. My thesis could also be disproved by COVID-19, if the pandemic does not shatter an important dimension of epidemiologic methods. I would speculate that its consequences will be comparable to those of the 1918 influenza. We have now schools of public health all over the world, and academic epidemiology is thriving. Still, population thinking remains esoteric outside of academia and public health. This may be changing. Never in the past have we collectively considered what an epidemic curve is, what herd immunity is, and how we can act together to affect the progression of the epidemic. Similarly, Covid-19 may have accelerated the understanding that health inequities can be better studied and corrected at a population rather than at an individual level. Population thinking may become mainstream and, if it does, this will have huge consequences on the society we will build when the ordeal is over.