Two cheers for a small giant? Why we need better ways of seeing data: A commentary on: 'Rising morbidity and mortality in midlife among White non-Hispanic Americans in the 21st century'.

Introduction

A recent Proceedings of the National Academy of Sciences (PNAS) article by Case and Deaton generated a lot of attention outside the journal’s usual readership. According to the social media tracker website Altmetric, by 30 December 2015 the paper was picked up by 84 news outlets and 42 blogs, and had been tweeted around 600 times, all in around a month. Altmetric rates on that date placed it as the 11th most impactful output of 37 873 PNAS items evaluated, placing it in the top 0.03% of all papers published in the journal.

The title and abstract of the article, ‘Rising morbidity and mortality in midlife among white non-Hispanic Americans in the 21st century’, was probably important factors in the mainstream media attention. Though it is perhaps not news that, in the USA, life has never been easy if you are African American, Hispanic or a young adult, the finding that life may not be so long for the White and the middle-aged perhaps came as more of a surprise. The Washington Post called the findings ‘startling’, Public Broadcasting Service called them ‘stunning’ and Al Jezeera America called them ‘devastating’.

In parallel with traditional media, the paper also drew interest within the new media of Facebook and bloggers. And it was from the new media rather than the old, and in particular from the personal blog of statistician Andrew Gelman, that some of the most insightful commentaries and critiques of the paper emerged.

Blogging and the sunlight of statistical scrutiny

In a series of seven blog posts, published between 6 and 18 November 2015, Gelman shone an increasingly sharp light, narrowing from a flashlight to a laser, on the methods, findings and claims made by Case and Deaton. Responding quickly to comments and suggestions from other bloggers, as well as initially helpful responses to queries from Case and Deaton themselves, Gelman was able to carefully document, describe and correct for a methodological flaw in Case and Deaton’s paper. The flaw was severe enough to require one of the central claims of the paper to be modified, but simple enough that peer review and editorial oversight ought to have caught the problem earlier.

Central to the flaw identified by Gelman is aggregation bias: aggregating not just the death counts and population counts of males and females into a single population ‘pool’ (even though males and females tend to have quite different mortality and morbidity profiles), but also pooling all individuals within the age range 45–54 years of age. At each single year of age within this 10-year age stratum, the underlying risk of dying within the next year increases by approximately 8%, something which is normally accounted for (somewhat) more fully by dividing this age group into two during age standardization. These compounded increases in the risk of dying with each additional year of age are one of the earliest and most consistent findings produced by demographers, and give rise to the log-linear relationship long known to exist between mortality risk and age, for all people from around their mid-30s onwards. They form the right-hand side of the famous ‘bathtub curve’ produced by plotting log(risk of dying within the next year) against age from birth onwards, within any given year, and for almost any cohort.,

An implication of this underlying mortality risk profile is that death rates within the 10-year age stratum can increase through change in age composition alone. With the advice of a blog commenter, Gelman identified that the average age of White non-Hispanics within the 45–54 year age range had indeed increased, by more than half a year of age from around 1989 to 2013. Having identified changing age composition as creating a risk of bias, Gelman then corrected for it, estimating that the bias was responsible for around half of the reported increase in all-cause mortality rates. As well as quantitatively reducing the reported mortality rate increase, which had captured so much media attention, the correction also qualitatively changed the findings in two ways: first, male and female trends were no longer similar enough to be meaningfully grouped together, with still-worsening mortality trends for women over that period but a reversal in these adverse trends for men; second, if males and females are to be combined, the reported continuously worsening trend in death rates from 1999 onwards would need to be amended to describe a stabilization from around 2005, which may now even be reversing.

A small giant?

Case and Deaton’s paper is an important case study in the complex relationship and relative strengths and weaknesses of traditional academic journal publishing, mainstream old media and specialist new media. In this case, academic blogging provided a degree of methodological scrutiny that was missing from the PNAS ‘contributed submission’ route through which Case and Deaton’s paper entered a high-impact academic journal; their findings were then simplified, publicized and amplified by mainstream media. It seems likely the strong, clear, unequivocal message of the paper—things are getting worse for middle-aged White, non-Hispanic Americans—was both a reason for the media attention and partly an artefact of failing to correct for a statistical risk of bias known since at least the 19th century.,

Nevertheless, Case and Deaton’s paper has had ripple effects, as scientific papers should. Isaac Newton said that we see further by virtue of standing on the shoulder of giants. Perhaps the Case and Deaton phenomenon—the paper, the mainstream media attention, the careful social media scrutiny, the methodological flaw and later an apparent reluctance or slowness in acknowledging this flaw—is something of a small giant: something that has had a towering impact, yet is methodologically easy to climb onto and see beyond.

Like many others who have been alerted to Case and Deaton’s research through the mainstream media tidal waves, their research has inspired us to look carefully at extant data on all-cause and cause-specific mortality in the USA, and to look at these data in ways that avoid the aggregation bias, and more generally the selective focus, which partially impaired Case and Deaton’s claims. Like Gelman, we were therefore inspired by Case and Deaton’s research despite and because of its flaws. Academic blogging, together with the free availability of the data used by Case and Deaton, has therefore made it easier for a momentum, generated by both methodological shortcomings and substantive public health engagement, to be maintained and redirected. Following the newspaper articles discussing substance and blogs describing methodology, has been a flurry of e-mails, phone calls, coffee-room conversations and other activities from which new academic outputs emerge. There may have been substantial opportunity costs imposed by Case and Deaton’s paper on both academic researchers and mainstream media, not least that the attention paid by both communities to the worsening health trajectory of middle-aged White non-Hispanics (WNHs) as a result of the paper may have directed focus away from both positive and negative health trends in other age groups and ethnic groups. However, we hope the eventual outcome of such attention will be a net gain rather than net loss to both epidemiologists and the public at large.

Lexis surfaces: a better way of looking

The attention generated by the Case and Deaton paper has encouraged further analyses and visualizations of their source data. Kolata and Cohen, for example, explored trends in younger age groups, and Gelman looked at mortality trends for ages in single years, indexed against 1999 levels., But in both cases, simple line graphs or lattices of line graphs were used, limiting the amount of information that can be conveyed in a single figure. Using the same database used by Case and Deaton, the Center for Disease Control WONDER database, but extracting data by age in single years and for both sexes and each of the three ethnic groups defined in their paper, we were able to visualize more than 40 000 individual sex-, ethnicity-, age- and year-specific mortality rates on a single page, using Lexis surfaces. In a Lexis surface, data are arranged as ‘pixels’ on a rectangular canvas, by age on one axis and by year on the other. The shade (and, online, the colour) of these pixels is then determined by the specific mortality risk faced by people of that particular ethnicity, sex, year of age and in a particular calendar year. For one of the figures showing all-cause mortality, contour lines are added linking spaces across this canvas with equal mortality risk. In contrast, most of the figures and tables presented by Case and Deaton are equivalent to focusing on a small horizontal strip within these surfaces, just the 45–54 year age range, and then merging all the vertical strips of pixels, 10 separate age-specific mortality risks within each year, into a single value, namely the vertical position of a line on a graph. For convenience we use the shorthand WNH to refer to White non-Hispanics, H for Hispanics and BNH for Black non-Hispanics. This information-intensive method of visualization allows us to see what Case and Deaton saw, as well as a lot of additional information that they could not see with their methods.

All-cause mortality

We begin by looking at all-cause mortality, for ages in single years ranging from newborns to 80 years, for WNH males and females only. This is shown in Figure 1. Figure 2 places the pair of tiles in Figure 1 in a much broader context, showing not just all-cause mortality for WNH males and females (top row of Figure 2a) but also equivalent all-cause mortality rates for H males and females (middle row of Figure 2a) and BNH males and females (bottom row of Figure 2a) as well as various forms of cause-specific mortality for each of these six ethnic/gendered groups (additional subfigures in Figure 2). The purpose of this is to show the richness and range of information that can be presented on a single page, but readers may wish to look at Figure 1 in the first instance.

Figure 1.

Lexis surface visualization of Centers for Disease Control and Prevention (CDC) all-cause mortality (log 10 scale) for White Non-Hispanic males and females.

Figure 2.

Lexis surface visualizations of Centers for Disease Control and Prevention (CDC) all-cause and external cause specific mortality rates (colour online). Sub-figures from left to right are (a) all-cause mortality (log10 scale); (b) suicides; (c) drugs; (d) vehicles; (b-d are deaths per 100 000 population.) Within each sub-figure, the left column shows female and right column shows male mortality risks. The top row is for White non-Hispanics (WNH), the middle row for Hispanics (H) and the bottom row for Black non-Hispanics (BNHs).

Figure 2(a), the left-hand tile, shows all-cause mortality rates for each of the six populations, using a base 10 log scale. Contour lines are added linking cells in the Lexis surface with equal mortality risks; as base 10 is used, the labels on the contour lines indicate the ‘number of zeroes’ in 12-month death rates (−1.0 meaning 1/10, −2.0 meaning 1/100 and so on). By implication this means that contour lines which bank upwards from left to right indicate generally improving mortality rates over time, lines which bank downwards from left to right indicate worsening mortality rates over time and horizontal lines indicate no change in age-specific mortality rates over time.

We can therefore see that mortality rates in adulthood have been continuing to improve for BNHs and for Hs over this period. By contrast, the mortality rates for those who are aged > c. 35 years have been roughly flat since 1999. These findings can be largely inferred from data presented in the tables of Case and Deaton’s paper, which indicate that mortality trends have improved much faster for BNH in particular, than they have deteriorated for WNH. Two other important things to note about the WNH panes are that the contour lines for females within the 45–54-year age range indicate rising mortality at that age until 2003, before stabilizing; for males the mortality rates were flat or increased among 45–54 year olds age until around 2004, before declining. Over the age of 60 years, mortality rates declined continuously for WNH men and women. The greatest drops in the contour lines from left to right, indicating the greatest increases in mortality risk on the log scale for WNH, occurred at younger adult ages for both genders, as seen by following the contours marked −3.2 for females and −2.8 for males. Although the underlying mortality risks at these younger ages—20s and 30s—are lower than at older ages for the reasons discussed above, as they are further along the ‘right side of the bathtub’, the greatest relative increases in mortality seem to be in younger adults, rather than the 45-54 year age group which Case and Deaton focused on.

By comparing the mortality contours for adult WNHs with Hs and BNHs of the same age and gender, we see that mortality trends for WNHs, with contour lines horizontal or banking downwards throughout much of adulthood, are at odds with the upwards banking contour lines of these other ethnic groups. Case and Deaton compared mortality rates for 45-54-year-old WNHs with total population mortality rates in a number of other countries, using data from the Human Mortality Database (HMD); the trends compared with other ethnic groups within the USA are therefore similar.

Cause-specific mortality

The other three strips in Figure 2, each comprising six tiles, one for each sex and ethnic group, show trends on an identity scale for three external causes of death, i.e. deaths that do not result from broader and inevitable processes of ageing and decline. The legend to the right of each tile shows the scale, with the numbers indicating the number of deaths per 100 000 population denoted by different shades of grey, with darker shades indicating higher mortality rates and lighter shades indicating lower mortality rates. These plots support some of the trends highlighted by Case and Deaton, but also reveal other important patterns affecting both WNHs of other ages and other ethnic groups of all ages. Briefly, some important patterns revealed by these figures include:

Suicide rates (Figure 2b) differ markedly by gender, but also by ethnicity. High rates of male suicide increased for WNH aged around 45–54 years, but had also been raised for a broadly defined WNH cohort who were in their 40s around 1999. High rates of suicide are also seen at much higher ages (> 70 years) for male WNHs, and to a lesser extent for H males. For BNH males, suicide rates are lower, but highest in their 20s.

Drug-based deaths (Figure 2c) have increased for both males and female WNHs aged 45-54 years, consistent with the emphasis in Case and Deaton, but for both males and females WNH drug deaths have increased at younger adult ages too. For male WNHs aged around 30 years, in particular, there is evidence of a marked increase in such deaths, largely following the 2008 recession. For male BNH especially there is evidence of a cohort effect, with an increased risk of drug deaths for those aged around 40–50 years in 1999, diminishing after around 2008. It is worth noting that this broad cohort ‘drifted’ into and out of the 45–54 year age range over the period 1999-2012, again highlighting a difficulty with assuming this age group to remain homogeneous over time.

Vehicle deaths (Figure 2d) have historically tended to be much higher for males than females for all ethnic groups, peak around the age of 20 years, and have decreased markedly following the 2008 recession. This is one of the clearest period changes apparent in the data, and went uncommented on within the Case and Deaton paper because they focused on a different age group. However, the 2008 recession seems to be associated with both increased drug deaths among WNHs of middle age and decreased car deaths amongst WNHs of younger adult age, and so a better understanding of the health impact of the recession clearly requires looking at all age groups., By looking at all ages, we can also see for all ethnicities, and particularly for males, elevated car deaths amongst people in their mid to late 70s, which again appear to have reduced following the 2008 recession. Again the focus on the 45–54 year age range meant that Case and Deaton did not identify this pattern in the data.

Suicide rates have therefore clearly increased in recent years for WNH males, and Case and Deaton’s paper was valuable in highlighting this increase. However, being able to look at mortality rates at all ages shows that this increase, and the increase in drug-based deaths, was not just a middle-aged phenomenon. Looking at all ages shows the 2008 recession as having an exacerbating role in both causes of deaths for WNHs, as well as an inhibitory role for young adult WNHs.

Discussion

The mortality patterns and trends revealed by the data, and the range of public health narratives that can be told, are richer and more complex than those presented by Case and Deaton, which were the focus of dozens of newspaper reports. Rising deaths from certain external causes amongst middle-aged WNHs are, however, an important and troubling public health phenomenon in the USA, and Case and Deaton are to be commended in identifying and highlighting such issues.

What does the report of Case and Deaton show about the relationship between new and traditional media, and between the mainstream and academia? The answer, as always, seems to be ‘it’s complicated’. Online publication made it easier for the original article and news stories based on it to be tweeted and shared very quickly, for editorialized bottom lines to traverse the globe in minutes and hours rather than weeks and months, and for a simple summary of a depressing finding to become so widely shared as to be common knowledge. Information communication technologies (ICTs) therefore allowed for rapid amplification of the errors within the message, as well as the message itself, within mainstream media. But at the same time, ICTs allowed for very rapid and very careful scrutiny of the claims and correction of the findings by tech-savvy academics. We cannot therefore conclude that ICTs have simply helped or simply harmed the process of academic debates in this case. Yet we can conclude that ICTs, for good and for bad, have increased both the volume and the velocity with which such debates have occurred.

The Lexis surface visualizations presented here show that it is neither necessary nor desirable to cut out small strips from the much richer tapestries of US mortality phenomena. By using appropriate data-smoothing methods they have also been applied effectively to identifying age, period and cohort effects in suicide mortality in Scotland, a much smaller population than the USA (around 5.3 million compared with around 318.9 million, according to recent population estimates), whose population was then further subdivided into deprivation quintiles (slightly over one million) as well as by sex (slightly over 0.5 million); in that study, their use complemented intrinsic estimator models which indicated ‘statistically significant’ cohort effects for suicide risk for males from the most socioeconomically deprived areas within Scotland, who entered the labour market after the Neoliberal reforms undertaken by Conservative governments in the early 1980s (Parkinson J, et al. unpublished). We therefore suggest that the complex data visualizations presented here are complements to statistical models, useful in both large and small populations, and useful for supporting inductive reasoning and hypothesis generation just as statistical models and significance testing support deductive reasoning.

Better approaches for looking broadly at trends in mortality rates are both achievable and necessary for helping to identify and address public health issues that affect all US citizens, regardless of age, sex and ethnicity.

Funding

R.S. is funded by the Medical Research Council, UK [MR/K025023/1]; J.M. is funded by the Economic and Social Research Council [ES/K006460/1]; G.M. has received no specific funding related to this work and is an employee of NHS Scotland; K.P. has received no specific funding related to this work; L.V. has received no specific funding related to this work; M.G. has received no specific funding related to this work.