Nursing home quality, COVID-19 deaths, and excess mortality.

The COVID-19 pandemic in the US has been particularly devastating for nursing home residents. A key question is how have some nursing homes been able to effectively protect their residents, while others have not? Using data on the universe of US nursing homes, we examine whether establishment quality is predictive of COVID-19 mortality. Higher-quality nursing homes, as measured by CMS overall five-star rating, have substantially lower COVID-19 mortality through September of 2020. Quality does not predict the ability to prevent any COVID-19 resident or staff cases, but higher-quality establishments prevent the spread of resident infections conditional on having one. Preventing COVID-19 cases and deaths may come at some cost, as high-quality homes have substantially higher non-COVID deaths. The positive correlation between establishment quality and non-COVID mortality is strong enough that high-quality homes also have more total deaths than their low-quality counterparts and this relationship has grown with time. As of late April 2021, five-star homes have experienced 8.4 percent more total deaths than one-star homes.

Introduction

The COVID-19 pandemic in the US has been particularly devastating for residents of nursing homes. As of August 15th, 2021, there were a total of 634,179 COVID-19-related deaths in the US.1 We estimate that 21 percent of these deaths are among nursing home residents.2 Through the end of 2020, the death rate for non-nursing home residents was about 87 per 100,000. The death rate for nursing home residents is more than 108 times that number at roughly 9200 per 100,000. The death rate in nursing homes is more 23 times larger than the death rate for those 65 and over outside of nursing homes, which we calculate to be about 390 per 100,000. In at least five states, Rhode Island, Indiana, New Jersey, South Dakota, and Connecticut, more than 12 percent of the nursing home population at the beginning of 2020 had died by the end of the year with COVID-19.

COVID-19 deaths among nursing home patients is heavily influenced by infection and transmission rates in the surrounding community, which varies substantially by region. For example, cumulative non-nursing home COVID-19 deaths rates at the end of 2020 varied from a low of 8 per 100,000 in Vermont to a high of 159 per 100,000 in New Jersey. Yet, community risk alone cannot explain the vast variation in nursing home death rates. Even in the five states mentioned above with the highest nursing home death rates, 17 percent of homes had not experienced a single COVID-19 mortality by the end of 2020. A key research question is then how have some nursing homes been able to effectively protect their residents, while others have not?

In this work, we focus on one particular dimension of the problem: nursing home quality. Starting in 2008, the Centers for Medicare and Medicaid (CMS) began providing a “five-star” rating of nursing home quality based on three elements: health inspections, staff-to-resident ratios, and quality metrics such as rates of falls and bedsores, with the first element having the greatest weight in an “overall” rating. We merge this quality data with data on COVID-19 cases and mortality that is reported weekly by nursing homes to CMS starting the week ending May 24th, 2020. In count-data models that control for local risk factors and nursing home characteristics, we find that the overall rating is highly predictive of mortality, with five-star homes having 15 percent fewer resident COVID-19 deaths by September 13th, 2020 than one-star homes. We find no such relationship between home quality and COVID-19 deaths after September of 2020.

We identify several mechanisms through which quality nursing homes lowered mortality. First, we show that the chance of having any COVID-19 cases among residents or staff is not explained by nursing home quality, meaning even high-quality homes were unable to prevent COVID-19 from entering their facility. That said, in models that condition on having at least one confirmed case, we show that higher-quality nursing homes more successfully prevented the spread of the disease among residents. Unfortunately, we do not have detailed data on the policies and procedures that enabled these establishments to prevent the spread of the disease. We do show that higher-quality homes experienced fewer staff shortages, which may have helped with patient monitoring and isolation, and somewhat weaker evidence of fewer personal protective equipment (PPE) outages. Furthermore, we show that high-quality homes tested residents and staff more frequently and received test results faster than low-quality homes. Consistent with high-quality homes simply following CMS guidelines more closely, we also show that once vaccines were made available, high-quality homes vaccinated residents and staff at a higher rate.

A likely scenario is that higher-quality establishments also did a better job of isolating residents from risks associated with both outside visitors and other residents. On March 13th, 2020, CMS recommended that nursing homes (i) restrict visitors and non-essential personnel from entering the home and (ii) cancel in-person dining and other group activities (CMS, 2020a).3 These precautions very well may have prevented deaths from COVID-19, but many have questioned whether those gains came at a cost. An abundance of qualitative evidence from nursing home staff, administrators, and resident family members suggests that the lack of in-person contact with loved ones and other residents not only generated feelings of loneliness, isolation, and despair, but may have also expedited death (Aronson, 2020; Paulin, 2020; Graham, 2020). A nationwide survey of nursing home residents by Altarum, a non-profit healthcare company, documents similar concerns (Montgomery et al., 2020). Existing research shows that social isolation increases both dementia severity and the likelihood of adverse outcomes among those with Alzheimer's (Dyer et al., 2020) and mortality among the elderly (Steptoe et al., 2013). According to CDC provisional data, there were 56,464 excess deaths (i.e., realized deaths above that which is predicted using historical averages) among Alzheimer's patients through early October of 2021;4 however, only 22,709 or 40 percent had a corresponding positive COVID-19 diagnosis,5 leaving 60 percent of these excess deaths not directly related to the disease but to other aspects of the pandemic.6

To investigate this claim, we return to our original model, but change the dependent variable from COVID-19 deaths to non-COVID deaths. We find that higher-quality nursing homes have much higher non-COVID mortality. In particular, as of September 13th, 2020, five-star homes had experienced 11.4 percent more non-COVID deaths than one-star homes, all else equal; by April 15, 2021, this figure had grown to nearly 15 percent. Research by Levere et al. (2020) suggests that these excess deaths likely resulted from isolation and loneliness. Using resident-level assessment data from Connecticut nursing homes, the authors document substantial weight loss and increases in severe pressure ulcers among residents who did not contract COVID-19. The resident survey mentioned above also documents severe isolation, finding that only 5 percent of respondents had visitors three or more times per week, compared to 56 percent before the pandemic, and just 13 percent reported dining in a communal setting, compared to 69 percent before the pandemic. Another possibly is that resident contact restrictions may coincide with, or even cause, a reduction in interactions with healthcare providers, both inside and outside the home, which would be consistent with widely documented reductions in healthcare receipt overall during the early stages of the pandemic (Bosworth et al., 2020; Ziedan et al., 2020; Cantor et al., 2020; Clemens et al., 2021).

An alternative explanation of our findings is that there is incomplete reporting of COVID-19 or strategic use of defining COVID-19 deaths by nursing homes to mitigate the damage to their reputation from the CMS reports. To investigate this, in Fig. 1 we report weekly deaths in nursing homes as reported to CMS from the end of May 2020 through June of 2021. Were deaths misreported, we would expect spikes in non-COVID deaths during the summer and winter waves of 2020. Such spikes are not present in the data. Moreover, when we eliminate deaths prior to June of 2020, when misreporting is most likely, our results are unaffected. Another plausible explanation of our findings is “harvesting”; i.e., low-quality homes experience fewer non-COVID deaths because the most fragile residents die from COVID-19. To test this theory, we estimate the impact of nursing home quality on total deaths. We find that between January 1st, 2020 and April 25th, 2021, five-star homes experienced 8.4 percent more total deaths than one-star homes, an average difference of 2.7 lives.

Fig. 1

Deaths by Week Among Nursing Home Residents, May 31st, 2020 through May 9th, 2021

This figure reports the total number of nursing home resident deaths in the US by week, from May 31st, 2020 to May 9th, 2021. Total deaths have been decomposed into those related to COVID-19 or not. Weekly deaths are calculated using the CMS COVID-19 nursing home surveillance survey described in Section II.A .

Given the number of COVID-19-related deaths and an early understanding that the elderly die at higher rates, there is surprisingly little research on deaths in nursing homes. As a result, this paper contributes to the literature on the health effects of COVID-19 along several dimensions. A number of papers have examined the general relationship between CMS five-star ratings and COVID-19 cases and/or deaths in nursing homes but much of this work was either in a single state (Bui et al., 2020; Harrington et al., 2020; He et al., 2020; Li et al., 2020a) or occurred very early in the pandemic (Abrams et al., 2020). Some studies have used various versions of the CMS data used in this study. Gorges and Konetzka (2020) show that county incidence rates are the strongest predictor of resident mortality and staff levels have modest impacts on the spread of the disease. Li et al. (2020b) document much higher COVID-19 death rates in homes with a larger share of minority patients. Chen et al. (2021) show that contractors that service multiple nursing homes in a local area helped spread the disease. The data from the CDC showing large excess death rates for the elderly and especially Alzheimer's patients has helped put this issue into the public discussion, yet there is little research other than efforts to document the extent of the problem and almost no research on the factors that lead to excess mortality.

While much of the previous work studies determinants of outbreaks, only a few studies such as (Li, Cen, Cai and Temkin-Greener, 2020b) and Gorges and Konetzka (2020) consider mortality in a nationwide sample as we do. Moreover, we identify that the home quality/COVID-19 death relationship is most likely explained by high-quality homes reducing the spread of the virus once it enters the home and it is not due to a home's ability to prevent the entry of the disease. Importantly, our paper is the first to establish a negative link between nursing home quality and non-COVID mortality. This relationship is so pronounced that high-quality nursing homes are found to have significantly higher total mortality than low-quality homes.

The COVID-19 pandemic presented an almost unprecedented challenge for nursing homes. In response to CMS directives and various state regulations, it is without question that the pandemic made nursing homes much lonelier places to live. In this paper, we show that the highest quality nursing homes have witnessed the greatest amount of death during the pandemic. The means by which quality is measured may be important to understanding this finding. Inputs to the CMS five-star rating measure a nursing home's ability to abide by pre-determined guidelines (e.g., staffing ratios) and minimize objectively bad health outcomes (e.g., bed sores). In light of this, it is not clear that homes of high quality, as it is traditionally measured, should fare well in a highly uncertain environment where guidelines from the centralized authority (i.e., CMS), as well as public opinion, attach extraordinary weight to preventing a single bad outcome; namely, COVID-19 cases and deaths. This paper provides a clear example of high-quality homes underperforming in such an environment.7 We offer two takeaways.

First, an organization's capacity to perform well in the face of adversity may be an important dimension of its quality. Such capacity is difficult to measure across organizations because adversity may not be observable and is rarely evenly distributed. The COVID-19 pandemic offers CMS an observable shock that affected all nursing homes. CMS should consider capitalizing on this opportunity by evaluating nursing home responses to the COVID-19 shock and integrating this evaluation into their quality metrics.

Second, through September of 2020, CMS guidelines clearly prioritized reducing nursing home resident exposure to COVID-19. Guidelines focused on proper PPE use, testing protocols, and isolating residents from the outside world and one another. COVID-19 cases and deaths were publicized and widely reported in the news media. Our results suggest that in response, the best nursing homes “hit their marks,” i.e., they followed guidelines and, for a time, avoided the bad health outcome of focus. That said, we show that these homes witnessed the most total death in the long-run. In the future, we hope that these findings serve as a reminder that even early guidelines and objective health metrics should be cognizant of resident wellbeing at large.

Data

Data sources and reporting accuracy

Our primary data source is a weekly data set released by CMS that has COVID-19 surveillance information by nursing home. On May 8th, 2020, CMS released a final ruling that required nursing homes to report confirmed and suspected COVID-19 cases of residents and staff to both the residents and their representatives. The ruling also required that nursing homes report weekly totals of surveillance items to the Centers for Disease Control (CDC) National Healthcare Safety Network, starting with the week ending May 24th, 2020. Reporting was mandatory with a $1000 fine issued after four-weeks of non-reporting. Thereafter, fines increased by $500 for each week of non-reporting (CMS, 2020c).

The first weekly file was released by CMS in early June and updates are released on Thursdays, 11 days after the end of the Monday to Sunday reporting period. The initial release of this data set was in a word, messy. There were obvious key-punch errors and variables were frequently reported in the wrong columns. Subsequent releases of the data corrected many of these recording errors and the most recent releases of the data are relatively free from these obvious errors, although some naturally exist. Reporting has always been high. In the first release of the data, 13,162 nursing homes were included and 97 percent passed a data quality assurance check. In the following few weeks, many homes missing the original deadline submitted both current and historic data. The number of homes reporting in a single week peaks at 15,330 for the week ending September 20th, 2020, and 98 percent passed a data quality assurance check. The number never dips below 15,167 after that. Based on the size of the CMS five-star data set, this represents 97 to 98 percent of all nursing homes.

The CMS nursing home data reports weekly and cumulative (since January 1st, 2020) values for confirmed COVID-19 cases for residents and staff, suspected cases for these two groups, and COVID-19 deaths (which includes suspected and confirmed), plus all deaths, among residents. Deaths are reported regardless of the location, so if a nursing home resident is moved to a hospital and dies there, the death is counted as a nursing home resident death. The survey asks nursing homes if they have shortages of key staff including nursing staff, clinical staff, and aids, and whether they have adequate supplies of specific PPE such as N95 masks, surgical masks, gowns, etc. From August 16th through November 22nd (2020), the survey asks a series of questions about COVID-19 testing procedures. Not until late May of 2021 did the survey ask about resident and staff vaccinations.

The CMS nursing home data has an important limitation. At the time of the first report (May 24th, 2020), CMS allowed nursing homes the choice to report cases and deaths from the prior week or cumulative cases and deaths since January 1st. Thereafter, homes report weekly counts and a cumulative count is calculated by CMS; thus, if a home fails to report the cumulative count since January 1st on May 24th, the cumulative count that CMS calculates in future weeks is too small. In Appendix Section B, we compare CMS nursing home death counts to those from the COVID-19 Tracking Project, which measures COVID-19 deaths in nursing homes at the state level in 37 states. We calculate that the initial undercount is about 15%. In contrast, the change in COVID-19 deaths between May 24th, 2020 and other dates as reported by these two sources differs by less than 4 percent. In our robustness analysis, we re-estimate our main model using cases and deaths since May 24th, which appears to be more accurately reported across the two samples.

Using these two data sources, we estimate total nursing home COVID-19 deaths in the nation as of August 15, 2021 to be 137,318, which is 21 percent of aggregate mortality (see Table A1 and associated discussion in the Appendix Section B).

Table A1

Comparing State and CMS Reports of COVID-19 Deaths in Nursing Homes for 37 States.

Source	Deaths as of 5/24/2020 (a)	Deaths as of 3/7/2021 (b)	Δ deaths (b) – (a)
CMS (1)	21,189	104,374	83,185
State facility reports (2)	24,286	110,254	85,968
Difference (2) – (1)	3079	5880	2783
% difference (2) – (1)	14.5%	5.6%	3.3%

Calculations made for the 37 US states reporting COVID-19 deaths among nursing home residents on state dashboards. State facility reports refer to the death counts from these dashboards, while CMS refers to the deaths counts in the CMS COVID-19 surveillance data discussed in Section II.A.

The staggering consequences of the pandemic for nursing home residents can be seen when we calculate COVID-19 death rates for residents and compare to the general population through the end of 2020.8 The death rate (per 100,000 people) for non-nursing home residents is roughly 87. Dividing imputed COVID-19 deaths by the number of nursing home residents alive at the beginning of 2020,9 the death rate for nursing home residents is about 9200, or about 108 times the rate for the general population. We calculate that the COVID-19 death rate for people 65 and over living outside of nursing homes is 390,10 meaning the nursing home death rate is about 24 times this number.

Heterogeneity across nursing homes in COVID-19 mortality

There is tremendous variation across states in the severity with which the pandemic struck nursing homes. Using the CMS data as of the last week of 2020, deaths per 1000 nursing home residents varied from a low of 12.8 in Alaska to a high of 134 in Rhode Island (see Appendix Fig. A3). In the 14 states with the highest death rates, more than 10 percent of the nursing home population died from COVID-19 in 2020.

Fig. A3

COVID-19 Deaths Rates (deaths/1000) in Nursing Homes, as of End of 2020

Information taken from CMS COVID-19 surveillance data discussed in Section II.A.

The risk nursing home residents face from the disease is strongly correlated with the underlying risk in their state. The correlation coefficient between the non-nursing home COVID-19 death rate at the state level and the same value for nursing home residents (from CMS data) is 0.54.

Despite the strong correlation between underlying risk and nursing home deaths, many nursing homes successfully avoid high death rates, even in the hardest hit areas. In counties from the top 10th percentile of total COVID-19 death rates by May 24th, 2020, just over half of nursing homes with more than 100 beds had two or fewer COVID-19 deaths per 100 beds. By April 24th, 2021 this figure was still in excess of 20 percent. Moreover, a disproportionately small share of nursing homes account for the majority of COVID-19 deaths. As of September 2020, the top 5 (1) percent of nursing homes, which is just 754 (151) homes or 8.3 (2.4) percent of the nation's beds, accounted for 39 (14) percent of all COVID-19 deaths in nursing homes. By the end of April 2021, the top 5 (1) percent of nursing homes still accounted for 23 (8) percent of all COVID-19 deaths.

Analysis sample

The goal of this project is to explore whether observed nursing home characteristics can explain the variation in COVID-19 death rates. In particular, we test whether high-quality nursing homes, as measured by the CMS five-star ratings, did a better job of preventing deaths from COVID-19. Our initial sample contains 15,421 nursing homes reporting data to CMS at any point between May 25th, 2020 and July 18th, 2021. We lose 311 homes to inconsistent reporting and 205 homes because no five-star rating data was available; thus, our main analysis sample consists of 14,905 nursing homes. Sample construction is discussed in Appendix A and summary statistics for these data are shown in Appendix Table A3.

Table A3

Sample Characteristics, CMS Data on Nursing Homes, as of September 13th, 2020.

Variable	Mean	S.D.
Total beds	106.222	58.858
Share of female residents	0.663	0.119
Share of female residents, missing	0.086	0.280
Share of residents under 65 years old	0.226	0.177
Share of residents under 65 years old, missing	0.547	0.498
Share of black residents	0.165	0.220
Share of black residents, missing	0.446	0.497
Share of hispanic residents	0.049	0.134
Share of black residents, missing	0.421	0.494
Share of residents on Medicaid	0.599	0.230
For profit	0.702	0.457
Acuity index	12.189	1.479
Medicaid, profit, and acuity missing	0.062	0.242
Observations	15,110

The construction of this sample is discussed in Appendix Section A. Total beds is measured using the CMS COVID-19 surveillance data discussed in Section II.A. All other variables come from the LTC Focus database at Brown University.

Several variables used in our analysis require explanation. Data on nursing home characteristics comes from “Long-term Care: Facts on Care in the US”, which is provided by a research center at Brown University.11 Sample statistics are reported for observations with non-missing data. The acuity index, which ranges from zero to 23 in the data, measures of the amount of care needed by the average nursing home residents (higher values suggest more care). For the three variables – the share of residents using Medicaid, the acuity index, and for-profit status – the data contains values for all or none of them.

Nursing home star ratings come from data.medicare.gov. There are three separate ratings – inspection, quality measures (QM), and staffing – which are aggregated by CMS into an overall rating.12 All three ratings, as well as the overall rating, measure quality in integer “star” values, where five-star is the best possible rating and one-star is the worst. We discuss the construction of these ratings and briefly outline the literature that evaluates the ratings in Appendix Section C. In our empirical analysis, we measure quality using the overall five-star rating, reported by CMS in June of 2020. All data informing these ratings was collected prior to March, when it was first recognized that COVID-19 was present and spreading in the United States.

In Table 1 , we present cumulative case and death rates over time, starting January 1st, 2020. Throughout our analysis, we focus on cumulative deaths at and between four dates: (i) May 24th, 2020, which precedes the summer 2020 COVID-19 wave and is the first date we observe in the CMS data; (ii) September 13th, 2020, which follows the summer wave, but precedes a September 17th CMS memo that altered visitation protocols in nursing homes (discussed below); (iii) December 6th, 2020, which is the last date available prior to the start of vaccine distribution; and (iv) April 25th, 2021, a point in which nearly all US nursing home residents had a full four months to become vaccinated and COVID-19 death rates in nursing homes had flatlined (see Fig. 1). Resident and staff case variables measure confirmed cases reported to CMS. Nursing homes report both total deaths from any cause and COVID-19 deaths each week, which allows us to calculate deaths not from COVID-19.13 Note that case (and death) rates are calculated as (cases/total beds)*100. County-level COVID-19 case and death totals come from the same source used by the CDC (USAFacts, 2020).

Table 1

COVID-19 Cases and Death Rates by Cause over Time in Nursing Homes Means (standard deviations).

Variable	As of 5/24/20	As of 9/13/20	As of 12/06/20	As of 4/25/21
Cumulative case and death rates (per 100 beds)
Resident COVID-19 case rate	4.93(12.79)	12.94(19.31)	26.17(24.80)	40.39(27.39)
Staff COVID-19 case rate	2.86(7.21)	11.11(16.59)	24.27(23.94)	38.11(22.14)
Resident COVID-19 death rate	1.30(3.69)	3.00(5.77)	5.00(6.90)	8.10(8.57)
Resident non-COVID-19 death rate	3.12(7.69)	9.69(23.88)	14.60(41.85)	22.25(51.87)
County-level cumulative case and death rates (per 1000 residents)
COVID-19 cases rate	2.86(4.76)	15.35(9.97)	33.75(15.85)	94.50(24.77)
COVID-19 deaths rate	0.25(0.42)	0.54(0.54)	0.89(0/61)	1.86(0.86)

This table reports mean, cumulative case and death rates at four points in time. The nursing home case and death rates reported in the top panel come from the CMS COVID-19 nursing home surveillance survey described in Section II.A. Note that the number of resident non-COVID-19 deaths is calculated by subtracting the reported number of COVID-19 deaths from the reported number of total deaths in the home. The county-level case and death rates are calculated using counts from USAFacts (2020).

In Table 2 , we present nursing home characteristics and relevant COVID-19 summary statistics for nursing homes with different overall star ratings as of September 13th, 2020.14 Many nursing home characteristics change monotonically with the overall star rating including: total beds (decreasing); the share of female residents (increasing); the share of residents under 65, black, Hispanic, and on Medicaid (decreasing), and the acuity index (decreasing). The persistent relationship between these observed characteristics and the overall star rating might mean there is little informational content in the ratings. However, a simple regression of the overall star rating on observed characteristics only produces and R2 of 0.21. Regarding COVID-19, higher-quality homes have lower resident case rates, but slightly higher staff rates. Moreover, these homes have notably smaller COVID-19 death rates, but higher non-COVID death rates. On average, higher-quality homes have experienced fewer staff shortages and PPE outages than lower-quality homes since May 24th, 2020. COVID-19 testing is not strongly correlated with overall quality.

Table 2

Descriptive Statistics in Nursing Homes, Overall 5-Star Rating.

	1-star	2-star	3-star	4-star	5-star
NH characteristics
total beds	122.8	116.6	110.4	102.7	88.5
share of female residents	0.621	0.645	0.659	0.681	0.692
share of residents under 65 years old	0.277	0.252	0.233	0.195	0.159
share of black residents	0.271	0.206	0.177	0.127	0.084
share of Hispanic residents	0.072	0.063	0.053	0.038	0.032
share of residents on Medicaid	0.693	0.650	0.630	0.584	0.487
for profit	0.867	0.775	0.735	0.665	0.543
acuity index	12.25	12.22	12.20	12.16	12.14
Cumulative Cases and Deaths (9/13)
any resident COVID-19 cases?	0.683	0.680	0.650	0.613	0.560
resident COVID-19 cases/100 beds	15.06	15.15	13.35	12.39	10.37
any staff COVID-19 cases?	0.877	0.855	0.854	0.844	0.832
staff COVID-19 case/100 beds	10.40	11.30	11.16	11.02	11.51
any resident COVID-19 deaths?	0.509	0.492	0.455	0.415	0.359
resident COVID-19 death/100 beds	3.324	3.539	2.999	2.922	2.531
any resident deaths not from COVID-19?	0.846	0.857	0.874	0.863	0.831
resident death not from COVID-19/100 beds	8.15	9.35	9.74	9.85	10.87
NH Staff and Supplies (since week ending 5/24)
any nursing staff shortage?	0.477	0.410	0.383	0.359	0.311
any aide staff shortage?	0.522	0.452	0.428	0.399	0.346
any clinical staff shortage?	0.211	0.182	0.168	0.166	0.151
n95 mask outage?	0.208	0.185	0.177	0.157	0.155
surgical mask outage?	0.119	0.113	0.107	0.101	0.103
eye protection outage?	0.123	0.124	0.118	0.104	0.097
gown outage?	0.126	0.130	0.128	0.112	0.116
glove outage?	0.085	0.070	0.073	0.060	0.065
hand sanitizer?	0.093	0.075	0.068	0.061	0.065
COVID-19 Testing (week ending 9/13)
average time to test results
<1 day	0.104	0.086	0.079	0.074	0.076
1–2 days	0.397	0.431	0.460	0.480	0.503
3–7 days	0.465	0.459	0.436	0.430	0.408
>7 days	0.034	0.025	0.025	0.016	0.013
any resident test in past week?	0.556	0.612	0.617	0.619	0.607
any staff test in past week?	0.799	0.822	0.816	0.818	0.808
testing machine on site?	0.695	0.688	0.685	0.685	0.618
Observations	2279	2918	2684	3221	3803

This table reports average nursing home characteristics and COVID-19 metrics by the nursing home's CMS overall star rating. We measure overall star rating for each nursing home as reported in June of 2020, which is available for download on the CMS website. All nursing home characteristics, except total number of beds, are taken from the LTC Focus database at Brown University. Total number of beds and all COVID-19 metrics come from the CMS COVID-19 nursing home surveillance data described in Section II.A. Case/Death information and Staff/Supply information are measured through September 13th, 2020; later dates are available upon request. Testing variables are reported for a single week (the week ending September 13th, 2020), as most testing-related questions were only asked from August 16th through November 22nd of 2020.

Statistical model and results

We estimate the effect of nursing home quality on the total number of nursing home deaths due to COVID-19 using a negative binomial model. That is, we write the probability of nursing home having COVID-19 deaths aswhere is a gamma function and and are the shape and scale parameters, respectively, of a gamma distribution. We allow to vary with nursing home and county characteristics, , such that . The parameters are estimated via maximum likelihood. All models include state fixed effects and standard errors are clustered at the state level.

We are primarily interested in the impact that nursing home quality has on COVID-19 deaths; however, we also control for the following: (logged) total number of beds; the percent female, under 65 years old, black, Hispanic, and on Medicaid; whether the home is for profit; acuity index;15 (logged) county population; and county COVID-19 cases per 1000 residents.16

COVID-19 mortality results

We estimate Eq. (1) across several different time periods. We report parameter estimates for the impact of nursing home quality, measured using the overall star rating, on COVID-19 deaths in the first seven rows of Table 3. 17 The first four rows use cumulative counts of COVID-19 deaths, starting January 1st, 2020 through May 24th, September 13th, December 6th (of 2020), and finally April 25th (of 2021).

Table 3

Negative Binomial Estimates, Impact of Overall Star Ranking on Deaths in Nursing Homes.

		Coefficients (standard errors) on Overall star ranking
Period of analysis	Sample mean	2-star	3-star	4-star	5-star
		COVID-19 deaths
(1) As of 5/24/20	1.702	0.000(0.048)	−0.164(0.068)	−0.142(0.073)	−0.173(0.065)
(2) As of 9/13/20	3.670	0.033(0.034)	−0.073(0.052)	−0.092(0.047)	−0.154(0.056)
(3) As of 12/06/20	5.570	0.019(0.023)	−0.045(0.033)	−0.040(0.035)	−0.094(0.036)
(4) As of 4/25/21	8.704	0.026(0.024)	−0.019(0.034)	0.003(0.030)	−0.072(0.032)
(5) Δ 9/13/20 – 5/24/20	1.969	0.001(0.042)	−0.066(0.057)	−0.096(0.053)	−0.211(0.057)
(6) Δ 12/06/20 – 9/13/20	1.899	−0.005(0.039)	0.001(0.039)	0.008(0.046)	−0.004(0.048)
(7) Δ 4/25/21 – 12/06/20	3.135	0.045(0.034)	0.056(0.046)	0.058(0.044)	0.023(0.051)
		Non-COVID-19 deaths
(8) As of 5/24/20	3.381	0.013(0.050)	0.015(0.050)	0.030(0.050)	0.038(0.059)
(9) As of 9/13/20	10.146	0.034(0.029)	0.070(0.031)	0.088(0.034)	0.114(0.041)
(10) As of 12/06/20	15.383	0.032(0.025)	0.074(0.025)	0.105(0.030)	0.123(0.032)
(11) As of 4/25/21	23.170	0.055(0.019)	0.092(0.021)	0.127(0.025)	0.148(0.030)
(12) Δ 9/13/20 – 5/24/20	6.765	0.039(0.026)	0.079(0.028)	0.103(0.030)	0.137(0.033)
(13) Δ 12/06/20 – 9/13/20	5.236	0.059(0.023)	0.135(0.024)	0.175(0.029)	0.200(0.031)
(14) Δ 4/25/21 – 12/06/20	7.787	0.098(0.023)	0.150(0.028)	0.202(0.033)	0.239(0.036)
		All deaths
(15) As of 5/24/20	5.082	0.010(0.043)	−0.007(0.041)	0.013(0.046)	0.010(0.056)
(16) As of 9/13/20	13.816	0.025(0.028)	0.027(0.030)	0.045(0.033)	0.050(0.039)
(17) As of 12/06/20	20.952	0.027(0.022)	0.031(0.021)	0.064(0.027)	0.061(0.030)
(18) As of 4/25/21	31.874	0.047(0.017)	0.055(0.018)	0.084(0.023)	0.084(0.026)
(19) Δ 9/13/20 – 5/24/20	8.734	0.029(0.028)	0.035(0.030)	0.053(0.029)	0.066(0.036)
(20) Δ 12/06/20 – 9/13/20	7.136	0.049(0.023)	0.102(0.020)	0.134(0.029)	0.153(0.031)
(21) Δ 4/25/21 – 12/06/20	10.922	0.085(0.018)	0.121(0.028)	0.159(0.035)	0.175(0.035)

Standard errors allow for arbitrary correlation across observations within a state. Other controls in the models include logged total beds; percent of residents that are female, under 65, black, Hispanic, on Medicaid (along with corresponding indicators for missing variables); for-profit status; acuity index; county-level COVID-19 cases per 1000 residents (measured 23 days prior to death); logged county population; and a full set of state fixed effects.

.

.

Our results show that through September 13th, 2020, higher-quality nursing homes experienced fewer deaths from COVID-19. For example, nursing homes with a five-star rating had a COVID-19 death rate that was about 15 percent lower than those with a one-star rating. Rows 3 and 4 show that five-star homes experienced statistically fewer cumulative COVID-19 deaths than lower-quality homes as of December 6th, 2020 and April 25th, 2021; however, these effects are driven entirely by deaths leading up to September, as is shown in rows 6 and 7.18

We discuss above that the CMS data likely undercounts the true number of COVID-19 deaths in nursing homes, as homes were given the choice on May 24th to report deaths from the prior week or cumulative deaths since January 1st. To show that our results are not somehow driven by this measurement error, in row 5 we change the dependent variable to deaths between September 13th and May 24th (of 2020), which should be measured accurately in the CMS data. These results are similar to our cumulative September 13th findings. Estimated quality effects as of May 24th are also very similar to the results for mid-September (row 1).

We will return to the other two sets of results in Table 3 later in Section III.C

In Table 4 we show that our main findings – the impact of overall star quality on cumulative COVID-19 deaths as of September 13th, 2020 – are robust to a number of alternative empirical specifications, including (row 2) using a Poisson model with clustered standard errors, as suggested by Cameron and Trivedi (2005); using linear models with (row 3) ln(deaths+1), (row 5) the inverse hyperbolic sine of deaths, or (row 7) the inverse hyperbolic sine of death rates as the dependent variable; (rows 4, 6, 8) adding county fixed effects to any of the prior three models; (row 9) adding controls for staff hours per resident day, which may be correlated with home quality conditional on number of beds; and (row 10) adding controls for the county's republican vote share in the 2016 presidential election, which others have found is negatively correlated with COVID-19 precaution taking, such as compliance with stay-at-home orders (Charoenwong et al., 2020) and vaccination rates (Agarwal et al., 2021).19 We will discuss the results in row (11) and (12) below.

Table 4

Robustness Analysis, deaths measured 9/13 unless stated otherwise.

			Overall Star Rating
Model	Sample mean	Obs	2-star	3-star	4-star	5-star
(1) Baseline (Table 3, row 2)	3.67	14,905	0.033	−0.073	−0.092	−0.154
			(0.034)	(0.052)	(0.047)	(0.056)
(2) Model (1), but Poisson	3.67	14,905	0.052	−0.126	−0.100	−0.169
			(0.048)	(0.064)	(0.052)	(0.054)
(3) Model (1) but OLS with ln(deaths+1)	0.82	14,905	0.030	−0.036	−0.042	−0.074
			(0.022)	(0.031)	(0.024)	(0.031)
(4) Model (3) but add county FE	0.82	14,905	0.024	−0.068	−0.061	−0.096
			(0.028)	(0.040)	(0.030)	(0.036)
(5) Model (1) but OLS with inverse hyperbolic sine of deaths	1.02	14,905	0.034	−0.043	−0.053	−0.094
			(0.028)	(0.039)	(0.030)	(0.038)
(6) Model (5) but add county FE	1.02	14,905	0.026	−0.082	−0.076	−0.121
			(0.034)	(0.049)	(0.037)	(0.044)
(7) Model (1) but OLS with inverse hyperbolic sine of death rate (per 100 beds)	1.02	14,905	0.034	−0.043	−0.053	−0.094
			(0.028)	(0.039)	(0.030)	(0.038)
(8) Model (7) but add county FE	1.02	14,905	0.026	−0.082	−0.076	−0.121
			(0.034)	(0.049)	(0.037)	(0.044)
(9) Model (1) but add controls for staff hours per resident day	3.67	14,558	0.038	−0.071	−0.079	−0.124
			(0.034)	(0.051)	(0.047)	(0.056)
(10) Model (1) but add controls for republican share of county	3.67	14,888	0.033	−0.073	−0.091	−0.154
			(0.034)	(0.052)	(0.047)	(0.055)
(11) Model (1) but add controls for any staff and resident cases, as well as counts	3.67	14,905	−0.045	−0.082	−0.169	−0.229
			(0.038)	(0.031)	(0.046)	(0.058)
(12) Model (1) but add controls for shortage counts	3.67	14,905	0.043	−0.060	−0.076	−0.136
			(0.033)	(0.051)	(0.045)	(0.054)

Standard errors allow for arbitrary correlation across observations within a state. Other controls in the models include logged total beds; percent of residents that are female, under 65, black, Hispanic, on Medicaid (along with corresponding indicators for missing variables); for-profit status; acuity index; county-level COVID-19 cases per 1000 residents (measured 23 days prior to death); logged county population; and a full set of state fixed effects.

Mechanisms

How did high-quality nursing homes manage to prevent COVID-19 deaths? CMS first offered nursing homes and assisted living facilities guidelines for preventing and managing COVID-19 cases on March 13th, 2020. Since then, these guidelines have been updated and expanded repeatedly as the public health community has learned more about the virus. The CDC's advice is expansive but is linked by several common themes (CDC, 2020d): First, keep COVID-19 out by limiting visitors and encouraging staff to stay home when ill. Second, clean hands, surfaces, and equipment thoroughly and repeatedly. Third, staff should closely monitor residents for signs of the virus, test symptomatic individuals and close contacts, and isolate those who are symptomatic. Fourth, staff should use PPE at all times. Fifth, after vaccines became available in December of 2020, all residents and staff should be vaccinated.

These recommendations guide our exploration of the potential mechanisms that enabled higher-quality nursing homes to prevent COVID-19 deaths. First, we examine whether higher-quality nursing homes were better able to prevent COVID-19 from entering the home at all. In the first block of results in Table 5 we report results from linear probability models that regress an indicator of whether a home has a single COVID-19 case among its staff (row 1) or among its residents (row 3) as of September 13th, 2020 on the home's overall star rating and the same set of covariates from our earlier analysis. These results show that higher-quality nursing homes, despite lowering the death rate, were not able to prevent COVID-19 from entering the home. Second, we test whether higher-quality nursing homes were able to prevent the spread of the virus, conditional on having at least one case. In rows 2 (4) of Table 5, we report estimates from a linear regression of log staff (resident) cases on the overall rating and covariates, only for homes with at least one staff (resident) case. The results show that while higher-quality nursing homes were not more effective that lower-quality homes in preventing the spread of COVID-19 among their staff, these homes were more effective at preventing the virus’ spread among their residents. Conditional on having at least one case, all else equal, five-star homes saw roughly 15 percent fewer cases than one-star homes. As of September 13th, among homes with at least one case, the average home has 24.5 cases per 100 beds, meaning this difference amounts to about 3.7 cases per 100 beds.20

Table 5

Negative Binomial Estimates, Impact of Overall Star Ranking on Non-COVID Deaths in Nursing Homes.

			Coefficients (standard errors) on Overall star rating
Period of analysis	Obs.	Sample mean	2-star	3-star	4-star	5-star
			As of 9/13/2020
(1) Any staff cases?	14,905	0.850	−0.010(0.008)	−0.005(0.010)	0.000(0.009)	0.000(0.013)
(2) ln(staff cases)	12,669	2.044	0.088(0.027)	0.011(0.021)	0.046(0.029)	0.072(0.030)
(3) Any resident cases?	14,905	0.629	0.024(0.010)	0.014(0.010)	0.009(0.010)	−0.005(0.010)
(4) ln(resident cases)	9392	2.385	0.027(0.040)	−0.095(0.047)	−0.087(0.049)	−0.147(0.040)
			From 9/13/2020 to 4/25/2021
(5) Any staff cases?	14,891	0.993	−0.001(0.002)	0.001(0.002)	0.000(0.002)	0.002(0.001)
(6) ln(staff cases)	14,782	9.959	0.031(0.022)	0.084(0.025)	0.116(0.023)	0.146(0.030)
(7) Any resident cases?	14,891	0.910	0.009(0.008)	0.010(0.007)	0.018(0.009)	0.010(0.009)
(8) ln(resident cases)	13,516	2.849	−0.010(0.035)	0.018(0.041)	0.036(0.035)	−0.015(0.044)

Standard errors allow for arbitrary correlation across observations within a state. Other controls in the models include logged total beds; percent of residents that are female, under 65, black, Hispanic, on Medicaid (along with corresponding indicators for missing variables); for-profit status; acuity index; county-level COVID-19 cases per 1000 residents (measured 23 days prior to death); logged county population; and a full set of state fixed effects.

In the second block of results in Table 5, we re-estimate the models from rows (1) through (4) but use cases occurring between September 13th, 2020 and April 25th, 2021. The fraction of homes with any staff and resident cases is nearly one, as the winter wave is in the middle of this time period, so it is not a surprise that the quality measure explains little for these outcomes. Consistent with our COVID-19 deaths results earlier, past September there is no difference between low- and high-quality homes in terms of resident COVID spread.21

If high-quality nursing homes prevented deaths not by keeping COVID-19 out of the nursing home entirely, but by preventing spread among residents within the home, the next obvious question is: how? We test several plausible theories. First, both identifying and isolating residents with COVID-19 symptoms requires a capable staff that is of an adequate size; thus, we first test whether high-quality nursing homes have had fewer staffing shortages during the pandemic, which may explain their ability to prevent COVID-19 cases among their residents.

In Panel A of Table 6 , we present results from three linear probability models that regress indicators for self-reported staffing shortages (nurses, aides, and clinical staff) at any point between May 24th and September 13th of 2020 on the overall star rating and other controls. Row 1 shows that, all else equal, five-star facilities were 10.7 percentage points less likely to have a nursing shortage over this time period than a one-star facility. The gap is 11.8 percentage points for aides (row 2) and 3.9 percentage points for clinical staff (row 3). All effects are statistically significant at the one-percent level. The estimates are 28, 28, and 23 percent, respectively, of the sample mean for the outcomes.

Table 6

OLS/Maximum Likelihood where Outcomes are Different Measures of Nursing Home Quality.

			Coefficient (Standard error) on Overall starrating
Dependent variable		Sample mean	2-star	3-star	4-star	5-star
A: Staff shortage (any 5/24/20 – 9/13/20)
	Any nursing shortage?	0.378	−0.045(0.011)	−0.068(0.014)	−0.085(0.016)	−0.107(0.014)
	Any aide shortage?	0.419	−0.050(0.012)	−0.071(0.017)	−0.093(0.018)	−0.118(0.016)
	Any clinical staff shortage?	0.172	−0.021(0.010)	−0.033(0.014)	−0.032(0.012)	−0.039(0.012)
B: PPE shortage (any 5/24/20 – 9/13/20)
	N95 outage?	0.173	−0.009(0.009)	−0.012(0.011)	−0.027(0.011)	−0.020(0.013)
	Surgical mask outage?	0.107	−0.003(0.009)	−0.004(0.011)	−0.009(0.011)	−0.001(0.014)
	Eye protection outage?	0.111	0.009(0.008)	0.008(0.010)	−0.002(0.011)	−0.001(0.012)
	Gown outage?	0.121	0.006(0.008)	0.008(0.010)	−0.007(0.009)	0.000(0.013)
	Glove outage?	0.069	−0.012(0.007)	−0.006(0.008)	−0.020(0.008)	−0.014(0.010)
	Hand sanitizer outage?	0.071	−0.016(0.009)	−0.020(0.008)	−0.028(0.009)	−0.023(0.010)
C: Testing procedures (week ending 9/13/20)
	Time to test	2.396	0.003(0.064)	−0.061(0.089)	−0.052(0.062)	−0.103(0.072)
	Test asymptomatic residents after anew resident case?	0.509	0.067(0.026)	0.018(0.029)	0.087(0.033)	0.045(0.029)
	Test asymptomatic staff after a newresident case?	0.445	0.070(0.024)	0.033(0.028)	0.068(0.028)	0.060(0.026)
	Ever test non-exposed resident?	0.374	0.017(0.016)	0.034(0.018)	0.048(0.017)	0.050(0.022)
	Ever test non-exposed staff?	0.642	0.001(0.013)	−0.002(0.018)	0.018(0.014)	0.016(0.016)
	Have in-home testing machine?	0.669	0.006(0.015)	0.010(0.015)	0.018(0.018)	−0.007(0.019)
D: Vaccinations, as of 6/20/21
	Share of residents vaccinated	0.800	0.013(0.006)	0.024(0.006)	0.042(0.006)	0.056(0.008)
	Share of staff vaccinated	0.567	0.022(0.007)	0.039(0.008)	0.066(0.009)	0.083(0.009)

Standard errors allow for arbitrary correlation across observations within a state. Other controls in the models include logged total beds; percent of residents that are female, under 65, black, Hispanic, on Medicaid (along with corresponding indicators for missing variables); for-profit status; acuity index; county-level COVID-19 cases per 1000 residents (measured 23 days prior to death); logged county population; and a full set of state fixed effects. All models are estimated via OLS except “time to test”, which is modeled as an ordered logit that is estimated via MLE.

Nursing homes may also prevent the spread of COVID-19 by following the CDC recommendation that all nursing home staff use PPE and wash their hands frequently. While these behaviors cannot be observed in our data, we are able to measure shortages in PPE (n95 masks, surgical masks, eye protection, gowns, and gloves) and hand sanitizer; thus, we test whether higher-quality facilities were less likely to have experienced such shortages between May 24th and September 13th of 2020.22 We again use linear probability models and control for the same set of potential confounders as above. Results are presented in Panel B of Table 6. For all five forms of PPE and hand sanitizer, the impact of quality is modest. There is suggestive evidence that higher-quality facilities have fewer shortages of all equipment except gowns, but effects are only statistically significant for a subset of coefficients in the N95 mask, glove, and hand sanitizer regressions.23

Higher-quality homes may also do a better job of testing residents for the virus. Fortunately, the CMS data allows us to generate a number of measures of testing intensity and speed.24 First, nursing homes report for the week ending September 13th that receiving test results takes “less than a day”, “between one and two days”, “three-to-seven days”, or “more than seven days.” Second, nursing homes report whether they tested any asymptomatic residents during the week ending September 13th in response to a new positive case.25 Third, homes report whether they have tested asymptomatic staff or residents when there is no knowledge of exposure. Finally, homes report whether they have their own testing machine.

In Panel C of Table 6, we present estimates of the effect of nursing home quality on these testing measures. We find that higher-quality homes receive test results faster than lower-quality homes (row 1) but these effects are not precisely estimated. Higher-quality homes are statistically more likely to test asymptomatic residents (row 2) and staff (row 3) following a new case; they are also statistically more likely to have ever tested non-exposed residents (row 4), but are no more likely to test non-exposed staff (row 5). There is no relationship between home quality and the likelihood of having an in-home testing machine (row 6).

Finally, we examine whether higher-quality homes were more likely to vaccinate residents and staff. Starting with the May 30th, 2021 survey, CMS began to ask nursing homes about the number of residents and staff that were fully vaccinated. In Panel D of Table 6, we report results from regressions that have the share of current residents and staff vaccinated as of June 20th, 2021 as the outcomes of interest. Here, there is a monotonic relationship between facility quality and vaccination rates. All else equal, resident and staff vaccination rates are 5.6 and 8.3 percentage points higher in five-star homes than in one-star homes. Both results are statistically significant at conventional levels and represent 7 and 14.6 percent increases, respectively, over the sample means of these outcomes.

Overall, higher-quality nursing homes prevent COVID-19 deaths, not by preventing the disease from entering the home, but by preventing its spread among residents. The methods by which homes prevent the spread are consistent with CDC guidelines. Higher-quality homes had more testing, got tests results faster, and had fewer PPE and staff outages. Given that the negative association between home quality and COVID-19 deaths does not persist past September of 2020, higher vaccination rates at high-quality homes clearly cannot explain the relationship. That said, we take the vaccination results as further evidence that high-quality homes followed CDC guidance along observable dimensions and, thus, likely followed along unobservable dimensions as well.

Non-COVID and total mortality

In addition to PPE use, adequate staffing, and robust testing, an early prevention method used by virtually all nursing homes was to refuse all outside visitors. On March 13th, 2020, CMS issued memorandum QSO-20–14-NH recommending that all facilities nationwide “should restrict visitation of all visitors and non-essential health care personnel, except for certain compassionate care situations, such as an end-of-life situation” (CMS 2020a). The memorandum also advised cancelling “communal dining and all group activities, such as internal and external group activities.” On May 18th, CMS issued memorandum QSO-20–30-NH, that provided a three-phase reopening plan for nursing homes (CMS, 2020d). The plan did not allow for outside visitors until a nursing home entered phase three, which (loosely) required (i) that COVID-19 cases in the outside community have declined for 14 consecutive days, (ii) no new cases within the nursing home for 28 days, (iii) no staff or PPE shortages, and (iv) homes have the capacity and supplies to test residents and staff weekly. In light of the continued spread of the virus and the strict reopening criteria, many nursing homes were still closed to visitors months later. Finally, on September 17th, 2020, CMS issued memorandum QSO-20–39-NH, that relaxed visitation guidelines, citing resident distress (CMS, 2020b). This memorandum was updated again on March 10th, 2021 in light of vaccine distribution.

In Fig. 2 , we use data from SafeGraph to document aggregate foot traffic to US nursing homes from the start of 2019 through mid-2021.26 The figure shows (i) a large drop in nursing home foot-traffic in late January of 2020, approximately when the first COVID-19 case was discovered in the US and the World Health Organization (WHO) declared COVID-19 a global health emergency, followed by (ii) another drop in early March, when WHO declared COVID-19 a pandemic and President Trump declared a national emergency. Foot traffic rose slowly through August of 2020, but surprisingly, has remained mostly unchanged since then, despite several changes in the CMS visitation guidelines outlined above.

Fig. 2

Total Visits to US Nursing Homes by Calendar Day, Safegraph

This figure plots the total number of visitors to US nursing homes (NAICS code 623,110) by calendar-day, from the first of January 2019 through the end of June 2021. The data come from SafeGraph's Weekly Places Patterns data series, which uses cell phone location services to produce hourly counts of foot traffic to about 4 million points of interest in the US. The aggregate counts provided here are adjusted for the number of devices per person in the state, as is recommended by SafeGraph.

There may be some unfortunate downsides to these early policies that could have negatively impacted nursing home resident health. First, these policies may have generated extreme isolation among facility residents. A survey of nursing home residents from early fall of 2020 documents massive declines in resident interactions with outside visitors and fellow residents, trips off site, and even trips outside for fresh air (Montgomery et al., 2020). In the introduction, we noted that some observers caution that among individuals with Alzheimer's, isolation may be deadly. Second, without group meals or meals supervised by staff, residents may not have been eating as well. In their analysis of Connecticut nursing homes during the early stages of COVID, Levere et al. (2021) found a large decline in patient weight among residents, a change they attribute to the isolation caused by facility safety protocols. Third, the lack of communal activities may have reduced exercise and increased the time patients spent in bed. Levere et al. (2021) also found increases in bed sores among nursing home residents during the early stages of COVID. Fourth, a number of authors have documented large declines in medical care use, especially in the Medicare population, as a result of the COVID-19 pandemic (Bosworth et al., 2020; Ziedan et al., 2020; Cantor et al., 2020; Clemens et al., 2021). We suspect declines in medical care use were mostly generated by precautionary behavior on the part of residents and their families; even the most stringent stay-at-home orders provided exemptions for medical care. There could be some supply response if providers limited the types of visits or reduced hours. Within nursing homes, staff or residents themselves may have been less likely to travel off-site to visit a medical care provider.

Given the potential downsides of efforts to control the spread of the virus, we next examine whether high-quality nursing homes also managed the non-COVID risks created by isolation. In particular, we estimate Eq. (1) above, but with non-COVID deaths as our dependent variable. Like our analysis of COVID-19 deaths, we estimate effects across seven different time periods. The second panel of Table 3 (rows 8 through 14) contains parameter estimates. As of May 24th, 2020 (row 8), nursing home quality had no statistical effect on non-COVID deaths in nursing homes; however, by September 13th, 2020 (row 9), we document a large, positive, statistically significant impact of nursing home quality on non-COVID deaths. Five-star homes have about 11 percent more non-COVID deaths than one-star homes and deaths are increasing monotonically with quality. Rows 10 and 11 shows that the relationship between quality and cumulative non-COVID deaths has strengthened with time, as do rows 12 through 14, which look at non-COVID deaths occurring in early, middle, and more recent time intervals. That the relationship has strengthened with time is consistent with the narrative above. The longer patients are exposed to isolation from others and/or regular medical care is not received, the greater risk they face. Moreover, the patterns in Fig. 2 show that despite growing vaccination rates through early 2021, traffic in and out of nursing homes remained well below pre-pandemic levels. We repeat all robustness tests performed for COVID-19 deaths (reported in Table 4) for non-COVID deaths in Appendix Table A6, where non-COVID deaths are measured on September 13th, 2020. Results are robust to all tested specifications.

Table A6

Robustness Analysis, non-COVID deaths measured 9/13/2020 unless stated otherwise.

			Overall star rating
Model	Sample Mean	Obs	2-star	3-star	4-star	5-star
(1) Baseline (Table 3, row 9)	10.15	14,905	0.034	0.070	0.088	0.114
			(0.029)	(0.031)	(0.034)	(0.041)
(2) Model (1), but Poisson	10.15	14,905	0.045	0.051	0.043	0.059
			(0.052)	(0.085)	(0.072)	(0.057)
(3) Model (1) but OLS with ln(deaths+1)	8.84	11,532	0.033	0.091	0.118	0.123
			(0.038)	(0.038)	(0.042)	(0.053)
(4) Model (3) but add county FE	1.84	14,905	0.041	0.073	0.099	0.127
			(0.034)	(0.034)	(0.036)	(0.047)
(5) Model (1) but OLS with inverse hyperbolic sine of deaths	1.84	14,905	0.069	0.071	0.127	0.159
			(0.046)	(0.046)	(0.049)	(0.064)
(6) Model (5) but add county FE	2.29	14,905	0.047	0.089	0.117	0.148
			(0.041)	(0.040)	(0.044)	(0.056)
(7) Model (1) but OLS with inverse hyperbolic sine of death rate (per 100 beds)	2.29	14,905	0.080	0.088	0.151	0.186
			(0.055)	(0.055)	(0.059)	(0.077)
(8) Model (7) but add county FE	2.29	14,905	0.047	0.089	0.117	0.148
			(0.041)	(0.040)	(0.044)	(0.056)
(9) Model (1) but add controls for staff hours per resident day	2.29	14,905	0.080	0.088	0.151	0.186
			(0.055)	(0.055)	(0.059)	(0.077)
(10) Model (1) but add controls for republican share of county	10.15	14,558	0.031	0.064	0.081	0.112
			(0.030)	(0.033)	(0.035)	(0.042)
(11) Model (1) but add controls for any staff and resident cases, as well as counts	10.15	14,888	0.033	0.071	0.090	0.118
			(0.029)	(0.031)	(0.034)	(0.041)
(12) Model (1) but add controls for shortage counts	10.15	14,905	0.032	0.072	0.087	0.112
			(0.029)	(0.031)	(0.034)	(0.039)

Standard errors allow for arbitrary correlation across observations within a state. Other controls in the models include logged total beds; percent of residents that are female, under 65, black, Hispanic, on Medicaid (along with corresponding indicators for missing variables); for-profit status; acuity index; county-level COVID-19 cases per 1000 residents (measured 23 days prior to death); logged county population; and a full set of state fixed effects.

There is some concern that our findings can be explained by higher-quality nursing homes intentionally misreporting COVID-19 deaths as non-COVID deaths, potentially in an effort to protect their reputation. This concern is somewhat mitigated by Fig. 1. Were COVID-19 deaths consistently reported as non-COVID, we would expect stark rises in non-COVID deaths during the summer and winter waves of 2020; we see neither. Moreover, misreporting is most likely early in the pandemic and results using cumulative deaths as of September are nearly identical to those using deaths between May and September. Finally, the result is present in nursing homes even after COVID-19 vaccines were introduced and disease mortality fell to the lowest level in months.

Yet another alternative explanation of our findings is that COVID-19 “harvests” the most fragile residents from low-quality homes, meaning they are not around to die of non-COVID causes.27 We address this concern, and further address the misreporting concern, by re-estimating Eq. (1) with total nursing home deaths as our dependent variable. We report parameter estimates in the third panel of Table 3 (rows 15 through 21). Our results show that there is not any period of time over which high-quality homes experienced fewer total deaths than low-quality ones, even very early in the pandemic (row 15) when statistically, high-quality homes were experiencing far fewer COVID-19 deaths (row 1).28 By December 6th, 2020, high-quality homes had experienced statistically more total deaths than low-quality homes (row 17) and the gap between the two has grown over time (row 18). The last three rows of Table 3 show the urgency of the problem. From summer 2020 (row 19), to fall 2020 (row 20), and finally winter/spring 2021 (row 21), the positive relationship between nursing home quality and total deaths has increased in magnitude. In the latter of these three periods (row 21), five-star homes experienced 17.5 percent more total deaths than one-star homes. On a base of 11 deaths per home, this amounts to roughly two additional deaths in five-star homes over just a four-and-a-half-month period. By our estimates, all of these excess deaths are due to non-COVID causes.

A related concern to the one above is that due to patient churn in and out of nursing homes over time, the composition, and therefore underlying health, of residents may change. For example, it is plausible that fear of COVID-19 led the wealthiest, healthiest potential residents to avoid nursing homes. For this behavior to drive our findings – specifically, our finding that the relationship between quality and total deaths increases over time – it would need to be the case that the average resident at higher-quality homes became sicker, without a similar change occurring at lower-quality homes. Unfortunately, as we do not observe changes in average patient acuity within the home over time, we cannot test this theory. Alternatively, selection could lead higher-quality homes to have higher occupancy rates than lower-quality homes over time. Indeed, over the life of the surveillance data, occupancy rates at high 5-star homes go from three percentage points higher than 1-star homes to just under five percentage points higher. That said, when we control for the home's occupancy rate in our total death models, our results are very similar.29

In a final attempt validate the important role that visitation restrictions play in our findings, we look for empirical evidence that high-quality nursing homes allowed fewer visitors during the COVID-19 pandemic. To this end, we matched the individual nursing homes in our main data file to the foot traffic data in the SafeGraph point of interest series. Using zip-code and longitude/latitude coordinates in both data files, we were able to identify 12,300 homes with both foot traffic and nursing home quality data. Among these homes, we calculated average daily foot traffic (i) in January 2020 (i.e., pre-pandemic) and (ii) between January 1st, 2021 and April 25th, 2021. We then calculated the percent change from (i) to (ii). On average, foot traffic in nursing homes declined by 34 percent (s.d. 18). We then regress the percent decline in foot traffic on home quality, using a variety of specifications (e.g., including additional controls, state fixed effects, and county fixed effects, and with sample limitations for nursing home size and measurement error).30

We find that while quality is typically negatively associated with the change in foot traffic, the effects are never statistically different from zero; thus, we do not report results here, but they are available upon request. In addition to measurement error, the fact that SafeGraph does not distinguish between the cell phones of visitors, staff, and residents makes the foot traffic data an imperfect proxy for “number of visitors.” This feature of the data poses a challenge to our validation exercise if, for example, high-quality homes experience a decline in visitors, while low-quality homes experience staff shortages (a reality that we document above.) Another concern is that the residents of high-quality homes are both younger and wealthier, meaning they and their visitors are more likely to possess cell phones. As SafeGraph expanded their network over the course of the pandemic, higher cell phone densities in higher-quality nursing homes could boost foot traffic counts, even as the number of visitors declined.

Conclusion

The COVID-19 pandemic has ravaged residents of nursing homes with roughly one fifth of COVID-19 deaths coming from this group. Not surprisingly, the impact of the pandemic varied in some systematic ways across homes. Initially, higher-quality homes were much more successful at limiting the impact of the pandemic, primarily by preventing the spread of the disease once it entered the nursing home, but these differences declined over time. Between January of 2020 and April of 2021, cumulative COVID-19 mortality was lower in higher-quality nursing homes, all else equal; however, starting sometime in the Fall of 2020, the marginal death counts from COVID-19 were no different across nursing home of different quality levels. This finding suggests that higher-quality homes adapted quickly at the start of the pandemic, while lower-quality homes took more time to understand how to effectively contain the virus among residents.

Our results raise a new concern about higher-quality facilities, in that they have higher non-COVID mortality than lower-quality places. This finding is not due to a misclassification of deaths, as the relationship persists even after COVID-19 vaccines were introduced and COVID-19 mortality fell to a fraction of the levels seen at the height of the pandemic. It is also not due to harvesting – the notion that lower COVID-19 deaths in higher-quality homes might mean more residents are available to die from other causes – as higher-quality homes have higher aggregate mortality. A more troubling aspect of our findings is that as the home quality/COVID-19 mortality gradient was eliminated over time, the quality/non-COVID-19 mortality gradient has steadily increased as the pandemic has aged.

Our paper is less successful at identifying the reason for the quality/non-COVID mortality gradient. Our results indicate that higher-quality homes were better at following CMS guidelines designed to control the spread of the virus, such as having PPE equipment on hand, not having staff shortages, more frequent testing of both residents and staff, and having higher staff and resident vaccination rates. It is logical to assume then that higher-quality homes were also better at generating more distance between residents and the outside world by preventing building entry and isolating residents from one another. Anecdotal reports from doctors, nurses, and resident family members (Aronson, 2020; Paulin, 2020; Graham, 2020), as well survey data from residents themselves (Montgomery et al., 2020), document frightening levels of depression, loneliness, and hopelessness. Consistent with these reports, CMS updated their visitation guidelines in mid-September of 2020 to combat the mental and physical distress of isolation.31 Analysis of the Minimum Data Set by Levere et al. (2021) during the early stages of the pandemic suggests that nursing home residents declined in health along dimensions consistent with increased isolation, such as unexplained weight loss, declines in cognitive function, and increases in depressive symptoms. This is, however, not the only pathway by which the pandemic could have altered non-COVID mortality. Isolation policies may coincide with, or even cause, reductions in routine medical care, residents’ physical activity, or food consumption. The rise in bed sores and drop in weight found in Levere et al. (2021) suggest these other mechanisms may play a role.

The good news from Fig. 1 is that once vaccines became available, mortality declined considerably. Weekly deaths of nursing home residents with COVID-19 peak at 6082 the week ending December 20th, 2020. By the week ending May 21st, 2021, this number was 179, a 97 percent drop. Fig. 1 provides some hope that things might be returning to normal within these group quarters. Despite this positive trend, not all has returned to normal. The decline in visits to nursing homes in the early stages of the pandemic as measured by cell phone movements was dramatic. By April 1st of 2020, visits to nursing homes were down 51 percent compared to January of 2020. As vaccines became available and COVID-19 deaths in nursing homes fell considerably in early 2021, visits to nursing homes increased but never returned to anywhere near pre-pandemic levels. By the end of June 2021, visits were still down by 35 percent compared to the January 2020 levels. These numbers are imperfect measures of visits because they include counts of family and friends visiting residents plus, entrances by workers, traveling nurses and aids such as physical therapists, etc. It is also not clear whether the persistent decline in visits to nursing homes is supply driven (continue restrictions on the part of nursing homes) or demand driven (friends and family do not want to visit the nursing homes). Despite these caveats, the fact that foot traffic in nursing homes never returns to anywhere near normal levels could explain another feature of our findings – that the impact of higher-quality homes on non-COVID mortality continues to grow with time.

The COVID-19 pandemic presented a unique challenge for nursing homes. Early CMS directives and various state regulations for nursing homes prioritized reducing resident and staff exposure to COVID-19. There was little discussion about the downside risks associated with reducing visitors, communal activities, and resident travel out of the home. Our results suggest that more balanced policies and guidelines that emphasize maximizing the health of residents, rather than just minimizing risk to one disease, may have improved outcomes. For a period of time, CMS and the news media at large measured nursing home COVID-19 performance using cases and deaths only, meaning the logical response on the part of the nursing home was to minimize these counts regardless of the cost. In retrospect, the tone of the discussion and the measurement of outcomes may have led to some deadly consequences. As economists continually stresses, there are benefits and costs to all regulations.

Author statement

This is the author statement for the paper referenced above. The two authors of this paper were engaged in all aspects of the production of this manuscript including: Conceptualization; methodology; software; validation; formal analysis; investigation; resources; data curation; writing;, visualization; supervision; project administration; and funding acquisition.

Table A2

National Estimates of COVID-19 Deaths in Nursing Homes from Inflated CMS Data,As of 8/15/2021.

Number	Death Count
(1) National estimates as of 5/24/2020	25,354
(2) Inflate by 14.5%	29,030
(3) Change between 8/15/2021 and 5/24/2020	108,288
(4) total (3) + (2)	137,318

Row 1 measures death counts among nursing home residents in all 50 states as reported in the CMS COVID-19 surveillance data discussed in Section II.A. Row 2 inflates this figure by the estimated 5/24/2020 undercount calculated in Table A1. Row 4 adds to this figure deaths reported to CMS after 5/24/2020 (Row 3) which we've show are more accurately reported.

Table A4

Distribution of Inspection Ratings in CMS Nursing Home Data.

Star rating	Overall rating	Inspection rating	QM rating	Staff rating
1	0.151	0.193	0.049	0.075
2	0.193	0.236	0.126	0.247
3	0.178	0.224	0.196	0.276
4	0.213	0.233	0.251	0.215
5	0.252	0.102	0.363	0.114
missing	0.014	0.014	0.014	0.074
Observations	15,110

Sample construction is discussed in Appendix Section A. Star ratings are taken from the CMS website.

Table A5

Correlation if the Inspection Ratings in CMS Nursing Home Data.

Star rating	Overall rating	Inspection rating	QM rating	Staff rating
Overall rating	1.000
Inspection rating	0.860	1.000
QM Rating	0.504	0.233	1.000
Staff rating	0.478	0.221	0.212	1.000

Sample construction is discussed in Appendix Section A. Star ratings are taken from the CMS website.