JUE Insight: Is Hospital Quality Predictive of Pandemic Deaths? Evidence from US Counties.

In the large literature on the spatial-level correlates of COVID-19, the association between quality of hospital care and outcomes has received little attention to date. To examine whether county-level mortality is correlated with measures of hospital performance, we assess daily cumulative deaths and pre-crisis measures of hospital quality, accounting for state fixed-effects and potential confounders. As a measure of quality, we use the pre-pandemic adjusted five-year penalty rates for excess 30-day readmissions following pneumonia admissions for the hospitals accessible to county residents based on ambulance travel patterns. Our adjustment corrects for socio-economic status and down-weighs observations based on small samples. We find that a one-standard-deviation increase in the quality of local hospitals is associated with a 2% lower death rate (relative to the mean of 20 deaths per 10,000 people) one and a half years after the first recorded death.

Introduction

The COVID-19 pandemic has highlighted the importance of quality of medical care. There is a large variation in this across geographical space in the US (Finkelstein, Gentzkow, Williams, 2016, Skinner, 2011, Kibria, Mancher, McCoy, Graham, Garber, Newhouse, 2013). Yet relatively little of the research examining COVID outcomes at the spatial level has considered the association with the quality of hospital care accessible to residents. Far more studies have focused on the association with community-level or state measures to control the spread of the pandemic (e.g., stay-at-home orders) and features of the local community (e.g., trust, socio-economic status of residents, political affiliation, ethnic composition).

This paper examines whether the quality of medical care at the US county level is associated with better outcomes. Specifically, we ask whether the quality of hospital care accessible to communities before the pandemic is associated with mortality due to COVID-19. We examine this from the date of the first documented occurrence in the US (22nd January 2020) up to 531 days later, a period covering learning about the new disease, announcement and implementation of county and state level mobility restrictions to control its spread, and the large-scale vaccine roll-out. We use measures of the quality of hospital care derived from those used in the national flagship Hospital Readmission Reduction Program [HRRP]. This program applies financial penalties to hospitals that are deemed to be of poor quality, as measured by excess readmission rates for three key conditions: acute myocardial infarction (heart attack) [AMI], heart failure [HF], and pneumonia [PN]. For our purposes, pneumonia is particularly relevant as COVID-19 is a respiratory illness.1 We adjust the published annual measures to derive a medium-run fixed measure of pre-pandemic quality, which measures the probability of being penalized in pneumonia treatment (Kunz et al., 2020). Our measure addresses concerns in using the published rates, including the extent of risk adjustment, regression towards the mean, and the influence of low volumes on the rate precision, and also accounts for the fact that some hospitals are never penalized whilst others always are. Importantly, our analysis deals with the fact that hospital markets in the US are larger than counties. The appropriate market definition is one that is based on the travel patterns of individuals who have used emergency care, and that may cross county boundaries (e.g. Chandra, Finkelstein, Sacarny, Syverson, 2016, Wennberg, Cooper, 1998). Our measure accounts for variation in quality at this market level. We also account for a large array of factors at the county level shown to be associated with the spread of the virus, including the number of confirmed cases, access to health insurance, the extent of vaccination, county political affiliation, and social capital.

Given the extensive literature on COVID 19 and community-level differences, we hypothesize that the relationship between quality and death rates will vary by spatial and social characteristics of communities. We also explore potential mechanisms by which quality matters: in particular, are there aspects of the quantity and composition of healthcare supply available to county residents that are particularly relevant? We, therefore, examine heterogeneity in the association between deaths and quality by a large set of characteristics of the country and its healthcare supply.

We find that pre-pandemic hospital quality correlates strongly with higher death rates from COVID-19 at the county level. However, our quality of care measure is not predictive of cases (i.e., the spread) of COVID-19, vaccination rates, or both all-cause and AMI pre-pandemic mortality after accounting for potential county-level confounders and state-level fixed-effects. As these outcomes are all likely to be associated with unobserved measures of health and thereby with COVID-19 death rates, this supports our contention that we are isolating a hospital quality effect. Assessing spatial heterogeneity in this relationship, we find that the association of quality and lower death rates is stronger in counties where the population is less economically advantaged, has less access to health insurance, and has lower levels of social capital. In terms of potential mechanisms, we find evidence that the relationship between higher quality and lower deaths is stronger where the population has access to more hospital-based doctors.

Our finding that quality is associated with the COVID-19 death rate contrasts with a handful of studies examining deaths and various measures of hospital quality early on in the pandemic in the USA. Kanter et al. (2020) and Knittel and Ozaltun (2020) find no association at the county level between deaths and current ICU beds per capita. Desmet and Wacziarg (2020) find no association with hospital risk-adjusted 30-day mortality rates for AMI, HF, and PN. Our longer time period allows us to show that the association between deaths and hospital quality was initially zero and then become negative (i.e., better quality was associated with fewer deaths) after about a year after the first death was recorded. This change over time mirrors findings by Das Gupta et al. (2021) for nursing homes. Desmet and Wacziarg (2021) and Allcott et al. (2020) more generally show changing pandemic patterns over time. This time pattern may have confounded studies that examined only the initial stages of the pandemic. In addition, other studies of hospital quality and outcomes for the USA have not taken into account travel patterns that cross county boundaries in measuring quality; rather, they relied on within-county measures. Our finding that the association between quality and deaths is larger in less economically and socially advantaged counties potentially indicates a higher return to the quality of medical care during the pandemic in disadvantaged communities. Our finding that hospital doctors, but not other hospital facilities, matter for the treatment of COVID-19 echoes the detailed hospital-based study of Souza-Silva et al. (2022).2

The paper is structured as follows. In Section 2, we discuss how we assign hospital referral region quality to counties, measure quality, the data we use, our empirical specification, and examine heterogeneity and possible healthcare supply-side mechanisms that may determine quality. We present the results in Section 3 and discuss these in Section 4.

Estimation strategy and data

Our aim is to examine the association, at the county level, between deaths per 10,000 persons and the quality of care accessible to residents. To do this, we need to assign quality measured at the relevant market level (i.e., Hospital Referral Regions - HRR thereafter) to each county and use a measure of hospital quality that is not driven by year-on-year variation, which may primarily reflect noise. We begin by describing how we assign hospital quality measured at the HRR level to a county and then define our preferred measure of quality.

Assigning a HRR measure to the county level

First, we classify an HRR by the average quality, , of the hospitals it covers. Second, we assign to each county a weighted average of the quality exposure, which is equal to the fraction of zip codes that belong to the referral network.3 Weights are the number of zip codes in a county that merge into an HRR, divided by the total number of zip codes in the county, i.e.For example, Autauga County comprises 13 zip-codes, 4 of which belong to HRR Birmingham and 9 to HRR Montgomery, with hospital quality averages of 0.34 and 0.28, respectively (where lower numbers reflect poorer quality). In this example, the assigned weighted quality exposure is 0.30. This places Autauga County in the 62.94th percentile of the county quality distribution, which has a mean of 0.38 and a standard deviation of 0.12. We refer to this approach as “zip-weighted”. We average across hospitals within a given HRR to derive quality at the HRR level and use Eq. (1) to calculate exposure at the county level.4 We use three other weighting methods as robustness checks. First, we calculate quality exposure as the average of all HRRs a county has referral ties to without weighing by how many zip codes in a county belong to an HRR. In this case, Autauga would have a quality of .5 We refer to this as an “equally-weighted” quality exposure. Second, we calculate a zip-code population-weighted approach. Third, we use a recently developed crosswalk between HRRs and counties that also accounts for the population from the Dartmouth Atlas (Nanda et al., 2021).6 7

HRR-level quality measure

Our measure of medium-term HRR quality, , and variations thereof is based on the publicized annual penalty status of hospitals due to excess pneumonia readmissions, administered by the Centers for Medicare & Medicaid Services [CMS]. Readmissions for emergency conditions are frequently used as a measure of hospital quality (e.g. Gaynor et al., 2011; Axon and Williams, 2011). From 2012 onwards, Medicare reduced reimbursements to hospitals if their risk-adjusted excess readmission ratio is one of three emergency conditions (PN, AMI, and HF) was larger than one in a three-year period, that is, if the hospital had more readmissions than the average hospital with a similar case-mix (McIlvennan et al., 2015). This published CMS measure of excess readmissions predicts market share (Chandra et al., 2016), is highly correlated with other important measures of hospital quality (Kunz et al., 2020) and has been validated by an instrumental variable approach based on rotational random ambulance-assignment (Doyle et al., 2019). Risk-adjusted readmissions may not measure latent quality if there is selection by patients (Doyle et al., 2015) or hospitals (Gupta, 2021). However, as the measure is based on admissions for emergency conditions, there is limited scope for self-selection (Chandra, Finkelstein, Sacarny, Syverson, 2016, Garthwaite, Ody, Starc, 2020). In our context, patient and hospital selection are possibly even less of a concern since we aggregate the average quality of hospitals to the HRR level. In addition, it is unlikely that patients with emergency conditions travel beyond HRR borders. However, as the official CMS risk adjustment is only based on age, sex, and co-existing conditions, we further risk adjust for local socio-demographic variation by extracting a medium-run measure of quality, using a panel fixed-effect approach for the financial years 2011–2015 as in Kunz et al. (2020). In brief, Kunz et al. (2020) derive a (condition)-hospital-specific measure of medium-run quality from longitudinal observations of the penalty status. Further adjustments are undertaken so that the measure does not suffer from regression-to-the-mean and the measure shrinks automatically to a no-quality benchmark.8

In robustness tests, we examine three other measures of hospital quality, which we expect to be less appropriate than our preferred measure. The first is the average of three penalized conditions constructed in the same way as the measure for pneumonia and measures the conditional hospital propensity to be fined in any of the three conditions (AMI, HF, and PN), see Kunz et al. (2020). As this is the average of three quality measures, two of which are not primarily respiratory conditions, it is a noisier signal of quality related to COVID treatment and we, therefore, expect the association to be less precise. The second and third measures do not have the regional risk adjustment and peer benchmarking embedded in our derived quality measure. These are the CMS published raw excess readmission ratio for pneumonia readmissions (risk adjustment only based on age, gender, and comorbidities) and the CMS published 30-day mortality rate following pneumonia admissions (no risk adjustment).

Outcome measures

Our primary outcome is the cumulative county-level deaths per 10,000 individuals. The data are from the Centers for Disease Control and Prevention [CDC] and state- and local-level public health agencies and are aggregated by USAFacts. We use data from January 22nd, 2020 (first recorded death) to the 1st of July 2021 (531 days later).9 Our data are cumulative deaths for every day from day 1 to day 531.

We examine four other outcomes in robustness tests. These are the cumulative number of cases per 10,000, the cumulative number of fully vaccinated persons per 10,000 for our sample period, and the CDC all-cause and AMI mortality rate for each year between 2016 and 2019.

Empirical specification

We use OLS to estimate the association between HRR quality and monthly cumulative COVID-19 deaths since the first observed COVID-19 death in the US:where is the cumulative deaths in county divided by population in month , the covariate vector contains county level and HRR-weighted level correlates, are state fixed-effects and is the error term. We use county-level clustered standard errors throughout. We begin by estimating this for each month separately to show the time trends in the association. In our main analyses, we focus on the last date in our data and run a single cross-sectional regression for this date, using robust standard errors.

The main concern in estimating the spatial correlation between hospital network quality and county-level death rates is the presence of omitted variables that correlate with both the death rate and weighted quality. We take several steps to overcome this. First, as noted above, our measure of hospital quality is the fixed-effect extracted from measures for the years 2011–2015, i.e. quality before the onset of the COVID-19 pandemic, after accounting for several potential determinants of quality of health care at the hospital, county, and HRR level. Second, we include a large number of controls at the county level that have been examined in previous research on the association between cases (or deaths) and county-level characteristics (detailed in Section 2.5). Third, we include state fixed-effects to control for differences in policy responses across states (Knittel and Ozaltun, 2020).

Finally, we undertake three additional tests. We first examine the cumulative number of confirmed cases to assess whether our quality measure is actually measuring hospital quality, as opposed to potential confounders that affect deaths via an increase in the spread of the virus. If hospital quality is predictive of cases, it may indicate that we are not isolating the hospital quality but instead have a measure correlated with underlying conditions that also affect the spread of the virus. Second, we assess whether higher pre-pandemic health care quality predicts vaccination rates. A relationship would suggest that unobserved factors correlated with quality may be driving our results. Third, to examine (absence of) pre-trends, we examine the association between pre-pandemic all-cause mortality (from the CDCs) and our quality measure for the years 2016–2019. This provides an assessment of whether pre-morbidity levels in certain areas are higher than in others, conditional on our covariates. If this were the case, then hospitals in these areas would already be serving populations with poorer pre-pandemic health, and this, rather than hospital quality, could be driving our results. As the second test of pre-trends, we examined the relationship between our quality measure and mortality rates from heart attacks (AMI). As COVID, these require timely hospital care, and AMI mortality is often used as measures of quality of hospital care, e.g., Gaynor et al. (2011).

To test for differences in the relationship between quality and death rates by spatial and social characteristics of communities and to explore the mechanism by which quality matters, we examine heterogeneity in the association between deaths and quality by a large set of characteristics of the country and its healthcare supply. We replace the quality measure in Eq. (2) with two interactions,where is a set of area characteristics (detailed below) and indicates the median. This is similar to estimating separate regressions but is more efficient as the fixed-effects, and county-level measures are estimated using the full sample.10

Control variables

At the county level, we include a large number of local controls, as informed by the extensive research on the correlates of COVID-19 at this spatial level.11 Our controls are measures of the economic and social status of the population (poverty rate, median household income, shares of different levels of educational attainment) and demography (population density, urbanity, average household size, single-adult households ages 65 and older, share aged 65 and older; residential segregation); ethnicity of the population (share of foreign-born residents, minority shares – non-Hispanic black, Hispanic, and other); measures of community health (population share in poor or fair health, population life expectancy, and premature deaths); measures of the population access to health care and primary care (share without health insurance, access to local primary healthcare); health behaviors (share with flu vaccinations and smoking rates); air pollution; commuting patterns (driving alone to work or having a long commute); and measures of local social capital (community and institutional capital and Republican vote share in 2020).12 At the HHR level, we include the availability of hospital care and hospital market concentration.13 Descriptive statistics for the controls are in Table A3.14

For our assessment of spatial heterogeneity and possible mechanisms, we examine three sets of area characteristics. The first two sets are subsets of the control variables. The first contains county measures of demographic composition and socio-economic status (SES). These include the population share living in poverty, share without health insurance, age-adjusted premature mortality rate, residential segregation, and share of minorities (i.e., 1 - share of white non-Hispanic) in the county. The second set is the county’s social and institutional capital and political affiliation, measured by the Republican vote share in 2020. The third set focuses on healthcare supply features to characterize what aspects of healthcare supply may be driving any associations we find. The measures include the number, type (teaching or not), and staffing mix of hospitals accessible to the county and whether the state implemented financial COVID related support for hospitals during the period we examine.15

Results

Fig. 1, Panel A, shows the spatial distribution of our measure of the quality of hospital care. It shows substantial variation in quality across counties, with lower quality (darker colors) being concentrated on the eastern side of the USA. Panel B presents the measure after conditioning the control variables, allowing for differences in the underlying health of the population. The impact of the inclusion of controls is to spread the locations with poorer quality outside the eastern side of the USA.

Fig. 1

County-level spatial distribution of HRR quality exposure and deaths per 10T population Note: Panel A presents our zip-code weighted measure of county-level exposure to high quality Hospital Referral Regions (HRR), measured as the risk-adjusted longitudinal hospital penalty status for readmissions from pneumonia admissions based on the years 2011–2015. Panel B presents the residualised measure regressing out covariates considered in the main analysis, see Fig. 2 for the full set of covariates. Appendix Figure A1 shows analogous maps for the outcome measure of deaths from Covid-19 per capita. Source: CMS 2011–2015, Dartmouth Atlas of Health Care, and others described in Appendix Table A3, own calculations.

Fig. 2

Regression coefficients and standard errors of quality exposure on cumulative deaths from Covid-19 Note: Figure shows coefficients and 95 (dark) and 90 (light) percent confidence intervals (cluster-robust standard errors on the county-level) from pooled regressions (higher quality corresponds to higher values) for 30 day intervals since pandemic began, conditional on cases, from day 331 (vertical line) on-wards also on vaccinations, and on county-level covariates (percent poverty (all ages), median household income, share of people people uninsured, premature deaths, poor or fair health, poor physical health days, poor mental health days, physical inactivity, life expectancy, air pollution particulate matter, flu vaccinations, preventable hospital stays, adult smoking, drinking water violations, driving alone to work, long distance commute driving alone, share non-Hispanic black, share Hispanic, share other minorities, population density, urban area, long commute driving alone, percent age 65 and older, foreign born, less than high school, high school, some college, associate degree, average household size, households 65 and older living alone, community health index, institutional health index, county index, republican vote share 2020) and HRR exposure measures using the same weighting as the outcome (number of hospitals and hospital competition (HHI) in number of beds). See A2 for analogous results for each day and A3 for descriptives and further sources. Source: USFacts (Jul 7, 21), CMS 2011–2015, Dartmouth Atlas of Health Care, and others described in Appendix Table A3, own calculations.

Fig. 2 presents the evolution of the association between quality and deaths over time. It presents estimates of Eq. (2) for each 30-day window since the pandemic began, including as controls cumulative cases, state fixed-effects, and our extensive set of covariates. From day 331, the regressions further include the county-level vaccination rates. There is no statistically significant association between hospital quality and deaths for about a year after the first death. However, a little before the introduction of vaccinations, hospital quality becomes positively associated with lower death rates. This association becomes statistically significant after a year or so after the first deaths.

Table 1 present the association of deaths (Panels A and B), cases (Panel C), and vaccinations (Panel D) at day 531. For deaths, the estimates are conditional on cases, and state fixed-effects in Panel A and on all covariates in Panel B. Panel B also presents the predicted difference relative to the mean for a one-standard-deviation change in our quality exposure measure (i.e., 0.12). Column (1) presents our preferred measure of quality. Columns (2) - (4) show robustness to different weighting methods using our preferred measure of pneumonia readmissions as the measure of quality, and Columns (5)-(7) show robustness to alternative quality metrics.

Table 1

Main results and alternative weighting and quality measures.

Dependent variable: Cumulative deaths and cases on July 1, 21 (531 days after first death)
		Alternative weighting			Alternative quality metric
		Zipcode	Dartmouth	Equal	Pooled	Readmission	Mortality
	Base	Population	Atlas	weight	quality	ratio (PN)	rate (PN)
	(1)	(2)	(3)	(4)	(5)	(6)	(7)
Panel A. Death per 10,000 capita, conditional on cases and state fixed effects
quality	-3.382	-2.903	-3.277	-5.955	0.540	-20.127	-131.393
	(1.976)	(1.954)	(1.867)	(2.102)	(2.977)	(7.095)	(33.194)
R^2	0.299	0.299	0.289	0.300	0.299	0.300	0.303
Panel B. Death per 10,000 capita, conditional on full set of controls
quality	-5.282	-4.929	-4.442	-7.105	-4.731	-13.258	-74.027
	(1.791)	(1.770)	(1.700)	(1.871)	(2.665)	(6.830)	(29.132)
Pre. change rel. to mean	-0.031	-0.029	-0.027	-0.040	-0.020	-0.025	-0.026
R^2	0.508	0.508	0.505	0.509	0.508	0.508	0.508
Panel C. Cases per 10,000, conditional on full set of controls
quality	-0.009	-0.022	-0.107	-0.033	0.380	0.002	-3.192
	(0.155)	(0.148)	(0.093)	(0.211)	(0.305)	(0.549)	(1.799)
Panel D. Vaccination per 10,000, conditional on full set of controls
quality	0.605	0.480	-0.813	-1.572	3.395	-11.784	44.534
	(1.172)	(1.112)	(0.873)	(1.224)	(2.677)	(5.000)	(23.001)
N	3,140	3,140	2,940	3,140	3,140	3,140	3,140
Mean deaths	20.52	20.52	20.60	20.52	20.52	20.52	20.52
Std. dev. quality	0.120	0.121	0.127	0.114	0.085	0.038	0.007

Notes: Table presents association between quality and deaths from COVID-19. Panel A shows cumulative deaths (conditional only on cases and state fixed effects), B adds all controls, C replaces the outcome in B with cases, and D with vaccination rates. Column (1) is our main specification corresponding to the last point in Fig. 2. In Columns (2)-(4), we use alternative ways to weigh the HRR quality. In (2), using population in each zip-code (based on census 2010 counts), in (3) using a newly developed crosswalk also based on a population derived by Nanda et al. (2021). There is a lower number of observations as their approach to assigning quality is slightly more restrictive but similar overall. In (4), we use the broadest measure that weights each HRR a county has access to equally. Columns (5)-(7) again use our preferred weighting approach but varies the measure of HRR quality. In (5), we use the same approach as in our main quality measure, but all emergency conditions are covered by HRRP, including acute myocardial infarction [AMI] as well as heart failure [HF]. In (6), we use the official but uncorrected for regional differences and small counts, thus raw, excess readmission ratio for pneumonia readmissions. In (7), we present the 30-day mortality rate following PN admissions (a non-risk-(nor regional differences)-adjusted indicator). In Panel B, we also present the Predicted change relative to the mean is calculated as (std = standard deviation of quality metric, which is 0.12 in Column 1 for example), both mean of deaths and standard deviation of quality are shown at the bottom of the Table. Source: USFacts (Jul 7, 21), CMS 2011–2015, Dartmouth Atlas of Health Care, and others described in Appendix Table A3, own calculations.

Panel A shows, as a benchmark, the raw correlation between measures of quality and deaths, allowing only for cumulative cases and state fixed effects. This is generally negative though the association with our preferred measure in column (1) - and that of two of the three differently weighted variants (columns 2 and 3) - is not statistically significant.

Panel B presents results conditional on all controls. It shows that exposure to higher pre-pandemic hospital quality in treating pneumonia patients is highly predictive of total lower realized death rates for all methods of assigning quality as measured by adjusted pneumonia admissions to a county (columns 1–4). Columns (5)-(7) show the results for other quality measures. The estimates in column (5) use our medium-run, pooled, a fixed quality measure for AMI, HF, and PN (Kunz et al., 2020). The association with deaths is negative but much less precisely estimated. As COVID-19 materializes as a respiratory condition, this weaker association with a broader quality measure (i.e., not just quality of treatment of a respiratory condition) provides support that we are showing a hospital quality-death relationship. Columns (6) and (7) show higher estimates of the association with the CMS published rates of readmissions and mortality from pneumonia, respectively. These measures have less risk adjustment than our measure, and these associations thus may be picking up a positive relationship between population health status and outcomes.

Using our preferred measure in column (1), a one standard deviation increase in quality exposure is estimated to decrease deaths by 3.1 percent from the mean (), or 21.2 deaths per 10,000 people. Whilst the coefficient estimates are higher for the other quality measures, the estimated magnitude of the percentage change relative to the mean is, in fact, very similar across the other weighting methods and measures of quality ranging between 2.5-3.1 percent (perhaps column 4 excepted, which is likely an upper-bound as it disregards any distance to hospitals).

In Panels C and D, we test whether unobservables drive these estimates by examining the cumulative association between cases (Panel C) and vaccinations (Panel D) and quality exposure, conditioning on the same controls as in the quality-death regressions. Panel C shows no significant association with cases for any of the weighting methods or measures of quality. Panel D shows no association between vaccination rates and our risk-adjusted quality measure (regardless of weighting or whether it is combined with measures of quality for other emergency conditions) conditional on the controls, though there is an unconditional association. As most confounding factors in the quality-death association would also confound the quality-case and the quality-vaccination associations, these results indicate that our primary analysis identifies a negative association of hospital quality with deaths. The significant associations with the two quality measures in columns (6) and (7) that do not correct for regional differences and small counts indicate that these measures are perhaps picking up unobserved heterogeneity in case-mix.

Table 2 presents the tests of pre-trends. Panel A presents results for all-cause mortality. Unconditionally, high hospital quality exposure is associated with substantially lower all-cause mortality rates. Conditionally, the pre-pandemic trends are very small and insignificant. Panel B examines AMI mortality. As the counts of this are low at the county level, we estimate a Poisson model. Panel A again shows an unconditional high correlation. Panel B shows that there is no significant association in any pre-pandemic years once we control for our full set of covariates. These tests suggest that the unobservable health status of the county population is not driving our results.16

Table 2

Pre-pandemic quality’s predictiveness of trends in all-cause and AMI mortality.

Dependent variable: Yearly all cause and AMI mortality
per 10,000 capita
	Pre-pandemic trends
	2016	2017	2018	2019
	(1)	(2)	(3)	(4)
	I. All cause mortality
Panel A. Raw correlation
quality	-9.90	-11.18	-14.71	-15.66
	(4.38)	(4.33)	(4.32)	(4.43)
Panel B. Conditional on full set of covariates
quality	2.37	-4.43	-1.40	-0.50
	(2.45)	(2.39)	(2.62)	(2.81)
N	3,103	3,107	3,102	3,105
Mean outcome	108.72	111.29	111.79	112.47
	II. AMI mortality - Poisson model
Panel A. Raw correlation
quality	-1.39	-1.14	-1.32	-1.41
	(0.20)	(0.20)	(0.21)	(0.21)
Panel B. Conditional on full set of covariates
quality	-0.39	0.39	0.08	0.17
	(0.23)	(0.23)	(0.23)	(0.24)
N	3,103	3,107	3,102	3,105
Mean outcome	4.20	4.10	3.99	3.72

Notes: See Table 1 notes. Table is constructed analogously, presenting as outcomes all-cause mortality and AMI – acute myocardial infarction or heart attack – from 2016 to 2019 – in the years after the quality measure was taken (2011–2015) and the beginning of the pandemic. CDC left censors for counties with 9 or less deaths. Here we use 0 as imputation, but the results are indistinguishable when imputing 9. Panel II uses Poisson regressions due to the much lower death counts and elevated frequency of 0 occurring. Source: CDC 2020, CMS 2011–2015, Dartmouth Atlas of Health Care, and others described in Appendix Table A3, own calculations.

Our results are also robust to the estimation method. Table A1 shows results for various non-linear estimators. The results confirm those above. Finally, we examine the relationship between quality and excess deaths in Table A2 Alsan et al. (2021). Calculating excess deaths at the county level results in very volatile estimates (see also, Ruhm, 2021). However, we find a negative but poorly estimated relationship with the quality.

Heterogeneity by county type

Having established a robust predictive value for pre-pandemic hospital quality on COVID deaths, we assess whether this predictive value differs by county-level characteristics, in Fig. 3 . The left-hand panel shows differences in estimates by measures of local area demographics (the top set) and social capital (the bottom set), the split being at the median of the relevant variable. The darker bars indicate the county is above the median on the relevant variable (or 1 for indicator variables); -values for the test statistic of significant differences are in parentheses.

Fig. 3

Heterogeneity by hospital market, county environment, and political and social cohesion Note: Figure shows coefficients and 95 and 90 percent confidence intervals from regressions of deaths on quality on day 531 of the pandemic, including an indicator for above or below median (or 0/1 for indicator variables, ie. a teaching hospital in catchment or state-level financial health support), interacted with this indicator variable. The regressions are otherwise analogous to those in Table 1, -values are based on F-test for the equality of the coefficients. The hospital capacity variables are based on pre-pandemic HRR-level information and aggregated to the county-level by zip-code population weights and are not used as controls in the main model. The State financial health support is based on ‘announced short term spending on healthcare system, e.g. hospitals, masks, etc.’ complied by the Oxford Covid-19 Government Response – note, it only record amount additional to previously announced spending, we define an indicator if there was any increase). Source: USFacts (Jul 7, 21), CMS 2011–2015, Dartmouth Atlas of Health Care, and others described in Appendix Table A3, own calculations.

In general, all estimates are either insignificantly different from zero or negative. For the demographic measures, quality matters significantly more in counties with a higher share of uninsured, a higher share of minorities, and ones located in metro areas. For the social capital measures, communities with lower levels of social and institutional capital have lower death rates where hospital quality is higher. There is no difference in the relationship by political affiliation of the county.

The right-hand panel examines possible mechanisms, examining whether and which health services supply features may be driving the association between higher quality and lower death rates. The results show that the effect of quality is significantly higher in counties where there are fewer hospital facilities, as measured by the number of hospitals, number of acute care beds, staff per head of population, and nurses (the latter difference is significant only at the 10 percent level). In contrast, counties with more hospital-based physicians have a stronger negative association between hospital quality and deaths. The bottom panel shows that there are no differences in state-level financial support, but in counties where vaccination rates were lower by day 531, quality matters more.

Taken as a whole, our heterogeneity results indicate that quality matters more where the population is more disadvantaged, has higher shares of minority populations, is less protected against the virus by vaccination and levels of social capital, and has access to fewer hospital resources. Put another way; it appears that hospital quality is a substitute for community capital. However, even conditional on these factors, quality has a larger association where there are more hospital physicians, suggesting one mechanism driving the association we find is access to a larger number of highly trained clinicians.

One further suggestive piece of evidence on the importance of what happens inside hospitals is the timing of the negative association between hospital quality and deaths. In Fig. 2 the estimated association begins to become negative and remains significantly below zero from around one year after the first death. This first suggests that it takes time to learn how to treat a new disease. The effect of such learning may take a while to manifest itself, particularly as the strain of the virus, the number, and the type of patients presented at hospitals change over time. Second, the date at which the association becomes negative also close to the FDA approval of two monoclonal antibodies as treatments for severe COVID cases.17 These treatments are almost exclusively used in hospital settings, and better quality hospitals may have been faster to adopt these, leading to the negative association between higher quality and lower deaths after their introduction. However, exploring both these ideas requires detailed data on treatments at the hospital level, and much of this data is not publicly available at scale. Within the confines of the present paper, we are thus not able to explore these hypotheses in more detail.

Discussion and conclusion

At the county level, we show that access to higher-quality hospitals correlates with lower cumulative death rates from COVID-19 by the middle of 2021, after many non-pharmaceutical interventions and the roll-out of vaccines. These associations are robust to many county-level factors shown to be associated with the spread of the virus (McLaren, 2020, Furzer, Miloucheva, 2020, Borgonovi, Andrieu, 2020, Friedson, McNichols, Sabia, Dave, 2020, Knittel, Ozaltun, 2020, Desmet, Wacziarg), and to controls for fixed state-level differences. The lack of association between our measure of hospital quality and COVID cases, vaccination rates and deaths from all causes, and AMI pre-pandemic suggests that the results are not driven by unobserved heterogeneity in health status. This finding is suggestive of a potentially causal interpretation of our results.

There are caveats. First, we assign to each county the quality of hospitals in the HHRs used by the county’s residents pre-COVID. This has the advantage that our quality measure is free from the endogeneity that would potentially arise if there is hospital selection based on ill-health (e.g., if the sickest patients go to the lowest quality hospitals). However, travel patterns might have changed as a result of COVID-19. As COVID-19 has been accompanied by changes in the financial viability of hospitals, even if hospitals did not close, as admissions of COVID-19 patients increased, ICU capacity constraints could have resulted in patients being sent to hospitals that are not normally used by the residents of a county.18 If hospitals are still within the HRR for each county, this would be accounted for in our measure of quality (which is another advantage of our approach of assigning quality from the HRR to the county, particularly in the context of the COVID-19 pandemic). Nevertheless, it is reassuring that our results are robust to different quality assignment algorithms. Second, if the information on how to appropriately treat COVID-19 patients spreads across hospitals, then our measure of quality, based on the quality of treatment of pneumonia patients pre-COVID-19, would be less likely to capture the current quality differentials across hospitals as the pandemic progresses. However, we show our results hold out to the 531st day after the first recorded case in the US and, in fact, increase in magnitude over time, thus reducing this concern.

In conclusion, our results suggest that access to high-quality hospital care has mattered during the COVID-19 pandemic and has mattered more for communities that are more deprived and have access to fewer hospitals, suggesting that community capital, broadly defined, may have been a substitute for higher quality hospital-based care during the COVID pandemic. In terms of mechanisms, the number of doctors (but not other clinical staff) available within hospitals increases the association, supporting the idea that one of the mechanisms bringing about this association is the availability of the most skilled group of hospital clinicians.

Finally, hospital care is an essential part of health care provision for vulnerable and disadvantaged populations in the USA (e.g. Bhatt and Bathija, 2018). The stronger association we find for more disadvantaged communities between the quality of hospital care and fewer COVID deaths may thus also extend to outcomes for other medical conditions for which hospital care is particularly important.

CRediT authorship contribution statement

Johannes S. Kunz: Conceptualization, Writing – review & editing, Formal analysis. Carol Propper: Conceptualization, Writing – review & editing, Formal analysis.