Patient-level and hospital-level variation and related time trends in COVID-19 case fatality rates during the first pandemic wave in England: multilevel modelling analysis of routine data.

BACKGROUND: A report suggesting large between-hospital variations in mortality after admission for COVID-19 in England attracted much media attention but used crude rates. We aimed to quantify these variations between hospitals and over time during England's first wave (March to July 2020) and assess available patient-level and hospital-level predictors to explain those variations.
METHODS: We used administrative data for England, augmented by hospital-level information. Admissions were extracted with COVID-19 codes. In-hospital death was the primary outcome. Risk-adjusted mortality ratios (standardised mortality ratios) and interhospital variation were calculated using multilevel logistic regression. Early-wave (March to April) and late-wave (May to July) periods were compared.
RESULTS: 74 781 admissions had a primary diagnosis of COVID-19, with 21 984 in-hospital deaths (29.4%); the 30-day total mortality rate was 28.8%. The crude in-hospital death rate fell in all ages and overall from 32.9% in March to 13.4% in July. Patient-level predictors included age, male gender, non-white ethnic group (early period only) and several comorbidities (obesity early period only). The only significant hospital-level predictor was daily COVID-19 admissions in the late period; we did not find a relation with staff absences for COVID-19, mechanical ventilation bed occupancies, total bed occupancies or bed occupancies for COVID-19 admissions in either period. Just 4 (3%) and 2 (2%) hospitals were high, and 5 (4%) and 0 hospitals were low funnel plot mortality outliers at 3 SD for early and late periods, respectively, after risk adjustment. We found no strong correlation between early and late hospital-level mortality (r=0.17, p=0.06).
CONCLUSIONS: There was modest variation in mortality following admission for COVID-19 between English hospitals after adjustment for risk and random variation, in marked contrast to early media reports. Early-period mortality did not predict late-period mortality. © Author(s) (or their employer(s)) 2022. No commercial re-use. See rights and permissions. Published by BMJ.

Introduction

Many studies have shown that COVID-19 case fatality rates vary by country, region within country1 2 and patient factors such as age, gender, ethnic group, body mass index, creatinine, socioeconomic deprivation, smoking and comorbidities,1 3–6 but there has been little published on variations between hospitals. Unpublished results from National Health Service (NHS) England in July 2020 that attracted much media attention suggested wide variations in mortality. A US study of patients admitted to the intensive therapy unit (ITU) found the risk-adjusted mortality to vary from 6.6% to 80.8% across the 65 study hospitals.3 A larger US study of risk-adjusted 30-day mortality or referral to hospice rates found a range from 5.7% to 24.7% and that all but one hospital improved its rates between January and July 2020.7 There have been differences in how hospitals have responded to the pandemic in terms of increasing capacity (converting operating rooms into makeshift ITUs, retraining staff) and treatment options (use of oxygen, ventilation, ITU admission, steroids and other medications),8 and it is likely that this will translate into variations in patient outcomes.

Using national hospital data for England, we compared crude and risk-adjusted in-hospital case fatality rates by hospital in the first 5 months following the introduction of the two specific International Classification of Diseases 10th Revision (ICD-10) codes for the disease by the WHO in March 2020 and covering the first wave of infection. Our research questions were:

How did early-wave period (March to April) and late-wave period (May to July) mortality and predictors compare?

How did each hospital’s early-period mortality compare with their late-period mortality?

How much non-random variation existed between hospitals?

Methods

Data

Hospital Episode Statistics (HES)9 covers all NHS patients treated in either NHS or private hospitals and also private patients treated in NHS hospitals in England. Information includes dates of admission and discharge, demographics, up to 24 procedures, one primary diagnosis (main problem treated) and up to 19 secondary diagnoses (comorbidities and complications) with ICD-10 coding, and in-hospital outcomes such as length of stay (LOS) and death. The basic unit of the database is the finished consultant episode, representing the time during which the patient is the responsibility of a senior doctor (consultant); there can be multiple episodes per admission. For each patient ID, we linked records into admissions to take account of transfers between consultants and between hospitals. We searched for either new ICD-10 COVID codes U07110 and U07210 11 in first just the primary diagnosis field (either first or second episode). Data are routinely linked each month to the national death register, and we therefore also report 30-day total mortality (ie, deaths in and out of hospital within 30 days of admission).

We augmented the HES records with several hospital-level variables. One was simply its daily number of COVID-19 admissions, hypothesising that a large case load could overwhelm the hospital’s capacity and lead to worse outcomes. In addition, COVID-19 Situation Reports (SitReps) published by NHS England12 were used to calculate the mean weekly number of occupied beds, number of beds occupied by patients with confirmed COVID-19, number of mechanical ventilation (MV) beds occupied by patients with confirmed COVID-19 and number of COVID-19-related staff absences, either through sickness or self-isolation. SitReps data were not available for March; we imputed values using those for the first week of April. The hospitals identified within the reports were matched against the hospitals within HES: we were able to match 124 out of a possible 128 acute, non-specialist hospitals. All analyses are based on the 124 hospitals.

Statistical analyses

We calculated crude death rates by hospital trust and then applied two-level logistic regression due to the clustering of patients within hospitals, with random intercepts for hospitals. These had the following patient (level 1) predictors: age (one-knot spline), gender, diabetes, hypertension, coronary heart disease, chronic obstructive pulmonary disease (COPD), obesity, cancer, renal disease, dementia, area-level Carstairs socioeconomic deprivation, emergency admission flag, source of admission (from own home, transferred from another provider), ethnic group, number of emergency admissions for any reason in the previous 12 months and month of admission. Comorbidities were taken from any diagnosis field in the index admission. Hospital-level total bed occupancies, confirmed COVID-19 bed occupancies, MV bed occupancies for confirmed COVID-19 cases, staff absences related to COVID-19 and COVID-19 admissions were also included as hospital-level (level 2) variables in the full model, each of them as one-knot splines (see online supplemental appendix figures A6–A11 for plots). Patient comorbidity was also described (but not modelled) using the Charlson Comorbidity Score calibrated to the NHS.13

To derive adjusted mortality ratios and rates by trust, predicted probabilities per patient were calculated from the fixed effects part of the model, that is, excluding the random hospital effects14 and summed to give total expected deaths by hospital; the observed deaths were divided by these expected totals to give a standardised mortality ratio (SMR). To convert this into risk-adjusted rates, it was multiplied by the national crude rate. This was done for the early and late periods separately.

To estimate the variation in mortality between hospitals, we first inspected the covariance estimate of the random effects and intraclass correlation coefficient (ICC), which indicates how much of the total variation in patient mortality is accounted for by the hospitals. To estimate this, we used a common assumption that the dichotomous outcome comes from an unknown latent continuous variable with a level 1 residual that follows a logistic distribution with a mean of 0 and a variance of 3.29.15 SMRs were plotted on funnel plots and the numbers of mortality outliers at 2 and 3 SDs noted.

To assess whether hospitals changed their mortality between the early and late periods, the two sets of SMRs were first compared with Pearson’s correlation coefficient. In each period, hospitals were then put into quartiles based on their SMRs. The weighted kappa statistic was calculated from the resulting 4×4 table.

For context, LOS was summarised overall and at a hospital level as medians and IQRs. The Pearson’s correlation coefficient was calculated between the SMRs and median LOS.

Sensitivity analyses

Sensitivity analyses included expanding the admissions to cover COVID-19 codes in any diagnosis position during the admission. Rather than impute SitReps data for March, we also ran the models without March admissions for the early period (see online supplemental appendix table A1). SAS software V.9.4 was used to run the analyses, and the significance threshold was 0.05.

Results

There were a total of 174 hospital trusts with one or more COVID-19 admissions, but we limited this to 124 acute, non-specialist hospital trusts that also had hospital-level published figures, thereby excluding 1568 COVID-19 cases and 425 deaths. We excluded 170 admissions where patients were discharged from hospital before the introduction of ICD-10 codes for COVID-19 in March 2020. Since the introduction of those codes, we found 74 781 admissions with COVID-19 as the primary diagnosis field and a further 117 369 admissions with it in any of the secondary diagnosis fields at the 124 hospitals. Table 1 describes the patient characteristics of the former, and all analyses were based on them.

Table 1

Patient characteristics, counts and proportions of in-hospital deaths among 124 English hospital admissions for early-period and late-period COVID-19 admissions

Feature	Value	Early period (March, April)		Late period (May, June, July)
		n=59 054		n=15 727
		n (%)	In-hospital deaths (crude rate as %)	n (%)	In-hospital deaths (crude rate as %)
Age (years)	Mean (SD)	67.7 (18.2)	–	68.4 (20.1)	–
	0–9	Suppressed* (<5)	Suppressed* (<5)	Suppressed* (<5)	Suppressed* (<5)
	10–19	236 (0.4)	8 (3.4)	Suppressed* (<5)	Suppressed* (<5)
	20–29	1141 (1.9)	30 (2.6)	Suppressed* (<5)	Suppressed* (<5)
	30–39	2644 (4.5)	114 (4.3)	751 (4.8)	11 (1.5)
	40–49	5022 (8.5)	347 (6.9)	1213 (7.7)	53 (4.4)
	50–59	8795 (14.9)	1259 (14.3)	1875 (11.9)	177 (9.4)
	60–69	9652 (16.3)	2554 (26.5)	2162 (13.7)	375 (17.3)
	70–79	12 395 (21.0)	4803 (38.7)	3164 (20.1)	897 (28.4)
	80–89	13 919 (23.6)	6618 (47.5)	4154 (26.4)	1521 (36.6)
	90+	4761 (8.1)	2555 (53.7)	1647 (10.5)	656 (39.8)
Gender	Male	34 462 (58.4)	11 377 (33.0)	8348 (53.1)	2160 (25.9)
	Female	24 592 (41.6)	6911 (28.1)	7379 (46.9)	1536 (20.8)
Ethnic group	Black or Black British	3763 (6.4)	1002 (26.6)	343 (2.2)	51 (14.9)
	Asian or Asian British	5367 (9.1)	1390 (25.9)	1396 (8.9)	208 (14.9)
	White	39 828 (67.4)	13 270 (33.3)	12 011 (76.4)	3079 (25.6)
	Other including Mixed	3079 (5.2)	694 (22.5)	519 (3.3)	58 (11.2)
	Unknown	7017 (11.9)	1932 (27.5)	1458 (9.3)	300 (20.6)
Deprivation quintile	1 (least deprived)	8850 (15.0)	2796 (31.6)	2450 (15.6)	632 (25.8)
	2	9740 (16.5)	3147 (32.3)	2802 (17.8)	688 (24.6)
	3	11 063 (18.7)	3557 (32.2)	3034 (19.3)	822 (27.1)
	4	12 210 (20.7)	3825 (31.3)	3417 (21.7)	746 (21.8)
	5	16 894 (28.6)	4909 (29.1)	3962 (25.2)	804 (20.3)
	6 (unknown)	297 (0.5)	54 (18.2)	62 (0.4)	Suppressed* (<5)
Method of admission	Emergency	58 174 (98.5)	18 081 (31.1)	15 390 (97.9)	3658 (23.8)
	Non-emergency	880 (1.5)	207 (23.5)	337 (2.1)	38 (11.3)
Admission source	Home	52 632 (89.1)	16 090 (30.6)	14 198 (90.3)	3290 (23.2)
	Transfer from acute hospital	391 (0.7)	142 (36.3)	123 (0.8)	24 (19.5)
	Transfer from non-acute hospital	92 (0.2)	43 (46.7)	26 (0.2)	12 (46.2)
	Transfer from unknown hospital	4002 (6.8)	1158 (28.9)	676 (4.3)	123 (18.2)
	Other/unknown	1937 (3.3)	855 (44.1)	704 (4.5)	247 (35.1)
Emergency admissions in previous 12 months	0	35 277 (59.7)	9118 (25.8)	7262 (46.2)	1295 (17.8)
	1	11 456 (19.4)	4233 (37.0)	3768 (24.0)	1057 (28.1)
	2	5362 (9.1)	2179 (40.6)	2001 (12.7)	576 (28.8)
	3+	6959 (11.8)	2758 (39.6)	2696 (17.1)	768 (28.5)
Admission month	March	18 985 (32.1)	6246 (32.9)	–	–
	April	40 069 (67.9)	12 042 (30.1)	–	–
	May	–	–	10 899 (69.3)	2783 (25.5)
	June	–	–	3652 (23.2)	756 (20.7)
	July	–	–	1176 (7.5)	157 (13.4)
Charlson Comorbidity Score	Mean (SD)	8.9 (10.0)	–	10.5 (10.7)	–
Comorbidity	Diabetes	16 239 (27.5)	5946 (36.6)	3887 (24.7)	1033 (26.6)
	Hypertension	26 143 (44.3)	9528 (36.4)	6510 (41.4)	1739 (26.7)
	Coronary heart disease	6581 (11.1)	3274 (49.7)	2436 (15.5)	943 (38.7)
	COPD	15 176 (25.7)	5079 (33.5)	4474 (28.4)	1159 (25.9)
	Obesity	5160 (8.7)	1481 (28.7)	1275 (8.1)	185 (14.5)
	Cancer	3406 (5.8)	1528 (44.9)	1265 (8.0)	465 (36.8)
	Renal disease	9713 (16.4)	4500 (46.3)	2694 (17.1)	1015 (37.7)
	Dementia	4666 (7.9)	2417 (51.8)	1444 (9.2)	559 (38.7)

*Data suppressed due to small numbers.

COPD, chronic obstructive pulmonary disease.

In the 74 781 admissions with COVID-19 as the primary diagnosis, there were 21 984 in-hospital deaths (a rate of 29.4%; the 30-day total rate was 28.8%): this rate fell each month from 32.9% of the 18 985 admissions in March, 30.1% of the 40 069 admissions in April, 25.5% of the 10 899 admissions in May, 20.7% of the 3652 admissions in June and 13.4% of the 1176 admissions in July (see online supplemental appendix figure A1).

Early-wave (March to April) vs late-wave period (May to July) mortality and predictors

Table 2 gives the logistic regression results for each period. For categorical variables in the two-level model, the p values in table 2 reflect the significance test relative to the reference category, but we use the overall significance p value to determine if the variable is significant.

Table 2

Multilevel logistic regression results for in-hospital mortality for early-period and late-period COVID-19 admissions

Feature	Value	Early period (March, April)		Late period (May, June, July)
		OR (95% CI)	P value	OR (95% CI)	P value
Age: OR per year	<45 years	1.08 (1.07 to 1.09)	<0.0001	1.10 (1.07 to 1.14)	<0.0001
	45+ years	1.05 (1.04 to 1.07)	0.0002	1.05 (1.01 to 1.08)	0.003
Gender	Male	1.44 (1.39 to 1.50)	<0.0001	1.40 (1.29 to 1.52)	<0.0001
	Female	1		1
Ethnic group	Black or Black British	1.07 (0.97 to 1.17)	0.171	1.09 (0.77 to 1.52)	0.637
	Asian or Asian British	1.21 (1.12 to 1.31)	<0.0001	1.11 (0.92 to 1.33)	0.268
	White	1		1
	Other including Mixed	1.10 (0.99 to 1.21)	0.075	0.89 (0.66 to 1.21)	0.470
	Unknown	1.14 (1.07 to 1.21)	0.0001	1.15 (0.99 to 1.34)	0.066
Deprivation quintile	1 (least deprived)	1		1
	2	1.07 (1.00 to 1.14)	0.064	0.98 (0.85 to 1.12)	0.726
	3	1.08 (1.01 to 1.15)	0.025	1.13 (0.99 to 1.29)	0.078
	4	1.10 (1.03 to 1.18)	0.004	0.97 (0.85 to 1.11)	0.660
	5	1.12 (1.05 to 1.20)	0.001	1.13 (0.98 to 1.30)	0.087
	6 (unknown)	0.76 (0.55 to 1.06)	0.104	0.49 (0.17 to 1.46)	0.202
Method of admission	Emergency	1		1
	Non-emergency	0.87 (0.72 to 1.05)	0.149	0.54 (0.35 to 0.83)	0.005
Admission source	Home	1		1
	Transfer from acute hospital	1.77 (1.39 to 2.26)	<0.0001	1.77 (1.00 to 3.12)	0.051
	Transfer from non-acute hospital	1.14 (0.74 to 1.75)	0.559	1.99 (0.88 to 4.50)	0.097
	Transfer from unknown hospital	0.99 (0.90 to 1.10)	0.901	0.89 (0.70 to 1.13)	0.325
	Other/unknown	1.16 (1.04 to 1.28)	0.006	1.22 (1.02 to 1.46)	0.034
Emergency admissions in previous 12 months	0	1		1
	1	1.12 (1.06 to 1.18)	<0.0001	1.14 (1.03 to 1.27)	0.010
	2	1.15 (1.08 to 1.23)	<0.0001	1.07 (0.94 to 1.21)	0.324
	3+	1.11 (1.04 to 1.18)	0.001	1.09 (0.97 to 1.22)	0.151
Admission month	March	1			–
	April	0.78 (0.74 to 0.81)	<0.0001	–	–
	May	–	–	1
	June	–	–	0.85 (0.75 to 0.96)	0.010
	July	–	–	0.63 (0.49 to 0.79)	<0.0001
Comorbidity	Diabetes	1.17 (1.12 to 1.22)	<0.0001	1.09 (0.99 to 1.19)	0.080
	Hypertension	0.95 (0.91 to 0.99)	0.013	0.82 (0.75 to 0.89)	<0.0001
	Coronary heart disease	1.40 (1.32 to 1.48)	<0.0001	1.50 (1.36 to 1.66)	<0.0001
	COPD	1.04 (0.99 to 1.08)	0.108	1.11 (1.01 to 1.21)	0.023
	Obesity	1.60 (1.49 to 1.72)	<0.0001	0.92 (0.77 to 1.10)	0.363
	Cancer	1.47 (1.36 to 1.58)	<0.0001	1.80 (1.58 to 2.05)	<0.0001
	Renal disease	1.29 (1.23 to 1.36)	<0.0001	1.37 (1.24 to 1.51)	<0.0001
	Dementia	1.41 (1.32 to 1.50)	<0.0001	1.28 (1.13 to 1.45)	<0.0001
COVID-19 daily admissions: OR per admission	<4 admissions	0.96 (0.92 to 1.01)	0.087	1.07 (1.02 to 1.12)	0.009
	4+ admissions	1.01 (0.96 to 1.05)	0.051	1.02 (0.96 to 1.07)	0.088
Bed occupancy	Per 10 extra occupancies	1.00 (1.00 to 1.00)	0.144	1.00 (1.00 to 1.00)	0.051
COVID-19 bed occupancy	Per 10 extra occupancies	1.01 (1.00 to 1.01)	0.090	1.00 (0.98 to 1.02)	0.908
COVID-19 MV bed occupancy: OR per bed	<4 beds	1.00 (0.97 to 1.02)	0.912	1.03 (0.98 to 1.08)	0.225
	4+ beds	1.00 (0.97 to 1.03)	0.940	0.99 (0.94 to 1.04)	0.122
COVID-19-related staff absences	Per 10 extra absences	1.00 (1.00 to 1.00)	0.502	1.00 (1.00 to 1.00)	0.546

COPD, chronic obstructive pulmonary disease; MV, mechanical ventilation.

In the multiple logistic regression model, statistically significant predictors of in-hospital mortality for both periods were age, male gender, deprivation quintile, source of admission, month of admission, hypertension, coronary heart disease, cancer, renal disease and dementia (see table 2, online supplemental appendix table A2). Statistically significant predictors for the early period only were ethnicity, emergency admissions in the previous 12 months, diabetes and obesity; those for the late period only were method of admission, COPD and hospital daily COVID-19 admissions of less than 4 (see table 2, online supplemental appendix table A2). Due to the spline, the odds of death for daily COVID-19 admissions rose by 7% per admission until four admissions but plateaued thereafter.

Model discrimination was fair, with a c statistic of 0.74 (95% CI 0.738 to 0.744) and 0.75 (95% CI 0.739 to 0.752) for the early and late periods, respectively; calibration was reasonable as assessed by the Hosmer-Lemeshow plot (see online supplemental appendix figures A12–A13). The ICC from the multilevel model for early and late periods was 1.6% and 1.4%, respectively; the covariance parameters for the random effects were 0.038 for the early period and 0.048 for the late period, both p<0.001, showing a statistically significant but small variation in mortality between hospitals for both periods after adjusting for available patient factors.

Hospital’s early-period mortality vs their late-period mortality

The Pearson’s correlation coefficient between early and late SMRs was 0.17 (p=0.06). Table 3 shows how early-period quartiles fared with late-period admissions.

Table 3

Risk-adjusted mortality rates and number (% of total) of hospitals by hospital mortality quartile in the early and late periods

Early period	Late period
	Quartile 1 (risk-adjusted rate 1.6%)	Quartile 2 (risk-adjusted rate 2.2%)	Quartile 3 (risk-adjusted rate 2.5%)	Quartile 4 (risk-adjusted rate 3.1%)
Quartile 1 (risk-adjusted rate 2.6%)	12 (9.7%)	7 (5.7%)	7 (5.7%)	5 (4.0%)
Quartile 2 (risk-adjusted rate 2.9%)	7 (5.7%)	6 (4.8%)	9 (7.3%)	9 (7.3%)
Quartile 3 (risk-adjusted rate 3.2%)	5 (4.0%)	12 (9.7%)	7 (5.7%)	7 (5.7%)
Quartile 4 (risk-adjusted rate 3.6%)	7 (5.7%)	6 (4.8%)	8 (6.5%)	10 (8.1%)

The majority of hospitals moved between quartiles from early to late periods, but only 4.0% moved from the lowest to the highest, and only 5.7% moved from the highest to the lowest quartile (table 3). After applying the weighted kappa to the table, the weighted kappa coefficient was only 0.097 (95% CI 0 to 0.228).

As age is the most important predictor, table 4 gives the average crude death rates by age group for quartile 1 and quartile 4 hospitals in the early-wave and late-wave periods.

Table 4

Average crude rates by age group for lowest and highest mortality hospital quartiles in the early and late periods

Lowest quartile				Highest quartile
Age	Crude rate% (95% CI)	OR cf <50	OR cf previous age	Age	Crude rate% (95% CI)	OR cf <50	OR cf previous age	OR cf lowest quartile
Early period				Early period
<50	3.9 (3.0 to 4.8)	1	1	<50	7.3 (5.9 to 8.9)	1	1	1.94
50–59	11.3 (9.8 to 12.9)	3.1	3.1	50–59	18.7 (16.5 to 21.1)	2.9	2.6	1.81
60–69	22.8 (20.8 to 24.9)	7.3	2.3	60–69	32.9 (30.4 to 35.5)	6.2	1.8	1.66
70–79	34.3 (32.2 to 36.4)	12.9	1.8	70–79	45.0 (42.6 to 47.4)	10.4	1.4	1.57
80–89	42.8 (40.7 to 44.9)	18.4	1.4	80–89	52.6 (50.4 to 54.8)	14.1	1.2	1.48
90+	48.4 (44.8 to 52.0)	23.1	1.3	90+	59.5 (55.9 to 63.1)	18.7	1.1	1.57
Late period				Late period
<50	1.1 (0.3 to 2.8)	1	1	<50	3.5 (1.8 to 6.1)	1	1	3.26
50–59	4.6 (2.2 to 8.3)	4.3	4.3	50–59	14.1 (10.1 to 19.1)	4.5	4.0	3.40
60–69	13.8 (9.7 to 18.9)	14.4	3.3	60–69	22.9 (18.5 to 27.7)	8.2	1.6	1.86
70–79	17.4 (13.5 to 21.9)	18.9	1.3	70–79	36.5 (32.2 to 40.9)	15.8	1.6	2.73
80–89	25.3 (21.2 to 29.8)	30.5	1.6	80–89	45.4 (41.4 to 49.4)	22.9	1.2	2.46
90+	33.8 (26.9 to 41.2)	45.9	1.5	90+	48.6 (42.1 to 55.2)	26.1	1.1	1.85

cf, compared with.

There were huge mortality differences by age in both mortality quartiles and wave periods. Crude rates fell for every age in every mortality quartile, but the age gradient became steeper in the later period and was more extreme for the lowest mortality hospital quartile.

Non-random variation between hospitals

Crude death rates for acute, non-specialist trusts varied from 0.16 to 0.46 in the early period (see online supplemental appendix figure A2) and from 0.04 to 0.46 in the later period (see online supplemental appendix figure A3). Among the hospitals, there were 54 (43.5%) and 27 (21.8%) mortality outliers at 2 or more SDs for early and late periods, respectively. For 3 SD (99.8% control limit), there were 25 (20.2%) and 13 (10.5%) mortality outliers for early and late periods, with 11 (8.9%) and 7 (5.6%) hospitals as high-mortality outliers and 14 (11.3%) and 6 (4.8%) hospitals as low-mortality outliers for the early and late periods, respectively.

Following risk adjustment, SMRs varied from 60 to 139 and 23 to 192 between hospitals for early (see online supplemental appendix figure A4) and late periods (see online supplemental appendix figure A5), respectively, with 25 (20.2%) and 10 (8.1%) mortality outliers at 2 or more SDs, and 9 (7.3%) and 2 (1.6%) at 99.8% control limit or more (3 SD). Thirteen (10.5%) and 6 (4.8%) hospitals were high-mortality outliers for their SMR, while 12 (9.7%) and 4 (3.2%) were low-mortality outliers for their SMR at 2 or more SDs for early and late periods, respectively. At 3 SD (99.8% control limit), there were 4 (3.2%) and 2 (1.6%) hospitals as high-mortality outliers and 5 (4.0%) and 0 (0%) hospitals as low-mortality outliers for early and late periods, respectively. Figure 1 is the funnel plot for the whole first wave. For the whole first wave, 25 (20.2%) hospitals were mortality outliers at 2 or more SDs, and 9 (7.3%) were low-mortality and 3 (2.4%) were high-mortality outliers at 3 SD.

Figure 1

Funnel plot for the adjusted COVID-19 standardised mortality ratio for the entire first wave (March to July 2020).

Overall LOS for COVID-19 admissions ranged from 0 to 311 nights for the early period and from 0 to 152 nights for the late period across trusts, with a national overall median of six nights for early and late periods. Early and late IQRs were similar (3–12 for early, 2–12 for late). This was similar for survivors and deceased in each period (data not shown). The median hospital-level stay was six nights for both periods (early-period IQR was 6–7 and late-period IQR was 5–7).

There were 117 054 admissions with COVID-19 in any diagnosis position, with 87 032 admissions in the early period and 30 022 admissions in the later period. Where it occurred in a secondary position, 26.0% of the time, the first one was occupied by a wide range of conditions, most commonly ICD-10 A419 (unspecified sepsis, 0.9% for the whole wave), N390 (urinary tract infection, site not specified, 0.9%), J181 (lobar pneumonia unspecified, 0.8%), S7200 (fracture of neck of femur, 0.7%) and N179 (unspecified acute renal failure, 0.6%).

Hospital SMRs for the adjusted model ranged from 68 to 136, with 9.7% high-mortality and 9.7% low-mortality outliers at 2 SD and 4.0% high-mortality and 6.5% low-mortality outliers at 3 SD on the funnel plots for the early period. For the late period, hospital SMRs for the adjusted model ranged from 32 to 171, with 9.7% high-mortality and 11.3% low-mortality outliers at 2 SD and 1.6% high-mortality and 4.0% low-mortality outliers at 3 SD on the funnel plots. Correlation with the SMRs derived from COVID-19 as the primary diagnosis was high at 0.86 (p<0.0001) for the early period and 0.82 (p<0.0001) for the late period. The adjusted SMR ranges for each period were similar to those from the main analysis.

As SitReps data were unavailable for March, we tried excluding March admissions and ran the model for the early period (see online supplemental appendix table A1). Regression coefficients were largely unchanged.

Discussion

Summary of main findings

Crude in-hospital mortality rates fell greatly during the first wave. We found wide variation between hospitals in England in these rates, which fell but remained statistically significant following risk adjustment and multilevel modelling. Hospitals’ early-period mortality correlated very weakly with their late-period mortality. Of our hospital-level variables, only COVID-19 daily admissions in the late period showed both a statistically and non-negligible association with mortality, in this case a 7% increase in odds per extra COVID-19 admission up to four admissions, plateauing thereafter. Mortality improved for all age groups and at both low-mortality and high-mortality hospitals over time, but the age gradient was steeper in the late period and in the lowest mortality quartile hospitals. This implies that, compared with the early period, the biggest relative improvements in survival were made in younger patients. Patient-level variables had consistent associations with mortality over time, with the main exception of obesity, diabetes, emergency admissions in the previous 12 months and ethnicity (significant only in the early period) and method of admission and COPD (both significant only in the later period).

Explanations for the interhospital variations include chance and differences in coding (use of the new ICD-10 U codes), case mix, case finding (testing for COVID-19 on admission) and treatment (triage, ventilation, extracorporeal membrane oxygenation, ITU, steroids and other medications). We discuss each in turn.

Chance, case mix, coding and case finding

Chance is highly unlikely as an explanation given the tiny p value for the ICC and our use of 3 SD control limits. We adjusted for many available key risk factors such as age and several comorbidities, obtaining a reasonable level of risk adjustment as measured by the c statistic, but did not have all the known risk factors1 3–6 such as renal function, disease severity and other physiological data. Given the big impact of our risk adjustment on interhospital variation, differences in such unmeasured confounding are likely to explain at least some of the remaining differences in mortality between hospitals. None of England’s hospitals would be expected to lie outside 3 SD control limits if there was only random variation in the death rates. We found the proportion of mortality outliers to be 9.7%, showing more than mere random variation. Such ‘overdispersion’ is also regularly found in metrics of overall hospital mortality based on administrative data such as the standardised hospital mortality indicator (SHMI) and hospital standardised mortality ratio (HSMR). In the comparable period for which the SHMI is published,16 June 2019 to May 2020, figures were given for 125 trusts: at 2 SD and despite adjustment (widening) of the control limits for overdispersion, 13 (10.4%) still had more deaths than expected and 16 (12.8%) fewer than expected. Comorbidity measurement relies on secondary diagnosis coding in HES, which is known to vary between hospitals. While comorbidities in administrative data can be under-recorded despite improvements in recent years, there is evidence that this does not necessarily lead to bias in their ORs17 and has limited impact on HSMRs.18 Nonetheless, this will contribute an unknown but probably modest amount to our results.

A related issue is in which diagnosis field in HES should COVID-19 be recorded. We ran the analysis in two ways: one using just the primary diagnosis and a second using COVID-19 mentioned in any field. Other variants are possible, such as to take pneumonia in the first diagnosis position and COVID-19 in the second, though with administrative data there is no obvious right approach for each admission. When comparing our sensitivity analysis hospital admissions against published Public Health England figures,19 20 we found that 96% of COVID-19 cases for April, 88% for May, 80% for June and 86% for July were accounted for. Direct mortality comparisons were less comparable as published mortality rates included COVID-19 deaths occurring within the community, while our study focused solely on deaths within English hospitals.

The accuracy of our denominators depends not just on the coding of diagnosed COVID-19 cases but of course on the diagnosis of those cases, which in turn depends on testing. In the first few months, some hospitals struggled to acquire sufficient testing capability. In our local hospitals in the early days of our response, hospitals used a combination of symptom identification and testing to diagnose COVID-19 in individual patients. Viral PCR tests were the most commonly used and during the study period took 12–24 hours to obtain a result.

Treatment

Robust evidence for the effectiveness of treatments such as dexamethasone (preprint published on 22 June 2020, first paper 17 July) and remdesivir (final report 20 May) only became available after the bulk of admissions during our study period,21–23 meaning that during our study period clinicians applied a range of treatments both in terms of medications and in the use of pathways such as ITU. Many patients were involved in trials in the UK. The use of ventilators and ITU beds depended on their availability at each hospital, which ran out at many units during the peak, and the medical teams had to make difficult decisions regarding where to place patients. Like Italy and France, the UK was hit hard early by the virus but had limited ITU facilities compared with other European countries. In England, Wales and Northern Ireland, only 20.3% of patients with COVID-19 being treated in intensive care units were older than 70 years (and only 2.6% were older than 80 years),24 compared with 54% (and 23%) in Germany, where their ITU registry data showed that capacity was never exceeded during the study period.8 The ITU admission rate in England was 17%,25 however, similar to that reported in Italy26 and New York.27

We found an association between COVID-19 daily admissions (when less than 4) in the later period and mortality. There were no appreciable relations for any of our other hospital-level variables, including case numbers and staff absences, which was surprising. This could either be because the variables were not consistently recorded or that, despite the enormous pressures on staff and hospitals, the NHS managed to mitigate them and provide a relatively consistent service across England.

LOS depends on many factors including COVID-19 severity, case mix and treatment. We found that median LOS varied only modestly between hospitals, with very limited correlation with mortality rates (data not shown).

Strengths and limitations

The study benefits from national data and a large sample size but relies on accurate data recorded by clinicians and then coded by clinical coders in hospitals. The primary diagnosis and procedure fields in administrative data are known to have high accuracy (>95%)28 though secondary diagnoses are subject to some under-recording. There will be likely variation between hospitals when the WHO COVID-19 coding advice was implemented. Our sensitivity analyses used admissions with COVID-19 recorded in any diagnosis position and found similar results to our main analyses. Some of those admissions would have COVID-19 as an incidental finding, whereas an unknown number, most likely those with pneumonia as the primary diagnosis, would be admissions due to COVID-19. However, due to the lack of presence of admission codes in HES, an unknown number with COVID-19 in a secondary position would have caught the virus in hospital. An early study of 1564 patients admitted up to 28 April 2020 from 10 UK hospital sites and one Italian site found that 12.5% of COVID-19 infections were acquired in hospital.29

Our overall death rates are in line with those from the International Severe Acute Respiratory and Emerging Infections Consortium WHO Clinical Characterisation Protocol UK study, a large prospective cohort study of patients with COVID-19 in 208 acute care hospitals in England, Wales and Scotland.25 Their data covered admissions before 19 April 2020 and found that 26% of admitted patients died. Regarding predictors of death, they found significant positive associations with age, male gender, chronic lung disease, chronic heart disease, chronic kidney disease, obesity, dementia, chronic neurological disorders, cancer and moderate or severe liver disease; the association with diabetes did not meet the conventional 5% cut-off for significance (p=0.087). Our study also found significant increased associations with age, male gender and several comorbidities, although the significant relation with obesity seen in the early period disappeared in the late period. As noted earlier, HES lacks physiological data such as heart rate and oxygen saturation. We also used hospital-level information on ventilation, whereas it would have been preferable to have this at patient level, but we found HES recording levels to be much lower than in non-HES studies.

As with most administrative databases, HES has limited ability to describe the severity of illness, and this is not directly possible with COVID-19. We did not know the need for MV on admission, and there are also no laboratory or physiological variables. We expect these factors may vary by hospital and for these aforementioned reasons, it is likely that our variations in risk-adjusted mortality between hospitals are overestimates.

By design, HES only covers admitted patients, so variations in admission thresholds will contribute to the variations we observed. In England, most hospitals were screening patients for COVID-19 symptoms on admission, but only a minority were swabbing. Diagnosis and hence diagnosis coding rates will therefore differ between centres. We focused on early mortality, but longer term outcomes would also be of interest.

Conclusion

In-hospital mortality for COVID-19 admissions fell greatly for all ages between March and July 2020 and showed statistically significant but modest variation between hospitals after risk adjustment. There was little correlation between hospitals’ early-period and late-period mortality. Further research is required to see whether differences in approach to treatment are associated with these variations and, particularly going forward, with adoption of new treatments. Further surveillance of in-hospital mortality would be useful to help identify good practice.