Evaluation of modernisation of adult critical care services in England: time series and cost effectiveness analysis.

OBJECTIVE: To evaluate the impact and cost effectiveness of a programme to transform adult critical care throughout England initiated in late 2000.
DESIGN: Evaluation of trends in inputs, processes, and outcomes during 1998-2000 compared with last quarter of 2000-6.
SETTING: 96 critical care units in England. PARTICIPANTS: 349,817 admissions to critical care units.
INTERVENTIONS: Adoption of key elements of modernisation and increases in capacity. Units were categorised according to when they adopted key elements of modernisation and increases in capacity. MAIN OUTCOME MEASURES: Trends in inputs (beds, costs), processes (transfers between units, discharge practices, length of stay, readmissions), and outcomes (unit and hospital mortality), with adjustment for case mix. Differences in annual costs and quality adjusted life years (QALYs) adjusted for case mix were used to calculate net monetary benefits (valuing a QALY gain at pound20,000 ($33,170, euro22 100)). The incremental net monetary benefits were reported as the difference in net monetary benefits after versus before 2000.
RESULTS: In the six years after 2000, the risk of unit mortality adjusted for case mix fell by 11.3% and hospital mortality by 13.4% compared with the steady state in the three preceding years. This was accompanied by substantial reductions both in transfers between units and in unplanned night discharges. The mean annual net monetary benefit increased significantly after 2000 (from pound402 ($667, euro445) to pound1096 ($1810, euro1210)), indicating that the changes were relatively cost effective. The relative contribution of the different initiatives to these improvements is unclear.
CONCLUSION: Substantial improvements in NHS critical care have occurred in England since 2000. While it is unclear which factors were responsible, collectively the interventions represented a highly cost effective use of NHS resources.

Introduction

Politicians, managers, clinicians, and the public have strongly held views as to the costs and benefits (or lack of benefits) of the additional investment and the attempts to “modernise” the English National Health Service (NHS) since 2000. There is little rigorous evidence to inform such debates.

In 2000 the Department of Health in England advocated the “modernisation” of adult critical care1 and, as a parallel initiative, funded a 35% increase in the number of beds.2 Responsibility for leading the transformation of services lay with the newly created NHS Modernisation Agency and focused on integrating critical care with other acute services in the hospital, thus creating “comprehensive critical care.”

Key elements were to be the creation of 29 clinical networks, covering the whole country and sharing agreed protocols, and, within hospitals, the establishment of outreach services and the adoption of care bundles. Care bundles are groups of clinical guidelines that, when implemented and monitored together, are believed to produce better outcomes than would be the case with individual implementation.3 4 Implementation of one—the ventilator care bundle, which has four components that aim to prevent ventilator acquired pneumonia—was supported by the Modernisation Agency. Others include the central venous line bundle to prevent infection5 and the sepsis, resuscitation, and management bundles.6 In addition, hospitals were encouraged to undertake local improvement projects, which resulted in many diverse activities.

Although there has been no previous attempt to evaluate the impact of the changes instituted since 2000, some constituent elements have been investigated. There is mixed evidence about the impact of outreach services on mortality, unplanned admissions, and length of stay.7 8 Four studies of the ventilator care bundle reported reductions in the rate of ventilator associated pneumonia,4 9 10 11 another reported a reduction in mortality,12 and one reported shorter lengths of stay and duration of ventilation but no impact on crude mortality.13

To evaluate the impact of these initiatives it was necessary to define a point in time when transformation started. The Modernisation Agency’s critical care programme got under way at the end of 2000, which coincided with the substantial increase in investment (fig 1). We therefore compared the periods before and after the final quarter of 2000.

Fig 1 Peak periods of implementation of modernisation activities and capacity expansion

We compared trends in inputs, organisation, processes, and outcomes of critical care in a representative sample of English hospitals before and after the final quarter of 2000; explored associations of trends with each component intervention; and estimated the cost effectiveness of the interventions overall.

Methods

Data

Inputs

The Department of Health provided data on the number of available beds since 1999,14 the NHS reference costs provided the number of bed days and unit costs,15 and the case mix programme database (CMPD) from the Intensive Care National Audit and Research Centre (ICNARC) provided estimates of the number of occupied beds since 1998.16 Units were categorised by their increase in capacity from 2000 to 2004: <20% growth (40% of units), 20-50% (25% of units), or >50% (35% of units).

Classification of critical care networks

The Modernisation Agency categorised networks on five occasions during 2002-3 as excellent, ready to receive, developing, at risk, or at significant risk. We classified seven networks (24%) that achieved “ready to receive” or “excellent” status at the first assessment as “earlier adopters” and 16 (55%) as “later adopters”—those that did not achieve this status until later assessments. Six networks that had not achieved the “ready to receive” threshold by the final assessment were classified as “other.”

Classification of critical care units

A previous survey in 2005 showed that by the end of 2001 half the units had established outreach services (table 1).17 We conducted a second survey in November 2007 to identify whether and when a unit had implemented the ventilator care bundle, with the level of compliance for each year categorised as >90%, 75-90%, or <75% of eligible patients. By 2004 half of the units reported they had implemented the bundle (table 2). Three quarters of units that implemented the bundle reported compliance levels above 90%, with only seven (6%) below 75%.

Table 1

 Implementation of critical care outreach services by year in case mix programme units (response rate 80%)

	Discharge* (n=131)			Support† (n=130)			Track and trigger‡ (n=129)
	No of units	Cumulative§ No (%)		No of units	Cumulative§ No (%)		No of units	Cumulative§ No (%)
≤1996	5	5 (4)		2	2 (2)		1	1 (1)
1997	0	5 (4)		0	2 (2)		0	1 (1)
1998	0	5 (4)		0	2 (2)		0	1 (1)
1999	2	7 (5)		2	4 (3)		2	3 (2)
2000	26	33 (25)		30	34 (26)		27	30 (23)
2001	45	78 (60)		38	72 (55)		38	68 (53)
2002	15	92 (70)		17	88 (68)		24	92 (71)
2003	8	97 (74)		4	89 (68)		16	106 (82)
2004	9	106 (81)		8	97 (75)		14	120 (93)
Not implemented	21	—		29	—		7	—

*Discharge: follow-up of patients’ post-critical care.

†Support: direct bedside clinical support on wards.

‡Track and trigger: use of physiological track and trigger warning system on general wards.

§Cumulative figures adjusted for units that stopped providing specific outreach services by 2004 (4 discharge, 4 support, and 2 track and trigger).

Table 2

 Survey of implementation of ventilator care bundle in case mix programme units (response rate 82%)

	No of units (%)	Cumulative No (%)
2001	0 (0)	0
2002	5 (4)	4
2003	24 (19)	23
2004	35 (28)	51
2005	27 (22)	73
2006	9 (7)	80
2007	11 (9)	89
Not implemented	14 (11)	—

Of the 96 English units with case mix programme data, 31 (32%) were classified as “later adopters”—they had not implemented an outreach service by the end of 2003 or had not implemented the ventilator care bundle by the end of 2005, or both—and 58 (60%) were classified as “earlier adopters”—they had implemented both by these dates.

There was no strong association between earlier adopter units and earlier adopter networks. In earlier adopter networks, 19 (73%) units were earlier adopters whereas in later adopter networks 30 (59%) were so classified (P=0.12). There was also no strong association between a unit’s classification and its change in bed capacity between 2000 and 2004.

Processes and outcomes

The case mix programme provided data on case mix, transfers, discharges, length of stay, readmissions, and unit and hospital mortality for patients admitted to critical care units. Data for 1998-2006 were available for 349 817 admissions in 96 English units.

Analysis

Comparison of trends

We compared data for 1998-2000 with data for 2000-6. We used regression analysis to compare trends in processes: Poisson regression for dichotomous dependent variables to estimate risk ratios for the average annual change in each period and linear regression for continuous dependent variables to estimate mean change per year and its 95% confidence intervals. The two time periods were fitted as an interaction to test if the trends differed between them. The time periods were based on quarterly data adjusted for quarterly seasonality with the final quarter of 2000 representing the end of the first time period and the beginning of the second. We used predicted mortality from the ICNARC model to adjust for case mix.18 Units were included as random effects to allow for clustering of admissions within units. For the Poisson regression analysis we estimated bias corrected 95% confidence intervals using bootstrap resampling with 1000 replications.19

Cost effectiveness analysis

We adopted a hospital perspective that included all costs for hospital admissions during an episode, including the index admission and any readmissions. Length of stay was valued using NHS reference costs (1998-2005) to give an episode cost for each patient. For 1998-2003, a single unit cost for intensive care unit bed days was available for each trust, whereas for 2004 and 2005 the unit costs were taken as the mean across three levels of care weighted by the relative number of bed days in each category. The unit cost for general wards was the weighted mean for all healthcare resource groups for non-elective bed days from NHS reference costs. All unit costs were inflated to 2006-7 prices.20

Mortality data were extrapolated to calculate lifetime QALYs. We assumed that the life expectancy for survivors was 80% that of the general population matched for age and sex21 with their quality of life downweighted by 20%. Those patients who died in hospital were assigned a QALY of zero. Each individual’s lifetime QALYs (discounted at 3.5%) were valued at £20 000, (€22 100, $33 170) and their costs subtracted to give a net monetary benefit.22 Similar regression models (to those described above) were used with annual costs, QALYs, and net monetary benefit as dependent variables and separate components of the ICNARC risk prediction model as independent variables.

Association with organisational changes

We used similar regression models to evaluate the impact of three organisational changes after 2000: increase in capacity, adoption of clinical networks, and adoption of outreach and ventilator care bundle. Interactions were fitted between each type of change to test whether trends in processes and outcomes were associated with any particular variable.

Results

Trends in inputs

The annual expenditure on critical care increased in real terms from £700m (1999-2000) to £1bn (2005-6). This was associated with a 35% increase in the number of staffed beds in general intensive care units (fig 2), with more of the increase in high dependency (106%) than in intensive care beds (23%). The main increase occurred during winter 2000-1, when high dependency beds increased by 57.5% and intensive care beds by 7.2%, after which there was an average 9.0% rise per year for high dependency and 1.4% rise per year for intensive care beds. Over the period 1999-2006, the mean cost of an intensive care bed day rose slightly from £1551 to £1647 (2006-7 prices).

Fig 2 Number of critical care beds in England located in general units providing intensive care 1999-2006

Trends in processes

Case mix of admissions

From 1998 to 2006 the proportion of women in the case mix increased, as did the mean age of those admitted (from 59.6 to 60.5) (table 3). Although there was no consistent change either in the proportion with at least one chronic condition or in the mean physiology score, the mean predicted risk of mortality rose from 30.5% in 1998 to 32.1% in 2000 but subsequently fell to 31.4% in 2006, indicating that less severe cases were being admitted. Analysis by 10ths of predicted risk of mortality in England showed no widening in the distribution of cases.

Table 3

 Characteristics of patients admitted to case mix programme units in England (1998-2006)

	1998	1999	2000	2001	2002	2003	2004	2005	2006
Mean age (years)	59.6	59.8	60.0	60.0	60.3	60.2	60.6	60.6	60.5
Female (%)	41.3	41.7	41.5	42.2	43.0	42.6	43.0	43.4	43.8
At least one previous chronic condition (%)	13.4	13.6	13.4	13.5	13.6	13.4	13.4	13.5	13.8
ICNARC model physiology score:
Mean	18.0	18.1	18.6	18.5	18.3	18.1	18.1	18.1	18.0
Median	16	16	17	17	17	16	16	16	16
ICNARC model predicted mortality:
Mean (%)	30.5	30.7	32.1	32.5	32.0	31.3	31.3	31.5	31.4
Median (%)	19.5	20.1	22.0	22.5	21.9	21.1	21.2	21.7	21.7

ICNARC=Intensive Care National Audit and Research Centre.

Transfers and discharges

Transfers out of units to another unit to receive the same level of care declined by 11.0% a year after 2000 (table 4) compared with annual increases of 2.6% before 2000 (P<0.001). Similarly, transfers into units declined 8.7% a year after 2000 compared with an annual 3.8% increase beforehand (P<0.001) (fig 3).

Table 4

 Comparison of average annual changes (adjusted for case mix) in transfers and discharges between 1998-2000 and 2000-6 (96 units)

	No (%)*	Relative risk (95% confidence interval)	P value†
Transfers out of units:
1998-2000	2018 (2.7)	1.03 (0.99 to 1.07)	<0.001
2000-6	3560 (1.9)	0.89 (0.87 to 0.90)
Transfers into units:
1998-2000	3357 (3.6)	1.04 (1.01 to 1.07)	<0.001
2000-6	6236 (2.8)	0.91 (0.90 to 0.92)
Early discharges:
1998-2000	4981 (7.1)	0.92 (0.89 to 0.94)	0.101
2000-6	7151 (4.0)	0.89 (0.88 to 0.90)
Unplanned discharges at night:
Discharged midnight to 4:59 am:
1998-2000	1048 (1.4)	1.03 (0.97 to 1.09)	0.008
2000-6	2016 (1.1)	0.92 (0.90 to 0.94)
Discharged 10 pm to 6:59 am:
1998-2000	1973 (2.6)	1.02 (0.98 to 1.07)	<0.001
2000-6	3695 (2.0)	0.92 (0.91 to 0.93)
Delayed discharges:
1998-2000	2500 (3.6)	1.38 (1.33 to 1.43)	<0.001
2000-6	16846 (9.5)	1.16 (1.15 to 1.17)
Discharges directly to normal place of residence:
1998-2000	581 (0.8)	1.11 (1.04 to 1.18)	0.218
2000-6	2676 (1.5)	1.17 (1.14 to 1.19)

*No adjustment for case mix; fourth quarter in 2000 included in 1998-2000; denominators based on unit discharges except transfers in (unit admissions).

†For difference in trends.

Fig 3 Critical care transfers as proportion of admissions (transfers in) and discharges (transfers out) 1998-2006

Early discharges because of a shortage of critical care beds declined from 7.1% in 1998 to 3.3% in 2006 (fig 4). Although the rate of night discharges (midnight to 4:59 am) steadily increased from 2.8% in 1998 to 4.2% in 2006 (fig 5), the proportion reported as being because of a shortage of critical care beds declined (44.5% to 21.8%), suggesting the proportion deemed “unplanned” fell from 1.2% to 0.9%. Before 2000, unplanned night discharges were increasing by 3.1% a year whereas afterwards they decreased by 7.7% a year (P=0.008) (table 4). A similar pattern emerged when we considered discharges between 10 pm and 6:59 am.

Fig 4 Early discharges, reported delayed discharges, and discharges directly to normal place of residence as proportion of all discharges 1998-2006

Fig 5 Night discharges and unplanned night discharges as proportion of all discharges 1998-2006

In contrast, the number of discharges delayed due to a shortage of ward beds rose steadily (fig 4) from 2.7% in 1998 to 14.2% by 2006, though the average annual increase after 2000 (16.2%) was lower than before (38.0%) (P<0.001) (table 4). There was an increase in discharges directly to patients’ normal place of residence (from 0.6% to 2.1%). The trend was not significantly greater after 2000 than before.

Length of stay and readmissions

After adjustment for differences in case mix, the mean length of stay increased before 2000 by 0.243 days a year but by only 0.036 days a year afterwards (P<0.001) (table 5). There was no significant difference in the decline in readmissions within 24 hours before and after 2000 (table 6). There was, however, a faster decline in readmissions within 48 hours after 2000 (P=0.006) (fig 6).

Table 5

 Comparison of average annual changes in mean length of stay (days) before and after 2000 (n=96 units)

Mean length of stay	Average annual change (95% CI )	P value*
1998-2000	0.24 (0.19 to 0.30)	<0.001
2000-6	0.04 (0.02 to 0.06)

*For difference in trends.

Table 6

 Comparison of average annual changes in readmission rates before and after 2000 (n=96 units)

	No (%)*	Relative risk (95% confidence interval)	P value†
Readmissions within 24 hours:
1998-2000	867 (1.1)	0.99 (0.94 to 1.05)	0.405
2000-6	1961 (1.0)	0.96 (0.94 to 0.98)
Readmissions within 48 hours:
1998-2000	1729 (2.3)	1.03 (0.98 to 1.07)	0.006
2000-6	3917 (2.1)	0.95 (0.94 to 0.97)

*No adjustment for case mix; fourth quarter in 2000 included in 1998-2000; denominator is unit discharges.

†For difference in trends.

Fig 6 Readmissions within 24 and 48 hours as proportion of all discharges 1998-2006

Trends in outcomes

While unit mortality adjusted for case mix did not change between 1998 and 2000, subsequently it fell by an average of 2.0% a year (P<0.001) (table 7), indicating a total fall of 11.3% in the six years after the final quarter of 2000. A similar pattern was observed for hospital mortality adjusted for case mix, with an average annual decline after 2000 of 2.4%, indicating a total fall of 13.4% over six years. Annual unit and hospital mortality adjusted for case mix also show a similar pattern (fig 7).

Table 7

 Relative risk for annual change in unit and hospital mortality adjusted for case mix before and after 2000 (n=96 units)

	No (%)*	Relative risk (95% CI)	P value†
Unit mortality:
1998-2000	19 427 (21.2)	1.00 (0.99 to 1.01)	<0.001
2000-6	46 119 (20.6)	0.98 (0.98 to 0.98)
Hospital mortality:
1998-2000	28 746 (32.1)	1.00 (0.99 to 1.01)	<0.001
2000-6	67 644 (30.7)	0.98 (0.98 to 0.98)

*No adjustment for case mix; fourth quarter in 2000 included in 1998-2000; denominators exclude readmissions to critical care within same hospital stay.

†For difference in trends.

Fig 7 Relative risk (95% confidence interval) of hospital mortality and unit mortality adjusted for case mix, 1998-2006

Cost effectiveness of changes in critical care

The decrease in the annual change in the mean length of stay, both in critical care (by 0.18 days, P<0.001) and for the subsequent ward stay (by 0.35 days, P<0.001), meant the mean annual cost increases after 2000 (£196) were smaller than before (£391). The mean incremental cost (adjusted for case mix) therefore fell by £195 (P<0.001).

Annual improvements in the mean lifetime QALYs (adjusted for case mix) were slightly greater after 2000 and so the mean incremental QALYs were positive (0.025), though not significant (P=0.06) (table 8). Valuing a QALY at £20 000, coupled with the decline in incremental costs (−£195), resulted in a positive incremental net monetary benefit of £692 (P=0.008). Hence, if the differences in costs and QALYs can be attributed to the interventions, then they were relatively cost effective.

Table 8

 Incremental cost effectiveness (mean predicted costs*, QALYs, and net monetary benefit) for 2000-6 compared with 1998-2000

	Mean annual change (before 2000)	Mean annual change (after 2000)	Increment (95% CI)	P value†
Mean QALY	0.040	0.064	0.025 (−0.001 to 0.050)	0.057
Mean cost (£)	391	196	−195 (−330 to −59)	<0.001
Net monetary benefit (£)	402	1096	692 (176 to 1208)	0.008

QALY=quality adjusted life year.

*Adjusted for case mix, season, and admission quarter.

†For whether difference in incremental change (before−after 2000) differs from zero.

Sensitivity analysis had little effect on the main findings. Firstly, the effect of taking mean unit costs from all English NHS trusts, rather than just participants from the case mix programme, led to a relative decrease in costs after 2000, lower incremental costs, and a higher incremental net monetary benefit (table 9). Secondly, re-weighting mean unit costs to include a 20% high dependency component led to a relative reduction in mean costs after 2000 and a higher incremental net monetary benefit. Re-weighting to reflect the gradual increase in the size of the high dependency component resulted in similar results to the base case. Thirdly, when the summary baseline probability of death was used for case mix adjustment rather than the separate components of the ICNARC model, this led to greater QALY gains, smaller incremental costs, and a larger incremental net monetary benefit. Fourthly, assuming the long term survival and the quality of life of patients was the same as for the general population23 yielded a higher incremental net monetary benefit. Finally, extending the analysis from 96 to all 159 units in the case mix programme also led to a higher incremental net monetary benefit as did valuing a QALY at £30 000.

Table 9

 Sensitivity analyses reporting incremental costs (£), incremental QALYs, and incremental net monetary benefit (NMB) (£20 000 per QALY) for different scenarios

	Mean incremental QALY (P value)	Mean incremental cost (P value)	Mean incremental NMB (P value)
Base case	0.025 (0.057)	−195 (<0.001)	692 (0.008)
Unit costs:
All trusts	0.025 (0.057)	−278 (<0.001)	773 (0.003)
20% HDU	0.025 (0.057)	−510 (<0.001)	1004 (<0.001)
13-23% HDU	0.025 (0.057)	−226 (<0.001)	720 (0.006)
Summary case mix adjustment	0.04 (0.027)	−307 (<0.001)	1112 (0.002)
No ICU decrement for QALY	0.038 (0.037)	−195 (<0.001)	955 (0.009)
All cases	0.091 (<0.001)	−256 (<0.001)	2072 (<0.001)
Value £30 000 per QALY	0.025 (0.057)	−195 (<0.001)	940 (0.016)

QALY=quality adjusted life year; HDU=high dependency unit; ICU=intensive care unit.

Impact of organisational changes

The benefit of being in a “later adopter” network was greater than being in a “earlier adopter” network: faster declines in the proportion of admissions that were transfers in (10.0% v 6.2% a year), transfers out (13.4% v 4.4% a year), early discharges (12.6% v 4.5% a year), and unplanned night discharges (8.6% v 1.4% a year); smaller increases in delayed discharges (12.2% v 23.6% a year); less increase in mean length of stay (0.015 v 0.089 days a year); and a faster decline in unit (2.5% v 1.3% a year) and hospital mortality (2.9% v 1.9% a year).

The effects of the time at which units adopted outreach and the ventilator care bundle were less clear. Later adopter units were associated with some benefits: faster declines in transfers in (11.8% v 8.2% a year) and unplanned night discharges (8.9% v 3.0% a year); slower increase in delayed discharges (8.1% v 18.4% a year); less increase in length of stay (0.024 v 0.072 days a year); faster decline in hospital mortality (2.8% v 2.2% a year). These units, however, experienced a slower decline in transfers out (9.6% v 13.0% a year) and in early discharges (7.7% v 11.8% a year).

Units with the largest increases in capacity were associated with faster declines in transfers in (13.6% v 5.8% a year), transfers out (17.9% v 8.5% a year), early discharges (17.5% v 6.5% a year), and unplanned night discharges (12.1% v 6.1% a year); and slower increases in delayed discharges (14.4% v 17.7% a year). There were, however, smaller declines in unit mortality (1.8% v 2.7%), though no significant differences for hospital mortality.

Discussion

Main findings

After 2000, unit mortality adjusted for case mix in England fell dramatically by 2.0% a year and hospital mortality by 2.4% a year (compared with no change between 1998 and 2000). This was accompanied by a decrease of 11.0% a year in transfers out (for the same level of care) to other units and a fall of 8.7% a year in transfers in, whereas previously both proportions had been rising. In addition, the proportion of unplanned night discharges declined by 7.7% a year. Despite small increases in average unit costs, the cost effectiveness of critical care increased after 2000, partly as a result of the improvements in outcome and partly because of smaller increases in the mean length of stay.

If such changes can be attributed to the initiatives to transform critical care (increased capacity, clinical networks, outreach services, and the ventilator care bundle) this package can be regarded as a more cost effective intervention than many other healthcare interventions. Improvements were associated with later, rather than earlier, adoption of organisational changes. There are two possible explanations for what seems to be a counterintuitive finding: later adopters might have benefited from the experiences of earlier adopters and also might have spent longer preparing for change and thus established greater commitment from staff. While increased capacity was not directly associated with improvements in outcomes, it was associated with declines in the rate of transfers and early discharges and might have contributed indirectly via unmeasured effects, such as improvements in staff morale.

Despite striking improvements in processes, outcomes, and cost effectiveness in England, the contribution of the explicit programme promoted by the Modernisation Agency is unclear. It is possible that concurrent changes—such as the introduction of “hospital at night,”24 new staff contracts,25 and nurse consultants26—which were also being implemented during this period contributed to the improvements observed.

Methodological limitations

Defining and measuring interventions

Inclusion of interventions had to be restricted to those that were feasible to measure. While this included the principal elements, it was not comprehensive. Creating simple dichotomies (earlier and later adopters) for networks and units might have masked some associations, and data on when interventions were adopted might have been subject to recall bias. Both limitations, however, would have reduced the likelihood of identifying associations rather than created spurious ones.

Measuring inputs

Given the increasing proportion of lower cost high dependency beds over time, use of overall bed numbers would have overestimated costs in the later period and thus underestimated cost effectiveness. The use of average NHS reference costs is justified by their similarity to deriving average unit costs for critical care with micro-costing techniques.27 Changes in the reporting of unit costs in 2004 made little difference given that the main determinants of the incremental net monetary benefit were the relative reduction in length of stay and the gain in QALYs.

Analysis

While the final quarter of 2000 represented the optimum time point to mark the start of the interventions, adoption did not occur until later in many units. This will have tended to reduce the observed impact. Our approach also assumes that any changes in processes and outcomes observed after 2000 are linear.

There is some uncertainty as to the longevity and quality of life of survivors of critical care. Recent research suggests that the assumption of a 20% deficit might overstate the relative decrement,28 and other analyses have assumed the same survival and quality of life as the general population.29 The sensitivity analysis, however, showed that our findings were robust.

It was important to recognise the hierarchical nature of the data in the analysis.30 The main analyses allowed for clustering within critical care units by fitting units as random effects. Networks were not treated as a separate level because of potential biases in those with a lower participation in the case mix programme. Some caution is therefore required in interpreting the findings for network adoption.

Conclusions

This attempt at a nationwide evaluation of policies pursued since the publication of the NHS Plan in 2000, albeit limited to one specific part of health care, suggests that the interventions represent a highly cost effective use of resources. In any time series analysis the attribution of causality is a challenge. This is particularly so for evaluations of complex interventions at a national level. The ability to make causal inferences about the benefits of “modernisation” is limited by the lack of a comparable control group of critical care units in which there were no modernisation interventions. Attribution is also challenged by the presence of other concurrent interventions, such as the introduction of hospital at night and a myriad local modernisation projects. As a consequence, conclusions must be restricted to the observation that significant improvements in outcomes and processes occurred in 2000-6 without being able to link such changes to specific interventions.

Whatever the reasons for the improvements, considerable additional expenditure on critical care combined with an explicit centrally driven programme of modernisation has resulted in dramatic improvements in outcomes.

Since 2000, there have been considerable increases in funding for adult critical care in the NHS in England

A specific attempt has been made to “modernise” services by establishing critical care networks, introducing outreach services in hospitals, and adopting clinical guidelines in the form of care bundles

There have been conflicting claims as to the impact of these developments on the outcome of care (intensive care unit and hospital mortality) and the processes of care (transfers, early discharges, delayed discharges)

In the six years since 2000 there have been major improvements in outcome, with a 13.4% fall in hospital mortality and in processes of care (fewer transfers, fewer early discharges)

Collectively the interventions associated with the modernisation of critical care represent a highly cost effective use of NHS resources