Performance of the National Early Warning Score in Hospitalized Patients With Kidney Failure on Maintenance Hemodialysis.

To the Editor:

Early warning scores have become a standard clinical decision support tool for hospitalized patients. They are designed to predict decompensation, defined typically as unplanned transfer to the intensive care unit or mortality in patients on a medical floor. There are dozens of early warning scores, of which one of the most commonly used is the National Early Warning Score (NEWS), a simple score consisting of 7 vital signs.,

Although studies have shown the NEWS and other early warning scores to have adequate predictive performance,, the utility of these scores in clinical subpopulations is less well established. Early warning scores have been examined in kidney transplant recipients, but their use in patients with kidney failure treated on maintenance hemodialysis has only been examined as subpopulation analyses in larger studies. Because these patients have unique hemodynamics and often abnormal vitals captured during dialysis treatment, it is unknown whether the NEWS is useful in this population. We sought to evaluate the performance of the NEWS in hospitalized patients on maintenance hemodialysis. All analyses were performed in R 4.0.3. This work was approved by the Duke University Institutional Review Board under Pro00108813, which waived the requirement for informed consent given the nature of the research.

Detailed methods are available in Item S1. In brief, we extracted electronic health record data on patients hospitalized at a large, >1000-bed academic medical center. We identified all patients in 2019 who were admitted to a medical or surgical ward. We used the presence of International Classification of Diseases, Tenth Revision code N18.6 in the index encounter and Current Procedural Terminology codes 90935 and 90937 to identify patients on maintenance hemodialysis. The outcome was death while on the medical or surgical floor or escalation of care to the intensive care unit. The NEWS is a points-based clinical decision support using 7 vital signs: systolic blood pressure, respiratory rate, temperature, pulse oximetry, heart rate, use of supplemental oxygen, and level of consciousness. The points system assigns more points for vitals both higher and lower than normal with total scores represented as integers between 0 and 20 (Table S1). We abstracted time-varying vital signs necessary to calculate the NEWS. Our data included 28,905 admissions over the study period, of which 1,343 (4.6%) were patients with kidney failure on maintenance dialysis. Demographics, comorbidities, and outcomes are shown in Table 1. Patients with and without kidney failure did not have a significantly different rate of the primary outcome at 12 hours (1.3% vs 1.0%, P = 0.32).

Table 1

Baseline Characteristics of Inpatient Encounters by Kidney Failure Status

	Patients With Kidney Failure on Maintenance Dialysis (N = 1,343)	Patients Without Kidney Failure (N = 27,562)	P
Age, y, median (IQR)	60.6 (49.6-69.8)	63.2 (50.1-72.8)	<0.001a
Male sex, n (%)	756 (56.3%)	13,298 (48.2%)	<0.001b
Race/ethnicity <0.001b
Hispanic	52 (3.9%)	819 (3.0%)
NH Black	818 (60.9%)	7,647 (27.7%)
NH White	341 (25.4%)	15,797 (57.3%)
Other	132 (9.8%)	3,296 (12.0%)
Comorbid conditions
Any malignancy	130 (9.7%)	7,626 (27.7%)	<0.001b
Chronic pulmonary disease	308 (22.9%)	6,197 (22.5%)	0.70b
Congestive heart failure	764 (56.9%)	6,608 (24.0%)	<0.001b
Diabetes mellitus	846 (63.0%)	8,324 (30.2%)	<0.001b
Myocardial infarction	129 (9.6%)	1,176 (4.3%)	<0.001b
Code status: DNR	117 (8.7%)	2,885 (10.5%)	<0.001b
Admission source
Direct admission	483 (36.0%)	14,124 (51.2%)	<0.001b
Emergency department	860 (64.0%)	13,438 (48.8%)

Abbreviations: DNR, do not attempt resuscitation; IQR, interquartile range; NH, non-Hispanic.

Kruskal-Wallis test

χ2 test

To emulate continuous monitoring, we calculated the NEWS in 2-hour intervals using last known vital measurement. We evaluated the predictive performance of the time-varying risk score, stratified by kidney failure status, in the next 12 hours, measuring performance using the area under the receiver operating characteristic curve and area under the precision recall curve. The precision recall curve calculates the trade-off between sensitivity and positive predictive value, and the area under the precision recall curve can be interpreted as the average positive predictive value. To account for the repeated measurements on a given patient overtime, we calculated the metrics within independent cross-sections and then generated a weighted mean. We used the bootstrap method to estimate standard errors for confidence intervals and difference testing.

Overall, the NEWS better predicted intensive care unit transfer or mortality over 12 hours in patients with kidney failure compared to patients without (area under the receiver operating characteristic curve of 0.749 vs 0.702, and area under the precision recall curve of 0.081 vs 0.048, P < 0.001, Table 2).

Table 2

Performance of National Early Warning Score by Kidney Failure Status

	Patients With Kidney Failure Maintenance Dialysis (N = 1,343)	Patients Without Kidney Failure (N = 27,562)	P
Outcome rate (death or ICU transfer)	1.3%	1.0%	0.32a
AUROC, 95% CI	0.749 (0.718, 0.774)	0.702 (0.694, 0.709)	<0.001b
AUPRC, 95% CI	0.081 (0.062, 0.100)	0.048 (0.044, 0.051)	<0.001b

Abbreviations: AUROC, area under the receiver operating characteristic curve. AUPRC, area under the precision recall curve; ICU, intensive care unit.

χ2 test

Bootstrap method was used to account for repeated measurements over time.

In order to further understand the predictive performance of NEWS, we assessed the relationship between each of the individual vital signs with the outcome. We fit separate time-varying covariate time-to-event models with penalized splines for each of the NEWS vital signs. Figure 1 shows relative hazards for the physiological parameters for patients with and without kidney failure. A relative outcome rate of 1 is assigned to the median vitals value. Values above 1 indicate increased risk of deterioration at that vital sign level, whereas values below 1 indicate decreased risk. The patterns of association were similar, though generally stronger for patients with kidney failure on maintenance dialysis. Notable differences were that (1) high systolic blood pressure tended to be relatively protective against decompensation in patients with kidney failure, and (2) a high heart rate was more predictive of decompensation in kidney failure patients. We noted that low pulse oximetry was nominally protective, though this effect attenuated after accounting for supplemental oxygen status (Fig S1).

Figure 1

Association of National Early Warning Score components and risk of deterioration by kidney failure status. Cut points defining thresholds for assigning points for each of the National Early Warning Score components are shown by vertical black lines. A relative outcome rate above 1 represents increased risk of deterioration whereas an outcome rate less than 1 represents decreased risk of deterioration. Dotted red and blue lines represent 95% CI.

This study is the first to establish that the widely used NEWS not only performs well in patients with kidney failure on maintenance dialysis but better than in other hospitalized patients. Patients with kidney failure undergoing dialysis may have frequent vital sign abnormalities such as hypotension and hypothermia. Despite these vital sign abnormalities, the well-established and broadly used NEWS is clinically pertinent and useful in this population.

Although the NEWS had better predictive performance in patients with kidney failure compared to those without, it is important to note that the NEWS functions as a triage score rather than an absolute risk prediction. Therefore, the overall utility of the NEWS depends upon institution-specific thresholds and associated action steps. The pattern of association between each vital sign component and risk of deterioration was generally similar between the 2 groups, though the association was often stronger in patients with kidney failure. Some of the NEWS components, such as heart rate, may even be more predictive in kidney failure patients, likely contributing to the score’s superior predictive performance in patients with kidney failure.

Given the NEWS’s predictive performance in this patient population, a customized early warning score for patients with kidney failure on maintenance dialysis is not necessary. Institutions can use the NEWS to reliably help predict decompensation in patients with kidney failure on maintenance dialysis.