Adjusting Early Warning Score by clinical assessment: a study protocol for a Danish cluster-randomised, multicentre study of an Individual Early Warning Score (I-EWS).

INTRODUCTION: Track and trigger systems (TTSs) based on vital signs are implemented in hospitals worldwide to identify patients with clinical deterioration. TTSs may provide prognostic information but do not actively include clinical assessment, and their impact on severe adverse events remain uncertain. The demand for prospective, multicentre studies to demonstrate the effectiveness of TTSs has grown the last decade. Individual Early Warning Score (I-EWS) is a newly developed TTS with an aggregated score based on vital signs that can be adjusted according to the clinical assessment of the patient. The objective is to compare I-EWS with the existing National Early Warning Score (NEWS) algorithm regarding clinical outcomes and use of resources. METHOD AND ANALYSIS: In a prospective, multicentre, cluster-randomised, crossover, non-inferiority study. Eight hospitals are randomised to use either NEWS in combination with the Capital Region of Denmark NEWS Override System (CROS) or implement I-EWS for 6.5 months, followed by a crossover. Based on their clinical assessment, the nursing staff can adjust the aggregated score with a maximum of -4 or +6 points. We expect to include 150 000 unique patients. The primary endpoint is all-cause mortality at 30 days. Coprimary endpoint is the average number of times per day a patient is NEWS/I-EWS-scored, and secondary outcomes are all-cause mortality at 48 hours and at 7 days as well as length of stay. ETHICS AND DISSEMINATION: The study was presented for the Regional Ethics committee who decided that no formal approval was needed according to Danish law (J.no. 1701733). The I-EWS study is a large prospective, randomised multicentre study that investigates the effect of integrating a clinical assessment performed by the nursing staff in a TTS, in a head-to-head comparison with the internationally used NEWS with the opportunity to use CROS. TRIAL REGISTRATION NUMBER: NCT03690128. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

RCT Entities:   Population Interventions Outcomes

A comparison of Early Warning Scores in a large prospective, randomised study.

This study will strengthen the current evidence of the effectiveness of track and trigger systems.

The objective vital sign-based National Early Warning Score (NEWS) and the nursing staffs’ subjective clinical assessment is combined into one unified score.

The effect of the Individual Early Warning Score is limited by the direct comparison to NEWS and the absence of a control group not using any scoring system for systematic observation and risk assessment.

Background

Clinical deterioration and serious adverse events (SAE), such as cardiac arrest or unexpected death are preceded by deviations in vital signs in 60%–84% of cases.1–3 This has led to a focus on early identification of deviations in vital signs in an attempt to identify patients at risk in order to initiate treatment.4 5

Rapid response systems are therefore implemented internationally.6 7 The systems consist of an afferent limb to detect patients at risk and an efferent limb to subsequently alert for a higher level of observation and assistance, for instance, a medical emergency team (MET). Track and trigger systems (TTSs) represent the afferent limb that estimates the risk of deterioration. TTSs are generally well validated to provide prognostic information on SAE.8–11 A widely accepted TTS is the National Early Warning Score (NEWS), which combines vital signs in an aggregated weighted score and has shown superior performance to other systems in predicting SAE.12 13 Several observational, but only two randomised control trials (RCTs) have investigated the effect of implementing an TTS based on an aggregated weighted score.14–19 Results from the studies are conflicting and the evidence of clinical usefulness of these widely implemented TTSs is low. The two RCTs did not detect a lower mortality during hospitalisation with TTSs.16 17 Therefore, it has been suggested that further large prospective, multicentre studies must be performed to investigate the effectiveness of TTSs.20 21

To date, early and reliable identification of high-risk patients remains challenging. Several reasons for why these logical TTS models do not always work in clinical practice have been discussed.22–24 One being a possible danger of over-relying on TTSs in place of clinical judgement. Currently a TTS as NEWS does not actively include or encourage the use of staff observations and clinical assessment. The intention with NEWS was ‘an aid to clinical decision making—it is not a barrier or alternative to skilled clinical judgement’.25 Even though the intention of NEWS never was to exclude nursing staff’s clinical assessment a concern has been raised of a culture that is more task-oriented and is referred to in clinical practice as ‘doing the obs’.24 26 27 Prior to the introduction of the systematic observation and risk stratification, identification of deterioration was based on clinical assessment, experience and reflection during the daily clinical practice.28 In fact, health staff can identify patients at increased risk of death and nurses’ concern can contribute to identify early stages of deterioration.29–32 A study investigated the combination of the objective vital sign-based risk prediction score with registered nurses’ subjective clinical assessment into one unified Copenhagen Triage Algorithm, used at admission. The Copenhagen Triage Algorithm was found non-inferior regarding 30-day mortality compared with traditional triage, and significantly reduced the level of acuity.33

A standardised TTS like NEWS is a ‘one size fits all’ system. It does not differentiate between different types of disease or the patient’s individual physiological response. Therefore, there is a potential risk that the system fails to detect a patient with an abnormal stress response. Additionally, patients suffering from chronic illness might have baseline values that are different from other patients, leading to redundant observations, measurements and suboptimal usage of limited staff resources. In its current form, TTSs only allow minimal or no room for individual clinical assessment.

Study objectives

The Individual Early Warning Score (I-EWS) study is designed to investigate the impact of a new TTS; I-EWS and to compare it to the existing NEWS system. The study will investigate the impact on clinical outcomes and use of resources.

Methods

Study hypothesis

The main hypothesis is that I-EWS will be non-inferior to the existing NEWS algorithm regarding 30-day mortality but the addition of clinical assessment in the I-EWS results in lower scores, fewer escalations, and consequently that patients on average are scored fewer times per day when I-EWS is used rather than NEWS.

Setting and intervention

The I-EWS study is a prospective, multicentre, cluster-randomised, cross-over, non-inferiority study with participation of eight hospitals from two regions in Denmark. Each hospital will complete a control period and an intervention period. All hospitals are, prior to study start, randomised to start as either control, with continued use of NEWS, with the opportunity to use the Capital Region of Denmark NEWS Override System (CROS) (see online supplementary material), or intervention with implementation of I-EWS. After 6.5 months, a cross-over will occur resulting in a total study period of 13 months. Local research staff defined as representatives from every participating ward will ensure education of staff, implementation and adherence (figure 1).

Figure 1

Implementation of I-EWS and NEWS in the intervention and control group before and after cross-over. I-EWS, Individual Early Warning Score; NEWS, National Early Warning Score.

Eligible hospitals and randomisation

Clusters are defined as hospitals. To be eligible, hospitals must meet the predefined cluster criteria (figure 2). All hospitals located in the Capital Region and Region Zealand were assessed for inclusion and if eligible, randomised using a stratified randomisation based on the size of the hospitals (number of unique admissions). Hospitals characteristics are shown in table 1.

Figure 2

Assessment of eligible clusters and the following randomisation. I-EWS, Individual Early Warning Score; NEWS, National Early Warning Score.

Table 1

Characteristics of eligible clusters

Catchment area (approx.)	Number of beds	Unique admissions in 2016*
Capital region of Denmark
Herlev-Gentofte Hospitaltwo units	480 000	949	41 420
Nordsjaellands Hospitaltwo units	378 000	552	34 159
Bispebjerg Hospitalone unit	460 000	456	13 013
Glostrup Hospitalone unit (Medical ward)	139 000	83	6598
Amager-Hvidovre Hospitaltwo units	532 000	698	26 466
Region Zealand	836,000†
Zealand University Hospitaltwo units	629	34 125
Slagelse Hospitalone unit	–	327	17 999
Holbaek Hospitalone unit	–	338	16 125

*Number of unique admissions is based on the wards that are participating in the study.

†The catchment area in Region Zealand is not defined for each hospital.

To balance the size difference between intervention and control group, hospitals with more than 25 000 admissions were defined as large hospitals, less than 25 000 as small hospitals (table 1). Our stratified randomisation based on the size of the hospitals (large and small) would ensure that these were equally distributed between the intervention and control group. The randomisation was performed using computer-generated numbers (random.org) allocating hospitals to start as either control or intervention in a 1:1 ratio.

Eligible patients

All patients ≥18 years of age, admitted for more than 24 hours to one of the participating hospitals are included in the study. Patients admitted to departments of obstetrics or transferred to the intensive care units are not included during their time of admission there, as other systems for observation are used. Patients will be included at the first admission in the study period (index admission) and will remain in this group (intervention/control) for a follow-up period of 30 days. All subsequent admissions (readmissions) will be ignored. If a ‘do not resuscitate’-order or ‘no NEWS/I-EWS registration’ are identified, patients will be excluded for further analysis at the time of registration.

Endpoints

The primary non-inferiority endpoint is all-cause mortality at 30 days and the primary superiority endpoint is the average number of times per day a patient is NEWS/I-EWS-scored. Secondary outcomes are listed in box 1.

Primary endpoint

All-cause mortality at 30 days after admission.

Coprimary endpoint

Average number of times per day a patient is National Early Warning Score (NEWS)/Individual Early Warning Score (I-EWS)-scored.

Secondary endpoints

Length of hospital stay.*

All-cause mortality at 48 hours after admission.

All-cause mortality at 7 days after admission.

The following will also be reported:

Cardiac arrests† during hospital stay, reported in numbers (%) and per 10 000 ward days.

Scores generating a call for mobile emergency team reported in absolute number (%) and per 10 000 ward days.

Scores generating a call for the attending doctor, reported in absolute number (%) and per 10 000 ward days.

Frequency of changes in I-EWS leading to an escalation or de-escalation in the escalation protocol.

Comparison of changes in NEWS due to I-EWS modification (intervention group) and due to temporary or chronic acceptable values (control group).

All endpoints will be studied in predefined subgroup analyses (table 2).

Table 2

Subgroups

Herlev-Gentofte Hospital
Nordsjaellands Hospital
Bispebjerg Hospital
Glostrup Hospital (medical ward)
Amager-Hvidovre Hospital
Zealand University Hospital
Slagelse Hospital
Holbaek Hospital
Age ≤39 years
Age 40–69 years
Age ≥70 years
Cancer*	Chapter II: neoplasms (C00-C97+D37-D48).
Cardiovascular disease*	Chapter IX: diseases of the circulatory system (I09-I52+I70-I89)
Pulmonary disease	Chapter X: (DJ00-DJ99)
Infections*	Chapter I: (A00-B99+J00 J22+N10 N11+N30 N31+G00 G09)
Neurological disease*	Chapter VI: diseases of the nervous system (G09-G47+I60-I69)
Surgical conditions†	KF+KG+KJ+KK+ KN

*Prespecified subgroups based on diagnosis (ICD-10 codes) at discharge.

†Surgical conditions: presence of the defined surgical procedure codes.

*Length of hospital stay is defined as number of consecutive days in hospital, including days after transfer to a higher-lever or lower-lever hospital.

†Clinical Cardiac Arrest as defined in the European resuscitation guidelines in patients without a ‘do not resuscitate’-order.

National Early Warning Score

NEWS is an aggregated weighted score of 0–20, based on measurements of heart rate, systolic blood pressure, arterial oxygen saturation, respiratory rate, level of consciousness, temperature and supplemental oxygen.12 Observation intervals and escalations of care are defined by the escalation protocol (see online supplementary material). A TTS has been standard since 2012 in Denmark.34 In the eastern part of Denmark, defined by the Region Zealand and The Capital Region of Denmark with a population of 2 662 974 people NEWS is used at all hospitals. NEWS can be used in combination with CROS. CROS gives the doctors an opportunity to define acceptable temporary or chronic values for NEWS variables which will automatically adjust all aggregated scores within the next 24 hours or until the chronic value is revised or the patient is discharged (see online supplementary material).

Individual Early Warning Score

I-EWS is based on NEWS.12 Nursing staff, following the algorithm for rescoring, record vital signs systematically, and patients are assigned an aggregated score of 0–20, subsequently the nursing staff can adjust the score based on a clinical assessment. If an adjustment is considered necessary, the I-EWS score can be modified with a maximum of −4 or +6 points. An I-EWS modification of 0 indicates that the score, based on vital signs, is acceptable and relevant for the patient. The available range for modification was based on an expert discussion within the steering committee. Clinical assessment is based on experience and the immediate clinical impression of the patient. In this process, the nursing staff take the individual patient characteristics and their knowledge of department-specific diseases into account. The patient’s or relatives’ concern can also be included. I-EWS protocol state that all nursing staff can adjust the score. Few wards have had concerns with this procedure and have introduced local limitations, for example, all modifications have to be discussed with a medical doctor or two experienced nurses must agree on an adjustment. We will assess such limitations in the statistical analyses.

Data and follow-up

NEWS and I-EWS registrations are integrated into the Electronic Patient Journal (EPJ) used by all participating hospitals. After measurement of the mandatory vital signs an aggregated score is calculated automatically. To ensure use of I-EWS, a warning is triggered in case the I-EWS modification is not addressed. During control periods, I-EWS is inactive in the EPJ system.

All patients will be followed from admission to 30 days after and will be accessed through the EPJ and the Danish Central Registries (The Danish National Patient Registry, the Civil Registration System, DanArrest). After extraction, all data will be pseudonymised and stored in a secured network in accordance with the current guidelines for data management in the Capital Region of Denmark. Data from the first 2 weeks and final 4 weeks of each period will be excluded (figure 3). Any patient lost to follow-up will be censored at the last time known to be alive.

Figure 3

Study period including periods with censored data. I-EWS, Individual Early Warning Score.

Sample size calculation

The study is a non-inferiority study. Based on the number of admissions in 2016, approximately 189 000 unique individuals are expected to be admitted within 12 months with an estimated 30-day mortality of 2.9%. During inclusion approximately 150 000 unique individuals are expected to be admitted. Due to the low incidence of the primary endpoint, the difference between the NEWS and I-EWS group (non-inferiority margin (∆)) is set to 0.5%. A variation in mortality of 25% over the course of the year is estimated. Based on these assumptions the study has more than 90% power to confirm non-inferiority. In a previous cohort of 103 733 patients the mean (SD) number of times per day a patient was NEWS/I-EWS-scored was 2.3 (2.0) and 15.4% of patients had one NEWS score of ≥6.34 With the expected inclusion, we will be able to detect a difference of 0.04 in mean number of scores and a difference of 0.6% in the number of patients with one NEWS ≥6 with a power >90%.

Statistics

Statistical analyses of the primary and secondary endpoints regarding mortality will be performed in accordance to the intention-to-treat principle. In addition, per-protocol analyses will be performed excluding patients with missing I-EWS modifications in more than 1/3 of the total number of measurements. The primary endpoint regarding all-cause mortality at 30 days after admission will be analysed using logistic regression that accounts for clustering with adjustment for hospitals, age and sex. The cluster intraclass correlation coefficient with 95% CIs will be reported and an assessment of cluster-level differences will be performed. Survival plots using Kaplan-Meier method will be presented. Subgroup analysis will also be performed (table 2). Average number of times per day a patient is NEWS/I-EWS-scored will be compared using linear regressions. Secondary endpoints regarding mortality will be assessed using the same analyses as the primary endpoint. Length of hospital stay between groups will be compared by linear regression following the same adjustment principles as for the primary outcome. In addition, the frequency of changes in I-EWS that led to an escalation or de-escalation in the escalation protocol will be reported. Changes in NEWS due to CROS values and changes in I-EWS due to clinical assessment will be compared using Poisson regression models. The number of cardiac arrests, scores generating a call for mobile emergency team or doctor will be reported in absolute numbers and per 10 000 ward days and compared using Poisson regression models. Sensitivity analyses will be performed for the last 3 months of each intervention period and excluding departments where I-EWS protocol was not followed. All endpoints are prespecified in the statistical analyses plan which is published at clinicaltrials.gov.

The findings will be reported in compliance with the Consolidated Standards of Reporting Trials. Analyses will be performed with R (R Project for Statistical Computing) and SAS software, version 9.4 (SAS Institute).

Patient and public involvement

Patients and the public had no other involvement in study design. Nursing staff from different clinical specialties were invited to a 2-hour meeting on systematic observation and prevention of deterioration during hospitalisation. Ideas for the study were here presented and discussed. A feasibility study was also performed, results are discussed below.

Substudies

Vital signs, demographics, diagnoses and information regarding admissions will be stored in a database for future research projects. Furthermore, a study will investigate registered nurses’ clinical observation, assessment and decision-making when using NEWS and I-EWS. An understanding of their role and experience in using these systems is important for future development of TTSs, implementation and ensuring compliance to these. A study investigating the inter-rater reliability of I-EWS will also be performed. Nurses will be included after oral and written informed consents have been obtained and pseudonymisation will be done.

Feasibility study

I-EWS and the expected implementation set-up was tested in a 2-week period at eight medical or surgical wards at Herlev-Gentofte Hospital, a 949-bed University Hospital in the Capital Region of Denmark in June 2018. The aim was to test the use of I-EWS in a clinical setting and to explore the nursing staff’s experience with the implementation of I-EWS. A total of 5669 complete vital sign registrations were recorded and I-EWS was registered correctly in 4585 (80.9%).

Of the total 4585 I-EWS, 876 (19.6%) were modified to a lower score and (116) 2.6% to a higher score. A questionnaire was sent electronically to the staff to assess if the information about I-EWS was sufficient and to assess the perception of I-EWS. Eighty-one of the 181 questionnaires (45%) were returned and 65.4% were very satisfied/satisfied. The registration of I-EWS was found easy by 80.3% of the nursing staff. We concluded that implementation of the new I-EWS was feasible and well accepted among staff to continue with a large-scale study.

Study status and surveillance

Randomisation was performed in August 2018. I-EWS was thereafter activated and inclusion began on 1 October 2018. Crossover was performed on 15 April 2019, as expected. In the first inclusion period (figure 3) a total of 548 129 complete vital sign registrations were registered in the intervention group. I-EWS was used in 520 630 (95%), with either the score being accepted or adjusted. Inclusion and observation periods were completed on 31 October 2019.

Study organisation

The study is an investigator-initiated clinical study. The protocol was written by the steering committee and is composed representatives from each of the participating hospitals as well as experts in acute medicine, intensive care and Rapid Response Systems in Denmark. The steering committee is responsible for study design, overseeing the conduct of the study and data acquisition, supervision of study personnel, approval of substudies, publication and communication of results, and for ensuring study compliance with Danish and European laws regarding personal data.

Ethics and dissemination

The results will be published in an international peer-reviewed journal, regardless of the results.

Discussion

The I-EWS study is a large investigation to compare different TTSs. The study investigates the effect of adding individual clinical assessment in a TTS. A strong association between vital sign abnormalities and poor outcomes is shown in several studies.1 2 35 A logical preventive step appears to be frequent and accurate measurements and reporting of vital signs, providing the opportunity to intervene and possibly prevent deterioration. Hence, it seems only clinically useful if it leads to improved care—ideally measured by improved patient outcomes. Few studies have been performed to quantify the clinical performance of NEWS and other aggregated weighted TTSs.14–19 Most studies are retrospective before-and-after studies and only two RCT exist. The results from these studies are conflicting, and their effectiveness on health outcomes compared with utilisation of resources remain uncertain.14 If applying standard methods to evaluate the evidence for the widely implemented TTSs, the evidence at its best will be level II-2 defined as comparisons between times and places with or without the intervention by the Canadian Task Force on the Periodic Health Examination. Thus, TTSs have been implemented based on low levels of evidence and without a general conclusion on their impact. As the impact of TTS on both patients and resources is enormous, it is essential to close this evidence gap and determine the optimal way to identify those patients at risk of clinical deterioration who would benefit from early intervention. Ideally, I-EWS or even NEWS should be tested and assessed in an RCT with a control group not using any scoring system for systematic observation and risk assessment. Due to the widespread implementation of NEWS and for practical and political reasons this is not possible. A head to head comparison between existing and newly developed scoring systems is however, both feasible and essential to obtain optimal risk assessment.

In 2017 NEWS2 was presented, which included a correction in oxygen saturation for patients with a history of hypercapnia.25 NEWS2 has largely in the UK superseded NEWS. NEWS2 is not available at the participating hospitals. In our control group the clinicians have the opportunity to include this correction in permissible oxygen saturation for patients with a history of hypercapnia through CROS. The beneficial effect of NEWS2 over NEWS is stilled being discussed.36 Of course, NEWS in combination with CROS is not directly comparable to NEWS2 but is an acceptable alternative which we believe can contribute to further causative evidence of benefit.

The most used TTSs show high specificity and an Area under the Receiver Operating Characteristic above 0.80 for death within 24 hours.37 Possibly, this results in over-observation and aggressive management in patients with a low-risk of death and even ‘alarm fatigue’.38 More advanced TTSs have been developed, such as eCART or adjusted scores for patients with chronic respiratory conditions.37 Temporary or chronic acceptable vital signs can be registered in CROS.39 Two studies have found fewer alerts with these systems, but this is at the cost of lower sensitivity for mortality and intensive care unit (ICU) admission.39 40

Patients at risk of deterioration might benefit from focusing on other aspects than vital signs. A physician-based Patient Acuity Rating (PAR) was found to be more accurate to predict a composite outcome of ICU Transfer, Cardiac Arrest or Rapid Response Team Activation within 24 hours, than either the TTS or PAR alone.41 A Dutch-early-nurse-worry-indicator-score has also been successful in alerting in early stages of deterioration.32

I-EWS is based on deviating vital signs as nursing staff frequently may be unaware of abnormalities when data are not systematically collected.42 I-EWS does not merely reduce complex disease to a number but invites the nursing staff to continuously reflect on the many clinical signs they observe when interacting with the patients. All these factors discussed above can affect the applicability of a TTS and hence their effectiveness in a clinical context.

NEWS was originally developed to predict a patient’s risk of death, cardiac arrest, or unplanned ICU admission within 24 hours of a scoring.13 As stated previously, TTSs are clinically useful if it leads to improved care—ideally measured by improved patient outcomes. Our objective was to compare I-EWS to NEWS in a Danish setting and to investigate the impact on selected clinical outcomes as well as the use of resources. We have chosen the primary endpoint to be 30-day mortality, which has been used previously to assess the efficacy of TTSs. Thirty-day mortality was chosen over in-hospital mortality, since the latter is affected by organisational factors with impact on length of stay. All-cause mortality at 30 days after admission is studied in a non-inferiority design since the number of preventable deaths have been reported to be low and a superiority design will be difficult to assess.43 The primary endpoint is combined with the average number of times per day a patient is NEWS/I-EWS scored were we have the power to detect small differences and assess superiority.

Admission to Intensive care unit will not be reported. Exactly which patients, who end up getting referred, are greatly affected by patient’s comorbidities, availability and the anaesthesiologist’s assessment of the need for ICU care more than the patient’s risk of severe adverse events illustrated by a scoring-system. Instead we have chosen to report number of scores generating a call for attending doctor or mobile emergency team as a measure of the need for further escalation in treatment.

One might argue that there is a risk of a carry-over effect when one group is randomised to an intervention followed by a control time-period. We find this limited by the fact that I-EWS is integrated into the EPJ and it is not possible to adjust the NEWS score after the intervention period has ended. Furthermore, a 14-day censoring period after cross-over will be excluded from the analyses. Education of NEWS/I-EWS research staff is repeated in both groups and they will be responsible for emphasising the importance of recognising deterioration as well as compliance to NEWS guidelines among their colleagues.

Conclusion and clinical implications

The I-EWS study investigates the effect of integrating a clinical assessment on mortality in a head to head comparison with the internationally used NEWS, which includes the opportunity to use CROS. Caring for the right patient at the right time could possibly allow for reallocation of resources. If our hypothesis is confirmed, considerations should be given towards allowing nursing staff to include their clinical assessment in TTS.