Can Routinely Collected, Patient-Reported Wellness Predict National Early Warning Scores? A Multilevel Modeling Approach.

OBJECTIVE: Measures exist to improve early recognition of and response to deteriorating patients in hospital. However, management of critical illness remains a problem globally; in the United Kingdom, 7% of the deaths reported to National Reporting and Learning System from acute hospitals in 2015 related to failure to recognize or respond to deterioration. The current study explored whether routinely recording patient-reported wellness is associated with objective measures of physiology to support early recognition of hospitalized deteriorating patients.
METHODS: A prospective observation study design was used. Nurses on four inpatient wards were invited to participate and record patient-reported wellness during every routine observation (where possible) using an electronic observation system. Linear multilevel modeling was used to examine the relationship between patient-reported wellness, and national early warning scores (NEWS), and whether patient-reported wellness predicted subsequent NEWS.
RESULTS: A significant positive relationship was found between patient-reported wellness and NEWS recorded at the next observation while controlling for baseline NEWS (β = 0.180, P = 0.033). A significant positive relationship between patient-reported wellness and NEWS (β = 0.229, P = 0.005) recorded during an observation 24 hours later while controlling for baseline NEWS was also found. Patient-reported wellness added to the predictive model for subsequent NEWS.
CONCLUSIONS: The preliminary findings suggest that patient-reported wellness may predict subsequent improvement or decline in their condition as indicated by objective measurements of physiology (NEWS). Routinely recording patient-reported wellness during observation shows promise for supporting the early recognition of clinical deterioration in practice, although confirmation in larger-scale studies is required.

BACKGROUND

Clinical patient deterioration that is not promptly responded to can result in admission to the intensive care unit, cardiac arrest, and increased morbidity and mortality.[1-3] Timely recognition of and response to early signs of clinical deterioration may prevent these serious adverse events.[4,5] Measures have been introduced to support staff to recognize and respond to deteriorating patients in hospitals in countries including the United Kingdom, United States, and Australia.[6] National early warning scores (NEWS) allow health professionals to monitor and detect changes in patients’ vital signs.[5] If a patient’s NEWS is outside of the reference range, this indicates clinical deterioration and prompts health professionals to trigger an escalation of care, ultimately to a critical care outreach team who provide the deteriorating patient with timely treatment on the ward.[7] Despite the use of these measures, the management of critical illness remains a problem globally.[8-10] Analysis of patient safety–related hospital deaths in England in 2015 demonstrated that 7% related to a failure to recognize or respond to deterioration.[8]

Increasingly, patients are being encouraged to become involved in their healthcare.[11,12] Systematic reviews of studies exploring the involvement of patients and relatives in detecting and escalating clinical deterioration identified that in some hospitals, patients, and relatives are invited to escalate patient care by triggering a critical care outreach team if they suspect patient deterioration and have not received a response from the ward team.[13,14] However, patient- and relative-led escalation systems implemented in the reviewed studies did not consider the extent to which patients and relatives can effectively monitor and detect changes in their condition, and there is scant empirical evidence exploring the ability of patients to recognize signs of their deteriorating condition.[13] Therefore, it is important for this evidence gap to be addressed, to better understand the potential safety benefits, or challenges, of involving patients in the detection and escalation of clinical deterioration.

The current study is the first to explore the potential clinical effectiveness of involving patients in the recognition of clinical deterioration, by inviting nurses and healthcare assistants to record patient-reported wellness during routine observations. This was explored via the following research questions: (a) Does patient-reported wellness add to the prediction of NEWS recorded during the next observation while controlling for baseline NEWS? (b) Does patient-reported wellness add to the prediction of NEWS recorded at an observation approximately 24 hours while controlling for baseline NEWS?

METHODS

Setting and Participants

The study was conducted on four oncology wards within a community hospital in Northern England in September 2017. Cancer accounts for 28% of deaths in the United Kingdom[15] and cancer patients can experience poor clinical outcomes, including clinical deterioration in their condition.[16] Pragmatic reasons to invite these wards to participate included their use of an electronic observation system, which allowed for the efficient collection of patient wellness ratings during observation with minimal disruption to the usual routine of healthcare staff.

A total of 73 nurses and healthcare assistants who conducted routine patient observations on four sampled wards were invited to participate in the study. An opportunity sampling strategy was used to recruit healthcare assistants and nursing staff. The researcher visited the participating wards every week day in the study period to introduce nurses and healthcare assistants to the study during the mid-shift huddle. Staff were given at least 24 hours to consider participation and were recruited to the study by the researcher during the next mid-shift huddle they attended. An opt-out approach to patient recruitment was used. A leaflet was given to all patients on admission to the ward. The leaflet informed patients that the study was taking place and that they could opt out of it by informing a member of nursing staff. Participating in the study was low burden to patients as what they experienced was consistent with usual care. Nurses and healthcare assistants often ask patients how they are feeling in practice. An opt-out approach also allowed for the inclusion of more unwell patients who may have had the capacity to talk to healthcare staff about how they were feeling during routine observation in the study but may have been too unwell to follow the informed consent process.

Measures

Patient Wellness Questionnaire

The Patient Wellness Questionnaire (PWQ) was developed in conjunction with patients and healthcare assistants during focus groups and piloted with patients on inpatient wards in a previous study.[17] The measure includes two questions each with the following five response options: Question 1: How are you feeling? Very poor (5), poor (4), fair (3), good (2), very good (1), and Question 2: How are you feeling compared with the last time you were asked? Much worse (5), worse (4), no change (3), better (2), and much better (1).

Patient wellness rating 1 refers to the response to question 1 and patient wellness rating 2 refers to the response to question 2. A combined patient wellness rating was calculated to explore an interactive effect to identify when patients report that they are feeling unwell and their wellness is declining and to differentiate these patients from those who report they are feeling unwell but are stable.

The combined rating was created by multiplying patient wellness rating 1 by patient wellness rating 2. The higher the combined patient wellness rating, the worse the patient has rated their wellness. The highest possible combined rating was 25 for a patient who responded with “very poor” and “much worse” in response to the PWQ.

National Early Warning Score

National early warning scores are based on routine physiological measurement of patients’ vital signs including their respiratory rate, oxygen saturation, temperature, blood pressure, and heart rate from which a score is calculated and recorded. A NEWS of five to six triggers a medium-level clinical alert (an urgent clinical review) and a NEWS score of 7 or more triggers a high-level clinical alert (an emergency clinical review). An extreme score of 3 in any one physiological parameter also triggers a medium-level alert.[18] The normal ranges for vital sign measurements used in the NEWS observation chart, issued by Royal College of Physicians[19] are as follows: a patients’ respiratory rate should fall between 12 and 20 breaths per minute, oxygen saturation should be between 96% and 100%, temperature should range between 36°C and 38°C, systolic blood pressure should fall between 110 and 220 mm Hg and heart rate should be between 50 and 90 beats per minute.[19]

Procedure

Ethical approval for this study was granted by National Health Service Health Research Authority Yorkshire and Humber Research Ethics Committee (Reference: 17/YH/0210). The study used a prospective observational design. For the 4-week study period, as part of routine care, nurses and healthcare assistants were invited to ask patients the 2-item PWQ during each routine observation. They then recorded patients’ responses by entering their two numerical patient wellness ratings into the electronic observation application along with the patients’ vital sign measurements. To prompt nurses and healthcare assistants to ask patients the PWQ during observation, a laminated printout of the questionnaire was displayed on the observation trolley. The ward sisters assessed the capacity of patients to ensure that they were well enough to answer the PWQ during the study. Where patients did not want to answer or were not well enough, the following options were available for nurses and healthcare assistants to input into the electronic observation application: (a) the patient refused to answer or (b) unable to ask PWQ because the patients’ medical condition makes it difficult for them to answer.

Data Analysis

Our analysis focused on testing whether patient-reported wellness (measured using the PWQ) predicted subsequent NEWS. This analysis was conducted for both the next observation (ie, taken between 2 and 24 hours after the patient wellness rating) and the corresponding observation taken the next day (i.e., taken 24 hours after the patient wellness rating). We also explored whether patient-reported wellness predicted NEWS at a subsequent observation when controlling for baseline NEWS. This allowed us to understand whether patient-reported wellness added predictive value to the ability of baseline NEWS to predict subsequent NEWS. Raw measurements of patients’ vital signs taken during observations were scored using the NEWS scoring system. The predictors (patient wellness ratings) were mean centered.

The analyses used hierarchical multivariate linear modeling implemented in the package HLM7. Hierarchical multivariate linear modeling allows different models of temporal variance-covariance structure to be assessed and is a suitable modeling approach for a varying occasion design.[20] Patient wellness ratings and NEWS were level 1 variables in this analysis, whereas patient age, sex, and acuity of their illness were level 2 variables. The model used patient wellness ratings and baseline NEWS as predictors. For the model, we report the deviance statistic to indicate model of fit plus unstandardized coefficients, standard errors, standardized coefficients, and significance (all based on the population-average model with robust standard errors). There were 18% missing data overall. Listwise pair deletion was used to address missing data. Simple slopes analysis was conducted where interactions were found.

RESULTS

Participants

Sixty-nine nurses and healthcare assistants provided informed consent and participated in the study (95% of nurses and healthcare assistants approached). Of the four nurses declining to participate, three did not have enough time to participate because of a heavy workload and one nurse was on annual leave during the study period. They were invited to collect patient wellness ratings from all patients (who had not opted out) for a 1-month trial period alongside NEWS during routine observation. The four participating wards used an electronic observation system that allowed patients’ vital sign measurements to be input on a smartphone device. The patient’s National Health Service number was used to identify their vital sign measurements and notes electronically.

At least one patient wellness rating was recorded during observation for 103 of the 125 patients cared for on the four participating wards during the study period. The mean age of patients was 66 years with an age range of 39 to 90 years, and 62% of the patients were male.

Descriptive Statistics

Descriptive statistics for all variables are reported in Table 1. The lower and/or upper ranges for all vital sign measurements (used to calculate NEWS) fell outside of the reference range for vital signs, indicating that some patients in the sample had abnormal vital sign measures. A total of 28 (28%) of 103 patients had a NEWS of 3 or greater during the study period. A NEWS of three triggers a low-level clinical alert where a registered nurse assesses the patient and decides whether increased frequency of monitoring and escalation of care is required.[18]

TABLE 1

Descriptive Statistics for All Variables

	M	SD	Min	Max
PWR 1	2.65	0.81	1	5
PWR 2	3.13	0.74	1	5
PWR 1*2	8.07	2.63	1	16
Same observation (baseline)
NEWS	1.45	1.61	0	11
RR	16.76	2.26	10	32
O2 Sats	97.21	1.51	90	100
Temp	36.56	0.82	32.60	46.10
BP Systolic	121.21	17.35	65	195
HR	84.28	14.18	46	129

Abbreviations: BP systolic, blood pressure systolic; HR, heart rate; O2 Sats, oxygen saturation; PWR 1, patient wellness rating 1; PWR 2, patient wellness rating 2; PWR 1*2, combined patient wellness rating; RR, respiratory rate; Temp, temperature.

Ability of Patient-Reported Wellness to Predict Subsequent NEWS

Table 2 reports the analyses of the relationship between patient wellness rating 1, 2, the combined patient wellness rating, and NEWS taken at the next observation or during an observation approximately 24 hours later. Analysis was based on a sample of 103 patients, 575 pairs of observations for next observation analysis and 500 pairs of observations for 24-hour later analysis. The model reports the relationship while controlling for baseline NEWS. Controlling for baseline NEWS allows for a prediction of change between NEWS recorded at baseline and NEWS recorded at a subsequent time point (at the next observation or an observation 24 hours later).

TABLE 2

Within-Person Associations Between Patient Wellness Rating 1, 2, Combined Patient Wellness Rating, and NEWS Recorded During the Next Observation and Approximately 24 Hours Later

	Next Observation				24-h Observation
Predictor	Coefficient	SE	Standardized B	P	Coefficient	SE	Standardized B	P
Predicting NEWS
PWR 1	0.180	0.083	0.360	0.033	0.028	0.067	0.029	0.683
PWR 2	0.032	0.057	0.070	0.571	0.091	0.107	0.150	0.396
PWR 1*2	−0.014	0.050	−0.026	0.778	0.229	0.105	0.370	0.005
Baseline NEWS	0.305	0.064	0.303	<0.001	0.087	0.079	0.106	0.322

n = 103.

Abbreviations: Coefficient, unstandardized coefficient; PWR, patient wellness rating; SE, standard error; Standardized B, standardized coefficient.

Next Observation

No significant association was observed between patient wellness rating 2 or the combined patient wellness rating and NEWS. However, we observed a significant positive relationship between patient wellness rating 1 and NEWS while controlling for baseline NEWS (β = 0.180, P = 0.033). The time interval between the baseline observation and the next observation varied between patients (M = 6 hours, 13 minutes; SD = 3 hours, 46 minutes; Min = 17 minutes; Max = 22 hours, 55 minutes) and was controlled for in the analysis.

Observation 24 Hours Later

There were no significant associations between patient wellness rating 1 or patient wellness rating 2 and NEWS. However, we did find a significant positive relationship between the combined patient wellness rating and NEWS while controlling for baseline NEWS (β = 0.229, P = 0.005). Based on simple slopes analysis, the relationship between patient wellness rating 1 (feeling very poor – very good) and NEWS recorded 24 hours later varied as a function of patient wellness rating 2 (feeling much better – much worse). Where patients felt “much better” and “better,” the relationship between patients’ rating of their current wellness and NEWS was nonsignificant (M − 1SD: β = −0.142, SE = 0.078, P = 0.073). Where patients felt “fair,” the relationship between patients’ rating of their current wellness and NEWS was nonsignificant (M: β = 0.028, SE = 0.067, P = 0.683). Where patients felt “much worse” or “worse,” the relationship between patients’ ratings of their current wellness and NEWS was significant (M + 1SD: β = 0.197, SE = 0.099, P = 0.049).

Associations between patient-reported wellness, and NEWS and vital signs, recorded within the same observation were explored, but no significant associations were found.

Associations between patient-reported wellness and individual vital signs recorded during the next observation and an observation approximately 24 hours later were also conducted. Additional file A shows the within observation and individual vital signs analysis.

Missing Data

Of the total number of observations conducted for all patients during the study period, a patient wellness rating was recorded during 14% of observations with a range of 3% to 55% across individual patients.

To address the distribution of missing data, we examined whether there were any differences in patient wellness rating 1 or patient wellness rating 2 baseline scores depending on the presence or absence of patient wellness rating 1 or patient wellness rating 2 scores at the next time point or the 24-hour time point. This indicated little or no difference in baseline scores based on the availability of subsequent scores (either at the next observation or 24-hour observation). For patient wellness rating 1, the baseline score was M = 2.65 (SD =0.819) when the next observation score was not available and M = 2.63 (SD = 0.783) when the next observation score was available and analysis of variance (ANOVA) indicated that this difference did not approach statistical significance (F(1,595) = 0.088, P = 0.767). For patient wellness rating 2, the baseline score was M = 3.12 (SD = 0.738) when the next observation score was not available and M = 3.14 (SD = 0.759) when the next observation score was available and ANOVA indicated that this difference did not approach statistical significance (F(1,557) = 0.091, P = 0.764).

Similarly, for patient wellness rating 1, the baseline score was M = 2.65 (SD = 0.834) when the 24-hour observation score was not available and M = 2.63 (SD = 0.729) when 24-hour observation score was available and ANOVA indicated that this difference did not approach statistical significance (F(1,595) = 0.046, P = 0.830). For patient wellness rating 2, the baseline score was M = 3.13 (SD = 0.738) when the 24-hour observation score was not available and M = 3.10 (SD = 0.762) when the 24-hour observation score was available and ANOVA indicated that this difference did not approach statistical significance (F(1,557) = 0.195, P = 0.659).

However, when examining NEWS based on whether patient wellness rating 1 and 2 scores were available at the next or 24-hour observation some differences did emerge. In particular, the NEWS baseline score was M = 1.33 (SD = 1.599) when the next observation score was not available and M = 1.81 (SD = 1.600) when the next observation score was available and ANOVA indicated that this difference did reach statistical significance (F(1,594) = 10.298, P = 0.001). In contrast, the NEWS baseline score was M = 1.44 (SD = 1.688) when the 24-hour observation score was not available and M = 1.48 (SD = 1.340) when the 24-hour observation score was available and ANOVA indicated that this difference did not approach statistical significance (F(1,594) = 0.083, P = 0.773). This would indicate that patient wellness ratings were less likely to be collected at the next observation when baseline NEWS indicated that the patient was less well, but no effect on whether patient wellness ratings were collected at 24-hour observation.

DISCUSSION

In the current study, healthcare assistants and nurses were invited to record patient-reported changes in wellness using the PWQ during routine observation. The study sought to explore whether patient-reported changes in wellness were associated with objective, clinical measures of patient physiology, such as NEWS. Our preliminary findings provide insight into the clinical effectiveness of routinely recording patient-reported wellness in practice and the ability of patients to recognize genuine deterioration in their condition.

Previous studies have found that patients and relatives are aware of patient deterioration and use intuition related to changes in patient behavior and capabilities to identify deterioration.[17,21] Anecdotally, this awareness can occur before changes in quantifiable objective measures, such as NEWS measurements.[21,22] In the current study, significant associations were found between patient wellness rating 1 and NEWS during the next observation, and the combined patient wellness rating and NEWS during an observation 24 hours later. Consistent with prior anecdotal evidence, this empirical finding indicates that patients’ ratings of their wellness may be a precursor for subsequent improvement or decline in their condition that is borne out in NEWS measurements. The findings support evidence in the literature that patients who voice concerns about changes in how they feel are in the early stages of clinical deterioration.[23-25] Responding to patient-reported changes in wellness may prevent further deterioration if it reduces delays in appropriate escalation of care, before clinical indicators of deterioration being demonstrated in the observation chart.

In terms of the predictive ability of the individual and combined patient wellness ratings, our study findings suggest that it may be useful to record both patient wellness rating 1 to capture the patients’ current wellness and patient wellness rating 2 to capture a change in the patients’ wellness so that a combined patient wellness rating can be calculated. Patients’ ratings of how well they felt (eg, very poor, poor, fair, good, or very good) were only significantly associated with NEWS when they also felt “much worse” or “worse.” As such, patients with a high combined patient wellness rating may be most at risk of clinically deteriorating.

Further evidence is required on a larger scale to develop robust evidence about these relationships. In addition, future research would need to explore whether patient-reported wellness can help predict other important clinical outcomes, such as length of stay, unexpected transfer to intensive care, cardiac arrest, and mortality. This would establish whether asking patients routinely about their wellness during observation supports the early detection of clinical deterioration. Ultimately, it may be that routinely recorded patient-reported wellness could be considered as a data stream to be used alongside others, such as NEWS, laboratory tests, and clinical judgment to aid identification of deterioration. It has been suggested that the use of algorithms that combine several variables rather than the use of single parameter with simple cut offs may be more effective to enhance the detection of clinical deterioration.[26] If future robust, large-scale evidence suggests that patient-reported wellness, captured using the PWQ, is effective at supporting the detection of deterioration, it would be important to determine thresholds and triggers for patient-reported wellness and also to understand how patient-reported wellness could be integrated within NEWS scoring systems already used in practice.

Limitations

One limitation of the study was that patient wellness ratings were not recorded at a large number of observations during the study period. A patient wellness rating was recorded during only 14% of observations with a range of 3% to 55%. Analysis to explore the distribution of missing data suggested that there may be some bias in relation to patient wellness ratings being more likely to be missing at the next observation but not the 24-hour observation when baseline NEWS scores were higher. Nevertheless this should not have unduly influenced our analyses particularly in relation to the 24-hour observation data. Nurses and healthcare assistants rarely gave a reason for not recording patient-reported wellness, for instance, whether the patient refused to answer or were too unwell to answer the questions. It will be important to gain a greater understanding of why healthcare assistants and nurses did not record patient wellness ratings during most observations. It may be that recording this information during every observation is too frequent. Further research is needed to explore healthcare assistant and nurse experiences of using the PWQ in practice.

One key challenge of the study related to the method used to prompt healthcare assistant and nurses to ask patients the PWQ. Laminated paper prompts with the patient wellness questions and response options were attached to the observation trolley. Unfortunately, it was not possible to program the PWQ into the electronic observation application in the current study so that it appeared alongside where vital sign measurements were input. It may be that more patient wellness ratings would have been recorded during observation if the questions were programed into the electronic observation application because staff may be less likely to miss the prompt.

It should be acknowledged that this was a small-scale study producing preliminary findings and thus firm conclusions cannot be drawn about the relationships between patient-reported wellness and NEWS. Knowing the diagnosis and treatment received by patients in the study would have allowed greater interrogation and stronger interpretation of the findings as treatment may partially explain some of the findings, for instance, long-term chemotherapy can result in increased temperature.

CONCLUSIONS

The findings suggest that patients’ ratings of their wellness may be a precursor for subsequent improvement or decline in their condition, as indicated by objective measurements of their physiology. As such, future large-scale research should examine if routinely recording patient-reported wellness during observation is a clinically effective strategy to aid health professionals in the early recognition of clinical deterioration in practice.