Clinical implications and validity of nursing assessments: a longitudinal measure of patient condition from analysis of the Electronic Medical Record.

OBJECTIVES: This study investigates risk of mortality associated with nurses' assessments of patients by physiological system. We hypothesise that nursing assessments of in-patients performed at entry correlate with in-hospital mortality, and those performed just before discharge correlate with postdischarge mortality.
DESIGN: Cohort study of in-hospital and postdischarge mortality of patients over two 1-year periods.
SETTING: An 805-bed community hospital in Sarasota, Florida, USA.
SUBJECTS: 42 302 inpatients admitted for any reason, excluding obstetrics, paediatric and psychiatric patients. OUTCOME MEASURES: All-cause mortalities and mortality OR.
RESULTS: Patients whose entry nursing assessments, other than pain, did not meet minimum standards had significantly higher in-hospital mortality than patients meeting minimums; and final nursing assessments before discharge had large OR for postdischarge mortality. In-hospital mortality OR were found to be: food, 7.0; neurological, 9.4; musculoskeletal, 6.9; safety, 5.6; psychosocial, 6.7; respiratory, 8.1; skin, 5.2; genitourinary, 3.0; gastrointestinal, 2.3; peripheral-vascular, 3.9; cardiac, 2.8; and pain, 1.1. CI at 95% are within ±20% of these values, with p<0.001 (except for pain). Similar results applied to postdischarge mortality. All results were comparable across the two 1-year periods, with 0.85 intraclass correlation coefficient.
CONCLUSIONS: Nursing assessments are strongly correlated with in-hospital and postdischarge mortality. No multivariate analysis has yet been performed, and will be the subject of a future study, thus there may be confounding factors. Nonetheless, we conclude that these assessments are clinically meaningful and valid. Nursing assessment data, which are currently unused, may allow physicians to improve patient care. The mortality OR and the dynamic nature of nursing assessments suggest that nursing assessments are sensitive indicators of a patient's condition. While these conclusions must remain qualified, pending future multivariate analyses, nursing assessment data ought to be incorporated in risk-related health research, and changes in record-keeping software are needed to make this information more accessible.

This study investigates risks of mortality associated with entry and last predischarge nurses’ assessments of patients’ conditions by physiological system.

It is the first quantitative study of the validity and clinical implications of nurses’ head-to-toe clinical assessments.

Entry nursing assessments (other than pain) are strongly correlated with in-hospital mortality, and final predischarge nursing assessments (other than pain) are strongly correlated with postdischarge mortality, independent of diagnosis and medical history. It is evident that most nursing assessments are clinically meaningful and valid.

The dynamic nature of in-hospital nursing assessments and the large mortality OR associated therewith suggest not only are nursing assessments sensitive indicators of a patient's condition, but they may also aid in detection of clinical problems as they develop during the course of a patient's stay.

Nursing assessment data, which are now essentially unused, provide additional information on patients’ conditions and should allow physicians to improve patient care and reduce in-hospital mortality.

This is the first quantitative study of the clinical validity of nursing assessments. More than 42 000 patient visits over two 1 year periods give the study a strong statistical base.

All the OR reported are both statistically and clinically significant, with none of the 95% CI's overlapping one (except for pain).

It is internally consistent, with in-hospital and postdischarge OR's for all time periods yielding similar results, providing evidence of reliability of nursing assessments.

There are some limitations to our study. No multivariate analysis was performed, making the associations found subject to possible unknown confounders. The work has been done at a single site and that site has a population skewed older, which raises questions about generalisability.

Background

Nursing budgets constitute a major part of a hospital's operating costs, accounting for some 25% of the total operating budget and 44% of direct care costs.1 2 Recent studies have demonstrated that higher staffing ratios of registered nurses are associated with fewer hospital-related deaths, failures to rescue, cardiac arrests, hospital-acquired pneumonia and other adverse events, as well as having positive effects on patients’ safety in intensive care units and in surgical patients.3–6 Increased registered nurse hours spent on direct patient care were associated with decreased risk of hospital-related death and shorter lengths of stay.3–6 In the course of providing direct patient care, nurses assess each patient by physiological system and record those assessments in the electronic medical record (EMR). Once recorded however, nursing notes are not often read by attendants or residents.7

There have been several studies of the relationship between registered nurse staffing and in-hospital mortality.8 9 However, no previous studies have demonstrated a direct connection between nursing assessments and patient risk of mortality. Furthermore, while the nursing literature is replete with studies of nursing diagnostic terminology and its standardisation,10–19 there is a paucity of quantitative studies of the validity and clinical implications of nurses’ head-to-toe clinical assessments. In this article we investigate clinical associations of 12 simplified nursing assessments, one for each physiological systems, with both in-hospital and postdischarge mortality data. Our hypotheses are that nursing assessments performed at entry of in-patients are predictors of in-hospital mortality, and nursing assessments performed just before in-patient discharges are predictors of postdischarge mortality.

Methods

This research was initiated in an attempt to understand issues of continuity of care in hospitals. What piqued our interest in nursing assessments is the fact that other than laboratory tests and vital signs, they are the only clinical variables in the EMR that are reflective of patient condition that are not static, and their values are updated regularly. The results of this research are incorporated in the ‘Rothman Index’, which is a new measure of patient condition. This is the first foundational paper in a series of studies related to the scientific basis of the Rothman Index. Approval for the work was granted by the Sarasota Memorial Hospital Institutional Review Board.

Nursing assessment data for the periods 1/2004–12/2004 and 7/2005–6/2006 were extracted from the Electronic Hospital Record at Sarasota Memorial Hospital, an 805-bed community hospital. Our cohort for this study were all patients admitted for any reason during these periods, excluding obstetrics, paediatric and psychiatric patients, which determined the study size of 42 302 inpatient visits. For the discharge study, we had complete data for 39 964 inpatient visits in which the patient was discharged alive. Demographic data and diagnostic data have not been collected for this population; however, our subject community hospital serves a population skewed older than the US average.

In general, nurses’ assessments are entered into the Electronic Medical Record in one of two ways: either the nurse answers a series of detailed questions to document each assessment, such as for respiratory, ‘What are the breath sounds?’, ‘What colour are the nail beds’, etc; or, the ‘charting by exception’ method is used, where the nurse records a simple overall answer of ‘met’ or ‘not met’ for each physiological system, such as ‘Does the patient's respiratory function meet a minimum standard?’ (if the answer is ‘not met’, only then are detailed follow-on questions considered). In this study the binary charting by exception method was used, and the assessments were characterised as either having ‘met’ the standard or having ‘not met’ the standard in each of the following 12 areas: food, neurological, safety, skin, genitourinary, musculoskeletal, respiratory, cardiac, peripheral vascular, gastrointestinal, psychosocial and pain. Definitions of the relevant standards are shown in the appendix. Nursing assessments were generally performed at least once per shift. For each area of nursing assessment, the all-cause in-hospital death rates and mortality OR's, associated with patients’ entry nursing assessments, were computed and the all-cause death rates and mortality OR's, associated with patients’ last assessments prior to discharge, were computed for patients living at the time of discharge with deaths within the time periods 2 days, 30 days and 1 year from discharge. Mortality was established by comparison with the Social Security Administration Death File. The reproducibility of outcome measures between the 2004 data and the 2005–2006 data was assessed using an intraclass correlation coefficient. The data analysis was carried out utilising SAS V.9.2 (SAS Institute, Cary, North Carolina, USA) and Systat V.13 (Systat Corp., Chicago, Illinois, USA).

Results

The population studied had a mean of 4.7 days and median 3.1 days length of stay, with an SD of 5.1 days. An example of the data and associated results for 30-days postdischarge mortality is given in table 1, illustrating our calculations. Results for in-hospital mortality OR's associated with failing an entry nursing assessment are given in table 2 for 42 302 patients of whom 1086 died in the hospital. OR's for deaths within postdischarge periods of 2 days, 30 days and 1 year for 39 964 patients for whom we had data and who were living at the time of discharge are given in table 3. There were less than 0.3% missing data for any result. Although generally, about 90% of patients passed the assessments, in all categories and for both in-hospital and postdischarge deaths, not meeting standards for an assessment resulted in significantly higher death rates than meeting standards and very large mortality OR's, with the single exception of the pain assessment. Except for the pain assessments, all results are statistically significant (p<0.001) and none of the 95% CI's overlap one. The implications of pain assessments are examined in the discussion section below.

Table 1

Nursing assessment data with resulting 30-day mortality OR

Nursing assessment	Met live	Met died	Met mortality odds	Not met live	Not met died	Not met mortality odds	OR	p Value
Food	34769	705	0.020	3383	1084	0.320	16	<0.001
Neurological	34600	770	0.022	3561	1018	0.286	13	<0.001
Psychosocial	36327	1260	0.035	1834	525	0.286	8.3	<0.001
Safety	32449	781	0.024	5157	993	0.193	8.0	<0.001
Genitourinary	34214	1110	0.032	3926	679	0.173	5.3	<0.001
Skin	29627	585	0.020	8506	1199	0.141	7.1	<0.001
Musculoskeletal	24528	243	0.010	13630	1546	0.113	11	<0.001
Respiratory	26941	449	0.017	11223	1340	0.119	7.2	<0.001
Gastrointestinal	32365	1098	0.034	5797	690	0.119	3.5	<0.001
Peripheral vascular	28914	828	0.029	9240	961	0.104	3.6	<0.001
Cardiac	31947	1119	0.035	6228	670	0.108	3.1	<0.001
Pain	33618	1568	0.047	4436	218	0.049	1.1	0.474

Numbers of patients dead and living at 30 days from date of discharge who were denoted ‘met’ or ‘not met’ at last in-hospital assessment, and their associated all-cause mortality odds and OR's. The 95% CI's for the OR's are all less than ±15% of the values given and none overlaps one; p values for OR's are estimated by the Fisher Exact test (two-tailed), and listed in the last column. Note the only p value larger than 0.001 is for the pain assessment.

Table 2

In-hospital death OR associated with entry nursing assessments

Nursing assessment	OR	95% CI		p
Neurological	9.4	8.3	10.6	<0.001
Respiratory	8.1	7.0	9.3	<0.001
Food	7.0	6.1	7.9	<0.001
Musculoskeletal	6.9	5.9	8.1	<0.001
Psychosocial	6.7	5.9	7.7	<0.001
Safety	5.6	4.9	6.3	<0.001
Skin	5.2	4.6	5.9	<0.001
Peripheral vascular	3.9	3.5	4.4	<0.001
Genitourinary	3.0	2.6	3.4	<0.001
Cardiac	2.8	2.5	3.2	<0.001
Gastrointestinal	2.3	2.0	2.5	<0.001
Pain	1.1	0.9	1.2	0.530

Table 3

Postdischarge mortality OR for final nursing assessments

Nursing assessment	Deaths within postdischarge period
	2-day OR	30-day OR	1-year OR
Food	37	16	6.7
Musculoskeletal	28	11	4.6
Neurological	27	13	6.5
Psychosocial	15	8.3	5.3
Respiratory	13	7.2	4.2
Safety	13	8.0	5.0
Skin	10	7.1	4.3
Genitourinary	8.4	5.3	3.7
Peripheral vascular	5.9	3.6	2.7
Cardiac	5.4	3.1	2.3
Gastrointestinal	4.6	3.5	2.2
Pain	2.2	1.1	0.8

All-cause mortality OR for postdischarge deaths for periods of 2 days, 30 days and 1 year from the date of hospital discharge associated with last in-hospital assessments prior to discharge. The 95% CI for the OR's are less than ±10% (±15%, ±35%) of the values for the 1 year (30 day and 2 day) postdischarge values shown, and none overlaps the value 1 except for pain; p<0.001 for all OR's except for pain.

To evaluate the agreement between 2004 and 2005–2006 values, first we calculated all the OR's for the 2004 and 2005–2006 subsets of data, and then an intraclass correlation coefficient was calculated comparing all OR's for 2004 to their counterparts from 2005 to 2006 across all categories and all time points (ie, in-hospital and 2, 30, and 365 days postdischarge) and found to be 0.85; values greater than 0.75 indicate excellent reproducibility.20 Thus, there is excellent reproducibility between the results for 2004 and those for 2005–2006 across all our measures of interest.

Discussion

There is a growing body of evidence that suggests that nurses have a separate and identifiable effect on patient hospital outcomes, irrespective of other medical care (cfKane et al21 22 and references cited therein). An important subject area in the literature is whether nursing data collected during the hospital stay can be used to explain commonly studied variables such as costs/charges, lengths of stay and mortality. However, until now, nursing assessments have not been shown to have clinical predictive validity. Our study combines over 40 000 cases, encompassing all admissions for any reason except for paediatrics, psychiatric and rehab, and we have shown that patients failing to meet minimum standards for any nursing assessment had significantly higher in-hospital and postdischarge death rates than patients who did meet these standards, regardless of medical history and diagnosis. This is evidence of the clinical validity of nursing assessments. In fact, the medical literature is replete with studies of critically ill patients for whom variables with mortality OR in the range 2–4 are considered sensitive and important predictors of mortality.23–26 The OR's we report demonstrate that these measures of a patient's functionality are significant and sensitive indicators of a patient's condition. Thus, nursing assessments may aid in physician care and possibly reduce hospital patient mortality, and future risk-related health research ought to consider incorporation of nursing assessments. Further study is clearly warranted.

We have studied deaths without regard to diagnosis, age, gender or severity of illness that occur in-hospital as well as within short (2 and 30 days) and long (1-year) periods of time for two reasons: (1) in order to learn whether there might be similar or different short and long-term relationships of assessments to mortality and (2) as a way to test validity and reliability. We have found that failing a nurse's assessment has both short-term and long-term correlations with mortality. The fact that our results are consistent over short and long periods, over all assessments, and from a medically diverse patient population, demonstrates that nursing assessments have valid and significant clinical implications irrespective of medical histories and diagnoses. While we have not compared survivor demographics with those who died, it is unlikely that such large OR's would be explained by underlying demographic factors like age or race or gender.

One might ask whether the high OR's associated with these assessments may simply represent high-risk diagnoses. We have not built a multiple linear regression model to determine the added predictive power of nursing assessments to other indicators of mortality. However, this is an area of current investigation. Preliminary results, which we shall publish in subsequent studies, show that among patients who were admitted with a diagnosis of congestive heart failure, those who failed any of the food, neurological, musculoskeletal or skin nursing assessments at admission had a mortality OR of 6–9, versus those who passed. Further, failing multiple nursing assessments had a greater implication in terms of in-hospital mortality than failing just one. Generally, patients who fail nursing assessments may well be those who also have serious or even terminal diagnoses. Nonetheless, here we have demonstrated that nursing assessments at both entry and discharge capture the seriousness of the patient's condition irrespective of diagnosis. Given the size of the study, it is clear that these variables are meaningful indicators of both in-hospital and postdischarge mortality.

The highest in-hospital mortality OR's, 7 or greater, are associated with those patients not passing the food, neurological, psychosocial or musculoskeletal assessments. It is important to comprehend the simple clarity of these findings. For example, if a newly admitted patient is not able to move independently in his/her bed, or behaves inappropriately, or cannot chew and swallow food—that patient is almost seven times more likely to die in the hospital than patients who have none of these problems. If a patient is incoherent or is not oriented, that the patient is almost 10 times more likely to die in the hospital than those who are coherent and alert, no matter what the diagnosis may be. In contradistinction, pain is not a significant indicator of mortality. This may be because pain, unlike the other problems indicated by nursing assessments, can often be controlled independent of the patient's general condition, so one does not expect it to have similar clinical implications. Further studies may clarify this issue.

Since we have shown that initial and final nursing assessments contain clinical information, it is reasonable to infer that all nursing assessments (other than pain) gathered throughout the patient's stay in the hospital contain significant clinical information. Nursing assessments are generally recorded every 12 h, and sometimes more frequently, throughout the patient's stay. Therefore, these data represent a changing indicator of patient condition. Thus, the importance of this work is in alerting physicians to a source of longitudinal clinical information about the patient's condition that no static measurement, such as demographics or principal diagnosis, can provide and which is not currently being routinely utilised.

Although our results show excellent reproducibility across two 1-year time periods, we are aware of questions about the reliability of nursing data, specifically intra-rater and inter-rater reliability. However, reproducible and consistent results such as shown here are only possible with reproducible and consistent nursing assessments. One has to conclude that by and large, the nurses get it right, and that they provide an important and valuable tool for assessing patient condition, irrespective of medical diagnosis and history. Thus, it is suggested that hospital physicians make special effort to ascertain whether patients have passed or failed their nursing assessments, a practice not widely followed currently by attendants or residents.7

We have established that simplified nursing assessments, gathered throughout a patient's stay and which are now essentially unused by physicians, have clinical validity. If physicians were to utilise these data, they then would be adding an important and largely ‘new’ source of clinical information to their evaluations. For example, when a physician sees a patient in the hospital, he or she often consults with the bedside nurse. However, this consultation is not always possible or practical. In many cases, consulting with the nurse from the previous shift, or the previous day, or even the day before that, to understand changes in the patient's condition, is not feasible. And even if it were, getting a verbal report on a patient's condition about the previous several days would likely be incomplete. These circumstances make use of nursing assessments, recorded in the EMR, the only effective way to gain access to this clinically relevant information. And though immediate observations are the most important in determining care, the prior observations provide an important and meaningful context, allowing the physician to assess the changes in patient condition. It must be noted however, that current EMR technology does not facilitate quick and easy physician access to nurses’ observations and assessments. Either changes in current record-keeping software or adjuncts to it would make access to this information more accessible.

Conclusions

Entry nursing assessments of in-hospital patients are strongly associated with in-hospital mortality, and predischarge nursing assessments of in-hospital patients are associated with postdischarge mortality. No multivariate analysis has yet been performed, and will be the subject of a future study, and thus there may be confounding factors. However, it is difficult to hypothesise any alternative factors that might confound our results, given the high OR's and the variety, multiple time intervals, and extent of data considered. We infer that these nursing assessments have valid and significant clinical meaning irrespective of medical histories and diagnoses. It is reasonable to infer further that nursing assessments taken throughout a patient's stay are also clinically meaningful. These assessments, which are part of what is termed as the ‘head-to-toe’ patient assessment, and which are a standard part of nursing school curricula, are collected and recorded at all hospitals, and simplified summaries of assessments, as we have analysed, can be constructed. Since nursing assessments are recorded at least every 12 h throughout the patient's stay, they represent a changing indicator of patient condition. Thus, they make available real-time longitudinal sensitivity that no static measurement, such as demographics or principal diagnosis, can provide.

The large OR's suggest that nursing assessments are sensitive indicators of clinical problems during the course of a patient's hospital stay. This compact clinical data source in the EMR is a natural longitudinal source of information, providing physicians access to the insights of nurses as recorded throughout the patient's entire stay. Such dynamic information should allow physicians to improve patient care. While these conclusions must remain tentative, pending detailed multivariate analyses, we believe nursing assessment data ought to be incorporated along with standard diagnoses in future risk-related health research. Current EMR technology does not allow quick and easy access to nurses’ observations and assessments, so changes in current record-keeping software or adjuncts to it will be necessary to make this information more accessible.