Prediction of Mortality in the Medical Intensive Care Unit with Serial Full Outline of Unresponsiveness Score in Elderly Patients.

BACKGROUND: Advanced age is one of the key risk factors for mortality and morbidity in intensive care units. The full outline of unresponsiveness (FOUR) score has been developed and introduced to address the limitations of the Glasgow Coma Scale (GCS). The current study aimed to evaluate the ability of the FOUR score in predicting the outcomes (survivors, nonsurvivors).
MATERIALS AND METHODS: This observational study of 168 consecutive elderly patients admitted to medical intensive care during the 14 months carried out prospectively. FOUR score in the 24, 48, and 72 hours of admission, and demographic characteristics of all elderly patients were calculated, then recorded. The receiver operating characteristic (ROC) curve, logistic regression, and Hosmer-Lemeshow test were used (95% confidence interval) for statistical analysis.
RESULTS: FOUR scores in 24, 48, and 72 hours between survivors and nonsurvivors (p <0.0001, p <0.0001, and p <0.0001, respectively) were statistically different. The discrimination power of FOUR score 24 hours of admission was excellent [area under ROC (AUC): 85.7% [standard error (SE)]: 2.8%]; it was acceptable for 48 and 72 hours of admission [AUC: 76.3% (SE: 3.6%), AUC: 75/0% (SE: 3.8%), respectively]. The FOUR score of 24 and 48 hours (x2 = 10.06, p = 0.261, x2 = 6.82, p = 0.448, respectively) showed acceptable calibration.
CONCLUSIONS: The FOUR score is a suitable scoring system for prognostication of outcomes in critically ill elderly patients. The FOUR score 24 hours of admission was superior in terms of discrimination power than 48 and 72 hours, but better calibration power belonged to FOUR score 48 hours. HOW TO CITE THIS ARTICLE: Ramazani J, Hosseini M. Prediction of Mortality in the Medical Intensive Care Unit with Serial Full Outline of Unresponsiveness Score in Elderly Patients. Indian J Crit Care Med 2022;26(1):94-99.

Introduction

A large number of predictive models have been developed and are widely used in intensive care practice over the past 35 years. They have valuable advantages such as continuous improvement of care and management in the intensive care units.[1] In addition to their strengths, scoring systems have weaknesses, and we know there is no perfect model that can be used in all different settings.[2,3] Customizing the model coefficients to the specific population could be beneficial for obtaining more accurate probabilities or the development of new scoring systems specialized to the target population.[4] Developed in 1974, the Glasgow Coma Scale (GCS) used to objectively describe the neurological status. Although it is considered as the gold standard in determining the level of consciousness, but in the literature, limitations for this score have been mentioned several times.[5-8] From the past decade, the full outline of unresponsiveness (FOUR) score was validated and introduced to calculate four functional dimensions including eye, brainstem reflexes, motor responses, and respiration.[9] By using this new structured objective scoring model, the brainstem function can be examined, even for patients who cannot communicate verbally.[10,11] Decreasing FOUR score (with ranging from 0 to 16) is associated with a falling level of consciousness.[12,13]

The validity of the FOUR score with reference to GCS has been shown in many studies.[5,6,11,14] However, there is no consensus on which of these scoring systems is best.[15-17] External validation should be obtained when applying the scoring system to a new setting.[2]

Nyam et al.[18] examined whether the FOUR score is superior to GCS in predicting ICU mortality in traumatic brain injury (TBI) patients. They compared the ability of four scoring systems by using AUC-ROC curve. Of the 55 TBI patients, 72.73% were men, and the mean age of subjects was 63.1 ± 17.9. The overall mortality rate was 14.6%. The AUCs were 74.47, 74.73, 81.78, and 53.32% for the FOUR score, GCS, APACHE II, and TISS, respectively. They found that the FOUR score was similar to the GCS and APACHE II, in terms of predictive power, also the FOUR score could predict early mortality in TBI patients. In another study, Said et al.[19] compared the usefulness of GCS and FOUR score in predicting extubation failure. A total of 86 patients with impaired consciousness were included in their study. The median GCS and FOUR score were 7 (3–10) and 8.5 (2.3–11), respectively, on admission. The AUC with FOUR score was significantly greater than GCS score, AUC: 0.867 (0790–0.944), confidence interval (CI): 95% and AUC: 0.832 (0.741–0.923), CI: 95%; respectively, (p = 0.014). They found the FOUR has a better ability to predict successful extubation than GCS.

Khanal et al.[20] in a study of 97 patients (age >16 years), prospectively compared the outcome prediction of these two models. Measuring both scores within 24 hours of ICU admission, data analyzing showed that the mean scores of nonsurvivors were lower than survivors (p <0.001). The AUCs of 0.79 and 0.82 (for GCS and FOUR score, respectively) indicated to fair discrimination power for both. Considering the Youden index, 6.5 was the best cutoff point for both scores. The Hosmer–Lemeshow Chi-square coefficient test showed the FOUR score had better calibration power than GCS. The Spearman test of 0.91 pointed to a strong correlation between two scores (p <0.001). Some studies suggest that these two predictive models have the same ability in prognostication of the outcome; however, some studies have pointed to the superiority of either model. Accordingly, it is recommended that these models should be repeatedly calibrated.[2] As far as we know, there have been no studies assessing the use of the FOUR score in predicting outcomes in MICU for the elderly patients. The present study aimed to evaluate the FOUR score ability to predict outcomes in elderly patients admitted to the MICU.

Materials and Methods

In this cohort study (July 2016–October 2017), 168 consecutive elderly patients (age ≤65 years) admitted to the MICU were included to study. Patients with brain death and who stayed in the ICU for less than 24 hours were excluded.

Patients’ demographic data (including gender and age) were collected from the records of patients, and serial FOUR scores for the first 24, 48, and 72 hours of admission were calculated for each patient separately.

Eye and motor responses, respiration patterns and brainstem reflexes are the categories of the FOUR score. The ranges of score for each category is 0–4 points. The lower scores of the FOUR score denoting an increasing deviation from normal. Worst FOUR scores in 24 hours at the first 24, 48, and 72 hours after MICU admission were derived. After the initial recording of data in collection form, the data entered to the SPSS (IBM Corp., Version 22.0, New York). Then, using statistical tests, the relationship between patients’ outcomes and the FOUR score was assessed. To protect patients’ privacy, no information has been provided that could lead to patient identification. The primary outcomes for this nonintervention study were nonsurvivors and survivors.

Patients who died after 24 hours of admission to the MICU or were diagnosed with brain death were included in nonsurvivors. To summarize the data related to the study population, simple descriptive statistics were used for continuous variables and percentage for classified variables. The p <0.050 is considered significant. For assessing the discrimination power of the model, calculating the AUC-ROC curve, distinguishing between survivors and nonsurvivors, was evaluated. AUC of 0.5 means random chance; AUC more than 0.7 and 0.8 indicates a moderate and good prognostic models, respectively.[21] The agreement between actual outcomes and individual probabilities shown with calibration power that assessed by Hasmer–Lemshaw goodness of fit (GOF) test; if P is greater than 0.05, it indicates that the model has a good calibration.[22]

Results

A total of 168 elderly patients admitted to MICU, with a mean age of 74.97 ± 6.28 years (65–90 years), 70 (41.7%) men, and 98 (58.3%) women were enrolled in this study. In total, 31% (52 patients) died. Chronic obstructive pulmonary disease (COPD), gastrointestinal (GI) bleeding, cerebrovascular accident (CVA), sepsis, post-cardiopulmonary resuscitation (CPR), and cancer were the most diagnosed diseases, respectively. Table 1 shows the characteristics of the study population.

Table 1

Characteristics of the study samples

Characteristics	Total (n = 168)	Survivors (n = 116)	Nonsurvivors (n = 52)	p
Age (years, mean ± SD range)	74.97 ± 6.28	73.40 ± 6.16	78.48 ± 5.02	p <0.001
COPD	72	55	17	—
GI bleeding	25	20	5	—
CVA	20	16	4	—
Sepsis	23	15	8	—
Post CPR	17	5	12	—
Cancer	11	5	6	—
Sex (n, %)
Men	70 (41.7)	48 (41.4)	22 (42.3)	p = 0.91
Women	98 (58.3)	68 (58.6)	30 (57.7)
Length of MICU stay (days, mean ± SD)	19.11 ± 11.81	18.97 ± 12.93	19.42 ± 8.94	p = 0.79
FOUR score 24 hours (mean ± SD)	8.53 ± 4.18	10.07 ± 3.99	5.10 ± 1.92	p <0.001
FOUR score 48 hours (mean ± SD)	9.55 ± 4.11	10.71 ± 4.05	6.98 ± 2.93	p <0.001
FOUR score 72 hours (mean ± SD)	9.28 ± 3.89	10.34 ± 3.86	6.92 ± 2.78	p <0.001

MICU, medical intensive care; FOUR score, full outline of unresponsiveness score

The survivors showed significantly higher scores at 24, 48, and 72 hours after admission than nonsurvivors (p <0.001, p <0.001, and p <0.001, respectively) (Table 1).

There are two known methods to examine the performance of predictive scoring systems: calculating the calibration and discrimination power of them. In Table 2, the performance of the GCS and FOUR score has compared.

Table 2

Comparison of FOUR score and GCS between survivors and nonsurvivors

Variables	Cutoff score	Youden index	Sensitivity (%)	Specificity (%)	Positive predictive value (%)	Negative predictive value (%)	Accuracy (%)	Area under ROC curve	95% CI	SE	Significant
FOUR score 24 hours	8.5	0.565	96.15	60.34	52.08	97.22	71.43	0.857	0.80–0.91	0.028	0.000
FOUR score 48 hours	10.5	0.432	92.31	50.86	45.71	93.65	63.69	0.763	0.69–0.83	0.036	0.000
FOUR score 72 hours	9.5	0.364	76.92	59.48	45.98	85.19	64.88	0. 750	0.68–0.82	0.038	0.000

FOUR score, full outline of unresponsiveness score; GCS, Glasgow Coma Scale

AUCs indicated that the FOUR score's discrimination power for 24, 48, and 72 hours of admission was good (AUC = 0.857, AUC = 0.763, and AUC = 0.750, respectively). By the Youden index (sensitivity + specificity −1), the best cutoff point for models was determined. By cutoff point 8.5, FOUR score 24 hours of admission showed 96% sensitivity, 60% specificity, and 71% accuracy, with an AUC of 0.757 ± 0.028 standard error (SE) (95%; 0.80–0.91, p <0.001). A cutoff point 10.5 for FOUR score 48 hours of admission showed 92% sensitivity, 51% specificity, and 64% accuracy, also the AUC was 0.763 ± 0.036 SE (95%; 0.69–0.83, p <0.001), and a cutoff score 9.5 for FOUR score 72 hours of admission accompanied by 77% sensitivity, 59% specificity, and 65% accuracy, also the AUC was 0.750 ± 0.038 SE (95%; 0.68–0.82, p <0.001) (Table 2).

The calibration was good just for FOUR score 24 hours (χ2 = 10.06, p = 0.26) and 48 hours of admission (χ2 = 6.82, p = 0.45), and it was weak for FOUR score 72 hours. In Figure 1, the ROC curve shows the predictive accuracy of the two models. In terms of predictive accuracy, the FOUR score 24 hours of admission was superior to the FOUR score of other days although the FOUR score 48 hours showed a slightly better calibration than FOUR score 24 hours of admission. The survivors and nonsurvivors were homogeneous in terms of gender and LOS-MICU (p = 0.91, p = 0.79, respectively) (Table 1). In addition, the nonsurvivors were older than survivors (p <0.001).

Fig. 1

ROC curves for FOUR score 24, 48, and 72 hours of admission at the MICU. The AUC ROC was 0.857.0.763, and 0.750, respectively

Discussion

The performance of serial FOUR score (24, 48, and 72 hours of admission) has been evaluated in the MICU. The survivors had a higher mean serial FOUR scores compared to nonsurvivors (p <0.001, p <0.001, and p <0.001, respectively). Moreover, the serial scores showed good discrimination power (p <0.001, p <0.001, and p <0.001, respectively). Also, the AUC of 0.857 for FOUR score 24 hours, the AUC of 0.763 for FOUR score 48 hours, and the AUC of 0.750 for FOUR score 72 hours of admission indicated that the discrimination power of FOUR score was excellent for FOUR score 24 hours and good for FOUR score 48 and 72 hours. The better discrimination power of FOUR score 24 hours of admission may represent that the 24 hours of admission in the intensive care unit is a critical time for assessing the severity of illness, as many predictive models use this time for calculating the score. Compared to FOUR score 72 hours (weak calibration), FOUR score 24 hours (χ2 = 10.06, p = 0.26) and FOUR score 48 hours (χ2 = 6.82, p = 0.45) had good calibration. It may represent that the FOUR score is suitable in short-term settings, and the adjusted model is needed for long-term settings.

The best cutoff point for FOUR score 24, 48, and 72 hours was 8.5, 10.5, and 9.5, respectively. In Akavipat et al.'s[23] study, the cutoff points for in-hospital mortality and the poor outcome were 10 and 14, respectively.

In line with our results, many studies have reported that patients with low FOUR scores had a poorer prognosis or higher mortality rates.[5,11,18,24,25] Many studies have also pointed to the appropriate prognostication power of FOUR score (Table 3).

Table 3

Similar studies to findings

Authors	Year	FOUR score 24 hours (mean ± SD)	Sensitivity	Specificity	Positive predictive value	Negative predictive value	Area under ROC curve	Findings
Fugate et al.	2010	FOUR score ≤4FOUR score ≤8FOUR score ≤10	47.671.492.9	10097.880.0	10096.881.3	67.278.692.3	—	Two points improvement in FOUR score was associated with survival. In addition, it is a simple and accurate predictor of outcome in patients surviving cardiac arrest.
Kocak et al.	2012	Survivors: 12.9 ± 2.48Non-survivors: 8.6 ± 4.32	—	—	—	—	0.67	The FOUR score is a useful tool for evaluation of acute stroke patients and outcome estimation.
Mcnett et al.	2014	Median:15	—	—	—	—	0.91	FOUR is comparable to GCS in terms of predictive ability.
Sadaka et al.	2012	13	—	—	—		0.64	The accuracy of the FOUR score in predicting the outcome of TBI patients confirmed but they emphasized there is need further studies for standard use of this tool.
Okasha et al.	2014	Median:11	73.0	80.0	—	—	0.85	The predictive ability of FOUR score was better than GCS.
Khanal et al.	2016	Survivors: 9.13 ± 3.61 Non-survivors: 4.97 ± 2.76	79.31	79.41	62.16	90.0	0.82	Compared to GCS the sensitivity, specificity, positive predictive value, negative predictive value, and accuracy of FOUR score were better.
Ramazani and Hosseini	2018	7.21 ± 2.83	67.6	87.5	33.33	92.59	78.7	FOUR score sowed good performance for predicting outcomes in children admitted to medical/surgical ICUs.

FOUR score, full outline of unresponsiveness score; GCS, glasgow coma scale

Fugate et al.[24] in a prospective, a cohort study from June 2006 to October 2009 assessed the predictive ability of the FOUR score in patients’ postcardiac arrest and compared its performance to GCS. Hospital mortality was the primary outcome. The total mortality rate was 65% (89). None of the patients with a FOUR score ≤4 survived at days 3–5 after cardiac arrest, and only one patient with a GCS of 3 survived. On the 3rd–5th days after cardiac arrest, 91% patients with FOUR score >8 (p <0.0001) and 87% patients with GCS >6 survived (p <0.0001). In serial exams, two-point improvement in FOUR score was associated with survival. This relationship did not apply to GCS score. They observed the FOUR score has good accuracy, and a simple clinical tool can predict outcomes reliably.

Kocak et al.'s[26] study assessed the usefulness of the FOUR score in predicting the clinical outcomes of ICU patients with stroke. They also examined the associations between FOUR score and other coma scales like GCS and APACHE II. It was a prospective study of 100 acute stroke patients who were admitted to a neurology ICU. The mean age of the population was 70.49 ± 12.42 years. The FOUR scores of nonsurvivors on the 1st, 3rd, and 10th days of admission were significantly lower than survivors (p = 0.005, p <0.001, p <0.001, and p <0.001, respectively). Also, the AUC ranged from 0.675 (95% CI, 0.565–0.786) for the 1st day of admission, 0.922 (95% CI, 0.867–0.977) for the 3rd day, and 0.981 (95% CI, 0.947–1.015) for the10th day. They suggested that the FOUR score as a useful scale can be used in the evaluation of acute stroke patients and outcome estimation.

Mcnett et al.[27] evaluated the correlations between GCS and FOUR scores of 24 and 72 hours with cognitive/functional outcomes. Also, they assessed the correlation between both scores and mortality; 136 patients with a mean age of 53.1 enrolled in their study. In their study, the FOUR score was similar to GCS in terms of discrimination power.

In this study, FOUR score 24 hours of admission showed excellent discrimination power and for 48 and 72 hours of admission, the power was good. The FOUR score of 24 and 48 hours had acceptable calibration power, but calibration of FOUR score 72 hours was weak. Most of the studies, in line with our findings, have mentioned that FOUR score has an acceptable to excellent level of discrimination power.[16,18,28]

Sadaka et al.[28] in a prospective study examined the predictive value of FOUR score in TBI and compared its performance to GCS; 51 patients were enrolled in their study. In-hospital mortality and poor neurologic outcome in the 3rd to 6th months were the primary outcomes. The FOUR score showed an AUC of 0.67 for poor neurologic outcomes, it was 0.65 for GCS. Similarly, for in-hospital mortality it was 0.64 for FOUR score and 0.63 for GCS. They confirmed the appropriate accuracy of the FOUR score in predicting the outcome of TBI patients, also they emphasized further studies for standard use of this model.

In another prospective cohort study, Okasha et al.[16] compared the ability of these two models in predicting TBI outcomes; 60 patients with TBI were included in their study. Both scores were calculated on arrival to the emergency room. Unfavorable outcome, in-hospital mortality, endotracheal intubation, and LOS were primary outcomes. The overall mortality rate was 25% (15) and 58% (35) had an unfavorable outcome. The AUC for FOUR score was significantly higher than AUC for the GCS (0.850 vs 0.796, p = 0.025). The models were similar in predicting unfavorable outcome (p = 0.136) and endotracheal intubation (p = 0.06). The ability of the FOUR score was greater in distinguishing survivors and non-survivors.

In our study, the calibration power of FOUR score was acceptable in 24 and 48 hours, but it was weak for FOUR score 72 hours of admission (p = 0.26, p = 0/45, and p <0.001, respectively). The calibration power of FOUR score was varied in several studies, depending on the diagnosis of the patients and the setting in which the study was performed.[29,30] This inconsistency and dissimilarity in calibration can be explained in this way that Predictive models of outcome should be calibrated before being used in new settings to avoid compromising their accuracy.[2] Recalibrating outcome prediction models take into account the adjustments needed to cope with population changes, and the quality of care in different settings thus can improve the estimation of the outcome.

The overall mortality rate observed in our study was 31%, which was 65% in Fugate et al.[24] study and 14.6% in Nyam et al.[18] study. Differences in inclusion criteria, patients’ diagnosis, severity of their illness, and quality of care between studies can be the reasons for the existing differences in mortality rates. The sample size, quality of care, and institutional policies can influence model calibration. Therefore, it is necessary to adapt the model to these conditions. Ethical considerations have been considered in conducting this study.

Conclusion

The FOUR score showed good predictive performance in predicting mortality in elderly patients who were admitted to MICU. FOUR score 24 hours of admission had excellent discrimination and good calibration power. For 48 hours of admission, the FOUR score had acceptable discrimination with good calibration power. For FOUR score 72 hours of admission, just discrimination of model was acceptable, and calibration was weak. Utilizing such a valid predictive model in critically elderly patients is beneficial.

Orcid

Jamileh Ramazani https://orcid.org/0000-0003-4645-780X

Mohammad Hosseini https://orcid.org/0000-0002-5977-1921