Comparison of full outline of unresponsiveness score and Glasgow Coma Scale in Medical Intensive Care Unit.

Context: The Glasgow Coma Scale (GCS) is the most commonly used scale, and Full Outline of Unresponsiveness (FOUR) score is new validated coma scale as an alternative to GCS in the evaluation of the level of consciousness. Aim: The aim of the current study was to evaluate FOUR score and GCS ability in predicting the outcomes (Survivors, nonsurvivors) in Medical Intensive Care Unit (MICU). Setting and Design: This was an observational and prospective study of 300 consecutive patients admitted to the MICU during a 14 months' period. Materials and
Methods: FOUR score, GCS score, and demographic characteristics of all patients were recorded in the first admission 24 h. Statistical Analysis Used: A receiver operator characteristic (ROC) curve, Hosmer-Lemeshow test, and Logistic regression were used in the statistical analysis (95% confidence interval).
Results: Data analysis showed a significant statistical difference in FOUR score and GCS score between survivors and nonsurvivors (P < 0.0001, P < 0.0001; respectively). The discrimination power was good for both FOUR score and GCS (area under ROC curve: 87.3% (standard error [SE]: 2.1%), 82.6% [SE: 2.3%]; respectively). The acceptable calibration was seen just for FOUR score (χ2 = 8.059, P = 0.428). Conclusions: Both FOUR score and GCS are valuable scales for predicting outcomes in patients are admitted to the MICU; however, the FOUR score showed better discrimination and calibration than GCS, so it is superior to GCS in predicting outcomes in this patients population.

Introduction

There are too many prognostic models have been used for predicting the patient outcome for three decades. Selecting and using an appropriate diagnostic tool is critical in the early stage for an appropriate decision about primary diagnosis, medical care, and prognosis.[1] Even though these models are an essential part of the improvement in clinical decisions and in identifying patients with unexpected outcomes, they have, however, their weaknesses and there is no ideal scoring system.[2] The Glasgow Coma Scale (GCS) was developed in 1974 to objectively describe neurological status and predict outcome in neuroscience patients, and it is the most widely used scoring system for comatose patients in intensive care. Through the years, the GCS has become the gold standard for describing the level of consciousness. Despite its widespread use, the GCS has many limitations, including the impossibility to assess the verbal score in intubated or aphasic patients, and an inconsistent inter-rater reliability that are well documented in the literature.[34] The full outline of unresponsiveness (FOUR) score is a new coma scale that was recently developed and validated in adults as a proposed replacement for the GCS, and it is not reliant on the verbal response. Decreasing in GCS and FOUR score is associated with worsening level of consciousness.[5]

Although the FOUR score has been validated with reference to the GCS in several clinical contexts.[6789] However, there are still conflicting data concerning which of this two scoring systems has the best predictive value.[1011121314] External validation is an essential step before application of the predictive model in the group of patients who are different from that group originally used for model development.[15]

Büyükcam et al.[14] investigated whether the FOUR score is better than GCS in predicting mortality and morbidity in children with head trauma. A total of 100 children (2–17 years of old) who admitted to the emergency department with head trauma and presented with an altered level of consciousness were included in their study. The cutoff scores for predicting in-hospital mortality were 9 for FOUR score and 7 for GCS. The area under the receiver operator characteristic (ROC) curve (AUC) values was similar for GCS and FOUR score. The FOUR score provided no significant advantage over GCS in predicting mortality and morbidity in children with head trauma. Khajeh et al.[16] evaluated the ability of GCS and FOUR score to predict the mortality and discharge rate of 200 patients admitted to Pediatric Intensive Care Unit (PICU). Logistic regression analyses (FOUR score = OR: 0.13; 95% CI: 0.06–0.29; P < 0.001; GCS = OR: 2.49; 95% CI: 1.44–4.32; P < 0.001) showed that the FOUR score and GCS are good predictors for in-hospital mortality. Their findings indicated that the FOUR score is more capable than GCS in predicting the mortality and discharge of patients admitted to the PICU.

Surabenjawong et al.[17] in a prospective cohort study examined the prognosis of 60 acute stroke patients were admitted to emergency department. With a mean FOUR score of 14.05 (standard deviation [SD] 4.02) and mean GCS of 12.45 (SD 3.74), both models had an excellent correlation with r = 0.821 (P < 0.001). For predicting 3-month mortality and poor neurological outcome in acute stroke patients, the FOUR score was superior to GCS. Saika et al.[5] in a prospective observational study of 138 patients, compared the predictability of FOUR score and GCS for early mortality, after moderate and severe traumatic brain injury (TBI). Both FOUR and GCS scores were determined at admission. The mean FOUR score and GCS were 11 (range, 0–16) and 9.5,[345678910111213] respectively. The total FOUR score and GCS were significantly lower in patients who did not survive. At a cutoff point of 7 for FOUR score, the AUC was 0.97, with sensitivity of 97.5% and specificity of 88.2% (P < 0.0001). For GCS score, AUC was 0.95, with sensitivity of 98.3% and specificity of 82.4% with cutoff point of 6 (P < 0.0001). There was a correlation coefficient of 0.753 (P < 0.001) between the FOUR score and GCS. The predictive value of the FOUR score on the admission of patients with TBI was no better than the GCS score.

Some studies pointed to similarity and some to the superiority of one of these two predictive models. For that reason, it is recommended that regular re-calibration of models should be undertaken to provide a well-validated ones to predict mortality.[15] The aim of this study was to evaluate the FOUR score and GCS ability in predicting the outcomes (survivors, nonsurvivors) in Medical Intensive Care Unit (MICU).

Materials and Methods

Design

It was a prospective observational cohort study of patients from July 2016 to October 2017.

Population

The selected population was 300 consecutive patients admitted to medical ICU. Excluded from the study population were patients with a length of ICU stay <24 h and brain death at the time of admission.

Data collection

All data were collected concurrently from MICU admissions. Demographic information (including gender and age), FOUR score and GCS were collected for each patient separately. The GCS is composed of three categories, including eye-opening, verbal response, and motor response. The score is determined by the sum of the score in each of the 3 categories, with a maximum score of 15 and a minimum score of 3. The FOUR score covers eye and motor responses, brainstem reflexes and respiration patterns. Each category is given 0–4 points, 0 being the worst and 4 being the best. For both FOUR score and GCS, the lower scores denoting an increasing deviation from normal. From the first 24 h after admission to MICU, a mark adjusting for the FOUR score and GCS was calculated. Data were recorded initially on a standardized data collection form for FOUR score and GCS and then transferred to the SPSS statistical software (IBM Corp., Released 2013, IBM SPSS Statistics for Windows, Version 22.0, Armonk, NY, USA). After calculation of FOUR score and GCS, the relationship between patients’ outcomes and these scores studied. Patients’ privacy maintained by not publishing identifying information.

Intervention

There was no intervention in this study.

Outcome measures

The primary outcomes for this investigation were survivors and nonsurvivors.

Data analysis

In this study, patients who died or classified as brain dead were included in nonsurvivors and others who transferred from MICU to other wards of the hospital were included in the survivors. The data encoded using SPSS statistical software version 22 (©Copyright IBM Corporation and other(s) 1989–2013), then using simple descriptive statistics, the study population characteristics summarized. For continuous variables, data were presented as means with SDs and frequencies with percentages are used for categorical data. The association between FOUR score and GCS with patients’ outcomes was assessed by logistic regression. Hence, FOUR score and GCS were as independent continuous variables. P < 0.050 is considered statistically significant. To validate these two models, standard tests to measure discrimination and calibration were performed. Calculating the area under the ROC curve, distinguishing between survivors and nonsurvivors (discrimination) was assessed. An AUC of 0.5 is equivalent to random chance (a diagonal line), AUC >0.7 indicates a moderate prognostic model, and AUC value >0.8 (a bulbous curve) indicates a good prognostic model.[18] The agreement between individual probabilities and actual outcomes (calibration) was assessed using the Hosmer–Lemeshow goodness of fit test and P > 0.05 was considered as well-calibrated.[19]

Results

A total of 300 patients admitted to MICU were enrolled in this study. The mean age of the cohort was 63.36 ± 16.98 years (range 14–90 years), which 143 (47.7%) were men, and 157 (52.3%) were women. The overall mortality rate for all subjects was 28% (84 patients). The characteristics of the study population are shown in Table 1.

Table 1

The characteristics of the study samples

Characteristics	Total (n=300)	Survivors (n=216)	Nonsurvivors (n=84)	P
Age (years), mean±SD range	63.36±16.98	60.95±17.61	69.57±13.46	0.052
Sex, n (%)
Men	143 (47.7)	99 (69.23)	44 (30.77)	0.30
Women	157 (52.3)	117 (74.52)	40 (25.48)
Length of MICU stay (days) mean±SD	19.02±12.30	18.94±12.98	19.24±10.41	0.01
FOUR score, mean±SD	8.79±4.11	10.21±3.81	5.11±2.10	<0.001
GCS, mean±SD	8.58±3.67	9.74±3.51	5.61±2.10	<0.001

SD: Standard deviation, FOUR: Full outline of unresponsiveness, GCS: Glasgow Coma Scale, MICU: Medical Intensive Care Unit

For the entire cohort of patients, FOUR score and GCS, age and length of MICU stay were significantly different between the survivors and nonsurvivors. Survivors and nonsurvivors exhibited profound differences in FOUR score and GCS; the nonsurvivors showed significantly lower scores at the first 24 h of admission than survivors (P < 0.001, P < 0.001, respectively) [Table 1].

The performance of predictive models can be assessed using a variety of different methods such as calculating discrimination and calibration power of models. The performance of two models has compared in Table 2.

Table 2

Comparison of full outline of unresponsiveness score and Glasgow Coma Scale between survivors and nonsurvivors

Variables	Cut-off score	Youden index	Sensitivity (%)	Specificity (%)	Positive predictive value (%)	Negative predictive value (%)	Accuracy (%)	Area under ROC curve	SE	Significant
FOUR score	6.5	55.55	78.31	78.24	58.04	90.37	78.26	0.873	0.021	0.000
GCS	7.5	47.74	79.76	68.98	50.00	89.76	72.00	0.826	0.023	0.000

FOUR: Full outline of unresponsiveness, GCS: Glasgow Coma Scale, SE: Standard deviation, ROC: Receive operator characteristic

Based on the area under ROC curve, the discrimination power of FOUR score and GCS was good (AUC = 0.873, AUC = 0.826, respectively). The best Youden index (sensitivity + specificity − 1) was used to determine the best cutoff score point for both predictive models. By cutoff score 6.5, FOUR score predicted MICU mortality with a sensitivity of 78%, a specificity of 78%, and accuracy of 78%, with an AUC of 0.873 ± 0.021 standard error (SE) (95%; 0.833-0.913, P < 0.001). For GCS, a cutoff score 7.5 showed a sensitivity of 80%, a specificity of 69%, and accuracy of 72%, also the AUC was 0.826 ± 0.023 SE (95%; 0.781-0.872, P < 0.001) [Table 2]. Based on the Hosmer–Lemeshow Chi-Square statistic, the calibration power of FOUR score was good (χ2 = 8.06, P = 0.43) but it was weak for GCS (χ2 = 17.01, P = 0.03). To access the predictive accuracy of two models, the ROC curve was drawn [Figure 1]. Based on the findings of this study, the overall predictive accuracy of FOUR score was better than GCS. The length of ICU stay (LOS) was significantly longer for nonsurvivors in comparison to survivors (P = 0.01). Although there was the difference between the age of survivors and nonsurvivors and nonsurvivors were older than survivors, but this relationship was not statistically significant (P = 0.052).

Figure 1

Receiver operator characteristic curves for full outline of unresponsiveness score and Glasgow Coma Scale. The area under curve is 0.873 for full outline of unresponsiveness score and 0.826 for Glasgow Coma scale

Discussion

In this study, two predictive models (FOUR score and GCS) have been evaluated in the medical ICU. The mean FOUR score and GCS were significantly higher in survivors compared to nonsurvivors (P < 0.001 and P < 0.001, respectively). The area under ROC showed FOUR score and GCS were good at discriminating survivors and nonsurvivors (P < 0.001 and P < 0.001, respectively). The AUC = 0.873 for FOUR score and the AUC = 0.826 for GCS, pointed out that the discrimination power of FOUR score was slightly better than GCS. The difference between discrimination of two models, may arise from case-mix and need for short-term or long-term cares. Compared to GCS, based on the Hosmer–Lemeshow Chi-Square statistic, better calibration obtained for FOUR score (χ2 = 8.06, P = 0.43) but it was weak for GCS (χ2 = 17.01, P = 0.03). It might be explained by the suitability of FOUR score in long-term MICU cares. The cutoff score with the best Youden's index for FOUR score and GCS was 6.5 and 7.5, respectively, it was 6.5 for both models in Khanal et al.[20] study and ≤8 for GCS in Yousefzadeh-Chabok et al.[21] study. Both models showed acceptable overall accuracy.

The findings of our study are in agreement with several studies have been cited that higher FOUR score and GCS were significantly associated with lower mortality rate or poor prognosis.[172223]

Mouri et al.[24] in a prospective, observational study assessed the diagnostic value of the FOUR score for detecting overt hepatic encephalopathy (OHE) in 94 cirrhotic patients (70 males [75%], mean age 57 ± 11 years): 29 (31%) with OHE and 65 (69%) with No-OHE. The mean FOUR and GCS scores were lower for OHE than No-OHE patients (13.1 ± 0.4 vs. 16.0 ± 0.3, P < 0.0001; and 11.6 ± 0.4 vs. 14.9 ± 0.3, P < 0.0001, respectively). The FOUR score could distinguish between different grades of hepatic encephalopathy (P < 0.0001). Furthermore, it could accurately detect and quantify OHE with an area under the c-index of 0.88 ± 0.10. Based on their findings, the FOUR score was associated with outcome. They proposed it can be used to detect and quantify HE in cirrhotic patients.

Said et al.[11] compared FOUR score and GCS regarding their predictive value for successful extubation at 14 days after intubation as a primary outcome measure. The secondary outcome measures were the 28-day mortality and the neurological outcome at 3 months. Eighty-six patients with median age of 63 (50–77) years and sex-ratio (M/F) of 1.46 were included in their study. A GCS ≤7 predicted the extubation failure at 14 days after intubation with a sensitivity of 88.5% and specificity of 68.3%, whereas a FOUR score <10 predicted the same outcome with a sensitivity of 80.8% and a specificity of 81.7%. The AUC was significantly higher with the FOUR score than GCS (respectively 0.867 confidence interval [CI]: 95% [0790–0.944] and 0.832 CI: 95% [0.741–0.923]; P = 0.014). Both scores had similar accuracy for predicting 28-day mortality and neurological outcome at 3 months.

Zeiler et al.[25] prospectively studied the FOUR score value in predicting the outcomes of 64 aneurysmal subarachnoid hemorrhage patients. The mean age was 54.2 years (26–85). There was a statistically significant association between FOUR score and mortality (P < 0.05).

In this study, the discrimination power of FOUR score and GCS based on AUC-ROC was good, and based on Hosmer–Lemeshow test, had acceptable and weak calibration power, respectively. In agreement to our findings, most of the studies have pointed out at acceptable or excellent discrimination power of FOUR score and GCS.[172627] In Sepahvand et al.[28] study the discrimination power of both models was excellent. The area under ROC curve was 0.961 for FOUR score and 0.928 for GCS. They investigated the prognostic power of these models in a prospective study on 198 patients with TBIs. Of all patients, 65.2% survived and 34.8% died and accuracy power of FOUR score was 82%. FOUR score had 0.76 sensitivity and GCS had a sensitivity 0.85 (similar to our findings). Mean scores for mortality and survival rates were 3.15 ± 3.52 and 12.77 ± 2.43 in FOUR score and 4.59 ± 2.36 and 10.71 ± 2.24 in GCS, respectively. They concluded FOUR score is a valuable, sensitive and specific diagnostic criterion for predicting outcomes in patients with TBIs.

In this study, the calibration power was just acceptable for FOUR score, and it was weak for GCS (P = 0.43 vs. P = 0.03, respectively). There are few studies, noted to different calibration for these predictive models.[152930] These discrepancies and different sensitivity and specificity values in some studies[528] can be elucidate by the fact that a predictive model based on validation and testing set from one population when transferred to another population without modification will often lose its accuracy.[15] By recalibrating these models frequently, we may be overcome these problems with taking into account the changes in settings, quality of care and improved survival.

Overall mortality observed in our study was 28%, which was 28.5% in Khajeh et al.[16] study, 34.8% in Sepahvand et al.[28] study and 10% in Büyükcam et al.[14] study. The different mortality rate in studies may be due to different inclusion criteria and severity of illness of patients who were involved in those investigations.

The present study has several limitations: First, the sample size is known to have a significant influence on calibration. Second, case mix (different settings), different quality of care and policies, can lead to bias. Customizing an appropriate model, specific to the particular setting can improve the predictive ability of models. Ethical considerations have been considered in this study.

Conclusions

Both FOUR score and GCS showed acceptable discrimination power, but good calibration was seen just for FOUR score. More accuracy of FOUR score than GCS makes it as an advisable predictive model for patients who are admitting to medical ICUs.

Financial support and sponsorship

This study has been financially supported by Bojnourd Branch, Islamic Azad University.

Conflicts of interest

There are no conflicts of interest.