Interrater Reliability of Four Neurological Scales for Patients Presenting to the Emergency Department.

BACKGROUND: Altered mental status (AMS) comprises a group of clinical symptoms rather than a specific diagnosis. A variety of neurological scales have been used to monitor these patients, including the Glasgow coma scale (GCS), AVPU (alert, responds to verbal stimuli, responds to painful stimuli, unresponsive) scale, and ACDU (alert, confused, drowsy, unresponsive) scale. The simplified motor scale (SMS) has been found to be a potential replacement for the GCS. In this study, we compare the interrater reliability of the GCS (individual components), AVPU, ACDU, and SMS to patients presenting to the ED with AMS.
MATERIALS AND METHODS: This was a prospective observational study conducted in the emergency department of an urban tertiary-care hospital in New Delhi, India. Patients with AMS (traumatic and nontraumatic) presenting to the ED were assessed by two emergency physicians within 10 minutes of each other using the four neurological scales. Percentage agreement and kappa coefficient score were used to determine interrater reliability for SMS, individual components of GCS, AVPU, and ACDU.
RESULTS: The SMS had the best inter-rater percent agreement with 83.91% (95% CI 77.7-88.6%), followed by ACDU which was 76.44% (95% CI 69.0-81.6%), AVPU 75.29% (95% CI 67.8-80.6%), GCS Eye 74.14% (95% CI 66.6-79.6%), GCS Verbal 67.82% (95% CI 60.6-74.3%), and GCS Motor was 64.94% (95% CI 57.6-71.6%).The kappa coefficient for SMS was 0.75 (95% CI 0.67-0.83), followed by GCS Eye which was 0.63 (95% CI 0.54-0.72), AVPU 0.62 (95% CI 0.52-0.72), ACDU 0.60 (95% CI 0.49-0.71), and GCS Verbal 0.58 (0.49-0.66) and GCS Motor was 0.53 (95% CI 0.44-0.63).
CONCLUSION: It can be said that SMS has the best interrater reliability in the ED and can be recommended for clinical use in the emergency departments for patients presenting with AMS. HOW TO CITE THIS ARTICLE: Haldar M, Verma A, Jaiswal S, Sheikh WR. Interrater Reliability of Four Neurological Scales for Patients Presenting to the Emergency Department. Indian J Crit Care Med 2020;24(12):1198-1200.

Introduction

Altered mental status (AMS) is a cluster of clinical symptoms that includes reduced levels of consciousness and deranged attention, arousal, and cognitive abilities.[1] Patients with undifferentiated AMS in who the exact etiology is unknown present frequently to the emergency department (ED). They often manifest vague symptoms; thus, evaluation and management of such patients may become highly challenging for the emergency physician.

To assess AMS, various neurological scales have been developed which include the Advanced Trauma Life Support AVPU (alert, responds to verbal and painful stimuli or unresponsive) scale,[2] ACDU (alert, confused, drowsy, unresponsive) scale 3, Glasgow coma scale (GCS),[3] and Swedish reaction level scale.[4]

In patients with altered levels of consciousness from traumatic and nontraumatic causes, the GCS has been found to exhibit moderate interrater reliability when tested in the ED.[5] Neurological assessment scales other than the GCS have never been assessed or studied to ensure reliability and validity.[6] Gill et al. recently proposed the simplified motor scale (SMS) which is a statistically derived scale as a possible replacement for the GCS to overcome limitations of the GCS score without losing its prognostic performance.[7] SMS score is assigned as: 2—obeys commands, 1—localizes pain, and 0—withdrawal to pain or less response. In comparison to the total GCS score or its three individual components, SMS has shown to have equivalent performance for the prediction of adverse neurological outcomes after traumatic brain injury with varying mechanisms of injury and levels of injury severities.

It has been previously suggested to simplify the GCS score to one of its three components. The motor component of the GCS had similar accuracy as a total GCS score in predicting severe traumatic brain injury (TBI), as found by Ross et al.[8] In 200,000 patients, Healey et al. found that the motor component preserved the power of the total GCS to predict survival.[9] Similarly, Meredith et al. found that a motor score of less than 6 correlated with an increased risk of mortality in the out-of-hospital setting.[10]

The SMS was found to be equivalent to the GCS in predicting clinical outcomes in 8,000 patients of TBI.[7] Despite the abovementioned conclusions of various authors pointing toward the possibility of the SMS to replace the more complex GCS, it would require prospective validation in a new sample of patients with altered levels of consciousness before clinical application. In this study, we chose to test the interrater reliability of the SMS, GCS, AVPU scale and ACDU scale prospectively when applied to patients presenting to the ED with altered mental status.

Materials and Methods

Study Design and Setting

This was a prospective observational study design that adopted convenience sampling. The Institutional Ethics Committee Board of the hospital approved the study. Written formal informed consent was taken from the patient or next of kin for induction of the patient's data in the study. In addition, no patient-identifying information was collected. The study was not funded by any organization or institution, as GCS is used as a tool for neurological examination in patients presenting in altered sensorium.

The study was conducted over a period of 1 year in the ED of an urban tertiary-care center. In this prospective observational study, two emergency physicians who were on shift had independently assigned the GCS, SMS, AVPU, and ACDU to qualifying adult subjects. Patients with both traumatic and nontraumatic causes of AMS were included in the study. The rater pair was random for every patient included in the study. The range of SMS score from 0 to 2 was not found clinically comparable to the range of 3 to 15 in the total GCS score, and hence the comparison was not analyzed. Patients above 18 years of age presenting to the ED in altered mental status were enrolled by the treating emergency physicians soon after the initial stabilization (if required) of these patients. Both the physicians had to assign their scores to the patients within 10 minutes of each other.

Emergency medicine residents with matched levels of education were qualified to enter patients into the study. The residents were blinded to one another's assessments and returned their completed forms to a folder immediately.

Patients were excluded if they had been intubated in another center or if both residents did not complete their evaluations of the patient within the stipulated time.

Statistical Analysis

All statistical testing was two-sided and performed using a significance (alpha) level of 0.05. Percentage agreement (percentage of patients who received the same score by both the observers for the different scales) and kappa coefficient score were used to determine interrater reliability for SMS, individual components of GCS, AVPU, and ACDU. STATA System, version 9.0 was used to conduct all statistical analysis.

Results

In the 174 patients who were included in the study, 108 (62.06%) were males and 66 (37.93%) were females. The median age was 58 years (19–98 years).

The SMS had the best interrater agreement in terms of percent agreements with 83.91% (95% CI 77.7–88.6%), followed by ACDU which was 76.44% (95% CI 69.0–81.6%), AVPU 75.29% (95% CI 67.8–80.6%), GCS Eye 74.14% (95% CI 66.6–79.6%), GCS Verbal 67.82% (95% CI 60.6–74.3%), and GCS Motor was 64.94% (95% CI 57.6–71.6%) (Fig. 1).

Fig. 1

Interrater agreement (%)

The kappa coefficient score for SMS was 0.75 (95% CI 0.67–0.83), followed by GCS Eye which was 0.63 (95% CI 0.54–0.72), AVPU 0.62 (95% CI 0.52–0.72), ACDU 0.60 (95% CI 0.49–0.71) and GCS Verbal 0.58 (0.49–0.66) and GCS Motor was 0.53 (95% CI 0.44–0.63) (Fig. 2).

Fig. 2

Kappa coefficient score

The causes of AMS in the study population were found to be infections (26%), trauma (16%), metabolic encephalopathy (13%), and cerebral infarctions (13%). And 12% of the population presented with other causes such as non-traumatic bleed, alcohol and drug intoxications, tumors of the brain, and cerebral venous thrombosis (Fig. 3).

Fig. 3

Causes of altered mental status presenting to the emergency department (%)

Discussion

The study found the SMS outperform other neurological scales when tested prospectively for interrater reliability in ED patients presenting with AMS.

Subjective evaluation of patients in AMS may vary among physicians which may result in bias. Neurological scales should ideally provide information regarding arousability, level of awareness, and ability to interact.

The GCS was introduced in 1974 as a tool to use in patients with altered levels of consciousness by Teasdale and Jennett, but physicians and investigators have over the many years repeatedly questioned its reliability and practicality.[3,5,7] In acute-care settings, it has been proposed that the use of simpler scales may be more useful and reliable than the GCS.

The motor component of the GCS has been found to be as sensitive as the GCS total to predict patients with severe TBI.[8] The motor component alone has been shown to be equivalent to the total GCS at predicting mortality in the prehospital setting.[10] The GCS and AVPU scales were compared to each other in toxicology patients, and it was found that the AVPU was simpler for nurses to implement, as concluded by Kelly et al.[11] In the neurosurgical setting, ACDU and AVPU scales were compared to the GCS. It was found that the ACDU was more reliable in identifying deteriorating patient conditions than the AVPU and GCS, and a recommendation to follow the ACDU was made.[12]

In patients with head injury, the SMS has been shown to be superior than the GCS and its components.[5] It has also been shown to reliably predict traumatic brain injury and mortality.[13,14] In a study of trauma patients, it was observed that the SMS or AVPU can potentially replace the total GCS score for trauma triage.[15]

In our study, the interrater reliability of the neurological scales were examined for both traumatic and nontraumatic patients in AMS. The SMS displayed a higher reliability as compared to the ACDU, AVPU, and GCS scales. Further larger validation studies to evaluate the SMS for clinical use are recommended.

Since the investigators were not present at all times, possibility of physician bias cannot be overruled. The ease of using SMS compared to the other scales may have resulted in the apparent better performance, as it has fewer categories to be assigned during evaluation which could be a possible limitation of the study.

We conclude that the Simplified Motor Score possessed the best interrater reliability in comparison to the other neurological scales (GCS total and components, AVPU, and ACDU) in both traumatic and nontraumatic settings for patients presenting with AMS.

It may be recommended that the SMS could be used as the neurological assessment scale of choice in the ED setting. However, larger studies as mentioned before are required to validate our findings and to determine its utility as a prognostic tool.

Disclaimer

The views expressed in the submitted case report are the author's own and not an official position of the institution.

Contributors

Haldar M is the principal author; Verma A, Jaiswal S and Sheikh WR are coauthors.