VE-CAM-S: Visual EEG-Based Grading of Delirium Severity and Associations With Clinical Outcomes.

To develop a physiologic grading system for the severity of acute encephalopathy manifesting as delirium or coma, based on EEG, and to investigate its association with clinical outcomes.
DESIGN: This prospective, single-center, observational cohort study was conducted from August 2015 to December 2016 and October 2018 to December 2019.
SETTING: Academic medical center, all inpatient wards. PATIENTS/
SUBJECTS: Adult inpatients undergoing a clinical EEG recording; excluded if deaf, severely aphasic, developmentally delayed, non-English speaking (if noncomatose), or if goals of care focused primarily on comfort measures. Four-hundred six subjects were assessed; two were excluded due to technical EEG difficulties.
INTERVENTIONS: None.
MEASUREMENTS AND MAIN RESULTS: A machine learning model, with visually coded EEG features as inputs, was developed to produce scores that correlate with behavioral assessments of delirium severity (Confusion Assessment Method-Severity [CAM-S] Long Form [LF] scores) or coma; evaluated using Spearman R correlation; area under the receiver operating characteristic curve (AUC); and calibration curves. Associations of Visual EEG Confusion Assessment Method Severity (VE-CAM-S) were measured for three outcomes: functional status at discharge (via Glasgow Outcome Score [GOS]), inhospital mortality, and 3-month mortality. Four-hundred four subjects were analyzed (mean [sd] age, 59.8 yr [17.6 yr]; 232 [57%] male; 320 [79%] White; 339 [84%] non-Hispanic); 132 (33%) without delirium or coma, 143 (35%) with delirium, and 129 (32%) with coma. VE-CAM-S scores correlated strongly with CAM-S scores (Spearman correlation 0.67 [0.62-0.73]; p < 0.001) and showed excellent discrimination between levels of delirium (CAM-S LF = 0 vs ≥ 4, AUC 0.85 [0.78-0.92], calibration slope of 1.04 [0.87-1.19] for CAM-S LF ≤ 4 vs ≥ 5). VE-CAM-S scores were strongly associated with important clinical outcomes including inhospital mortality (AUC 0.79 [0.72-0.84]), 3-month mortality (AUC 0.78 [0.71-0.83]), and GOS at discharge (0.76 [0.69-0.82]).
CONCLUSIONS: VE-CAM-S is a physiologic grading scale for the severity of symptoms in the setting of delirium and coma, based on visually assessed electroencephalography features. VE-CAM-S scores are strongly associated with clinical outcomes.

Question: Can visually assessable EEG features be used to accurately predict the severity of delirium symptoms and coma?

Findings: In this prospective, observational cohort study, we use machine learning to develop the Visual EEG Confusion Assessment Method Severity (VE-CAM-S), a physiologic grading scale to predict the clinical severity of delirium or coma secondary to acute encephalopathy. VE-CAM-S scores are well calibrated with the severity of delirium symptoms and coma and also associated with important clinical outcomes, including inhospital and 3-month mortality and functional disability at hospital discharge.

Meaning: VE-CAM-S is a data-driven, accurate and interpretable grading scale derived from a comprehensive set of visually assessed features from standard clinical EEGs in patients with delirium and coma.

Delirium is an acute neuropsychiatric syndrome characterized by a disturbance of attention and awareness (1, 2). Even though more than 20% of hospitalized older adults experience delirium (3), delirium is often missed by healthcare professionals because of its variable presentation (4, 5). Delirium is a manifestation of underlying acute encephalopathy and exists on a continuum between subsyndromal delirium and coma (6). Within this spectrum, the severity of delirium symptoms is associated with increased mortality, longer hospital stays, and cognitive and functional deterioration (7–10). While clinical tools have been developed to standardize delirium evaluation, even validated delirium severity scales such as the CAM-S (10) are subjective and can be subject to inter-rater variability. Additionally, these scales often create inflexible distinctions between patients whose clinical manifestation of an underlying acute encephalopathy in a given moment is more consistent with delirium, a syndrome of impaired attention and awareness with clear diagnostic criteria (11), or coma, a syndrome primarily operationalized using decreased responsiveness to environmental stimuli in clinical scales. However, both delirium and coma are potential manifestations of the same acute encephalopathies and patients can fluctuate between these states (6, 12). A physiologically based measure of the full breadth of manifestations of acute encephalopathy, uniting delirium and coma, could overcome challenges to monitoring patients with acute encephalopathy and potentially could provide important prognostic information.

Numerous studies have documented characteristic electroencephalographic changes in patients with delirium (13–18). Unfortunately, the delirium literature to date has been relatively isolated from the long-standing physiologic literature on encephalopathy (6, 19), including several clinical neurophysiologic scales that have been proposed to grade the degree of encephalopathy as revealed by EEG (20–28). However, prior EEG grading studies have been limited by their focus on narrowly defined patient populations, small sample sizes, evaluation of only limited sets of EEG features, or the use of primarily qualitative analytic tools.

Here we use machine learning on a comprehensive set of visually assessable EEG features in a large and heterogeneous clinical cohort to develop the Visual EEG Confusion Assessment Method Severity (VE-CAM-S), a physiologic grading scale to quantify the symptom severity in the full spectrum of acute encephalopathy including delirium and coma. We identified a minimal subset of nine EEG features that reliably characterize symptom severity in the context of a grading system where weighted points are assigned for the presence of each EEG feature. We demonstrate that the VE-CAM-S scores are not only well calibrated with symptom severity across both delirium and coma but also associated with important clinical outcomes, including inhospital and 3-month mortality and functional disability at the time of hospital discharge.

MATERIALS AND METHODS

Study Design, Setting, and Participants

We conducted a single-center, prospective observational cohort study consisting of adult inpatients undergoing clinical EEG recording to assess brain activity. Adult inpatients were considered for evaluation from all wards, including medical, surgical, and neurologic floors, as well as ICUs. The study was conducted from August 2015 to December 2019, with a temporary pause in study recruitment from January 2017 to September 2018 due to research staff personnel availability. Patients were excluded prior to evaluation if deaf, severely aphasic, developmentally delayed, non-English speaking (if noncomatose), or if their goals of care focused primarily on comfort measures. Prior to data analysis, patients were also excluded if there were technical difficulties with EEG that precluded clinical interpretation (eFig. 1, http://links.lww.com/CCX/A884). Study design is compatible with STrengthening the Reporting of OBservational studies in Epidemiology guidelines.

Standard Protocol Approvals, Registrations, and Patient Consents

This study of human subjects was approved by the Mass General Brigham Institutional Review Board (approval number 2012P001929), including review of EEG and other clinical data. The Partners Healthcare Human Research Committee provided a waiver of written consent for this study.

Clinical Assessment

Patients were assessed at the bedside by study staff during active clinical EEG recording or as soon as possible if limited by patient or staff availability. Each evaluation was conducted by a single member of the study team. Study staff were unaware of the EEG results at the time of delirium assessment. Staff were trained to perform assessments through a combination of didactics, literature review, in-person case reviews, and ongoing discussions.

A one-time evaluation of mental status was conducted, using a structured interview to determine the severity of delirium symptoms using the Confusion Assessment Method-Severity (CAM-S, Long Form [LF]: 0–19) (10). The CAM-S scores the severity of 10 delirium related features: 1) acute change/fluctuating course, 0–1; 2) inattention, 0–2; 3) altered level of consciousness, 0–2; 4) disorganized thinking, 0–2; 5) disorientation, 0–2; 6) memory impairment, 0–2; 7) perceptual disturbances, 0–2; 8) psychomotor agitation, 0–2; 9) psychomotor retardation, 0–2; and 10) altered sleep-wake cycle, 0–2. Additionally, the Richmond Agitation-Sedation Scale (RASS; normal = 0)was used to assess level of arousal (29), and we collected the Age-adjusted Charlson Comorbidity Index (calculated via the medical record) (30).

For descriptive purposes only, patients were classified into various clinical states: delirium according to the CAM framework (31) and coma if they had a RASS score of –4 or –5. To analyze the severity of all patients collectively, patients not assessable due to deep sedation or coma were assigned a CAM-S score of 15 out of 19 for machine learning model development. This score was chosen a priori as in a hierarchical framework of consciousness, without a sufficient level of consciousness, that is arousal, it is not possible to have intact contents of consciousness, that is attention (32–34). We therefore have given maximal points for features that cannot be had in the absence of an appropriate level of consciousness (e.g., negative symptoms) but have not assigned points that can only occur with an appropriate level of consciousness (e.g., positive symptoms of specifically features 7: Perceptual disturbances such as hallucinations and 8: Psychomotor agitation), yielding a score of 15. This practice is consistent with our previously published work, in which patients with coma were assigned maximal CAM-S short form scores (35). Expanded information on evaluation questions and rules used for delirium symptom severity scoring for each category are given in eTables 1 and 2 (http://links.lww.com/CCX/A884).

EEG Recordings and Visual Interpretation

Clinical EEGs were recorded with Silver/Silver chloride scalp electrodes using the standard international 10–20 electrode placement by qualified EEG technicians and read and reported clinically by neurophysiologists using the 2012 American Clinical Neurophysiology Society Critical Care EEG terminology (36). As part of routine clinical practice, all EEG recordings were reviewed by two clinical experts (fellow and attending physician electroencephalographers) before reports were finalized and published in the electronic medical record. Patient evaluations were done prior to clinical interpretation of the EEGs. Although clinical EEG readers had access to routine clinical data, they were blinded to the results of the research evaluation.

Clinical EEG reports were reviewed to identify the presence of a wide range of findings (eTable 3, http://links.lww.com/CCX/A884), including background/rhythm abnormalities, periodic patterns, sporadic discharges, and seizure activity. For scoring, the reported EEG epoch containing the time of patient evaluation was chosen. If a patient was unable to be evaluated during the EEG recording, the nearest reported EEG epoch to the evaluation time was then chosen.

VE-CAM-S Model Development

For the VE-CAM-S model, the visual EEG features defined in eTable 3 (http://links.lww.com/CCX/A884) were used as inputs to predict the determined CAM-S LF score, 0–19. As our dataset included only three patients with CAM-S LF scores greater than 15, we capped all scores at 15 and made the model prediction range from 0 to 15.

The model was created by adapting its coefficients (i.e., points) so that for each pair of patients A and B, the model must discriminate whether the CAM-S LF for patient A is higher than that for patient B. We imposed several a priori constraints based on medical domain knowledge and to reduce collinearity among the inputs, including 1) ElasticNet penalty: encourages some points to be 0 when they do not improve prediction; 2) integer constraint: points had to be integers so they can easily be added by practitioners when an EEG feature is seen; 3) sign and severity constraints: certain points must be 0 or positive; certain patterns of severe encephalopathy were set a priori to receive maximal points, as specified in eTable 3 (http://links.lww.com/CCX/A884); and 4) ordinal constraints: focal/unilateral delta slowing was constrained to have points greater than or equal to focal/unilateral theta slowing.

The model was trained using five-fold nested cross validation (CV), consisting of outer and inner CV (eFig. 2, http://links.lww.com/CCX/A884). Outer CV reports an unbiased out-of-sample performance and inner CV selects the best model parameters. The outer CV splits the dataset into five-folds, where each fold was used to estimate out-of-sample performance (testing set), and the other four-folds combined were used to train the model (outer training set). In each step of the outer CV, we do inner CV by further splitting the training sets (80% of whole data) into five-folds to select model parameters. Note that these model parameters are not trainable and have to be specified before model training; hence, they are also called hyperparameters. There are two hyperparameters, including the strength of the ElasticNet penalty, selected from 10–3, 10–2, …, 101 (five choices), and the 0-point encouragement parameter over the range 0.5, 0.6, …, 0.9 (five choices). In total, there are 25 choices. The choice of hyperparameters that maximize the Spearman correlation, averaged across the five inner testing folds, were selected.

Next, data from the inner five-folds (80% of whole data) were combined to retrain the model with the selected hyperparameters. We then transformed the model output probability for each level of CAM-S LF to the actual occurrence frequency in the dataset (this is called calibration). Only at this point was the trained model applied to the outer testing set (20% of whole data held out in outer CV) to get a testing performance. The final reported performance was obtained from the average of the performances on the five outer testing folds. However, we now have five models, one for each outer CV fold. To get the final model, the most common hyperparameters from the five outer CV folds were used to retrain the model and calibrate on the whole dataset, from which we get the points for the VE-CAM-S model.

Measuring the Association of VE-CAM-S With Clinical Outcomes

To estimate the association of VE-CAM-S with clinical outcomes, we fit a generalized linear model with inputs as age, sex, and VE-CAM-S. We fit models separately for three outcomes: functional status at discharge, inhospital mortality, and mortality at 3 months postdischarge. Functional status at hospital discharge was scored with the Glasgow Outcome Scale (GOS; 1 = death to 5 = good recovery) (37), determined using a combination of physician documentation and physical/occupational therapy evaluations at discharge. The whole VE-CAM-S dataset was used to fit models for clinical outcomes without CV, as there were no hyperparameters in these models. For comparison, we also performed the same analysis with the clinically assessed CAM-S LF instead of the VE-CAM-S. We also conducted subset analyses, comparing associations of VE-CAM-S with each clinical outcome in the following groups: young (< 40 yr) versus middle-aged (40–59 yr) versus old (≥ 60 yr), male versus female, White versus Black race, ICU versus non-ICU patients, and noncomatose. All demographic information (age, sex, race/ethnicity) was obtained from the electronic health record.

Performance Metrics

We used three metrics to measure associations of VE-CAM-S with outcomes: Spearman R correlation; area under the receiver operating characteristic curve (AUC) to assess the ability of VE-CAM-S to discriminate between levels of delirium severity (CAM-S = 0 vs CAM-S ≥ X); and the calibration curve to assess the consistency of the predicted probability with the observed frequency for CAM-S LF less than or equal to 4 versus greater than or equal to 5 (10).

Statistical Analysis

Quantitative data are reported as medians (interquartile range) and compared using Kruskal-Wallis analysis of variance tests, followed by Dunn’s post hoc comparison. Categorical data are reported as n = counts (percent) and compared using chi-square tests, followed by pairwise comparisons with Bonferroni correction. The significance level for all tests was set at p value of less than 0.05. CIs were generated by bootstrapping 1,000 times, to obtain 2.5% and 97.5% percentiles as the lower and upper bounds, respectively. Analyzes were planned prior to conducting all statistical tests. Code used to develop the model and generate figures and tables are available at: https://github.com/mghcdac/VE-CAM-S.

RESULTS

Dataset Characteristics

In all, 406 subjects were assessed for delirium; two were subsequently excluded due to technical difficulties with the EEG that precluded interpretation (eFig. 1, http://links.lww.com/CCX/A884). In the remaining 404 subjects, three were evaluated more than once for a total of 407 timepoints of paired EEG and delirium assessments. Of the 404 subjects analyzed, 132 did not have delirium or coma (32.7%), 143 had delirium (35.4%), and 129 had coma (31.9%). Subjects with delirium or coma were older, had longer hospital stays, higher Charlson Comorbidity scores, more severe CAM-S scores, lower RASS, and lower GOS scores at discharge (Table ).

The majority of evaluations (59.7%) occurred during EEG recordings, 85.7% of evaluations were conducted during the active clinical EEG recording or within 1 hour, and all evaluations were conducted within 5 hours (Table 1). Most EEGs involved long-term monitoring (LTM; 237/407, 58.2%), with a mean reported epoch duration, used for scoring, of 11.0 hours (sd 7.4 hr). Differences in the prevalence of EEG findings between nondelirious, delirious, and coma subjects are shown in Table . Correlations among EEG features are shown in eFigure 3 (http://links.lww.com/CCX/A884).

TABLE 1.

Patient Characteristics Based on Confusion Assessment Method Defined Delirium

Quantitative Data: Unique Subjectsa	Total (n = 404)	No Delirium (n = 132)	Delirium (n = 143)	Coma (n = 129)	Post Hocb
Age, yr, mean (sd)	59.8 (17.6)	55.1 (18.6)	64.3 (16.1)	59.7 (17.0)	N < D
Age-adjusted Charlson Comorbidity Index, median (IQR)	4 (2–6)	3 (1–5)	5 (3–6)	5 (3–7)	N < D, N < C
Length of stay (d), median (IQR)	10 (5–19)	6 (3–11)	12 (6–18)	17 (9–29)	N < D < C
Glasgow Outcome Scale at discharge (1 to 5), median (IQR)	3 (3–4)	4 (3–5)	3 (3–3)	1 (1–3)	N > D > C
Categorical data: unique subjectsa, n (%)
Sexc
Female	172 (42.6)	56 (42.4)	65 (45.5)	51 (39.5)	NS
Male	232 (57.4)	76 (57.6)	78 (54.5)	78 (60.5)
Racec
Asian	13 (3.2)	4 (3.0)	2 (1.4)	7 (5.4)	NS
Black	34 (8.4)	9 (6.8)	16 (11.2)	9 (7.0)
Native American or other Pacific Islander	1 (0.2)	1 (0.8)	0 (0.0)	0 (0.0)
White	320 (79.2)	109 (82.6)	113 (79.0)	98 (76.0)
Other or unknown	36 (8.9)	9 (6.8)	12 (8.4)	15 (11.6)
Ethnicityc
Hispanic	16 (4.0)	3 (2.3)	3 (2.1)	10 (7.8)	N/C
Non-Hispanic	339 (83.9)	119 (90.2)	121 (84.6)	99 (76.7)
Unavailable	49 (12.1)	10 (7.6)	19 (13.3)	20 (15.5)
Disposition
Home, self-care	105 (26.0)	68 (51.5)	30 (21.0)	7 (5.4)	N/D, N/C, D/C
Home, with services	47 (11.6)	29 (22.0)	13 (9.1)	5 (3.9)
Acute rehabilitation	59 (14.6)	16 (12.1)	23 (16.1)	20 (15.5)
Skilled nursing facility	55 (13.6)	10 (7.6)	40 (28.0)	5 (3.9)
Short-term hospital	10 (2.5)	2 (1.5)	5 (3.5)	3 (2.3)
Long-term care	31 (7.7)	2 (1.5)	10 (7.0)	19 (14.7)
Hospice	16 (4.0)	3 (2.3)	9 (6.3)	4 (3.1)
Death	81 (20.0)	2 (1.5)	13 (9.1)	66 (51.2)
Three mo postdischarge
Alive	236 (58.4)	102 (77.3)	87 (60.8)	47 (36.4)	N/D, N/C, D/C
Deceased	113 (28.0)	9 (6.8)	29 (20.3)	75 (58.1)
Unknown	55 (13.6)	21 (15.9)	27 (18.9)	7 (5.4)
Quantitative Data: All Timepointsa	Total (n = 407)	No Delirium (n = 132)	Delirium (n = 145)	Coma (n = 130)	Post Hocb
Delirium severity (Confusion Assessment Method-Severity Long Form: 0–19), median (IQR)	11 (4–15)	2 (1–4)	11 (9–13)	NA	N < D
Richmond Agitation-Sedation Scale (–5 to +4), median (IQR)	–1 (–4 to 0)	0 (0–0)	–1 (–2 to 0)	––4 (–5 to –4)	N > D > C
Categorical data: all timepointsa, n (%)
ICU admission	172 (42.3)	15 (11.4)	46 (31.7)	111 (85.4)	N < D < C
EEG type, n (%)
Routine EEG (< 60 min)	170 (41.8)	83 (62.9)	80 (55.2)	7 (5.4)	N/C, D/C
LTM	237 (58.2)	49 (37.1)	65 (44.8)	123 (94.6)
EEG epoch scoring
Duration (min) used for clinical reports, mean (sd)	393.6 (459.2)	271.8 (450.2)	281.9 (399.0)	641.7 (434.5)	NA
Routine EEG (< 60 min), mean (sd)	26.5 (10.5)	26.5 (12.0)	26.0 (8.7)	31.0 (10.0)
LTM, mean (sd)	656.9 (442.6)	687.4 (522.1)	596.8 (419.0)	676.4 (420.7)
Evaluation time within reported EEG epoch, n (%)d	243 (59.7)	51 (38.6)	75 (51.7)	117 (90)	NA
Evaluation time 0–1 hr from reported EEG epoch	106 (26.0)	51 (38.6)	47 (32.4)	8 (6.2)
Evaluation time 1–2 hr from reported EEG epoch	32 (7.9)	20 (15.2)	12 (8.3)	0 (0.0)
Evaluation time 2–3 hr from reported EEG epoch	18 (4.4)	5 (3.8)	9 (6.2)	4 (3.1)
Evaluation time 3–5 hr from reported EEG epoch	8 (2.0)	5 (3.8)	2 (1.4)	1 (0.8)

C = coma, D = delirium, IQR = interquartile range, LTM = long-term monitoring, N = no delirium, N/D = no delirium and delirium are significantly different, NA = not applicable, NS = not significant.

aDataset consisted of 404 individual subjects with three having been evaluated > 1× (total of 407 timepoints of paired EEG and delirium assessments).

bQuantitative data are reported as medians (IQR) and compared using Kruskal-Wallis analysis of variance tests, followed by Dunn’s post hoc comparison. Categorical data are reported as n = counts (percent) and compared using χ2 tests, followed by paired comparisons with Bonferroni correction. The significance level for all tests was set at p < 0.05. We show the pairwise results from the post hoc comparison. If NS, the omnibus p is NS.

cDemographic data (age, sex, race, and ethnicity) is reported based on information obtained from the electronic health record.

dFor scoring, the reported EEG epoch containing the time of patient evaluation was chosen. If a patient was unable to be evaluated during the EEG recording, the nearest reported EEG epoch to the evaluation time was then chosen.

The VE-CAM-S Model

VE-CAM-S model development was based on a series of visual EEG features (Table 2), each of which could be assigned scores subject to model constraints (eTable 3, http://links.lww.com/CCX/A884). The sum of these scores for EEG features with nonzero coefficients, as determined by model development, yielded the VE-CAM-S, which ultimately could range from 0 to 20. Visual EEG features, from which scores are derived, were divided into two groups (Table ): patterns of severe encephalopathy, in which a maximum score is always assigned and nine additional features, whose point values ranged from 1 to 6. Features assigned 1 point are absence of sleep transients, generalized theta slowing, generalized rhythmic delta activity (GRDA), and lateralized rhythmic delta activity; two point features are lateralized periodic discharges, generalized low voltage, generalized delta slowing. Generalized periodic discharges were assigned four points, and intermittent brief attenuation (IBA) were assigned six points. We note that these last two features receive high-point values despite being uncommon (e.g., IBA occurs in only 5.4% of patients) because they are highly associated with severe encephalopathy. Nevertheless, these high-point values typically occur in combination with multiple other abnormal features; thus, they do by themselves determine the overall VE-CAM-S score. EEG examples of normal, low, mid, high, and worst VE-CAM-S scores are provided in eFigures 4–58 (http://links.lww.com/CCX/A884). We also provide a look-up table that converts the VE-CAM-S score to CAM-S LF score (eTable 4, http://links.lww.com/CCX/A884), where the probability of functional disability (GOS ≤ 3 at discharge) and mortality (inhospital and 3-mo) are also reported for each VE-CAM-S.

TABLE 2.

EEG Findings Based on Confusion Assessment Method Defined Delirium

Features	Total n = 407), n (%)	No Delirium (n = 132), n (%)	Delirium (n = 145), n (%)	Coma (n = 130), n (%)	Post Hoca
Any EEG abnormality	348 (85.5)	77 (58.3)	141 (97.2)	130 (100)	N < D, N < C
Background/rhythm abnormalities
Absence of a normal posterior dominant rhythm	279 (68.6)	38 (28.8)	114 (78.6)	127 (97.7)	N < D < C
Absent sleep transients (spindles, K-complexes, vertex waves)	350 (86.0)	93 (70.5)	131 (90.3)	126 (96.9)	N < D, N < C
Asymmetry	168 (41.3)	41 (31.1)	74 (51.0)	53 (40.8)	N < D
Focal/unilateral theta slowing	48 (11.8)	14 (10.6)	29 (20.0)	5 (3.8)	D > C
Focal/unilateral delta slowing	110 (27.0)	35 (26.5)	55 (37.9)	20 (15.4)	D > C
Lateralized rhythmic delta activity	18 (4.4)	4 (3.0)	8 (5.5)	6 (4.6)	NS
Generalized rhythmic delta activity	52 (12.8)	10 (7.6)	20 (13.8)	22 (16.9)	NS
Generalized/diffuse theta slowing	230 (56.5)	45 (34.1)	107 (73.8)	78 (60.0)	N < D > C
Generalized/diffuse delta slowing	223 (54.8)	29 (22.0)	91 (62.8)	103 (79.2)	N < D < C
Excess/diffuse alpha	27 (6.6)	5 (3.8)	8 (5.5)	14 (10.8)	NS
Excess/diffuse beta	37 (9.1)	12 (9.1)	7 (4.8)	18 (13.8)	D < C
Extreme delta brush	1 (0.2)	0 (0.0)	0 (0.0)	1 (0.8)	NS
Brief potentially ictal rhythmic discharges	1 (0.2)	1 (0.8)	0 (0.0)	0 (0.0)	NS
Intermittent brief attenuation	22 (5.4)	0 (0.0)	6 (4.1)	16 (12.3)	N < C, D < C
Low voltage: moderate (< 20 μV)	20 (4.9)	2 (1.5)	6 (4.1)	12 (9.2)	N < C
Low voltage: extreme/electrocerebral silence	5 (1.2)	0 (0.0)	0 (0.0)	5 (3.8)	NS
Burst suppression with epileptiform activity	9 (2.2)	0 (0.0)	0 (0.0)	9 (6.9)	N < C, D < C
Burst suppression without epileptiform activity	16 (3.9)	0 (0.0)	0 (0.0)	16 (12.3)	N < C, D < C
Unreactive EEG	17 (4.2)	0 (0.0)	3 (2.1)	14 (10.8)	N < C, D < C
Periodic discharges
Lateralized periodic discharges	44 (10.8)	8 (6.1)	17 (11.7)	19 (14.6)	NS
GPDs: not triphasic	61 (15.0)	0 (0.0)	16 (11.0)	45 (34.6)	N < D < C
GPDs: triphasic	13 (3.2)	0 (0.0)	7 (4.8)	6 (4.6)	N < D, N < C
TWs	13 (3.2)	0 (0.0)	7 (4.8)	6 (4.6)	N < D, N < C
GPDs (with or without triphasic morphology) or TWs	62 (15.2)	0 (0.0)	17 (11.7)	45 (34.6)	N < D < C
BIPDs	8 (2.0)	0 (0.0)	3 (2.1)	5 (3.8)	NS
GPDs or BIPDs	64 (15.7)	0 (0.0)	18 (12.4)	46 (35.4)	N < D < C
Sporadic discharges
Sporadic discharges (focal or generalized)	109 (26.8)	13 (9.8)	51 (35.2)	45 (34.6)	N < D, N < C
Seizure activity
Discrete seizures: focal	17 (4.2)	6 (4.5)	5 (3.4)	6 (4.6)	NS
Discrete seizures: generalized	2 (0.5)	0 (0.0)	1 (0.7)	1 (0.8)	NS
NCSE: focal	1 (0.2)	1 (0.8)	0 (0.0)	0 (0.0)	NS
NCSE: generalized	1 (0.2)	0 (0.0)	0 (0.0)	1 (0.8)	NS

BIPD = bilateral independent periodic discharge, C = coma, D = delirium, GPD = generalized periodic discharge, N = no delirium, NCSE = nonconvulsive status epilepticus, NS = not significant, TW = triphasic wave.

aCategorical data are reported as n = counts (percent) and compared using χ2 tests, followed by paired comparisons with Bonferroni correction. The significance level was set at p < 0.05. We show the pairwise results from the post hoc comparison. If NS, the omnibus p is NS.

As shown in Figure , the VE-CAM-S score correlated with the CAM-S LF, with Spearman correlation 0.67 (0.62–0.73; p < 0.001) on the aggregated testing sets. When discriminating CAM-S LF = 0 versus greater than or equal to 4, the AUC was 0.85 (0.78–0.92) (Fig. ); calibration slope was 1.04 (0.87–1.19), which included 1 (1 indicates accurate calibration) for CAM-S LF less than or equal to 4 versus greater than or equal to 5 (Fig. ). In subset analyses (eTable 5, http://links.lww.com/CCX/A884), VE-CAM-S applied to different groups achieved similar performance, by age (young, middle, old), sex (male, female), race (White, Black), clinical location (ICU, non-ICU), and noncomatose status.

Visual EEG Confusion Assessment Method Severity (VE-CAM-S) performance—correlation, discrimination, consistency. A, Boxplots of predicted scores for patients at each Confusion Assessment Method-Severity (CAM-S) Long Form (LF) score on the aggregated testing sets. The red lines indicate the median value; the lower and upper boundary of the box are the 25% and 75% percentiles of the bootstrapped values; the lower whisker is equal to the 25% percentile minus 1.5× the interquartile range (IQR), the upper whisker is equal to the 75% percentile plus 1.5× the IQR. Data points are color-coded according to clinical state: no delirium or coma (blue); delirium (red) via CAM framework; coma (black) via Richmond Agitation-Sedation Scale score of –4 or –5. B, The area under the receiver operating characteristic curve (AUROC) (y-axis) using VE-CAM-S for discriminating CAM-S = 0 (baseline) versus CAM-S more or equal to a certain level (x-axis). The inset shows the receiver operating characteristic for CAM-S = 0 versus CAM-S greater than or equal to 4. The shaded areas represent 95% CIs from bootstrapping. C, The calibration curve for CAM-S LF less than or equal to 4 versus greater than or equal to 5 on the aggregated testing sets.

Association of VE-CAM-S With Clinical Outcomes

We next investigated the independent association of VE-CAM-S with several clinical outcomes after adjusting for age and sex. For comparison, we calculated the same adjusted associations for clinically assessed CAM-S LF. The outcomes included inhospital mortality (binary), 3-month mortality (binary), and GOS at discharge (converted into binary of ≤ 3 vs ≥ 4). VE-CAM-S showed strong independent associations with the outcomes and overall behaved similarly to CAM-S LF (Fig. , and ); adjusted odds ratios and p values are shown in eTable 6 (http://links.lww.com/CCX/A884). When we compared individual predictive performance metrics, VE-CAM-S had a similar false positive rate and a higher false negative rate than CAM-S LF (significance indicated by the asterisk) (Fig. , and ). CAM-S LF and VE-CAM-S showed similar positive and negative predictive values (Fig. , and ).

Visual EEG Confusion Assessment Method Severity (VE-CAM-S) performance—clinical outcomes. The comparison of various performance metrics of using Confusion Assessment Method-Severity (CAM-S) Long Form (LF) versus VE-CAM-S to predict three outcomes: Discharge Glasgow Outcome Score (GOS) (≤ 3 vs ≥ 4), inhospital mortality, and 3-mo mortality. The performance metrics include (A) area under the receiver operating characteristic (AUROC); (B) Cohen’s kappa; (C) false positive rate (FPR); (D) false negative rate (FNR); (E) positive predictive value (PPV); and (F) negative predictive value (NPV). A threshold of 0.5 is used for outcome prediction to compute the FPR, FNR, PPV, and NPV. For each box, the middle orange line indicates the value without bootstrapping; the lower and upper boundary of the box are the 25% and 75% percentiles of the bootstrapped values; the lower whisker is equal to the 25% percentile minus 1.5× the interquartile range (IQR), the upper whisker is equal to the 75% percentile plus 1.5× the IQR. The asterisk represents significant difference by comparing the 95% CIs.

DISCUSSION

In this prospective observational cohort study, we developed a physiologic grading scale, VE-CAM-S, based on visually assessed EEG features, to grade the clinical severity of delirium or coma on the full spectrum of clinical manifestations of acute encephalopathy. Our results show that VE-CAM-S correlates strongly with the well validated CAM-S LF delirium severity score and shows similar associations across a diverse spectrum of adult patients encountered in a general inpatient clinical setting. We also demonstrate that the VE-CAM-S score is strongly associated with important clinical outcomes, including mortality and functional disability at the time of hospital discharge. Therefore, the VE-CAM-S is a clinically relevant physiologic approach to assess the clinical manifestations of acute encephalopathy in the framework of delirium severity.

Typically, mental status is assessed by clinicians via intermittent and subjective daily interactions. Due to the fluctuating nature of delirium and increasing concern that delirium severity is associated with worse prognosis (7–9), a strong clinical need exists for an objective method to report the severity of delirium as a manifestation of acute encephalopathy. However, prior literature on the relationship between qualitative EEG findings and delirium symptom severity is limited. Several EEG classification systems for encephalopathy have been proposed, most notably Parsons-Smith et al (20), Hockaday et al (21), Hughes et al (22), Synek (23), Rae-Grant et al (24), Young et al (25), among others (26–28), but these have not been applied to the full spectrum of EEG manifestations including delirium in the context of its severity. An expanded review of EEG classification systems for patients with EEG and their prognostic value is included in eTable 7 (http://links.lww.com/CCX/A884). VE-CAM-S expands upon this prior work by placing EEG findings reflective of an underlying pathobiological encephalopathy process firmly in the context of the severity of delirium symptomatology.

This work is a step toward helping unite physiologic investigations of delirium and coma, which can both be manifestations of acute encephalopathy. Several empirical considerations support a framework in which significantly impaired arousal in acute coma should be considered akin to severe delirium severity. For example, rates of delirium increase as RASS decreases from 0 to –3, making exclusion of delirium at RASS of –4 somewhat artificial (38). In other studies, patients with decreased arousal, even RASS –4 or –5, have high rates of delirium when they can be assessed (39). Additionally, altered arousal has at least as high an impact on prognostic outcomes as inattention (40). Additionally, the likelihood of generalized EEG slowing is associated not only with arousal but also all core features of delirium (16). Last, studies frequently use days free of delirium or coma, rather than just delirium, as a primary clinical outcome (41), recognizing the joint importance of careful assessment of these conditions.

Among the broad range of visual EEG features coded, some features were excluded from the model, including absence of a normal posterior dominant rhythm (PDR), excess/diffuse alpha and beta, sporadic discharges (focal or generalized), discrete seizures (focal or generalized), focal nonconvulsive status epilepticus (NCSE), and brief potentially ictal rhythmic discharges (BIRDs). Absence of a normal PDR was excluded for better interpretability, as it may be confusing to clinicians to have this separate from background slowing. Excess/diffuse alpha and beta were excluded because this finding is often due to medication effects (diffuse alpha is often observed with propofol; diffuse beta is often observed in normally mentating patients on low dose benzodiazepines [42]). Sporadic discharges were excluded as these are less tightly related to level of encephalopathy and may not generalize across patients with and without epilepsy. Discrete focal seizures, meaning seizures that are brief and well localized, were excluded because the cognitive changes associated with these events are short-lived and represent a process distinct from delirium, which is a comparatively more sustained state. Discrete seizures (generalized), focal NCSE, and BIRDs were excluded as there were too few cases observed in the dataset.

In line with previously described delirium-associated EEG abnormalities (14, 43), selected features included generalized/diffuse delta and theta slowing, GRDA, loss of reactivity, and triphasic waves, continuing to affirm the importance of generalized EEG features in the evaluation of delirium severity. Also comparable to prior studies, the following EEG features were found among the most severe clinical manifestations of acute encephalopathy including coma and were associated with poorer outcomes: extreme low voltage, burst suppression with or without epileptiform activity, unreactivity, extreme delta brush, and generalized NCSE. An expanded review of these EEG features and their prognostic value is included in eTable 8 (http://links.lww.com/CCX/A884).

Our study has several limitations. This observational cohort study was conducted at a single center; thus, further investigation via an external validation cohort is important. EEG referral for altered mental status was initiated by providers; thus, it is not yet clear to what extent our findings will generalize to patients without an acute/chronic neurologic condition warranting continuous EEG assessment. Nearly all patients were evaluated only once, precluding additional investigation of fluctuations. The characterization of EEG fluctuations in the setting of symptom fluctuations remains an area for future exploration. For a minority of patients (58/407, 14.3%), the time of CAM evaluation did not fall during or within 1 hour of the EEG report epoch used for scoring. EEGs included a mix of routine and LTM studies, and epoch durations may affect the possibility of observing some EEG features. Including long-term continuous EEG studies allowed for concurrent assessment; however, it also contributed to increased patients with higher CAM-S scores and coma based on clinical context. VE-CAM-S appeared prognostically comparable to the CAM-S LF but had minor prognostic differences, which could be due to variability in EEG interpretation, CAM-S assessments, or GOS evaluation. The VE-CAM-S model coefficients were constrained to be integers that trade potential quantitative sophistication for clinical simplicity. Last, the VE-CAM-S, like clinical delirium severity scales such as the CAM-S, is not specific to the cause of encephalopathy, rather reflecting severity of brain dysfunction in clinical circumstances where dysfunction is frequently multifactorial.

CONCLUSIONS

VE-CAM-S is a data-driven, accurate and interpretable grading scale of delirium symptom severity and coma based on visually assessed features from standard clinical EEGs. Our results further validate VE-CAM-S in terms of its associations with clinical outcomes. Additional work on VE-CAM-S should also further characterize finer gradations within patterns of background slowing, as well as the intermittency of slowing, presence of alpha activity, and spontaneous variability and reactivity. The consistency of identified EEG findings across multiple studies and contexts also suggests that further research is needed to identify the fundamental brain circuits giving rise to these prognostically important findings in order to develop new targeted therapies for neurocognitive vulnerability.

ACKNOWLEDGMENTS

We wish to acknowledge and express their appreciation to the patients who were a part of this study, as well as the EEG technicians and clinical neurophysiologists of the Massachusetts General Hospital Division of Clinical Neurophysiology who participated in their clinical care.

TABLE 3.

The Visual EEG Confusion Assessment Method Severity Scores

Visual EEG Features	Score
Absent sleep transients (spindles, K-complexes, vertex waves)	1
Generalized/diffuse theta slowing	1
Generalized rhythmic delta activity	1
Lateralized rhythmic delta activity	1
Lateralized periodic discharges	2
Low voltage: moderate (< 20 μV)	2
Generalized/diffuse delta slowing	2
Generalized periodic discharges (with or without triphasic morphology/triphasic waves) or bilateral independent periodic discharges	4
Intermittent brief attenuation	6
Extreme delta brush	20 (worst delirium severity)
Nonconvulsive status epilepticus: generalized
Low voltage: extreme/electrocerebral silence
Burst suppression (with or without epileptiform activity)
Unreactive EEG

aGGLS—GLV—GB—I—W “Things to take skiing: 1—goggles, 2—gloves, 4—good boots, 6—insurance, W—water.”

(1) GGLS: G = “GRDA, ” G = “Generalized Theta,” “L = “LRDA,” and S = “Sleep missing.”

(2) GLV: G = “Generalized Delta,” “L = “LPDs,” and V = “Low Voltage.”

(4) GB = “GPDs or BiPDs.”

(6) I = “Intermittent Brief Attenuation” and (W) Worst.