Clinical prediction rule for neurological sequelae due to acute encephalopathy: a medical community-based validation study in Harima, Japan.

OBJECTIVES: This study aimed to verify the screening performance of our clinical prediction rule for neurological sequelae due to acute encephalopathy (NSAE-CPR), which previously identified the following three variables as predictive of poor outcomes: (1) refractory status epilepticus; (2) consciousness disturbance and/or hemiplegia at 6 hours from onset and (3) aspartate aminotransferase >90 IU/L within 6 hours of onset.
DESIGN: Medical community-based multicentre retrospective cohort study.
SETTING: Six regional hospitals in Harima and one tertiary centre in Kobe, Japan, from 2008 to 2012. PARTICIPANTS: We enrolled a total of 1612 patients aged <16 years who met the diagnostic criteria for an initial diagnosis of complex febrile seizure. Patients with a history of neurological disease and those included in the derivation cohort were excluded. PRIMARY OUTCOME MEASURES: Univariate and multivariate analyses were performed to determine the association between each of the three predictor variables and poor AE outcome (Pediatric Cerebral Performance Category score ≥2). Receiver operating characteristic curve (ROC) analysis was also performed to assess the screening performance of the NSAE-CPR.
RESULTS: The ROC analysis identified at least one of the three predictive variables as an optimal cut-off point, with an area under the curve of 0.915 (95% CI 0.825 to 1.000). The sensitivity, specificity, positive and negative predictive values, positive and negative likelihood ratios and Matthews correlation coefficient were 0.867, 0.954, 0.149, 0.999, 18.704, 0.140 and 0.349, respectively.
CONCLUSIONS: Our findings indicate that the NSAE-CPR can be used for the screening and identification of patients with poor outcomes due to acute encephalopathy within 6 hours of onset. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2017. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

This is the first published clinical prediction rule for neurological sequelae due to acute encephalopathy validated via a medical community-based multicentre retrospective cohort study, which may closely resemble the general population.

Clinical symptoms such as altered consciousness may have been underestimated in some patients due to the retrospective design of the study.

Several parameters associated with neurological outcomes such as duration of seizure and detailed treatments were not assessed.

Introduction

Children with acute encephalopathy (AE) and febrile seizure (FS) exhibit symptoms of seizure and/or altered consciousness, which are accompanied by fever without central nervous system (CNS) infection.1,2 FS is a primarily benign seizure condition in which patients typically experience good outcomes, and intensive therapeutic interventions are often unnecessary in patients with FS. In contrast, AE is diagnosed based on prolonged altered consciousness with/without seizure and is associated with a mortality rate of 6%, according to a nationwide survey in Japan—far higher than that for FS.3 Furthermore, approximately 35% of patients with AE experience neurological sequelae such as an intellectual disability or vegetative state.4 Therefore, intensive therapeutic interventions are necessary for patients with AE.5,6 Because patients with both AE and FS often present with seizure and altered consciousness early after onset, accurate differential diagnosis between AE and FS is critical for ensuring prompt initiation of intensive treatment for patients with AE.5 Some epidemiological studies have proposed that AE is defined as disturbed consciousness lasting longer than 24 hours.7 8 However, as it is necessary to distinguish AE from FS within 24 hours of onset in clinical settings because the treatment methods are quite different, this definition is not clinically useful.

To address this issue, we previously conducted a single-centre (Hyogo Prefectural Kobe Children’s Hospital) retrospective study in which we identified three early risk factors for poor outcomes in this patient population: (1) refractory status epilepticus (RSE); (2) consciousness disturbance and/or hemiplegia at 6 hours from onset and (3) aspartate aminotransferase (AST) >90 IU/L within 6 hours from onset. A rule for the clinical prediction of neurological sequelae of AE within 6 hours of onset in patients at risk for poor outcomes (NSAE-CPR) was developed (derivation cohort).9 However, no studies have sought to validate the NSAE-CPR in other populations. The objective of the present study was to verify the screening performance of the NSAE-CPR in a medical community setting in Harima, Japan.

Patients and methods

Study design and setting

From January 2008 to December 2012, we conducted a multicentre retrospective cohort study involving six regional hospitals in Harima (Kakogawa Central City Hospital, Akashi City Hospital, Kita-Harima Medical Centre, St Mary’s Hospital, Nishiwaki City Hospital and Japanese Red Cross Society Himeji Hospital) and one tertiary centre (Hyogo Prefectural Kobe Children’s Hospital) in Kobe, Japan. The study was approved by the ethical committee of each participating facility, who waived the requirement for informed consent due to the retrospective nature of the study. The hospitals participating in this study accounted for 93.7% of individuals receiving inpatient care for AE or FS in this region in 2012 (validation cohort).

Inclusion and exclusion criteria

The present study used the same inclusion and exclusion criteria as the derivation study.9 FS was defined according to the guidelines provided by the National Institutes of Health consensus, with the exception of the age criterion.10 Patients under 16 years of age who met the diagnostic criteria for an initial diagnosis of complex FS were enrolled. Complex FS was defined when patients exhibited one or more of the following features: (1) partial onset or focal features during the seizure; (2) prolonged duration (>15 min) or (3) recurrent FS within 24 hours of the first episode.11 Patients who had a history of neurological conditions such as epilepsy, developmental delay, metabolic disorders or structural anomalies in the CNS were excluded. Patients included in the derivation study were also excluded.

Definitions

Onset time was defined as the time at which seizure or altered mental state was first described in the clinical chart. Blood tests were performed within 6 hours of onset. When two or more blood samples were available, the sample with the highest level of serum AST was chosen for subsequent analyses. RSE was defined as convulsive status epilepticus lasting longer than 60 min that failed to respond to first-line medications, such as diazepam. Disturbance of consciousness, defined as a Glasgow Coma Scale (GCS) score of 14 or lower, was diagnosed by the attending paediatricians and retrieved from each patient’s medical record. Since altered mental state or hemiplegia cannot be evaluated during continuous treatment with anaesthetic drugs such as barbiturates or midazolam, data from patients exhibiting these conditions were considered to represent those of patients without altered mental state or hemiplegia for the analyses. Patients who met NSAE-CPR criteria were classified into a group at high-risk for poor prognosis, while those who did not meet NSAE-CPR criteria were classified into a low-risk group. Although AE is defined as prolonged impaired consciousness, it is sometimes difficult to evaluate the level of consciousness in children with AE in the clinical settings due to sedation. Therefore, in the present study, prolonged impaired consciousness due to AE was defined as disturbed consciousness lasting longer than 24 hours without sedatives. Patients who had prolonged impaired consciousness accompanied by fever without CNS infection or neurological sequelae there after were diagnosed with AE. Patients with conditions due to other aetiologies such as specific inborn error of metabolism were excluded.

Prognosis

Prognosis was evaluated 0–3 months from onset using the Pediatric Cerebral Performance Category Scale (PCPC).12 Children were classified into two groups based on PCPC score, as follows: good outcome (PCPC=1) and poor outcome (PCPC=2-6 (2: mild sequelae 3: moderate sequelae 4: severe sequelae 5: vegetative state, 6: brain death)).

Study methods and statistical analyses

We performed univariate analyses between each of the three predictor variables (RSE, consciousness disturbance or hemiplegia at 6 hours from onset and AST >90 IU/L within 6 hours of onset)9 and poor outcome in the validation cohort using Fisher’s exact test. Variables significantly associated with poor outcome in the univariate analysis were further evaluated via multiple log-binomial analysis. To estimate the power of the NSAE-CPR, we calculated the screening performance of the NSAE-CPR in the validation and derivation cohorts (sensitivity, specificity, predictive values, likelihood ratios and Matthews correlation coefficient). Receiver operating characteristic curve (ROC) analysis was also performed to compare the area under the curve (AUC) between each cohort.

Several syndromes of AE characterised by variations in clinical course, neuroradiological findings and biochemical data have been established; such as acute encephalopathy with biphasic seizures and late reduced diffusion (AESD),13 acute necrotising encephalopathy,14 Reye syndrome15 and haemorrhagic shock and encephalopathy syndrome.16 Mizuguchi et al classified these syndromes into three major categories: metabolic error, systemic cytokine storm and excitotoxic neuronal injury (see online supplementary table 1).1 Our derivation study revealed that AST >90 IU/L within 6 hours of onset predicts AE caused by metabolic error and/or cytokine storm, while normal AST levels are associated with excitotoxic neuronal injury.9 In the present study, we further evaluated the association between AST level and AE outcome to assess the screening performance for each category of AE.

All statistical analyses were performed using EZR (SaitamaMedical Center, JichiMedical University, Saitama, Japan), which is a graphical user interface for R software (The R Foundation for Statistical Computing, Vienna, Austria). More precisely, it is a modified version of R commander designed to add statistical functions frequently used in biostatistics. A P value <0.05 was considered statistically significant.

Results

Patient characteristics

The present study included a total of 1612 (median age: 23 months; 59% male) patients who met the criteria for an initial diagnosis of complex FS. Among these, 15 patients (0.93%) experienced poor outcomes. RSE, consciousness disturbance or hemiplegia at 6 hours from onset and AST level >90 IU/L within 6 hours of onset were observed in 52 (3.2%), 35 (2.2%) and 10 (0.62%) patients, respectively. Eighty-seven patients (5.4%) met the NSAE-CPR criteria and were classified into high-risk group (table 1).

Table 1

Patient characteristics

	Derivation cohort n=86	Validation cohort n=1612
Age, months
Median (range)	22 (4–159)	23 (0–185)
Gender, n (%)
Male	51 (59.3)	950 (58.9)
Poor outcome, n (%)	17 (24.6)	15 (0.93)
Predictor variables, n (%)
Refractory status epilepticus	27 (31.4)	52 (3.2)
Consciousness disturbance or hemiplegia at 6 hours from onset	13 (15.1)	35 (2.2)
AST >90 IU/L within 6 hours of onset	5 (5.8)	10 (0.62)
High-risk group (fulfilling any of the predictor variables above, NSAE-CPR)	37 (43.0)	87 (5.4)

AST, aspartate aminotransferase; NSAE-CPR, clinical prediction rule for neurological sequelae due to acute encephalopathy.

Univariate and multivariate analyses

Univariate analysis revealed that all three predictive variables were significantly associated with poor outcome (RSE: P<0.0001; consciousness disturbance or hemiplegia P=0.0001; AST level >90 IU/L: P<0.0001). Multiple log-binomial analysis further revealed that RSE and level >90 IU/L within 6 hours of onset were independently associated with a poor outcome (RSE: relative risk=10.97 and P=0.0003; AST level >90 IU/L: relative risk=16.12 and P<0.0001, table 2).

Table 2

Univariate and multivariate analyses of the three predictor variables

	Good outcome n=1597	Poor outcome n=15	Relative risk (95% CI)	P value
Univariate analysis
Refractory status epilepticus, n (%)	44 (2.76)	8 (53.3)	34.29 (12.92 to 90.99)	<0.0001
Consciousness disturbance or hemiplegia at 6 hours from onset, n (%)	32 (2.00)	3 (20.0)	11.26 (3.33 to 38.15)	0.0001
AST >90 IU/L, n (%)	6 (0.38)	4 (26.7)	106.80 (46.81 to 243.69)	<0.0001
Multivariate analysis
Refractory status epilepticus			10.97 (2.97 to 40.47)	0.0003
Consciousness disturbance or hemiplegia at 6 hours from onset			1.31 (0.10 to 17.76)	0.8047
AST >90 IU/L			16.12 (4.62 to 56.26)	<0.0001

AST, aspartate aminotransferase.

ROC analysis

The ROC analyses of the NSAE-CPR and poor outcome in the derivation and validation cohort identified at least one of the three predictive variables as an optimal cut-off point. The AUC values of the derivation and validation cohorts were 0.971 (95% CI 0.913 to 1.000) and 0.915 (95% CI 0.825 to 1.000), respectively.

Screening performance

The screening performance of the NSAE-CPR in the validation cohort is shown in table 3. In the validation cohort, 13 (14.9%) of 87 patients in the high-risk group experienced poor outcomes. The sensitivity, specificity, positive and negative predictive values, positive and negative likelihood ratios and Matthews correlation coefficient were as follows: 0.867, 0.954, 0.149, 0.999, 18.704, 0.140 and 0.349 respectively.

Association between AST level and AE

Thirty-eight patients were diagnosed with AE. Of these, detailed characteristics of the 15 patients who experienced poor outcomes are included in online supplementary table 2. The pathomechanism of AE was identified as cytokine storm in four patients, excitotoxicity in six patients and undetermined in five. AST >90 IU/L within 6 hours of onset was significantly associated with AE caused by cytokine storm (sensitivity=1.000, specificity=0.996, positive predictive value=0.400, negative predictive value=1.000, positive likelihood ratio=268.000, negative likelihood ratio=0.140 and Matthews correlation coefficient=0.631, P<0.001; table 4). High-risk for poor prognosis without AST elevation was significantly associated with AE caused by excitotoxicity (sensitivity=0.667, specificity=0.948, positive predictive value=0.046, negative predictive value=0.999, positive likelihood ratio=12.900, negative likelihood ratio=0.351 and Matthews correlation coefficient=0.166, P<0.001; table 5).

Discussion

Both AE and FS are clinically diagnosed based on seizure and/or consciousness disturbance with fever. Although it remains unknown whether AE and FS represent different syndromes or phenotypes of the same spectrum disorder, evidence suggests that the latter is more likely. The present study is the first medical community-based validation study of the NSAE-CPR. Our findings indicated that all three predictive variables identified in the derivation study were also significantly associated with poor outcome in the validation cohort. In addition, although we were unable to conduct a multivariate analysis in the derivation study because of the small sample size, the results obtained using the validation cohort confirmed that two of three variables were independently associated with poor outcome.

In the present study, we observed high sensitivity and specificity of the NSAE-CPR in the validation cohort, indicating that the NSAE-CPR is suitable for use as a screening tool and for identification of patients with poor outcomes. The NSAE-CPR was unable to accurately predict prognosis for 2 (13.3%) of 15 patients in the validation cohort who experienced poor outcome, both of whom had been diagnosed with AESD. AESD is characterised by a biphasic clinical course: febrile status epilepticus early seizure) on the first day, followed by altered consciousness on the second day and clusters of brief seizures on the fourth or fifth day (late seizure). Since early seizures were brief, these two patients presented to the hospital following the appearance of late seizures, indicating that the NSAE-CPR may not be useful for patients who do not visit hospital immediately after onset.

AST elevation was observed in 6 of the 15 patients who experienced poor outcomes, 4 of whom were diagnosed with AE caused by cytokine storm. The results of our previous derivation study indicated that elevated AST level predicted this clinical situation with 100% sensitivity and specificity.9 Similarly, we also observed 100% sensitivity and 99.6% specificity in the present study, indicating that AST level >90 IU/L within 6 hours of onset can be used to accurately identify and screen for AE caused by cytokine storm in patients with poor outcomes. Previous studies have revealed that corticosteroid therapy may be effective for AE caused by cytokine storm.5 However, further studies are required to evaluate the efficacy of corticosteroid therapy in high-risk patients with AST elevation.

Six out of nine patients in the poor outcome group with AST level ≤90 IU/L received final diagnoses of AESD, which is a common type of AE in Japan. Although sensitivity was not as high as that observed in the derivation study (66.7% vs 91.7%), specificity was 94.8%, suggesting that the NSAE-CPR can be used to identify AE caused by excitotoxicity in patients with poor outcomes.

Strengths and limitations

Patients in the derivation cohort of our previous study were recruited from a tertiary centre, while those in this validation cohort were recruited from multiple centres in the Harima region of Japan. Thus, the present cohort may closely resemble the general population. However, the present study also possesses some limitations of note. Clinical symptoms such as altered consciousness may have been underestimated in some patients due to the retrospective design of the study, which may have influenced diagnosis. The present study was designed as a validation study of our previous findings.9The primary objective was to clarify whether the three clinical features associated with poor prognosis in our previous study9 are associated with poor prognosis in a medical community cohort likely to closely resemble the general population. A major limitation of our study was that several parameters associated with neurological outcomes such as symptoms of complex FS, duration of seizure and detailed treatments were not evaluated. Thus, further studies are indispensable.

Some patients with AE underwent specific forms of treatment. Pulsed steroid therapy, immunoglobulin therapy or edaravone treatment were performed in 28, 12 and 6 patients, respectively. Four patients underwent targeted temperature management. Although these treatments may have influenced outcomes, treatments were chosen ad libitum for each patient in each facility, and patients with poor outcome received intensive treatments. To obtain more precise results for the validation of the NSAE-CPR, additional prospective studies are required. In addition, clinical trials should be conducted using the same protocol to assess the effects of specific symptoms and treatments. Nonetheless, the findings of the present study indicate that the NSAE-CPR is useful for the screening and identification of patients at risk for poor AE outcomes within 6 hours of onset, which may aid clinicians in making early decisions regarding intensive treatment. Our findings further indicate that AST level >90 IU/L within 6 hours of onset can be used to identify patients at risk for poor outcomes associated with AE caused by cytokine storm.

Table 3

Screening performance for the prediction of poor outcomes

Derivation cohort (n=86)			Validation cohort (n=1612)
	Poor outcome	Good outcome		Poor outcome	Good outcome
High-risk group	16	21	High-risk group	13	74
Low-risk group	1	48	Low-risk group	2	1523

	Values	95% CI		Values	95% CI
SENS	0.941	0.713 to 0.999	SENS	0.867	0.595 to 0.983
SPEC	0.696	0.573 to 0.801	SPEC	0.954	0.942 to 0.963
PREV	0.198	0.120 to 0.298	PREV	0.009	0.005 to 0.015
PPV	0.432	0.271 to 0.605	PPV	0.149	0.082 to 0.242
NPV	0.980	0.891 to 0.999	NPV	0.999	0.995 to 1.000
LR+	3.092	2.123 to 4.504	LR+	18.704	13.881 to 25.201
LR−	0.085	0.013 to 0.570	LR−	0.140	0.038 to 0.508
MCC	0.512	0.325 to 0.561	MCC	0.349	0.246 to 0.390

LR−, negative likelihood ratio; LR+, positive likelihood ratio; MCC, Matthews correlation coefficient; NPV, negative predictive value; PPV, positive predictive value; PREV prevalence; SENS, sensitivity; SPEC, specificity.

Table 4

Association between AST level and acute encephalopathy caused by cytokine storm

Derivation cohort (n=86)			Validation cohort (n=1612)
	AE with poor outcome caused by cytokine storm	Others		AE with poor outcome caused by cytokine storm	Others
AST >90 IU/L	5	0	AST >90 IU/L	4	6
AST ≤90 IU/L	0	81	AST ≤90 IU/L	0	1602

	Values	95% CI		Values	95% CI
SENS	1.000	0.359 to 1.00	SENS	1.000	0.284 to 1.000
SPEC	1.000	0.934 to 1.00	SPEC	0.996	0.992 to 0.999
PREV	0.058	0.019 to 0.13	PREV	0.002	0.001 to 0.006
PPV	1.000	0.271 to 0.605	PPV	0.400	0.122 to 0.738
NPV	1.000	0.891 to 0.999	NPV	1.000	0.997 to 1.000
LR+	Inf	NaN–Inf	LR+	268.000	120.582 to 595.645
LR−	0.000	0.000–NaN	LR−	0.140	NaN
MCC	1.000	0.670 to 1.000	MCC	0.631	0.339 to 0.631

AE, acute encephalopathy; AST, aspartate aminotransferase; Inf, infinite; LR−, negative likelihood ratio; LR+, positive likelihood ratio; MCC, Matthews correlation coefficient; NPV, negative predictive value; NaN, not a number; PPV, positive predictive value; PREV prevalence; SENS, sensitivity; SPEC, specificity.

Table 5

Association between AST ≤90 IU/L and acute encephalopathy due to excitotoxicity in the high-risk group

Derivation cohort (n=86)			Validation cohort (n=1612)
	AE with poor outcome caused by excitotoxicity	Others		AE with poor outcome caused by excitotoxicity	Others
High-risk group with AST ≤90 IU/L	11	1	High-risk group without AST >90 IU/L	4	83
Others	1	73	Others	2	1523

	Values	95% CI		Values	95% CI
SENS	0.917	0.615 to 0.998	SENS	0.667	0.223 to 0.957
SPEC	0.986	0.927 to 1.000	SPEC	0.948	0.936 to 0.959
PREV	0.140	0.074 to 0.231	PREV	0.004	0.001 to 0.008
PPV	0.917	0.615 to 0.998	PPV	0.046	0.013 to 0.114
NPV	0.986	0.927 to 1.000	NPV	0.999	0.995 to 1.000
LR+	67.833	9.611 to 478.756	LR+	12.900	7.056 to 23.583
LR−	0.084	0.0130 to 0.552	LR−	0.351	0.113 to 1.090
MCC	0.903	0.682 to 0.973	MCC	0.166	0.067 to 0.230

AE, acute encephalopathy; AST, aspartate aminotransferase; LR−, negative likelihood ratio; LR+, positive likelihood ratio; MCC, Matthews correlation coefficient; NPV, negative predictive value; PPV, positive predictive value; PREV prevalence; SENS, sensitivity; SPEC, specificity.