Validation of central nervous system-induced seizures and other neurological variables in the Swedish Neonatal Quality Register.

AIM: We sought to validate neurological variables and relevant International Classification of Diseases, Tenth Revision (ICD-10) codes in the Swedish Neonatal Quality (SNQ) Register.
METHODS: Register data were collected for 351 neonates, born between January 2009 and December 2016, who were treated at a neonatal unit in the Stockholm region on 385 occasions. They were eligible if the check-box for central nervous system (CNS)-induced seizures was ticked. The Register data, including relevant ICD-10 codes, were validated by checking the patients' electronic medical charts.
RESULTS: Most of the neonates were born at term (76%) and weighed >2500 g (80%). The variable CNS-induced seizures had a positive predictive value of 46%. The ICD-10 diagnosis P90.9A had a positive predictive value of 90%. This comprises seizures validated with electroencephalography, amplitude-integrated electroencephalography or continuous function monitoring. The majority of the associated neurological variables in the Register had positive predictive values above 85%.
CONCLUSION: When the check-box was ticked for central nervous system-induced seizures, most of the neurological variables in the Register had high validity. However, the CNS-induced seizures variable per se had a lower positive predictive value. Future SNQ Register-based studies of such neonatal seizures should also include ICD-10 P90.9A.

amplitude‐integrated electroencephalogram

confidence interval

central nervous system

electroencephalogram

hypoxic ischaemic encephalopathy

intraventricular haemorrhage

negative predictive value

positive predictive value

Swedish Neonatal Quality Register

The Swedish Neonatal Quality (SNQ) Register may be useful for population‐based studies of central nervous system (CNS)‐induced seizure outcomes in neonates, but its neurological variables remain unvalidated.

The variable CNS‐induced seizures showed low validity, but the International Classification of Diseases – Tenth Revision (ICD‐10) diagnosis P90.9A, and several associated neurological variables, had high validity.

Future SNQ Register‐based studies of such neonatal seizures should also include the ICD‐10 diagnosis P90.9A.

BACKGROUND

The neonatal period has a higher risk for seizures than any other period of infancy, with an incidence of 1–3.5 per 1000 live births. The majority of the seizures during this period of life are acute provoked seizures. This is when there is an underlying disorder that directly affects the brain, such as a central nervous system (CNS) infection, hypoxic‐ischaemic encephalopathy (HIE) or stroke. HIE and stroke are the most common causes.2, 3, 4, 5 Importantly, both the detection of seizures and their differentiation from other abnormal movements and symptoms may be difficult in the neonatal period and the seizures may be modest or even subclinical. Neonatal seizures are known to be associated with adverse outcomes, such as intellectual disabilities, cerebral palsy and an increased risk of post‐neonatal epilepsy.6, 7, 8 Seizures are considered harmful to the brain.9, 10, 11 The risk also depends on the aetiology of the seizures.12, 13, 14, 15, 16

However, data on outcomes and risks for sequelae often originate from studies where the seizure diagnosis is based merely on clinical judgement. They also include smaller studies of neonatal cohorts at high risk for seizures carried out at tertiary centres.17, 18, 19 Therefore, there may be a knowledge gap regarding outcomes following neonatal seizures, verified by an electroencephalogram (EEG), as well as about which anti‐seizure medications are optimal in acute settings. Healthcare quality registers may be of great use for population‐based outcome studies, provided that the register has high validity.

The Swedish Neonatal Quality (SNQ) Register has been used in clinical practice in all neonatal units in Sweden since 2012 and provides a great deal of information about neonatal hospitalisations. These include seizures and EEG and amplitude‐integrated EEG (aEEG) results. This makes the SNQ Register a potentially important tool for performing population‐based studies of outcomes following CNS‐induced seizures in the neonatal period. However, the neurological variables in the SNQ have not yet been validated. The aim of this study was to validate the CNS‐induced seizures variable and other neurological variables of importance in the Register.

MATERIALS AND METHODS

Study population

We used the SNQ Register to identify all children born between January 2009 and December 2016, who were admitted to a neonatal ward in the Stockholm region. They were included if the check‐box for CNS‐induced seizures had been ticked in the Register. Details of each hospitalisation for these newborn infants were obtained from the Register, together with background data and variables of interest. The variables were validated by comparing the material from the Register with the electronic medical charts, including neurophysiology and radiology reports. Two authors (HW and HMW) extracted the data obtained from the Register and from the electronic medical charts using a structured extraction sheet. To ensure interobserver agreement, the authors discussed all cases if there were questions about interpretation, to achieve consensus. Information from the electronic medical charts was compared with the information in the Register for the variables included.

SNQ Register and variables

The SNQ Register has been used in clinical practice since 2001 and all 37 neonatal units in Sweden have used it since January 2012. The Register includes infants admitted for neonatal care within 27 days of birth. Data on stillborn infants and on newborn infants treated in maternity units, paediatric units or paediatric intensive care units are not included in the Register. The Register provides information such as delivery mode, sex, gestational age, birth weight, Apgar scores, infections, hyperbilirubinaemia, hypoglycaemia, treatments and different neurological variables. The data are added to the Register by clinicians at admission, during hospitalisation and at discharge. The Register also contains information about the mother, including her age and epilepsy diagnoses, if applicable. Information about anti‐seizure medication given during hospitalisation has been available in the Register since 2009 as it provides space to add details of these in the free text. We obtained neurological variables of interest for neonates registered in the Register if the CNS‐induced seizures check‐box had been ticked. For the variables in the Register, we validated CNS‐induced seizures as well as the following ICD‐10 diagnoses: P90.9A (seizures with EEG, aEEG or continuous function monitoring verification), P90.9B (seizures without EEG or aEEG verification) and P90.9C (subtle seizures with EEG, aEEG or continuous function monitoring, but not clinical verification). We also validated CNS haemorrhages other than intraventricular haemorrhage (IVH) and CNS infarction. Moreover, we evaluated whether EEG registration or aEEG registration was performed, as well as the relevant results. Cystic periventricular leukomalacia on the left and right side, hypoxic‐ischemic encephalopathy (HIE) grades 2‒3, and IVH grade on the left and right side were assessed. We validated anti‐seizure medication treatment during hospitalisation, and at discharge, and hypothermia treatment.

We only investigated the validity of neurological variables and not infection‐related variables. To validate the CNS‐induced seizures variable, we searched electronic medical charts for interictal epileptic activity and/or seizure activity in the EEG reports from the neurophysiologist and the aEEG results from daily notes and/or in the discharge chart. Positive cases were classified as those with CNS‐induced seizures, regardless of the presence or absence of clinical symptoms.

For the ICD‐10 P90 diagnoses, medical chart information concerning clinical seizures and EEG and/or aEEG registration results were combined to validate the diagnoses. For the EEG and aEEG variables, we defined seizure activity, interictal epileptiform activity, burst suppression and/or other abnormalities as pathological. A CNS haemorrhage was defined as parenchymal bleeding, excluding subarachnoid, subdural and epidural bleeding. We defined CNS infarction with a combination of ischaemic and haemorrhagic components as CNS haemorrhage and CNS infarction. An IVH was not regarded as CNS bleeding, in accordance with the design of the SNQ Register. The variable HIE grades 2‒3 were based on the Sarnat score.

In terms of treatment, a ketogenic diet, vitamin B6 and other metabolic treatments were not considered as anti‐seizure medication.

The missing SNQ Register data were analysed using the same definitions as above. Missing data were investigated to determine if this was indicative of a negative or positive result or if the missing data were randomly distributed. Missing data were excluded from the validation process. However, we analysed whether the missing data affected the results if they were included.

Statistical analyses

The positive predictive value (PPV) and negative predictive value (NPV) were calculated and 95% confidence intervals (95% CI) were obtained using the MedCalc Software (MedCalc Software Ltd). The cut‐off level was set to 85% prior to study initiation, with validation results below this value considered to be suboptimal for use in future research. For anti‐seizure medication treatment during hospitalisation, the level of coherence between the SNQ and electronic medical charts was categorised into four groups: 100% coherence on treatments, ≥50% coherence, <50% coherence or 0% coherence. The NPV for the CNS‐induced seizures variable could not be calculated, because ticking this checkbox in the SNQ Register was a pre‐requisite for defining the cohort. As all cases in the Stockholm area were studied, no power calculation was performed prior to the initiation of the study.

Ethical approval

This project was approved by the Stockholm Ethics Review Board. Due to the nature of the study, no informed parental consent was required.

RESULTS

We collected data on 391 hospitalisations from the SNQ, but charts for 6 children were excluded because the electronic medical charts could not be found, including 3 who were registered as deceased. Consequently, 385 hospitalisations of 351 patients were included in the study: 324 had 1 hospital admission, 20 had 2 admissions and 7 had 3 hospital admissions. There were 48 deaths during the neonatal period. Birth weight data were missing for 3 hospitalisations. Most of the included neonates were born at term (76%) and most had a birth weight of >2500 g (80%). A description of the cohort is presented in Table 1.

TABLE 1

Demographic characteristics of 351 neonates in the cohort

Demographic data	n (%)
Sex
Male	187 (53.3)
Female	164 (46.7)
Delivery mode
Vaginal	129 (36.8)
Caesarean section	151 (43.0)
Instrumental	71 (20.2)
Gestational age (weeks)
23‒27	29 (8.3)
28‒31	11 (3.1)
32‒36	43 (12.2)
37+	268 (76.4)
Birth weight (g)
<1000	28 (8.0)
1000‒2499	41 (11.8)
2500‒3499	130 (37.4)
3500‒4499	124 (35.6)
>4500	25 (7.2)
Apgar score <5
At 1 min (n = 346)	172 (49.7)
At 5 min (n = 347)	120 (34.6)
At 10 min (n = 347)	77 (22.2)

Agreement between SNQ and medical charts

Central nervous system‐induced seizures were verified in 384 of the 385 hospitalisations and the ICD‐10 diagnoses were P90.9A, P90.9B and P90.9C, as detailed in the Methods section. The remaining case was excluded, as there was no information on EEG or aEEG results in the electronic medical chart. The PPV was 46% for the CNS‐induced seizure variable. The level of agreement for the ICD‐10 diagnoses, when the CNS‐induced seizures variable box was ticked in the Register, showed a PPV of almost 90% for a diagnosis of P90.9A. The ICD‐10 diagnoses P90.9B and P90.9C had PPVs of 51% and 29%, respectively, and higher NPVs than P90.9A (Table 2).

TABLE 2

Validation results for the variable CNS‐induced seizures and ICD‐10 diagnoses in the SNQ Register and information obtained from the electronic medical charts after the missing data were excluded

SNQ variable/diagnosis	SNQ	Electronic medical chart		PPV (95% CI; %)	NPV (95% CI; %)
		Yes	No
CNS‐induced seizures	Yes	177	207	46.1	‒
	No	0	0
ICD‐10 P90.9A	Yes	176	20	89.8 (85.4‒93.0)	59.0 (54.1‒63.8)
	No	77	111
ICD‐10 P90.9B	Yes	54	51	51.4 (43.9‒58.8)	87.8 (84.6‒90.4)
	No	34	245
ICD‐10 P90.9C	Yes	2	5	28.6 (8.1‒64.5)	97.9 (97.1‒98.4)
	No	8	369

Abbreviations: CI, confidence interval; ICD‐10 P90.9A, neonatal seizures with EEG, aEEG, or continuous function monitoring verification; ICD‐10 P90.9B, seizures without EEG or aEEG verification; ICD‐10 P90.9C, subtle seizures with EEG/aEEG or continuous function monitoring, but not clinical, verification; NPV, negative predictive value; PPV, positive predictive value; SNQ, Swedish Neonatal Quality register.

Most of the associated neurological SNQ variables had PPVs and NPVs above 85%. The exceptions were CNS haemorrhage (56%), CNS infarction (80%) and anti‐seizure medication at discharge (73%; Table 3).

TABLE 3

Validation results for neurological variables in the SNQ Register and information obtained from the electronic medical charts after exclusion of missing data

SNQ variable	SNQ	Electronic medical chart		PPV (95% CI; %)	NPV (95% CI; %)
		Yes/pathological	No/Normal
EEG/aEEG performed	Yes	315	1	99.7 (98.0‒99.9)	9.7 (6.9‒13.9)
	No	56	6
EEG result pathological	Yes	146	2	98.6 (95.0‒99.6)	48.1 (39.2‒57.1)
	No	27	25
aEEG result pathological	Yes	143	17	89.4 (85.1‒92.5)	64.6 (52.6‒75.0)
	No	17	31
CNS haemorrhage	Yes	14	11	56.0 (38.6‒72.0)	94.6 (92.8‒95.9)
	No	19	331
CNS infarction	Yes	40	10	80.0 (67.7‒88.4)	88.8 (86.2‒91.0)
	No	36	287
cPVL left	Yes	0	0	‒	97.3 (97.3‒97.3)
	No	10	367
cPVL right	Yes	0	0	‒	97.6 (97.6‒97.6)
	No	9	368
HIE grades 2‐3	Yes	101	6	94.4 (88.4‒97.4)	88.1 (84.7‒90.9)
	No	33	245
IVH grade left	Yes	19	1	95.0 (73.9‒99.2)	88.9 (68.0‒96.8)
	No	2	16
IVH grade right	Yes	19	0	100	94.7 (72.7‒99.2)
	No	1	18
Anti‐seizure medication at discharge	Yes	98	37	72.6 (66.1‒78.3)	94.4 (91.1‒96.5)
	No	14	235
Hypothermia treatment	Yes	76	10	88.4 (83.6‒91.9)	100
	No	0	7

Abbreviations: aEEG, amplitude‐integrated electroencephalogram; CI, confidence interval; CNS, central nervous system; cPVL, cystic periventricular leukomalacia; EEG, electroencephalogram; HIE, hypoxic‐ischemic encephalopathy; IVH, intraventricular haemorrhage; NPV, negative predictive value; PPV, positive predictive value; SNQ, Swedish Neonatal Quality register.

We carried out subgroup analyses for the 301 term‐born infants, defined as ≥37+0 weeks of gestation, and the 84 preterm infants born before that gestational age. This revealed significantly higher PPVs for ICD‐10 diagnosis P90.9C, CNS haemorrhage, and IVH grade left versus right in the preterm group. In contrast, higher PPVs were observed in the term‐born group for aEEG results and CNS infarction. These results are summarised in Tables S1‐S4.

Phenobarbital was the most frequently used anti‐seizure medication, as it was used 344 times during hospitalisation. This was followed by midazolam (n = 167), lidocaine (n = 41), levetiracetam (n = 21), phenytoin (n = 9), clonazepam (n = 8), diazepam (n = 4), topiramate (n = 3), pentobarbital (n = 1) and tiopenthal (n = 1). One child was treated with a ketogenic diet and 18 received vitamin B6. Full agreement between the SNQ Register and the medical charts for anti‐seizure medication treatment during hospital stay was found in 67% of cases, whereas no agreement was seen in 13% of cases (Figure 1). In all cases where full agreement didn't exist, the medical chart always contained more anti‐seizure medications than the SNQ.

FIGURE 1

Coherence between the SNQ Register and the electronic medical charts for anti‐seizure medication in the acute setting

Missing data

Our analysis of the 385 hospitalisations revealed that there were no missing data for the variables CNS haemorrhage, CNS infarction, EEG/aEEG registration, cystic periventricular leukomalacia left or right or HIE grades 2‒3. For the variables IVH grade left, IVH grade right, anti‐seizure medication at discharge and hypothermia treatment, missing data corresponded to a normal/negative result in more than 94% of cases. In contrast, when it came to the EEG and aEEG results, missing data in the SNQ Register corresponded to a pathological result in 71% and 53% of hospitalisations, respectively (Table 4). When the missing data were included, the validity for the variables did not affect the PPVs, but it did increase the NPVs for the EEG results, aEEG results and IVH grade left. These results are shown in Table S5.

TABLE 4

Missing data for neurological variables in the SNQ Register compared with information obtained from the electronic medical charts

SNQ variable	Missing data (MD) total (%)	MD in SNQ and yes/pathological in the electronic medical chart (% of MD total)	MD in SNQ and no/normal in the electronic medical chart (% of MD total)	MD in SNQ and data not available in the electronic medical chart (% of MD total)
IVH grade left	344 (89.3)	8 (2.3)	328 (95.3)	8 (2.3)
IVH grade right	344 (89.3)	8 (2.3)	328 (95.3)	8 (2.3)
EEG result	160 (41.6)	113 (70.6)	31 (19.4)	16 (10.0)
aEEG result	159 (41.3)	84 (52.8)	43 (27.0)	32 (20.1)
Anti‐seizure medication at discharge	1 (0.3)	0	1 (100)	0
Hypothermia treatment	292 (75.8)	17 (5.8)	275 (94.2)	0

Abbreviations: aEEG, amplitude‐integrated electroencephalogram; EEG, electroencephalogram; IVH, intraventricular haemorrhage; SNQ, Swedish Neonatal Quality register.

DISCUSSION

This validation study of neurological variables in the SNQ found high validity for the majority of the variables studied, with PPVs above 85%. However, there were some important exceptions. A clear definition of appropriate cut‐off levels has been lacking in validation studies, but PPVs above 85% have previously been used and were chosen from the start in the present study. The CNS‐induced seizures variable had markedly lower validity than this cut‐off, with a PPV of only 46%. The NPV could not be calculated, as the presence of CNS‐induced seizures, according to the SNQ Register check‐box, was an inclusion criterion in this study. This variable alone was therefore unsuitable for defining a cohort of neonates with true CNS‐induced seizures, that is with seizures with EEG correlates. In contrast, the ICD‐10 diagnosis P90.9A, which was seizures with EEG, aEEG or continuous function monitoring, was highly valid, with a PPV of almost 90%. This validity was obtained under the pre‐requisite that the CNS‐induced seizures check‐box was marked in the SNQ Register. It demonstrated that this diagnosis effectively identified neonates with clinical symptoms of seizures and with corresponding EEG correlates in the Register. However, electroencephalographic, subclinical seizures would not be identified by definition, and the diagnosis of P90.9C, which covered subclinical seizures, had a very low validity, with a PPV of 28%. The diagnosis of P90.9B, which was clinical seizures without EEG or aEEG verification, also had a low PPV of 51%. Conversely, the NPV for the diagnosis P90.9A was low, at 59%, and a negative value in the SNQ would miss a substantial number of CNS‐induced seizures. This has implications for using this variable in studies with a population‐based approach, as it did not identify all cases.

Although most of the neurological SNQ Register variables in this study had a high validity, with PPVs above 85%, intracranial haemorrhage and CNS infarction had a low validity in the SNQ. It is possible that the lack of guidance on how to report subarachnoid bleeding or ischaemic infarction with haemorrhage led to differences in registration. The importance of coherence in interpretation was also illustrated in the validity of the EEG and aEEG results, which had high PPVs, but low NPVs. These were below 50% for the EEG results. We defined a pathological result as not only seizure activity, but also interictal epileptiform activity, burst suppression or other abnormalities that may differ from a clinician's interpretation. This led to lower validity for these variables. Adding guidance on the interpretation of individual variables would probably increase the validity of these variables. For example, an instruction sheet on the SNQ Register website or an appendix would be helpful in this regard.

However, differences in interpretation were not the only cause of the low validity of a variable in the Register. This was illustrated by the low validity of the binary variable anti‐seizure medication treatment at discharge, which had a PPV of only 72%. In general, since the PPV is dependent on the prevalence of the factor studied in a given population, this might have interfered with the PPV results of the variables investigated in this study. In the SNQ, data concerning anti‐seizure medication treatment during hospitalisation were in complete agreement with the electronic medical charts in two‐thirds of all cases. They were also above 50% in over 80% of the hospitalisations. Non‐coherence was, in all cases, due to missing information in the SNQ Register and there were no cases of over‐reporting anti‐seizure medication use in the Register. Of note, coherence below 50% was very rare. Therefore, when information regarding anti‐seizure medication treatment during hospitalisation was entered into the SNQ, namely if missing data were omitted, the agreement was high, although not complete.

The variables IVH left and IVH right had the highest number of missing data among all the variables. In more than 95% of cases, a missing value corresponded to a normal result in the electronic medical chart. The assumption that a missing value corresponded to a normal result was therefore adequate for the IVH left and right variables. However, the very low use of these variables in the SNQ Register was problematic, because it could provide important information on the aetiology of seizures. In contrast, more than 40% of the data for the EEG and aEEG results were missing, but the results for these variables were more unpredictable. Missing data corresponded to a pathological result in 70% of the EEG result and in 52% of the aEEG result variables. This indicated a more random distribution and may have been due to the design of the Register, as the check‐boxes for these two variables were located under Investigation of children with moderate to severe asphyxia/HIE. It is possible that the EEG and aEEG check‐boxes might have been missed if the newborn infant did not meet the asphyxia or HIE criteria. Although this would not have interfered with the PPV and NPV for the variables, it would have affected the sensitivity and specificity.

No previous study has validated the SNQ Register and its neurological variables. The Register has high coverage and demonstrates completeness, as shown by Norman et al, although it should be noted that previous data mainly concerned premature neonates. However, the present study indicated the usefulness of the SNQ Register for population‐based cohort studies. Nevertheless, further validation of the Register variables is needed.

Study limitations

One limitation of the current study was that all cases came from the Stockholm region and we cannot exclude the fact that there would be regional differences in the coverage or interpretation of individual variables. Thus, the external validity of our findings may be limited. However, the high validity of most of the variables in the present study indicate that the SNQ Register may be used for future research studies on neurological diseases and outcomes in neonates, if subjects are recorded as having CNS‐induced seizures. These could include post‐neonatal epilepsy, cerebral palsy and intellectual disabilities. As we sought to specifically validate the SNQ Register for its possible usefulness in future studies of outcomes following CNS‐induced seizures in neonates, we focused on variables relevant to that area. Somewhat surprisingly, the validity of the Register variable of CNS‐induced seizures was too low to be used as a sole variable for defining cohorts in future register and population‐based studies. In contrast, the ICD‐10 diagnosis of P90.9A, namely seizures with EEG, aEEG, or continuous function monitoring verification, had high validity in our cohort.

CONCLUSION

The CNS‐induced seizures variable in the SNQ had low validity for EEG verified seizures in neonates. However, when we combined this variable in the SNQ Register with the ICD‐10 diagnosis code P90.9A, the Register was able to identify newborn infants with clinical neonatal seizures of CNS origin. Moreover, our data showed that, with some exceptions, most neurological variables in the SNQ Register had high validity.

CONFLICT OF INTEREST

The authors have no conflict of interest to declare.

Table S1‐5

Click here for additional data file.