The utility of an automated and ambulatory device for detecting and differentiating epileptic and psychogenic non-epileptic seizures.

OBJECTIVE: Accurate differentiation between epileptic seizures (ES) and psychogenic non-epileptic seizures (PNES) can be challenging based on history alone. Inpatient video EEG monitoring (VEM) is often needed for a definitive diagnosis. However, VEM is highly resource intensive, is of limited availability, and cannot be undertaken over long periods. Previous research has shown that time-frequency analysis of accelerometer data could be utilized to differentiate between ES and PNES. Using a seizure detection and classification algorithm, we sought to examine the diagnostic utility of an automated analysis with an ambulatory accelerometer.
METHODS: A wrist-worn device was used to collect accelerometer data from patients during VEM admission, for diagnostic evaluation of convulsive seizures. An automated process, that involved the use of K-means clustering and support vector machines, was used to detect and classify each seizure as ES or PNES. The results were compared with VEM diagnoses determined by epileptologists blinded to the accelerometer data.
RESULTS: Twenty-four convulsive seizures, consisting of at least 20 seconds of sustained continuous activity, recorded from 11 patients during inpatient VEM (13 PNES from five patients and 11 ES from six patients) were included for analysis. The automated system detected all convulsive seizures (ES, PNES) from >661 hours of recording with 67 false alarms (2.4 per 24 hours). The sensitivity and specificity for classifying ES from PNES were 72.7% and 100%, respectively. The positive and negative predictive values for classifying PNES were 81.3% and 100%, respectively. There was no significant difference between the classification results obtained from the automation process and the VEM diagnoses. SIGNIFICANCE: This automated system can potentially provide a wearable out-of-hospital seizure diagnostic monitoring system.

An automated seizure detection and classification algorithm demonstrated an overall sensitivity of 100% and specificity of 72.7%, for classifying PNES.

PPV and NPV for PNES were found to be 81.3% and 100%, respectively.

McNemar's test suggests that there was no significant difference between the classification results obtained from the automation process and the VEM diagnoses.

These results demonstrate the potential utility of this automated, ambulatory, non‐invasive wearable device in differentiating between ES and PNES.

INTRODUCTION

Epileptic seizures (ES) are caused by abnormal oscillatory discharges of cortical neurons resulting in behavioral changes and/or an altered state of consciousness. In contrast, psychogenic non‐epileptic seizures (PNES) are involuntary behavioral events that occur in the absence of electroencephalographic (EEG) abnormalities.1, 2 PNES are generally considered to be physical symptoms of an underlying psychological disturbance that may be triggered by stress‐related or emotional events.3

Despite the fundamental differences in their pathophysiology, PNES often behaviorally mimic ES, making it challenging to differentiate them based on patient or witness descriptions in the outpatient setting.4 It has been reported that PNES is mistaken for and inappropriately treated as epilepsy in up to 30% of patients.5, 6 There is a potential for significant harm from the adverse side effects or teratogenicity of anti‐epileptic drugs (AEDs) inappropriately prescribed to patients with PNES, as well as morbidity and mortality from intubation and ICU admission for prolonged seizures.7 An inaccurate diagnosis may also result in delayed psychological treatment for the particular issues underlying the psychogenic seizures. Previous studies have indicated that due to these contributing factors, patients with PNES are not correctly diagnosed for an average of 5.6 years after the initial manifestations of their seizures.8

Inpatient video EEG monitoring (VEM) is often required to differentiate between PNES and ES.9 Diagnostic VEM is generally performed over several days or weeks and involves concurrent EEG and video recording of seizure behavior. Although the VEM is regarded as the “gold standard” in the differentiation of ES from PNES, its availability is limited, it is resource intensive, and it is inconvenient for patients.10 Furthermore, it may not be representative of a patient's normal, ambulatory environment and is impractical for patients with infrequent events.11

The US Food and Drug administration (FDA) has cleared the Embrace smart watch (Empatica inc) for seizure tracking and epilepsy management. The Brain Sentinel EMG device is another FDA‐approved device for monitoring convulsive seizures.12 Apart from these devices, there are no other commercially available continuously recording or reporting seizure detection tools to provide clinicians and clinical trialists with a true measure of the nature and frequency of a patient's seizures. Conventional seizure recording relies on patient and/or carer self‐reporting diaries which are inconsistent and significantly under‐report seizure occurrence.4 Alternative monitoring methods are needed that balance the level of invasiveness with economic feasibility and accuracy for prolonged use in ambulatory patients.

The use of limb‐worn accelerometry sensors that can record movements is one such avenue that is being explored by a number of groups.13, 14, 15 Our previous research has shown that the evolutionary pattern of the frequencies of rhythmic movement artifacts on EEG during PNES differs from that of ES.16 Convulsive PNES were demonstrated to display a characteristic pattern of rhythmic movement artifact that remains stable over time during the event, whereas the EEG activity during convulsive ES tends to evolve throughout.16 This finding was then applied to examine the utility of time‐frequency mapping of data from an accelerometer (continuous movement‐recording device) worn on wrists.17 The results of that study indicated that time‐frequency analysis of data from a wristband movement monitor has the potential to be utilized as a diagnostic tool to differentiate between ES and PNES with a sensitivity and specificity of 92.7% and 75.0%, respectively, and furthermore may be suitable to incorporate into a device for outpatient ambulatory monitoring.

This particular study extends this work by investigating the clinical utility of a wrist‐worn device that incorporates an automated algorithm to detect and classify seizures as PNES or ES, based on the time‐frequency mapping patterns. The automation in particular is what this study adds to existing published material as it enables “real‐time” diagnosis. This therefore has potential to be incorporated into seizure alert systems which would have great safety value for patients, particularly those who live alone, are at risk from injury during convulsive seizures, or having nocturnal seizures that are usually unwitnessed. The algorithm incorporated into the automated device has been developed by our group previously, using data acquired from motor seizures, both epileptic and psychogenic.18 This study aims to evaluate the accuracy of this automated and ambulatory diagnostic system, and validate its ability in differentiating PNES and ES in an independent cohort of patients.

METHODS

Participants

This was a prospective, observational study. Patients admitted for VEM for the investigation of possible epilepsy were eligible for inclusion. Patients were eligible for inclusion if they experienced one of their typical clinical events of at least 20 seconds (s) in duration in which there was sustained, rhythmic or arrhythmic movements affecting at least one limb. This included patients with purely tonic or hypermotor movements. Patients experiencing solely non‐convulsive seizures were excluded. All included patients provided written informed consent. The study was approved by the Human Research Ethics Committee of the Royal Melbourne Hospital (HREC Project 300.259).

Accelerometer data collection

The device utilized was an Apple iPod Touch (4th generation), with an in‐built micro‐electro mechanical system (MEMS) accelerometer. The MEMS accelerometer utilized had a full scale of ± 2.5 g, sampling at a frequency of 50 Hz, and recording the motion data on three axes (x, y, and z) along with a timestamp. The accelerometer was affixed to the patient's wrist for the duration of VEM. In our center, patients were typically admitted from Monday to Friday each week. The time on the devices was synchronized with the VEM computers to ensure exact comparison and analysis. Movement data were continuously recorded during this period other than when devices were changed over due to battery drainage (every 24 hours). A smart application was uploaded to the iPod Touch. This program was designed to continuously record patient movements, sampling at a frequency of 50 Hz. The devices were securely affixed by an elastic sweatband, to the posterior aspect of the forearm, as close to the wrist as the possible, with the iPod Touch facing up (Figure 1). Raw accelerometer data were stored using flash memory on the iPod Touch and later transferred to a computer for visualization. The motion data captured on each axis were combined and further analyzed.

Figure 1

Patient recruited in the VEM unit of Royal Melbourne Hospital. The red boxes display the iPods used for 3D accelerometry data collection

VEM diagnosis of seizures as epileptic or PNES

Each event captured by the device was determined as either ES or PNES by a team of at least three neurologists, who are qualified as consultants in the clinical department of neurology. A thorough review of their EEG recordings, clinical semiology as seen on video recordings, clinical history, neuropsychiatry, and neurology evaluation (including imaging such as MRI), was done on each patient during the VEM meeting. Following this, each independent neurologist would express their opinion on whether the event would be classified as ES or PNES. The group consensus review was determined by at least two out of three members of the team agreeing, at the minimum. In a case of disagreement or uncertainty, the decision was made to review the patient again during another VEM admission. These patients were not to be included in this study. However, such a case did not occur. The VEM diagnoses were made blinded to the classification results of these events made by the automated device, so as to eliminate any possible bias.

Visualizing accelerometer data using MATLAB

MATLAB was used to visualize the accelerometer data. Using the timestamps of every seizure, each event was visualized on MATLAB by an epileptologist to validate the accelerometer's ability to record motion data accurately. The visualization of each event that was captured on the accelerometer was done prior to the automated detection and classification. Therefore, if an event was not detected by the algorithm, the reason was attributed to the algorithm itself rather than the accelerometer not recording the event. An ES and PNES events, as visualized on MATLAB, are shown in Figures 2 and 3, respectively.

Figure 2

Time‐frequency plot of a typical epileptic event. Line 1: Frequency‐time Map: This demonstrates the frequency at which the limb (left in this figure) oscillates for the first 50 s after the start time entered into Main 7. Line 2: Acceleration‐time map for the 88 s following the start time. Line 3: Acceleration shown in 2.5‐s epochs, starting at different time points through the 88 s. Line 4: Power‐spectrum distribution. This displays how the frequency‐distribution varied in the corresponding 2.5‐s epochs shown in line 3

Figure 3

Time‐frequency plot of a typical psychogenic non‐epileptic event. Line 1: Frequency‐time map: This demonstrates the frequency at which the limb (left in this figure) oscillates for the first 50 s after the start time. Line 2: Acceleration‐time map for the 88 s following the start time. Line 3: Acceleration shown in 2.5‐second epochs, starting at different time points through the 88 s. Line 4: Power‐spectrum distribution. This displays how the frequency‐distribution varied in the corresponding 2.5‐second epochs shown in line 3

Inclusion criteria for seizure analysis

At the primary stage of the automated process of event detection, a resultant signal was calculated for all the data points across the x, y, and z axes which are calculated using This result was then filtered, where all signals <0.2 g (acceleration of movement) are considered as no activity or normal activity. 0.2 g represents the force required, where 0.2 is the mass and g is the acceleration of gravity (9.8 m/s2). The value of 0.2g was empirically chosen based on the lower bound of the collected seizure data. A fast Fourier transform (FFT) was done on these filtered signals and was calculated by dividing each 20‐second window into 20 blocks of 1 second each. These filtered signals were then pre‐processed. Further details regarding this process can be seen in Gubbi et al's study.18 The pre‐processing ensured that all subtle movements were excluded from any seizure‐like activity. Subsequently, a time filter was utilized, where any continuous movement data less than 20 seconds were eliminated. The 20‐seconds value was utilized as a cutoff during the development of the algorithm, based on feedback from clinical neurologists. Due to this, during this clinical study, any events that were <20 seconds had to be excluded. What is left at this moment are seizure‐like activity and persisting normal activity. Several time domain features were extracted from these signals, including slope sign changes, waveform length, and variance. Of these, signal power, zero crossing, and standard deviation were selected as key features.18

Using these key features, K‐means clustering was then used to partition the data into a large group of normal events and a small group of seizure‐like events, and the latter of which were formally detected as “seizures.”

Automated classification of seizure events

The second stage involves the use of discrete wavelet transforms and support vector machines to classify the detected seizures. The wavelet transform was performed on the Euclidian transformed single movement data stream. Support vector machines are a class of trained models used in data analysis and pattern recognition for classification. The algorithm itself was trained using a training data set. This data set was collected between 2012 and 2013. This study has utilized this algorithm in a clinical setting, on a separate data set, which has not been done previously. The training model was validated by a fivefold cross‐validation. Through these means, each seizure was then classified as either ES or PNES which appeared as the automated result during analysis on MATLAB. Figure 4 shows the flowchart of the automated detection and classification system used in the clinical setting. Details regarding the detection and classification process of the events captured, can be seen in Appendix S1.

Figure 4

Flowchart of the automated detection and classification system used in the clinical setting

Statistical analyses

The automated output from the accelerometer data for each seizure was compared with the corresponding VEM diagnosis. The variables being measured are categorical in nature; the results are presented in two‐way contingency tables of frequencies. In order to assess the validity of the automated device as a possible diagnostic tool, the overall sensitivity, specificity, positive, and negative predictive values were calculated for both ES and PNES. The outcomes were categorized as “ES” and “not ES,” where undetected and falsely diagnosed ES events were included in the latter group. The same analysis was performed for PNES events. Sub‐analyses for the automated device's ability to accurately classify the detected seizures were also conducted. From these results, the positive and negative likelihood ratios were calculated. The positive likelihood ratio (LR+) was defined as the ratio of the chance of a PNES result if the individual has PNES, to that of if the individual does not have PNES and receives a PNES result. It was calculated as sensitivity/(1 – specificity). The negative likelihood ratio (LR‐) was defined as the ratio of the chance of an ES result if the individual has PNES, to that of if the individual does not have PNES and correctly receives an ES result. It was calculated as (1‐sensitivity)/specificity. Statistical analyses were performed using Microsoft® Excel® (Microsoft Corp). McNemar's test was performed to determine whether there was a systematic difference between the VEM and automated results by using a significant level of 0.05. For the purpose of this study, a systematic difference was defined as a significant difference between the VEM and automated results, being attributed to the methodology of the VEM and automation, rather than due to random error in either system. Furthermore, an absence of a systematic difference implies that there is no bias toward a particular result in the automated process, when compared to the VEM.

RESULTS

A total of 26 patients were enrolled in the study. Among them, 11 experienced at least one convulsive seizure while having the accelerometers affixed to their wrists and were included in the analyses. Of the remaining 15 patients, 9 had no events during the VEM admission, 5 had only non‐convulsive events, and 1 had atypical events involving head shaking and no limb movements. Table 1 presents the clinical characteristics of the patients included in the analysis.

Table 1

Demographic and clinical characteristics of the patient cohort

	PNES	ES
Median age, years ± SD	20 ± 6.58	24 ± 6.59
Male (number, percentage)	4, 30.8%	6, 54.5%
Female (number, percentage)	9, 69.2%	5, 45.5%
Number of convulsive seizures included in analysis	13	11
Seizure type
Bilateral tonic‐clonic seizure at onset	N/A	8
Focal onset evolving to bilateral tonic‐clonic seizure	N/A	3
Generalized onset seizure	13	0

ES = 11; PNES = 13.

Abbreviation: N/A, not applicable.

Seizures selected for analyses

Of the 11 patients included, 5 (41.7%) experienced PNES and 6 (42.9%) experienced ES. A total of 23 ES (range, 4‐7 per patient) and 33 PNES events (range, 5‐8 per patient) were recorded from these patients. No patient had both ES and PNES. Of the 23 ES, 11 were included in the analysis and 12 were excluded (4 were excluded because the acceleration of movement was <0.2 g and 8 were <20 seconds in duration). Of the 33 PNES, 13 were included in the analyses and 20 were excluded (all due to duration <20 seconds). The mean duration (±standard deviation) of the 11 ES was 61.2 ± 20.6 seconds and for the 13 PNES was 86.5 ± 26.5 seconds (P = 0.23, Student's t test). All the seizures included in the analyses were visualized on MATLAB via the time stamp.

In addition, we have also performed analyses using a less stringent criterion where we have decreased the minimum seizure duration threshold from 20 to 5 seconds. Of the 23 ES events, 4 were still excluded due to the acceleration of movement being <0.2 g. The remaining 19 ES events could be included in the analyses as the 8 that were excluded with the 20‐seconds threshold were now included. However, 3 of the 8 ES events that are now included were still undetected by the algorithm as seizure activity. Of the 33 PNES events, all were detected as seizure activity and included in the analyses as all were >5 seconds in duration.

Validity of the automated device in event classification

The classification results by the automated device compared to the VEM diagnosis are presented in a two‐way contingency table (Table 2). All 13 PNES events from five patients were correctly classified as PNES by the automated classification (100% sensitivity in classifying PNES). Of the 11 detected epileptic events, 8 were correctly classified as epileptic while 3 were incorrectly classified as PNES (72.7% sensitivity for ES). PPV and NPV for PNES were found to be 81.3% and 100%, respectively. Specific to the classification results, McNemar's test produced an exact P‐value of 0.25. The positive and negative likelihood ratios in classifying PNES were found to be 3.67 and 0, respectively. As there were no patients in this study with both epilepsy and PNES, the likelihood ratios were not calculated for such cases and are thus not applicable to them. Figures 5 and 6 display ROC curves for the detection and classification of ES and PNES.

Table 2

Two‐way contingency table of the classification result of ES or PNES made by automation compared to the corresponding VEM result

Automation	VEM
	PNES	ES	Total
PNES	13	3	16
ES	0	8	8
Total	13	11	24

ES = 11, PNES = 13, VEM = 24

Figure 5

ROC curve using a 20‐s threshold The receiver operating characteristics curve (ROC) of the proposed system. The performance is in terms of area under the ROC curve (AUC). The red curve represents the performance of the seizure (ES and PNES) detection stage I. The blue curve shows the performance of the seizure classification stage II

Figure 6

ROC curve using a 5‐s threshold. The receiver operating characteristics curve (ROC) of the proposed system. The performance is in terms of area under the ROC curve (AUC). The red curve represents the performance of the seizure (ES and PNES) detection stage I. The blue curve shows the performance of the seizure classification stage II

When the time threshold was decreased to 5s, the algorithm correctly classified 12 of the 19 ES events (63.2% sensitivity for ES) and 28 of the 33 PNES (84.8% sensitivity for PNES). A summarized comparison of results with the 20 and 5‐seconds time threshold is shown in Table 3.

Table 3

Comparison of detection and classification results utilizing a 20 s and 5 s threshold

Event Characteristics	Time threshold
	20 s	5 s
ES events > Time threshold	11	23
ES events detected	11 (100%)	16 (69.6%)
Sensitivity for ES classification	72.70%	63.20%
PNES events > Time threshold	13	33
PNES events detected	13 (100%)	33 (100%)
Sensitivity for PNES classification	100%	84.80%
False alarm rate	2.43/d (67 false alarms in 661.58 h)	5.44/d (150 false alarms in 661.58 h)

DISCUSSION

The results of this study demonstrated that the automated, wearable, seizure detection system had a high level of sensitivity (100%) for non‐epileptic events that fulfilled the inclusion criteria, that is, continuous rhythmic arm movement lasting at least 20 seconds in duration and having a force of >0.2 g. It also had a high diagnostic specificity for PNES convulsive events (100%), but a moderate specificity (72.7%) for convulsive epileptic events with three of the seizures being incorrectly diagnosed as PNES. This is further reinforced with the positive and negative predictive values for PNES being 81.3% and 100%, respectively. The diagnostic accuracy of this automated system is consistent with the findings made in our previous study utilizing a non‐automated algorithm for convulsive ES and PNES using a time‐frequency–based analysis of a wrist‐worn accelerometer device.17 McNemar's test did not show any systematic difference between the results obtained from the automation process and the VEM diagnoses, although the sample size might be too small to enable definitive conclusion.

The result also demonstrated that the algorithm displayed no bias toward ES or PNES. The positive and negative likelihood ratios are clinically important as they indicate how trustworthy the automated results are for a clinician. The LR+ and LR− values demonstrate a patient with PNES is more likely to be classified as having PNES by automation, compared to those with ES. These results illustrate the potential that automation has in assisting a clinician to avoid a misdiagnosis of epilepsy. This will accelerate the commencement of appropriate treatment and management and eventually lead to a better quality of life for patients with PNES.

Three out of the 11 ES were incorrectly classified as PNES. The support vector machine used the differences in the dominant frequency patterns in ES and PNES to distinguish seizure types, with the assumption that ES shows evolving seizure frequency over time while PNES has a more stable pattern. However, these three epileptic events were more tonic and less clonic in nature, and therefore did not demonstrate the evolution of the motor activity as clearly. Thus, it is possible that these events did not have significant enough variations in the dominant frequencies to classify as ES. However, classifying an epileptic seizure as PNES can potentially be more harmful as it causes delay in providing appropriate treatment and places the patient at an increased risk of suffering an injury from a convulsive seizure.

The limitations of this study must be acknowledged as these are factors that will contribute to the trajectory that this particular automated system in detecting and classifying events takes. The number of events used to test this algorithm was low. A part of this can be attributed to the fact that the method pertains to only convulsive epileptic seizures and PNES events that are also tonic‐clonic or convulsive in nature, in that the event includes movements of one or both arms. Due to this, non‐convulsive epileptic and PNES events had to be excluded as naturally; these would have been classified as “no movement” by the algorithm due to the activity filter.

Another reason for the reduced number of events was the 20‐seconds exclusion criterion, which caused the elimination of a number of potential events that could have been otherwise analyzed. As mentioned earlier, this time limit was utilized when developing the algorithm on the training data set, as stated in Gubbi et al's study.18 Upon further consulting, clinical specialists recommended the 20‐seconds time limit be used as a threshold to ensure any continuous movement are most likely seizure‐like movements. Even though this resulted in a substantial lowering of event numbers in this clinical study, the point of this study was to assess the clinical application of this algorithm. This time limit can now be recognized as a significant limitation and thus can now be worked upon so that an event of any duration can be included for analysis.

As a result of the elimination of a number of events, many of the seizures that were valid for analyses came from the same patients. Due to the analysis being performed on different window lengths, a number of samples were still able to be utilized to test this algorithm in a clinical setting. Therefore, in a retrospective analysis, the algorithm was also tested using a less stringent threshold of 5 seconds, in order to investigate the impact this would have on the detection and classification results. The algorithm detected all PNES events whereas it could only detect 16 of the 19 ES events that were included for analysis. The ROC performance curves for 20 and 5‐seconds thresholds are shown in Figures 5 and 6. As expected, the overall performance of the algorithm decreases on using a less stringent threshold of 5 seconds (correctly classified 12 of 19 ES and 28 of 33 events as PNES). However, the number of false alarms during the monitoring duration of >661 hours increases to 150 (5.44 per 24 hours) for 5‐seconds threshold in comparison with 67 false alarms (2.43 per 24 hours) with 20‐seconds threshold. The three events that were missed by the algorithm were all <10 seconds in duration and had a VEM diagnosis of focal onset seizures with impaired awareness. The movements involved in these seizures were very subtle, non‐purposeful, and of short duration. The automated algorithm classified these events as normal movements or activities of daily living, which was also expected as the algorithm uses features that were engineered for activities with duration ≥20 seconds. Based on these findings, in future we will investigate a more robust feature set such as, non‐linear analysis of the multivariate time series data to capture patterns corresponding to seizures with duration up to 5 seconds.19, 20 However, based on the results of the proposed study it would be safe to assume that convulsive epileptic and non‐epileptic seizures can be detected and differentiated non‐invasively using wearable automated systems. Furthermore, the results of this study also validate and re‐enforce the fact that convulsive PNES can be differentiated from convulsive ES by capturing the rhythmic movement activity during the event using movement‐recording devices such as wearable accelerometers.

The clinical utility of the approach presented in this report may be further enhanced with additional sensors as with devices such as Empatica and Brain Sentinel.12 For instance, contraction of muscles during motor seizures can be measured using a surface electromyography (sEMG), which has been used to distinguish between GTCS and convulsive PNES.21 The incorporation of sEMG, pulse oximeter, heart rate, and accelerometer sensor, into an integrated non‐invasive, ambulatory device, may enable the detection and classification of non‐convulsive seizures.

The proposed study showed the utility of the automated wearable systems in detection and differentiation of convulsive epileptic and non‐epileptic seizures. In the past decade, such wearable systems have garnered a lot of attention as they present a potential alternative for continuous non‐invasive monitoring in patient home setting. However, the utility and requirements of such systems from clinical perspective are rarely considered. The findings of this study suggest that automated detection and differentiation of convulsive epileptic and non‐epileptic seizures can be done using accelerometer‐based wrist‐worn wearable systems. However, there are a few points that should be considered during the development of such algorithms. First, it is important that the proposed system should have a high differentiation specificity for convulsive ES events as misclassifying ES to PNES may prove misleading in diagnosing an epileptic patient and cause failure and delay in providing appropriate treatment. Secondly, the automated systems should minimize the trade‐off between performance and minimum seizure duration that can be detected by the system as it might be necessary for patients that have brief events.

CONFLICTS OF INTEREST

Neither of the authors has any conflict of interest to disclose. We confirm that we have read the Journal's position on issues involved in ethical publication and affirm that this report is consistent with those guidelines.

Click here for additional data file.