Seizure prediction: any better than chance?

OBJECTIVE: To test whether epileptic seizure prediction algorithms have true predictive power, their performance must be compared with the one expected under well-defined null hypotheses. For this purpose, analytical performance estimates and seizure predictor surrogates were introduced. We here extend the Monte Carlo framework of seizure predictor surrogates by introducing alarm times surrogates.
METHODS: We construct artificial seizure time sequences and artificial seizure predictors to be consistent or inconsistent with various null hypotheses to determine the frequency of null hypothesis rejections obtained from analytical performance estimates and alarm times surrogates under controlled conditions.
RESULTS: Compared to analytical performance estimates, alarm times surrogates are more flexible with regard to the testable null hypotheses. Both approaches have similar, high statistical power to indicate true predictive power. For Poisson predictors that fulfill the null hypothesis of analytical performance estimates, the frequency of false positive null hypothesis rejections can exceed the significance level for long mean inter-alarm intervals, revealing an intrinsic bias of these analytical estimates.
CONCLUSIONS: Alarm times surrogates offer important advantages over analytical performance estimates. SIGNIFICANCE: The key question in the field of seizure prediction is whether seizures can in principle be predicted or whether algorithms which have been presumed to perform better than chance actually are unable to predict seizures and simply have not yet been tested against the appropriate null hypotheses. Alarm times surrogates can help to answer this question.