Spike detection. I. Correlation and reliability of human experts.

A panel of 5 experienced electroencephalographers detected spikes in EEG trials from 40 epilepsy patients and 10 control subjects. 1952 spikes were detected, and detailed attribute scores were recorded. Statistics from the theory of measurement error are utilized to quantify the reliability and difficulty of the study. An extension of the Pearson correlation coefficient, called the detection correlation coefficient, is derived and used in recognition of the fact that readers agree on numerous non-spike regions. Spike perception is modeled with both dichotomous and continuous values. as expected, the study reliability is higher when using the continuous values. Standard sensitivity and specificity definitions are extended and applied to continuous-valued spike perception. A database of "panel scores" was created from the 5 reader scorings by merging spikes within 75 msec on each side. The average inter-reader correlation is 0.79 with a corresponding reliability of 0.95. Average spike attributes are calculated, and the resulting database can serve as a "gold standard" for testing computer algorithms or other readers.