Most Experts Agree … But What About Other EEG Readers?

Commentary

Electroencephalogram (EEG) interpretation is a complex skill. Although accepted definitions and terminologies are available, a gold standard is lacking and the final interpretation of EEG is subjective. Misinterpretation of EEG can have significant adverse consequences, particularly misdiagnosis of epilepsy predicated on benign variants, sharply contoured background activity, or artifacts mistaken for interictal epileptiform discharges (IEDs).[1-3] Common sense would indicate that most EEG reads must be reasonably accurate, given that EEG has proven clinical value. Nevertheless, studies dating back several decades have reported interreader agreement on EEG findings ranging from poor to substantial but have been generally limited by factors including small EEG sample size, potentially biased selection of EEG samples (either too easy or to complex), and small numbers of readers from single centers.[1,4,5] Factors that have been correlated with improved interrater agreement for IEDs include fellowship EEG training, subspecialty board certification, academic versus private practice, and greater time devoted to EEG reading.[1,5] However, a recent study found that even expert readers with high confidence in their interpretation were often not in agreement with their peers.[6] Therefore, despite the widespread use of EEG, one could still reasonably argue the reliability of an interpretation regardless of the qualifications of the reader.

In an effort to provide greater clarity on the issue of interrater reliability of identifying IEDs, Jing et al completed a study using a large sample size of routine scalp EEG recordings. Nine experts reviewed 1051 scalp EEG recordings of 30 to 60 minutes duration, rating them for the presence or absence of IEDs. A total of 991 consecutively obtained EEGs clinically interpreted as containing IEDs were used, along with 60 interpreted as IED-free. Both inpatient and outpatient studies were included. Experts had at least 1 year of fellowship training in clinical neurophysiology. Each study was first reviewed by one expert who marked potential IEDs. These potential IEDs were then sent to the remaining 8 experts for their review, with a total of 13 262 waveforms scored. Percentage agreement for interpretation of the presence or absence of IEDs in an entire EEG recording was 80.9% (mean κ of 69.4%), which is considered substantial. For individual IEDs, the percentage agreement was lower at 72.4% (mean κ of 48.7%), but this is still moderate agreement. Interestingly, individual readers were found to be consistent in under- or overcalling relative to the group. Compared to earlier studies looking at agreement in EEG interpretation, this study has many strengths. First, the EEG sample size was large, and full-length EEGs were included rather than single epochs. Because consecutive abnormal EEGs from the clinical practice at Massachusetts General Hospital were included, it is less likely that the samples were biased toward outliers that were atypically perfect or difficult. A potential limitation is that all but 2 of the expert readers trained at the same institution. Since EEG is taught from master to apprentice, readers from the same “guild” may be more likely to agree with each other than those who trained elsewhere.

While we can be reassured that experts generally agree on what represents a true IED on scalp EEG, the ultimate goal is to improve the skill and reliability of everyone who interprets EEG. Despite its importance as a core diagnostic test in neurology, there are no accepted standards of EEG training or proficiency. In a recent survey of graduating US neurology residents, only 37.3% of adult neurology trainees and 66.7% of child neurology trainees reported feeling confident about interpreting EEG independently.[7] The American Council of Graduate Medical Education competency target for EEG skills at the time of graduation includes interpreting common EEG abnormalities, recognizing normal EEG variants, and creating an EEG report. However, only two-thirds of graduating neurology residents are confident they had met these milestones, and 15% of program directors rated these milestones as unreasonable to achieve by the end of residency training.[8] We lack standardized EEG training curricula, and some neurology residency programs have no requirement for an EEG rotation.[8] Improving EEG education is an achievable goal. Even a short web-based training session has been shown to improve interreader agreement on EEG pattern recognition.[9] Furthermore, if a validated sample of IEDs now exists, every EEG reader could potentially benefit from assessing the reliability of their interpretation, regardless of their level of confidence, training, or experience, just as we would hope to validate a computer spike detection algorithm. Until we have reliable ways to measure our individual skills as EEG readers, it may be wise to recall the adage that more damage is likely done by overinterpretation than underinterpretation. Electroencephalogram results should be considered in the context of everything that is known about a given patient. If the results don’t make sense, a request for an additional review of the tracing or a repeat study may be reasonable.