Implementing the SCORE system improves the quality of clinical EEG reading.

Objective: To assess whether implementing the freeware version of the SCORE EEG system (Standardized Computer-based Organized Reporting of EEG) leads to improvement in the quality of clinical EEG reading, and whether EEG reports in SCORE EEG are understood and accepted by the referring physicians.
Methods: We generated EEG reports in the conventional, free-text style and then using SCORE EEG, in consecutive patients referred to routine EEG. We used the Georgian translation in the SCORE EEG Free Edition. We pre-defined quality indicators consisting of a list of 24 key features that need to be addressed in EEG reports. We compared these quality indicators in free-text reports with SCORE EEG. In addition, EEG reports in SCORE EEG format were assessed by ten referring physicians, who evaluated their usability on a 7-point Likert scale.
Results: We included and evaluated EEG reports from 157 patients (80 female; age: 1-75 years; median: 28 years). Fourteen features were reported exclusively in SCORE EEG, four were reported significantly more often in SCORE EEG than in free-text format, and six features were reported equally often in SCORE EEG and in free-text format. Usability aspects of SCORE EEG were highly rated by the referring physicians (median 6-7 on the 7-point Likert scale). Conclusions: The structured system of EEG reporting in SCORE EEG helped the experts reading clinical EEG to cover the important aspects and increase the quality of clinical EEG reports. Significance: Implementing the freeware version of SCORE EEG in underprivileged areas will help improving management of patients with epilepsy.

Introduction

Clinical EEGs are typically reported in free text format. This leads to high variability of the content and style, use of ambiguous / ill-defined terminology and uncertain quality. To circumvent this, a working group of the International Federation of Clinical Neurophysiology (IFCN) and International League Against Epilepsy (ILAE) developed a consensus guideline on common data elements which need to be included into clinical EEG reports (Beniczky et al., 2013, Beniczky et al., 2017). The system is known by the acronym SCORE (Standardized Computer-based Organized Reporting of EEG). Because the standardized features in SCORE are complex and context-sensitive, a software is needed to manage the database and the report generator. Programming of the software SCORE EEG was done by Holberg-EEG, and they offer a freely available version, available at holbergeeg.com (Fig. 1). SCORE EEG has been translated into 12 languages. The user can choose to score EEGs using any of these languages, and then issue the report in another language, also facilitating data exchange at international level. SCORE EEG has been used in seven centers in Scandinavia (Denmark and Norway) and it is currently under implementation in the UK. More than 50,000 clinical EEG recordings have been successfully reported using the SCORE EEG system.

Fig. 1

Graphical User Interface of SCORE EEG. The navigation window to the left contains the lists of EEG features (normal and abnormal). After selecting one of these items, the observed EEG features are described by clicking on the pre-defined items in the scoring window on the right. In this example, an epileptiform interictal activity is scored.

Numerous research projects and educational databases were completed using the SCORE system (Aanestad et al., 2020, Brogger et al., 2018, Meritam Larsen et al., 2021, Beniczky et al., 2020, Beniczky et al., 2018). However, one of the main goals of SCORE, namely improvement in quality of clinical EEG reading in underprivileged areas has not been investigated yet. Our goal was to assess this aspect in an EEG laboratory in Georgia. We compared the quality indicators in free-text reports with the SCORE EEG system, and we investigated whether the referring physicians find SCORE EEG reports understandable and useful in the clinical context.

Methods

Consecutive patients referred to routine EEG either diagnosed with epilepsy or having clinical suspicion of epilepsy or a seizure, in the period March-December 2020, at the Institute for Neurology and Neuropsychology (INN), Tbilisi, Georgia, were included. All patients had standard (routine) EEG recordings of 20 min duration. The clinical EEG was recorded using the standardized electrode array of the International Federation of Clinical Neurophysiology (Seeck et al., 2017).

EEG reports were written in both free-text format and the SCORE EEG format. The SCORE EEG Free Edition software (translated into Georgian) was used to generate EEG reports in the SCORE EEG format (Beniczky et al., 2013, Beniczky et al., 2017). We used a version of SCORE EEG which is freely available at holbergeeg.com. The content of the reports was then evaluated for the presence/absence of a list of key features, important for describing the EEG, which were used as quality indicators in this study. Table 1 shows the list of evaluated key features (quality indicators).

Table 1

The evaluated key features.

Indication for EEG

Information about the recording electrode array

Quality of hyperventilation procedure

Frequency of the Posterior Dominant Rhythm (PDR)

Symmetry of the PDR

The cause of non-identifiable PDR specified (when PDR was absent)

PDR classified as normal vs abnormal

Beta activity – defining if this is normal or not

Normal drowsiness and sleep activity during the recording – indicating that sleep recording is normal or not

Normal variants and patterns of uncertain significance

Implications of the artifacts on the quality of the assessment

Detailed location (at electrode-level) of abnormal slowing

Pattern-type (single discharge or runs) of abnormal focal slowing

Duration of the runs of abnormal focal slow activity

Frequency of the abnormal focal slow activity

Modulatory effect of hyperventilation on the abnormal slowing

Morphology of the Interictal Epileptiform Discharges (IEDs)

Pattern-type (single discharge or runs) of the IEDs

Duration of the runs of the IEDs

Modulatory effect of hyperventilation on IEDs

Separate estimation of single discharges and trains/ bursts in the same location

Abundance of IEDs

Detailed location (at electrode-level) of IEDs

Diagnostic significance

We used McNemar test to compare how often the key features listed in the Table 1 were reported in SCORE EEG vs free text format.

In addition, EEG reports written in the SCORE EEG format were given to referring physicians for evaluation by the 7 point Likert scale. They were asked to define if and in what extent they agree with the following statements: 1. SCORE EEG reporting is informative, 2. SCORE EEG reporting is easy to understand, 3. SCORE EEG reporting is useful, 4. Compared to the free text, SCORE EEG reporting is more refined (i.e. of higher granularity and precision). For this purpose, out of 157 EEG reports written in the SCORE EEG format, 20 were selected containing description of diverse EEG features as well as having different diagnostic significance (e.g., normal awake and sleep recordings, normal awake recordings without PDR or with artifacts, no definite abnormality, focal CNS dysfunction, Focal Epilepsy, Generalized Epilepsy).

Results

Fig. 2 shows an example of an EEG report in SCORE EEG: the original report in Georgian language and the automatically translated report in English.

Fig. 2

Example of an EEG report in SCORE EEG: to the left, the original report in Georgian language and to the right, the automatically translated report in English. An English translation of the free text report format is available in Supplementary Document 1.

EEG recordings in both free text and SCORE EEG formats of 157 patients (80 female patients) were investigated. The age of the patients was between one and 75 years (median 28 years). Indication for EEG was: clinical suspicion of Epilepsy or a seizure (88 patients), monitoring the effect of medication (66 patients), follow-up EEG (2 patients), part of a multinational research study (1 patient).

Comparison of reporting the key features in the free text and SCORE EEG formats is presented in Table 2. Most of the key features were reported either exclusively in the SCORE EEG format (14 features) or significantly more often in the SCORE EEG format than in the free-text report (four features). Six features only were reported as frequently in SCORE EEG as in free-text (no significant difference), while none was reported only in free text.

Table 2

Comparison of reporting of different variables in the free text and SCORE EEG formats.

Reported only in SCORE EEG, never in the free-text report:

Indication for EEG (n = 157)

Information about the recording electrode array (n = 157)

Quality of hyperventilation procedure (n = 131)

The cause of non-identifiable PDR specified (n = 4)

Beta activity – defining if this is normal or not (n = 4)

Normal drowsiness and sleep activity during the recording – indicating that sleep recording is normal or not (n = 38)

Separate estimation of single discharges and trains/ bursts in the same location (n = 9)

Abundance of IEDs (n = 9)

Detailed location of IEDs (n = 75)

Detailed location of abnormal slowing (n = 71)

Pattern-type (single discharge or runs) of abnormal focal slowing (n = 67)

Duration of the runs of abnormal focal slow activity (n = 62)

Frequency of the abnormal focal slow activity (n = 63)

Modulatory effect of hyperventilation on the abnormal slowing (n = 63)

Reported significantly more often in SCORE EEG than in the free-text report:

Symmetry of the PDR (120 vs 24, p < 0.0001)

Normal variants and patterns of uncertain significance (5 vs 2, p = 0.0026)

Implications of the artifacts on the quality of the assessment (32 vs 9, p < 0.0001)

Diagnostic significance (157 vs 2, p < 0.0001)

Reported as frequently in SCORE EEG as in free-text (no significant difference)

Frequency of the PDR (128 vs 125)

PDR classified as normal vs abnormal (145 vs 144)

Morphology of the IEDs (75 vs 75)

Pattern-type (single discharge or runs) of the IEDs (75 vs 75)

Duration of the runs of the IEDs (15 vs 11)

Modulatory effect of hyperventilation on IEDs (45 vs 37)

The last element of the SCORE EEG reports is the diagnostic significance (Table 1), containing the overall interpretation of the recording and automatically generating a report. This was specified as normal recording in 53 patients, no definite abnormality in 6 patients, focal CNS dysfunction in 28 patients, focal epilepsy and focal CNS dysfunction in 2 patients, focal epilepsy in 57 patients, generalized epilepsy in 9 patients, and epilepsy of unknown type in 2 patients.

Twenty EEG reports written in the SCORE EEG format were given to referring physicians for evaluation by the 7 point Likert scale. Data were collected from 10 referring physicians. From those, four were epileptologists, four pediatric neurologists and two adult neurologists. Evaluation by referring physicians according to Likert scale is shown in the Table 3.

Table 3

SCORE EEG – evaluation by referring physicians according to Likert scale. Median scores and interquartile ranges in parenthesis.

SCORE EEG reporting is informative	SCORE EEG reporting is easy to understand	SCORE EEG reporting is useful	Compared to the free text, SCORE EEG reporting is more refined
7 (7–7)	7 (6–7)	6 (6–7)	6.5 (6–7)

Note: 7 means strongly agree, 6 - agree.

All responders found the standardized SCORE EEG format informative, easy to understand and useful as well as more refined than the free text format, which suggests great acceptability by referring physicians.

Discussion

We have evaluated and compared the quality indicators in the SCORE EEG system and in free-text reports in an EEG laboratory in Georgia. We found that the majority of the key features were reported either in the SCORE EEG format only or significantly more often in the SCORE EEG format than in the free-text report (14 and four features, respectively). Six features were reported equally often in the SCORE and in free-text formats, and none was reported exclusively in free text. The key features, important for describing EEGs are included into the recommendations on EEG reporting (Tatum et al., 2016). Our results suggest a clear improvement in quality of clinical EEG reading in the SCORE format compared with the free text report, based on a better coverage of these items in the reports, when the user is guided by the SCORE system. As far as we know, this is the first study investigating improvement in quality of clinical EEG reading in underprivileged areas.

In addition, we investigated whether the referring physicians find SCORE EEG reports informative, understandable, useful, as well as more refined compared to the free text. The mentioned usability aspects were highly rated by the referring physicians (median 6–7 on the 7-point Likert scale) indicating great acceptability. This is in agreement with previously reported results of Guerrero-Aranda et al. (2022), the first study to assess the acceptability of a structured (standardized) EEG report format by referring physicians.

Reading EEG requires extensive training and experience, which is not available in all areas of the world. The SCORE EEG system guides the clinicians through the logical steps of EEG assessment, lists the clinically important features for the EEGs in a context-sensitive way, and uses standardized terminology. EEG features in SCORE EEG are provided with a definition. A systematic application of the standardized EEG assessment method in SCORE EEG has the potential of improving the quality of EEG assessment and the quality of the reports. A previously published study demonstrated that review and reporting times using the SCORE EEG system were reasonable in clinical practice (Brogger et al., 2018). Well-written free-text reports that follow all the recommendations of ACNS guideline-7 may be comparable to SCORE reports, although this was not formally assessed in this paper). The principal advantages of the SCORE system over free text reports are that it guides the user to include all these features into the report, leading to a better quality of the report, reduces the variability inherent in free text reports and may be especially useful in places where extensive EEG training is not available, such as in underprivileged areas.

In conclusion, the results of our study show definite benefits of using the SCORE EEG system in underprivileged areas. In particular, the standardized EEG reporting help clinical neurophysiologists to evaluate the key features important for describing the EEG increasing thus the quality of clinical EEG reports. This will result in improving management of patients with epilepsy.

Declaration of Competing Interest

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Harald Aurlien is Chief Medical Officer and minority shareholder in Holberg EEG.