Educational initiatives for electroencephalography in the critical care setting: a systematic review and meta-analysis.

PURPOSE: We systematically reviewed existing critical care electroencephalography (EEG) educational programs for non-neurologists, with the primary goal of reporting the content covered, methods of instruction, overall duration, and participant experience. Our secondary goals were to assess the impact of EEG programs on participants' core knowledge, and the agreement between non-experts and experts for seizure identification. SOURCE: Major databases were searched from inception to 30 August 2020. Randomized controlled trials, cohort studies, and descriptive studies were all considered if they reported an EEG curriculum for non-neurologists in a critical care setting. Data were presented thematically for the qualitative primary outcome and a meta-analysis using a random effects model was performed for the quantitative secondary outcomes. PRINCIPAL
FINDINGS: Twenty-nine studies were included after reviewing 7,486 citations. Twenty-two studies were single centre, 17 were from North America, and 16 were published after 2016. Most EEG studies were targeted to critical care nurses (17 studies), focused on processed forms of EEG with amplitude-integrated EEG being the most common (15 studies), and were shorter than one day in duration (24 studies). In pre-post studies, EEG programs significantly improved participants' knowledge of tested material (standardized mean change, 1.79; 95% confidence interval [CI], 0.86 to 2.73). Agreement for seizure identification between non-experts and experts was moderate (Cohen's kappa = 0.44; 95% CI, 0.27 to 0.60).
CONCLUSIONS: It is feasible to teach basic EEG to participants in critical care settings from different clinical backgrounds, including physicians and nurses. Brief training programs can enable bedside providers to recognize high-yield abnormalities such as non-convulsive seizures.

Use of electroencephalography (EEG) in critical care settings is recommended in multiple guidelines for a variety of indications.1–4 Owing partly to the expansion of evidence-based indications for EEG, there has been a rapid and sustained uptake in both intermittent and continuous EEG (cEEG) use in critical care environments.5 For cEEG alone, the rate of growth across adult and pediatric settings has been estimated at 30% per year.5,6 Nevertheless, even as monitoring capabilities and personnel are mobilized to meet the increase in need for EEG, current demands far outstrip availability. In a survey of neurophysiologists across 97 adult intensive care units (ICUs) in the United States, most respondents stated that if given additional resources they would monitor between 10 and 30% more patients with cEEG, and 18% would increase the duration of cEEG recordings.7

Among other logistical challenges, the expansion of EEG across ICUs has led to a substantial increase in the workload of epileptologists, neurophysiologists, and other trained experts who interpret raw EEG recordings. Interpretation of a 24- or 48-hr cEEG can be lengthy and burdensome. Moreover, while rates of EEG acquisition have risen dramatically, the number of trained experts has remained largely constant.8 Finally, even in large tertiary and quaternary centres, there are often delays of up to several hours between seizure occurrence and time to EEG interpretation.9 This can lead to therapeutic delays, which may increase patient morbidity and mortality in the case of subclinical seizures or non-convulsive status epilepticus.10–13

One potential method to expedite diagnosis and management is to train non-experts to interpret EEGs at the bedside. Multiple studies have described EEG educational programs for non-experts in adult and pediatric/neonatal acute care settings.14–16 Nevertheless, the overall structure, content, and efficacy of these programs remains unknown. We conducted a systematic review and meta-analysis of educational programs for EEG interpretation in adult and pediatric/neonatal critical care settings (emergency departments, ICUs, and postanesthetic care units), with the goal of describing important educational elements and outcomes across programs.

Methods

This systematic review was performed according to a published protocol17 and followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) recommendations.18 The protocol was also registered with PROSPERO: International prospective register of systematic reviews (CRD42020171208).

Literature search

We searched for relevant studies published on MEDLINE, Embase, Cochrane central (OVID), CINAHL (EbscoHost), and Web of Science (Clarivate Analytics) from inception to 30 August 2020, with an update performed before submission for publication. The search strategy was created by a health information specialist (ME) using a combination of free-text keywords and medical subject headings. A sample of the MEDLINE search is provided in eAppendix 1 in the Electronic Supplemental Material (ESM). We also searched the grey literature including http://clinicaltrials.gov to identify unpublished material. The reference lists of studies selected for inclusion were scanned to identify any remaining studies. Lastly, cited reference searching via the Web of Science was conducted on ten seminal papers. No language restrictions were applied.

Eligibility criteria

We searched for studies reporting educational methods to teach EEG to non-experts in the ICU, emergency department, or postanesthesia care unit. To be eligible for inclusion, studies were required to present sufficient details of their educational program. Randomized controlled trials, cohort studies, and descriptive studies were all considered. Studies from both adult and pediatric/neonatal settings were included with the goal of being broadly inclusive. Studies were excluded if participants had prior advanced expertise interpreting and reporting EEGs (e.g., neurophysiologists or epileptologists). Medical trainees, EEG technicians, neurocritical care nurses, and other individuals with exposure to EEG but without dedicated EEG specialization were all considered non-experts for the purpose of this review.

Finally, we considered studies describing multiple forms of EEG, including intermittent EEG, raw EEG (cEEG), and processed forms of cEEG with data displayed as compressed tracings (e.g., amplitude-integrated EEG [aEEG], colour density spectral array [CDSA]) or as a numerical output (e.g., bispectral index). Figure 1 presents a simplified schematic used to classify common forms of EEG for the purposes of this study. We decided a priori to include various forms of EEG (including both raw and processed), with the rationale that excluding certain modalities would potentially misrepresent important findings on the overall scope and landscape of existing EEG programs.

Figure 1

Simplified schematic of EEG forms. EEG = electroencephalogram

Study selection and data extraction

Studies were initially screened by title and abstract by one reviewer (S.T.). Articles selected for further consideration were reviewed independently and in full by two reviewers (S.T., W.A.). Data reported in different publications (e.g., conference abstract and full manuscript) but originating from the same study were included only once in the analysis, with the most comprehensive source being represented. Both reviewers (S.T., W.A.) compared their study lists at the end of the selection process to ensure concordance. Differences were resolved in consensus to generate the final list of studies.

Two reviewers (S.T. and W.A.) independently extracted data on study design, methods, and outcomes into a standardized data collection form. The full data form (available as ESM, eAppendix 2) was pre-piloted on five studies and iteratively modified to ensure completeness. We collected information on participant background, study setting, and details of the EEG learning program. Quantitative data were also extracted for the subset of studies reporting our secondary outcomes of interest. Study authors were contacted up to three times to obtain any missing data. Author responses are reported in eAppendix 3 (available as ESM). Differences in extracted data were resolved in consensus by the two primary reviewers.

Funnel plots were considered for assessing publication bias via inspection for asymmetry. Nevertheless, the small number of studies (< ten) in each meta-analysis precluded a formal assessment of publication bias and funnel plots were instead used to show an overall estimate of small-study effects, as per the Cochrane Handbook.19

Outcomes

Our primary outcome was to characterize educational methods to teach EEG to non-experts in critical care settings. For the assessment of this qualitative outcome, results were reported according to the following themes: 1) clinical background of the participants, 2) content of the training program, 3) teaching methods, 4) assessment methods, 5) duration of the program, and 6) participant feedback. Our secondary outcomes included 1) comparison of learners’ performance on written or bedside assessments before and after receiving the training program and 2) inter-rater agreement for seizure identification between non-experts receiving the EEG program and experts (i.e., neurophysiologists or epileptologists). For the assessment of secondary outcomes, we analyzed only the subset of studies reporting the relevant data.

Quality assessment of observational studies and risk of bias of randomized-controlled trials

Two reviewers (S.T. and W.A.) independently assessed the quality of observational studies using the National Institute of Health (NIH) study quality assessment tools (available from https://www.nhlbi.nih.gov/health-topics/study-quality-assessment-tools and also presented in the ESM, eAppendix 4). The optimal quality assessment tool is transparent, offers a comprehensive analysis of a study’s internal validity, is specific to the study design being evaluated, and avoids representing the study’s overall quality with a single numeric score.20 The NIH tools were chosen because they met all of these recommendations. Risk of bias was assessed at the study level and studies were assigned a quality of “good,” “fair,” or “poor” based on items considered essential for internal validity (full adjudication details are presented as ESM, eAppendix 4). One randomized-controlled trial was included in this systematic review and its risk of bias was evaluated with the Cochrane Collaboration’s Risk of Bias 2 (RoB 2) tool.21

Assessing the certainty of evidence

As discussed in our protocol, we anticipated difficulty providing Grading of Recommendations Assessment, Development and Evaluation (GRADE) summary statements for our chosen outcomes because of heterogeneity between studies. Our final review of the included studies confirmed this to be the case, so we did not generate GRADE recommendations for this systematic review.

Data analysis

The qualitative primary outcome was broken down by theme and described in the form of written summaries, graphs, and tables. Synthesized data for each of the two secondary outcomes were presented as written summaries and forest plots. A meta-analysis of studies contributing to our secondary outcomes was not initially planned because of anticipated clinical heterogeneity between studies.17 However, after finalizing our study list and reviewing their clinical methods, we felt there was sufficient similarity between studies to undertake a standard meta-analysis following the Cochrane Handbook’s recommendations.19

We used the standardized mean change for comparing evaluation scores pre-post EEG training program using patient-level data when available22–24 or the mean (standard deviation [SD]) of the overall cohort assuming a correlation of 0.5 between pre-post scores25–27 to calculate the SD of the mean change. The standardized mean change is a measure that allows meta-analysis of data across pre-post studies that have different scores reported across studies.28 Standardized mean changes were then pooled using the inverse variance method with DerSimonian–Laird random effects. We performed two sensitivity analyses to estimate the SD of the change using correlations of 0.3 and 0.7 between pre-post scores. Cohen’s kappa coefficients were pooled using similar methods. Cohen’s kappa assumes that subjects being rated are independent of each other, the categories of ratings are independent and mutually exclusive, and the two raters operate independently.29 The data available in some papers had the same patients’ EEGs evaluated by multiple non-experts (i.e., the subject independence assumption was incompletely met), leading to estimates of kappa’s confidence intervals that were potentially narrower than they should have been.

Results

The search yielded 7,486 studies, of which 6,035 were screened and 29 were included (Fig. 2). Table 1 presents the characteristics of included studies. Studies were all published in English and included 26 cohort studies,14–16,22–27,30–46 two descriptive studies,47,48 and one randomized-controlled trial.49

Figure 2

Study selection

Table 1

Characteristics of included studies

Author (year)	Number of centres	Study setting	Participant background	Number of participants	EEG program characteristics
					Recording	Data used for EEG teaching	Primary goal of training	Teaching methods	Assessment method	Length of program
RCT
Chari49 (2019)	3	ED	ED physicians	30	Intermittent EEG	Raw waveform	Seizure identification	Didactic lectures, PowerPoint	Written quiz	N/A
Cohort studies
Amorim30 (2017)	2	Adult Neuro-ICU	ICU nurses	33	Continuous EEG	Processed—CDSA	Seizure identification	Small group discussion	Written quiz	<3 hours
Chau26 (2014)	1	Adult Neuro-ICU	ICU fellows	9	Continuous EEG	Raw waveform	EEG theory and waveform interpretation	Bedside/interactive, didactic lectures, one-on-one review	Written quiz	3–6 hrs
Citerio14 (2017)	2	Adult Neuro-ICU	ICU physicians	86	Continuous EEG	Both raw (cEEG) and processed (CDSA, aEEG)	Recognition of seizures, artefacts, sedation, and symmetry	Case-based learning, didactic lectures	N/A	N/A
Dericioglu32 (2015)	1	Adult Neuro-ICU	ICU nurses, neurology residents	4	Continuous EEG with video	Processed—CDSA and aEEG	Seizure identification	One-on-one review, small group discussion	Off-line review of processed EEG	3–6 hrs
Fahy23 (2008)	1	Adult Neuro-ICU and intraoperative	Anesthesiology residents	33	Intermittent and real- time EEG	Raw waveform	EEG theory and waveform interpretation	Bedside/interactive, didactic lectures, one-on-one review	Written quiz	>1 day
Fahy22 (2009)	1	Adult Neuro-ICU and intraoperative	Anesthesiology residents	8	Intermittent and real-time EEG	Raw waveform	EEG theory and waveform interpretation	Bedside/interactive, one-on-one review	Written quiz	>1 day
Fahy33 (2019)	1	Adult Neuro-ICU and intraoperative	Anesthesiology residents	81	Intermittent EEG	Raw waveform	EEG theory	Didactic lectures	Written quiz	>1 day
Fahy24 (2020)	1	Adult Neuro-ICU	ICU fellows	9	Intermittent EEG	Raw waveform	EEG theory and waveform interpretation	Self-study, small group discussion	Written quiz	N/A
Seiler42 (2012)	1	Adult Neuro-ICU	ICU nurses	47	Continuous EEG	Raw waveform	EEG theory	PowerPoint, small group discussion	Written quiz	N/A
Kang39 (2019)	1	Adult Neuro-ICU	ICU nurses	20	Continuous EEG	Processed—aEEG and rhythmicity spectrogram (similar to CDSA)	Seizure identification	Bedside/interactive, PowerPoint	Real- time/bedside interpretation	<3 hr
Herta38 (2017)	2	Adult ICU	ICU nurses, other	18	Continuous EEG with video	Processed—data displayed as aEEG on Neurotrend monitor	Seizure identification	Didactic lectures	Written quiz	<3 hr
Kyriakopoulos45 (2020)	1	Adult ICU	Neurology residents	16	Intermittent EEG	Raw waveform	Seizure identification	Self-study, bedside/interactive, small group	Real- time/bedside interpretation	N/A
Legriel46 (2020)	6	Adult ICU	Attending physicians, fellows, residents, medical students, nurses	250	Continuous EEG	Raw waveform	EEG theory and waveform interpretation	Didactic lectures, self-study	Written quiz	<3 hr
Leira25 (2004)	1	Adult ICU	ICU fellows, ICU nurses, neurosurgery residents, neurology residents, other	50	Intermittent EEG	Raw waveform	EEG theory and waveform interpretation	Didactic lectures, PowerPoint	Written quiz	N/A
Lybeck40 (2020)	1	Adult ICU	ICU physicians	5	Continuous EEG	Both raw (cEEG) and processed (aEEG)	EEG waveform interpretation	Bedside/interactive, didactic lectures, case-based learning	Real- time/bedside interpretation	7 hr–1 day
Nitzschke41 (2011)	1	Adult ICU	ICU physicians	4	Continuous EEG	Processed—aEEG displayed on Neuroscreen monitor	Seizure identification	Bedside/interactive, didactic lectures, one-on-one review, question and answer	Real- time/bedside interpretation	<3 hr
Swisher43 (2015)	1	Adult ICU	ICU nurses, other	17	Continuous EEG	Both raw (cEEG) and processed (CDSA, aEEG) displayed on Persyst	Seizure identification	Didactic lectures, PowerPoint	Real- time/bedside interpretation	<3 hr
Topjian44 (2015)	1	Adult ICU	ICU physicians, ICU fellows, ICU nurses	39	Continuous EEG	Processed—CDSA using Persyst	Seizure identification	Didactic lectures, PowerPoint, question and answer	Real- time/bedside interpretation	N/A
Bourgoin31 (2020)	1	Pediatric ICU	ICU nurses	51	Continuous EEG	Processed—aEEG	Recognition of pathologic pattern (severely altered background or seizures)	Online modules, small group discussion	Real-time/bedside interpretation	3–6 hr
Du Pont-Thibodeau15 (2017)	1	Pediatric ICU	ICU physicians, ICU fellows, nurses	23	Continuous EEG	Processed—aEEG and CDSA	Seizure identification	Didactic lectures, PowerPoint	Written quiz	<3 hr
Lalgudi Ganesan34 (2018)	1	Pediatric ICU	ICU fellows, ICU nurses, other	12	Continuous EEG	Processed—aEEG and CDSA	Seizure identification	Bedside/interactive, one-on-one review	Written quiz	<3 hr
Goswami35 (2018)	2	Neonatal ICU	ICU nurses	230	Continuous EEG with video	Both raw (cEEG) and processed (aEEG) displayed using Persyst	EEG lead application and monitor setup	Bedside/interactive	N/A	N/A
Griesmaier36 (2011)	2	Neonatal ICU	ICU physicians, ICU nurses, other residents	60	Continuous EEG	Processed—aEEG	Seizure identification	Didactic lectures	Written quiz	3–6 hr
Guan37 (2016)	1	Pediatric ICU	ICU physicians, ICU nurses, other	6	Continuous EEG with video	Processed—aEEG	Seizure identification	Didactic lectures, textbook readings	Real- time/bedside interpretation	>1 day
Poon27 (2015)	1	Neonatal ICU	ICU physicians, ICU nurses	37	Continuous EEG	Processed—aEEG	EEG theory and waveform interpretation	Didactic lectures, online modules, PowerPoint, small group discussion	Written quiz	3–6 hr
Rennie16 (2004)	1	Neonatal ICU	ICU physicians	4	Continuous EEG with video	Processed—cerebral function monitor (aEEG)	Seizure identification	Bedside/interactive, one-on-one review, question and answer	Real- time/bedside interpretation	3–6 hr
Descriptive studies
Sacco47 (2016)	1	Neonatal ICU	ICU nurses	N/A	Continuous EEG	Processed—aEEG	Seizure identification	Didactic lectures, question and answer, simulation	Written quiz	7 hr–1 day
Whitelaw48 (2006)	1	Neonatal ICU	ICU physicians	N/A	Continuous EEG	Raw EEG	EEG theory and waveform interpretation	Bedside/interactive, case-based learning, didactic lectures, question and answer, simulation	Written quiz	7 hr–1 day

aEEG = amplitude-integrated electroencephalogram; CDSA = colour density spectral array; cEEG = continuous electroencephalogram; EEG = electroencephalogram; ICU = intensive care unit; N/A = not applicable; NR = not reported; RCT = randomized controlled trial

Study characteristics

All studies described a program for educating non-experts in EEG in the acute care environment. Twenty-two studies were single centre in design, 17 originated in North America, and 16 were published after 2016. Sample sizes ranged from four educational participants in the smallest studies16,32,41 to 250 in the largest.46 Only seven studies had more than 50 participants.14,25,31,33,35,36,46 One study was performed in the ED,49 18 in adult ICUs (either general medical-surgical ICU or neuro-ICU),14,22–26,30,32,33,38–46 four in pediatric ICUs,15,31,34,37 and six in neonatal ICUs.16,27,35,36,47,48 In three studies, participants learned intraoperative EEG from trained experts during various neurosurgical procedures, although this was in the context of a broader educational curriculum that largely occurred within the neuro-ICU.22,23,33 None of the studies were conducted in the postanesthesia care unit. Additional study characteristics are presented as ESM (eAppendix 5).

Four studies were conducted by the same primary author,22–24,33 but because the educational program and participants were different between the studies (confirmed by correspondence with the study author), all four were included and analyzed separately. Two studies outlined programs for EEG training of non-experts but did not pilot them in a group of participants.47,48 Both were classified as descriptive studies and included only in the assessment of the qualitative primary outcome.

Outcome analysis

Primary outcome

Clinical background of participants

Most EEG training programs were geared towards nurses. Overall, critical care nurses were the most common participants (17 studies),15,25,27,30–32,34–39,42–44,46,47 followed by critical care physicians (11 studies)14–16,27,36,37,40,41,44,46,48 and critical care fellows (seven studies).15,24–26,34,44,46 Medical residents comprised the minority of participants in EEG educational programs. Only one study included ED physicians.49

Content of EEG training

The most common learning objective among programs was seizure identification (Table 1). Few studies focused on recognition of other EEG patterns, such as normal background, sleep/wake patterns, and artefacts. In several studies, the learning program involved a mix of EEG theory, practical considerations (e.g., how to set up the recording system or apply electrodes), and a general approach to waveform interpretation.

Processed forms of EEG were more common in learning programs than raw cEEG or intermittent EEG. As shown in Table 1, although raw data were collected by cEEG in most studies, participants rarely reviewed these tracings and the educational program was mostly performed using simplified outputs (e.g., aEEG and CDSA). By far, the most common training program involved aEEG (15 studies);15,16,27,31,32,34–41,43,47 whereas raw cEEG14,26,35,40,42,46,48 and intermittent EEG22–25,33,45,49 were evaluated in seven studies each.

Teaching methods

EEG education programs involved a mix of large group didactic lectures, small group sessions, and self-learning or one-on-one review with EEG experts. Didactic lectures were common whereas small group simulations, case-based learning, and textbook readings were used in a smaller proportion of programs. Full details of the teaching methods for each study, including the content of the training program and teaching methods, are presented as ESM eAppendices 6 and 7.

Assessment methods

Assessments occurred in the form of written quizzes or bedside/real-time EEG interpretation. Among studies reporting assessment methods, 18 used written quizzes15,22–27,30,33,34,36,38,42,44,46–49 and nine used bedside interpretations.16,31,32,37,39–41,43,45 Only four studies assessed participants’ long-term retention of EEG concepts.33,42,46,49 Knowledge was assessed between two and 11 months in these studies and each indicated statistically significant improvement in EEG test scores compared with the baseline test administered before the start of the education program (data could not be meta-analyzed). Additional details are presented as ESM, eAppendix 6.

Duration of the learning program

In most studies, the overall length of the training program was one day or shorter. Among studies reporting training length, program duration was fewer than three hours in eight studies,15,30,34,38,39,41,43,46 between three and six hours in six studies,16,26,27,31,32,36 and between seven hours and one day in three studies.40,47,48 Only in four studies was the overall duration of the training program longer than one day.22,23,33,37

Participant feedback

Participant feedback was available in three studies.24,27,47 Results were reported using different measures and could not be meta-analyzed. In the first study, 70.4% of participants felt that the program allowed them to interpret a normal aEEG with confidence and 77.8% believed the duration of the teaching program was optimal.27 In the second study, satisfaction with the EEG curriculum was moderately high [mean (SD) score, 3.7 (0.9)] on a Likert scale of 1–5.24 In the final study, feedback was assessed qualitatively and indicated that participants felt more comfortable with EEG equipment and tracings.47

Secondary outcomes

Pre-post performance on structured assessments

Meta-analysis was performed for six pre-post studies.22–27 Each study compared trainee performance on a written assessment before and after receiving the EEG training program. There was a significant improvement in trainees’ test scores following the EEG program (standardized mean change, 1.79; 95% CI, 0.86 to 2.73), although statistical heterogeneity was considerable (I = 85.5%). For interpretability, a standardized mean difference (SMD) of 0.2 represents a “small” effect, an SMD of 0.5 represents a “medium” effect, and an SMD of 0.8 represents a “large” effect.50 The forest plot of pre-post studies is shown in Fig. 3. Subgroup analysis was not performed owing to the small number of available studies.

Figure 3

Forest plot assessing performance change after undergoing a training program. CI confidence interval, NIH National Institute of Health, RE random effects

Comparison between non-experts and experts for seizure identification

Six studies assessed agreement for seizure identification between experts and non-experts who had received brief EEG training.14,30,31,39,44,45 Meta-analysis indicated moderate agreement for seizure identification (Cohen’s kappa = 0.44; 95% CI, 0.27 to 0.60) although statistical heterogeneity was considerable (I = 98.0%). The forest plot is shown in Fig. 4. Meta-analysis could not be performed for agreement in other aspects of EEG interpretation (e.g., artefact, burst suppression), given that these were not assessed in most studies.

Figure 4

Forest plot for agreement for seizure identification between non-experts and experts. CI confidence interval, NIH National Institute of Health, RE random effects

Funnel plots for both secondary outcomes are displayed as ESM, eAppendices 8 and 9. There do not appear to be any small studies reporting large effect estimates in either meta-analysis. Assessment for publication bias was deferred because the number of studies was low.

Quality of reported studies

Summary results from the NIH quality assessment for non-descriptive observational studies are presented in Table 2. Study quality was assigned as “good”, “fair”, or “poor” based on items deemed crucial for strong internal validity, a qualitative approach that is in line with the broader recommendation to avoid assigning global numeric scores during quality assessment.20 The majority of studies in this review ranked as fair to good in quality. One randomized-controlled trial was included,49 and its overall risk of bias was judged to be low (ESM, eAppendix 10).

Table 2

NIH quality assessment of non-descriptive observational studies

N/A = not applicable; NIH = National Institute of Health

For observational cohort studies

1. Was the research question or objective in this paper clearly stated?

2. Was the study population clearly specified and defined?

3. Was the participation rate of eligible persons at least 50%?

4. Were all the subjects selected or recruited from the same or similar populations (including the same time period)? Were inclusion and exclusion criteria for being in the study pre-specified and applied uniformly to all participants?

5. Was a sample size justification, power description, or variance and effect estimates provided?

6. For the analyses in this paper, were the exposure(s) of interest measured prior to the outcome(s) being measured?

7. Was the timeframe sufficient so that one could reasonably expect to see an association between exposure and outcome if it existed?

8. For exposures that can vary in amount or level, did the study examine different levels of the exposure as related to the outcome (e.g., categories of exposure, or exposure measured as continuous variable)?

9. Were the exposure measures (independent variables) clearly defined, valid, reliable, and implemented consistently across all study participants?

10. Was the exposure(s) assessed more than once over time?

11. Were the outcome measures (dependent variables) clearly defined, valid, reliable, and implemented consistently across all study participants?

12. Were the outcome assessors blinded to the exposure status of participants?

13. Was loss to follow-up after baseline 20% or less?

14. Were key potential confounding variables measured and adjusted statistically for their impact on the relationship between exposure(s) and outcome(s)?

For pre-post studies

1. Was the study question or objective clearly stated?

2. Were eligibility/selection criteria for the study population pre-specified and clearly described?

3. Were the participants in the study representative of those who would be eligible for the test/service/intervention in the general or clinical population of interest?

4. Were all eligible participants that met the pre-specified entry criteria enrolled?

5. Was the sample size sufficiently large to provide confidence in the findings?

6. Was the test/service/intervention clearly described and delivered consistently across the study population?

7. Were the outcome measures pre-specified, clearly defined, valid, reliable, and assessed consistently across all study participants?

8. Were the people assessing the outcomes blinded to the participants’ exposures/interventions?

9. Was the loss to follow-up after baseline 20% or less? Were those lost to follow-up accounted for in the analysis?

10. Did the statistical methods examine changes in outcome measures from before to after the intervention? Were statistical tests done that provided p values for the pre-to-post changes?

11. Were outcome measures of interest taken multiple times before the intervention and multiple times after the intervention (i.e., did they use an interrupted time-series design)?

12. If the intervention was conducted at a group level (e.g., a whole hospital, a community, etc.) did the statistical analysis take into account the use of individual-level data to determine effects at the group level?

Cohort studies were assigned a score of “good” if they met all of questions 4, 9, and 11; fair if they met one or two of questions 4, 9, and 11; and poor if they met none of questions 4, 9, and 11

Pre-post studies were assigned a score of “good” if they met all of questions 6, 10, and 11; fair if they met one or two of questions 6, 10, and 11; and poor if they met none of questions 6, 7, and 11

Discussion

Our systematic review summarizes the published literature on EEG training programs for non-experts in critical care settings. We identified 29 studies spanning multiple countries. Most programs were targeted to critical care nurses, focused on processed EEG (e.g., aEEG and CDSA), and were shorter than one day in duration. Participants’ short-term knowledge of EEG theory significantly improved after the program, and analysis of studies assessing learners against experts showed moderate agreement for seizure identification. The overall quality of published studies was fair to good as measured by the NIH quality assessment scales.

Our study has several important implications. First, by systematically reviewing the existing content, structure, and design of EEG programs for critical care clinicians, our study provides comprehensive information to guide the development of future EEG programs. Educators will find the answers to important logistical questions in our review, such as who could deliver EEG content, which teaching methods should be considered, and how knowledge could be evaluated. Although we were unable to assess these concepts quantitatively, our study nevertheless provides curriculum designers with a framework to aid in the development of institution-specific programs.

Second, our study highlights the typically condensed and limited nature of EEG programs for non-experts. We found that most studies focused on seizure identification in critically ill patients but did not emphasize other indications for which EEG may be requested. This may reflect the specific nature of critical care environments, where an overarching priority is the diagnosis of non-convulsive seizures and non-convulsive status epilepticus (NCSE). Nevertheless, it also highlights the important point that EEG interpretation is a complex skill that cannot be mastered from a short educational program. In Canada and the United States, for example, the average length of neurophysiology fellowship is six months to one year and follows a four- or five-year neurology residency.51,52 Approximately 50% of overall training time in most fellowships is devoted to EEG or the epilepsy monitoring unit.52 Conversely, EEG training programs for non-experts cannot be nearly as comprehensive, leading to emphasis on topics that are easily teachable. Our study supported this notion with the observation that most programs focused on seizure identification with processed forms of EEG rather than raw cEEG. Processed EEG, such as CDSA or aEEG, allows for rapid detection of pathologies and may be particularly suited for non-experts owing to its condensed nature and more obvious demonstration of seizure activity.53 This observation may help educators maximize the yield of a brief EEG curriculum.

Third, our study draws attention to a wider and more recent trend in neurocritical care, in which non-neurologists are playing a larger role in EEG interpretation. In a recent survey of members of the American Clinical Neurophysiology Society, 21% of respondents indicated that non-experts took part in EEG interpretation at their hospitals.54 Another survey found that EEG technologists reviewed full cEEGs at 26% of respondents’ home institutions.7 With increasing participation of non-neurologists in EEG review, an important concern is that clinicians may interpret EEGs incorrectly and take clinical decisions based on erroneous information. This is compounded by the issue of overconfidence, as highlighted by the study from Poon et al. in which 70.4% of participants felt comfortable in their ability to diagnose seizures after receiving only a brief EEG curriculum.27 Nevertheless, as EEG programs become more standardized, it is foreseeable that non-experts will acquire a more robust and dependable skillset to recognize and act upon basic patterns. Such a transition might follow the example of critical care echocardiography. Whereas ten years ago echocardiography was a cardiology-led initiative, it is now recognized as a critical care competency that plays an important role in bedside patient management.55 A final additional advantage of EEG training programs is their potential to reduce the time to an initial diagnosis. Currently, even in large academic centres with specialized EEG expertise, there may be delays of several hours for EEG interpretation.9,56,57 For non-convulsive seizures and NCSE, delayed diagnosis is associated with increased morbidity and mortality.10,11,13 Bedside interpretation of EEG by non-experts could substantially shorten these delays and expedite patient management.

The strengths of our review lie in its transparent reporting process and rigorous methodology. As per best practices,18 our study was registered in PROSPERO and adhered to a pre-specified study protocol.17 Meta-analysis was performed following the Cochrane Handbook recommendations.19 Finally, our study was performed by a diverse team of clinicians (including intensivists, neurologists and neuro-intensivists), research scientists, and health information specialists who had specific subject matter and methodologic expertise.

This review also has important limitations. First, our quantitative analysis was limited to seizure identification. Only a few studies described other important EEG tracings such as normal background, burst suppression, sleep/wake cycle, or artefact.31,38 In the ICU, artefacts are common and may result from equipment (e.g., mechanical ventilator, hemodialysis machine), patient physiology (e.g., excessive sweating, cardiac arrhythmias), and monitoring devices (e.g., electrocardiogram).58 These are worth noting because non-experts might overcall seizures if they cannot distinguish true seizures from artefacts or other abnormalities.30 Second, the studies in our review implemented a wide range of teaching methods and multiple EEG modalities. Since we were unable to explore possible associations between outcomes and specific teaching methods or EEG modality, important differences in efficacy between various approaches remain unknown. Nevertheless, the fact that short-term knowledge still improved across a diverse set of EEG programs targeted to different participants is hypothesis generating and supports the need for further research into the best methods to teach EEG to non-experts. Third, our meta-analysis of pre-post studies was limited to short-term knowledge outcomes. We could not draw robust conclusions on long-term content retention since only a minority of studies assessed this variable. Fourth, the large effect (> 0.8) observed in the standardized mean change in pre-post studies may over-estimate the true effect of the EEG intervention as the meta-analysis included four small studies with fewer than ten participants and only two studies with 30 and 57 participants. Therefore, the SDs may not be representative of the true variability existing in the population.50 Finally, our meta-analysis of Cohen’s kappa may have been affected by the fact that not all studies met the subject independence assumption, leading to potentially narrower confidence intervals from each study. We proceeded with meta-analysis because the summary estimate for kappa would be unaffected (i.e., our kappa estimate of 0.44 is unlikely to be substantially changed); furthermore, in a foundational paper describing reporting methods for Cohen’s kappa, a meta-analysis was reported for a representative group of studies despite incomplete adherence to the subject independence assumption, similar to our case.29

Conclusions

Our systematic review and meta-analysis offers a comprehensive summary of existing EEG training programs for non-experts. It shows that EEG programs may improve short-term knowledge in mixed groups of participants and suggests that non-experts can be trained to detect seizures on EEG after receiving brief training. Finally, our review offers educators and curriculum developers a framework to develop institution-specific EEG programs.

Below is the link to the electronic supplementary material.

Supplementary material 1 (PDF 564 kb)