Teleneuropsychology in the time of COVID-19: The experience of The Australian Epilepsy Project.

PURPOSE: Traditional neuropsychological testing carries elevated COVID-19 risk for both examinee and examiner. Here we describe how the pilot study of the Australian Epilepsy Project (AEP) has transitioned to tele-neuropsychology (teleNP), enabling continued safe operations during the pandemic.
METHODS: The AEP includes adults (age 18-60) with a first unprovoked seizure, new diagnosis of epilepsy or drug resistant focal epilepsy. Shortly after launching the study, COVID-related restrictions necessitated adaptation to teleNP, including delivery of verbal tasks via videoconference; visual stimulus delivery via document camera; use of web-hosted, computerised assessment; substitution of oral versions for written tests; online delivery of questionnaires; and discontinuation of telehealth incompatible tasks.
RESULTS: To date, we have completed 24 teleNP assessments: 18 remotely (participant in own home) and six on-site (participant using equipment at research facility). Five face-to-face assessments were conducted prior to the transition to teleNP. Eight of 408 tests administered via teleNP (1.9 %) have been invalidated, for a variety of reasons (technical, procedural, environmental). Data confirm typical patterns of epilepsy-related deficits (p < .05) affecting processing speed, executive function, language and memory. Questionnaire responses indicate elevated rates of patients at high risk of mood (34 %) and anxiety disorder (38 %).
CONCLUSION: Research teleNP assessments reveal a typical pattern of impairments in epilepsy. A range of issues must be considered when introducing teleNP, such as technical and administrative set up, test selection and delivery, and cohort suitability. TeleNP enables large-scale neuropsychological research during periods of social distancing (and beyond), and offers an opportunity to expand the reach and breadth of neuropsychological services.

Introduction

The emergence of COVID-19 has resulted in significant changes in the day-to-day functioning of society the world over. Social distancing policies have had profound effects on people’s day-to-day movements and interpersonal interactions [1] with enormous implications for workplaces, organisations, institutions and social activities. In the health sector this has meant that, wherever possible, clinical consultations have moved from face-to-face to a virtual environment (e.g. telephone, videoconferencing). It also appears that fear of infection is leading to avoidance of the hospital system by individuals who would otherwise seek medical care for conditions such as stroke and cardiac arrest [2,3].

Thus, one might ask how can the health sector, including clinical research, best continue to operate in this environment? In this communication we describe how a large-scale clinical research project in epilepsy has adapted to the emergence of COVID-19, in recognition of the fact that face-to-face interactions in clinical research will have to be reduced for the foreseeable future. We focus on the use of teleneuropsychology (TeleNP) – the application of audiovisual technologies to enable remote clinical encounters with patients to conduct neuropsychological (NP) assessments [4] – to acquire research-based neuropsychological datasets. While our emphasis is on research-based data collection via teleNP, we also touch upon issues relevant to the clinical application of TeleNP in epilepsy.

The Australian Epilepsy Project

The Australian Epilepsy Project (AEP) is a large-scale clinical research project shortlisted for funding by the Medical Research Future Fund of the Australian Government. The vision of the AEP is to develop predictive epilepsy-specific decision support tools for use by clinicians. Machine learning / artificial intelligence (AI) methods will be applied to prospectively acquired neuropsychological, genetics and advanced imaging data obtained from 8000+ adults living with epilepsy, to predict their epilepsy-related two-year outcomes (Fig. 1 ). Sharing of de-identified datasets will further maximise breakthrough opportunities in research. We have commented elsewhere [5] on the role of machine learning/AI in the analysis of such datasets, and in the health sector more broadly, and do not consider this issue further here.

Fig. 1

Conceptual overview of multimodal data collection, integration, analysis, and clinical reporting in the Australian Epilepsy Project.

The AEP commenced a pilot study in February of 2020 to evaluate recruitment feasibility and participant tolerability of the protocols for collection of neuropsychological and imaging data. The first case of COVID-19 was reported in Australia in late January 2020, and a State of Emergency was declared in Victoria in mid-March, around six weeks into AEP recruitment. Despite the introduction of Government-mandated COVID-related restrictions, the AEP pilot study has been able to continue by switching to the use of TeleNP for all participants.

TeleNP can enable research in the era of COVID-19

Prior to the arrival of COVID-19 in Australia, the AEP Pilot Study relied almost exclusively on face-to-face interactions for its data collection and analysis activities. The institutional and governmental response to COVID-19 in Australia demanded a re-evaluation and adjustment to each of these activities to ensure the safety of all persons involved, while preserving the scientific integrity and health care objectives. All our activities accord with guidance from Federal and State regulatory authorities, Institutional clinical governance, and local human research ethics committee approval. Box 1 provides a condensed description of our current operating protocol, outlining the key conceptual features.

Referral

Recruitment occurs through neurologists during routine clinical practice. For the majority of patients this is now via telehealth consultation.

Recruitment

Information provision and the establishing of informed consent are conducted via email and telephone. Participant consent is confirmed verbally and documented electronically and in writing by the researcher.

Data collection

Cognition: Neuropsychological assessment is performed entirely via TeleNP. This includes a combination of neuropsychologist-supervised oral and computer-assisted testing, in conjunction with purely computer-administered web-based testing. Further detail is provided in the main text.

MRI: MRI is performed in-person, at research-dedicated scanners. All participants are screened for COVID-19 symptoms or risk factors before they attend the premises. On-site, physical distancing strategies, appropriate personal protective equipment use, and cleaning procedures are all applied according to up-dated research facility protocols. Images are transmitted electronically to the hospital radiology departments for standard reporting and clinical use. This approach eliminates the need for participants to physically attend hospital premises for clinical scans and reduces the burden on hospital radiology at a time of increased strain on the hospital system.

While genetics and epilepsy follow-up (e.g. seizure diaries, medications, psychological and quality of life questionnaires, adverse events, health economic data) are not collected in the AEP Pilot Study, in the full project genetic samples will be obtained by blood-draw at a local community pathology provider (typically at the same time as routine clinical blood tests), and epilepsy follow-up will occur via smart device app/web survey and telephone call.

Reporting

Data analysis and transfer is performed by research staff accessing secure server platforms remotely via encrypted network connections, enabling this work to be safely performed from home. Key team decision making activities are supported via teleconferencing.

Alt-text: Box 1

Selecting epilepsy-relevant tools for TeleNP

The most substantive protocol changes have involved the transition to collecting all neuropsychological data via TeleNP. Traditional, face-to-face neuropsychological testing carries elevated COVID-19 risk, both for participants and the neuropsychologist, and is clearly unacceptable from both a community safety and occupational health viewpoint. The examiner and examinee may spend several hours in close proximity, passing materials back and forth (e.g. stimulus materials, response forms), usually in a small enclosed room for privacy. Further, the examinee must also travel to the physical premises for the assessment, which can necessitate additional interpersonal interactions (e.g. public transport, waiting areas).

The neuropsychological measures used in our TeleNP protocol are listed in Table 1 . These measures were selected for their evidence base in epilepsy (as acquired through traditional, face-to-face assessments), and their compatibility with TeleNP administration. The experience gained from the AEP Pilot Study will be used to further empirically refine instrument selection.

Table 1

Neuropsychological measures used in the AEP Pilot TeleNP protocol.

Test of Premorbid Functioning [6]: an estimate of premorbid intellect based on irregular word reading, used here both as a measure of intellect, and due to the elevated incidence of reading disorders in epilepsy [7,8].

WASI-II FSIQ 2 (Matrix Reasoning, Vocabulary) [9]*: a short form intelligence measure with excellent psychometric properties.

Oral Symbol Digit Modalities Test [10]: the SDMT is a sensitive measure of processing speed and, in its written form, has been used widely in epilepsy trials [11].

Oral Trail Making Test [12]: a measure of divided attention and speed, sensitive (in its original written form) to dysfunction in epilepsy [11,13].

Reverse Digit Span [14]*: a measure of working memory, sensitive to dysfunction in epilepsy [13].

Verbal Fluency [15]*: letter and category based verbal fluency are sensitive markers of dysfunction in epilepsy, both in focal epilepsy [16] and as an effect of antiseizure medications [13], and also useful as a predictor of cognitive risk from surgery [17].

Boston Naming Test [18]*: a measure of confrontation naming, sensitive to lateralised dysfunction in focal epilepsy [19,20], and also useful as a predictor of cognitive risk from surgery [21].

Rey Auditory Verbal Learning Test (learning and delayed recall trials)*: a word list learning task sensitive to memory dysfunction across a range of epilepsy syndromes [22], to lateralisation in temporal lobe epilepsy [19,23], and predictive of post-operative cognitive outcome [24].

CANTAB Connect (web-based testing) [25]: a computerised assessment battery with an emphasis on executive functions. While the CANTAB does not have an extensive research base in epilepsy [26], it has been used widely in other neurological conditions and is sensitive to frontal and temporal lobe dysfunction [27]. We include the following measures from the CANTAB:• Spatial Working Memory: a measure of spatial working memory; shown to predict postoperative psychological outcomes in epilepsy [28]. • Rapid Visual Information Processing: a measure of information processing speed and sustained attention; computerised assessment of sustained attention has been recommended for large scale epilepsy research [29]. • Intra/Extra Dimensional Set Shift: a measure of cognitive flexibility and set shifting • One Touch Stockings of Cambridge: a measure of planning and problem solving • Paired Associate Learning: a non-verbal measure of arbitrary associate learning. While numerous different measures of non-verbal memory have been used in epilepsy there is no broadly agreed upon, recommended measure [23,[30], [31], [32]]. Arbitrary associate learning has been shown to be a sensitive marker of mesial temporal lobe function in temporal lobe epilepsy [23,[33], [34], [35]]. The paired associate learning subtest of the CANTAB has been shown to be a sensitive marker of pathology in the mesial temporal region in other forms of neurological disease targeting the mesial temporal lobe [36].

The following psychological measures, administered via REDCap [37,38] online surveys, are also reviewed by the neuropsychologist during the TeleNP testing session:

Neurological Disorders Depressions Inventory for Epilepsy (NDDI-E) [39]*: a widely used mood screening tool developed specifically for epilepsy, validated against psychiatric interview determined clinical diagnosis [39].

Generalised Anxiety Disorder 7-item scale (GAD7) [40]*: a brief generalised anxiety screening tool that has been validated in epilepsy against psychiatric interview determined clinical diagnosis [[41], [42], [43]].

Epilepsy Anxiety Survey Instrument (EASI) [43]: an anxiety screening tool designed specifically for use in epilepsy, validated against psychiatric interview determined clinical diagnosis.

* Recommended measure in the NINDS Epilepsy Common Data Elements [30].

In our pre−COVID face-to-face protocol we had been administering EpiTrack [13] and the Reaction Time task from the CANTAB. The trail making test (TMT), inhibition task and maze task within EpiTrack cannot be administered via telehealth, and the Reaction Time task is not available via CANTAB Connect (indeed the variability in the hardware possessed by participants would almost certainly preclude accurate measurement of reaction times in any home delivered, web based platform). We include the oral version of the TMT in our telehealth protocol, as a measure comparable to the written TMT [44]. We have also trialled various versions of the Stroop task (Victoria Stroop [45], Dodrill Stroop [11]), to use as a measure of inhibition, but have ultimately abandoned it due to insensitivity (Victoria version) and difficulty presenting the stimuli appropriately via videoconference (Dodrill version).

Practical considerations for TeleNP

The potential benefits of TeleNP have long been recognised, including convenience, user satisfaction, potential cost-reductions and improved access (geographic; availability of interpreter services [46]). Nonetheless, the neuropsychological community has not uniformly embraced the adoption of TeleNP necessitated by the emergence of COVID-19. One of the most obvious concerns relates to whether TeleNP departs sufficiently from standardised face-to-face administration to invalidate test results and interpretation. There is accumulating evidence that telehealth delivered neuropsychological assessments can yield reliable and valid evaluations [[47], [48], [49]]. Since the emergence of COVID-19, a number of journal articles [4,[50], [51], [52]] and statements from professional bodies have provided guidelines and experience-based recommendations regarding the use of TeleNP (via position papers [53], webinars and online resources; see, for instance, the Australian Psychological Society [https://www.psychology.org.au/Event/21454], the International Neuropsychological Society [https://www.the-ins.org/webinars/], the Inter Organizational Practice Committee [https://iopc.online]). While the purpose of the present paper is not to provide a comprehensive review of TeleNP, we do provide a brief discussion on some issues relevant to our implementation.

The best TeleNP evidence concerns the use of tasks that are predominantly verbal in nature [47]. This encompasses the majority of measures we have selected for use (Table 1), and includes measures such as paragraph and word list learning tasks [[54], [55], [56], [57]], verbal span/working memory tasks (such as digit span) [[54], [55], [56], [57]], verbal fluency tasks [[54], [55], [56], [57], [58]], and measures of crystallized intelligence (e.g. measures of word reading and vocabulary [[57], [58], [59], [60]]). There is also evidence for tasks that rely upon verbal responses to visually presented stimuli, such as visual confrontation naming [54,55] and visuoperceptual reasoning tasks (e.g. WAIS Matrix Reasoning [57,58]). While supported by good evidence, it is worth noting that purely verbal tasks do not guarantee immunity to issues when administered via telehealth. Transient interruptions of the connection can interfere, especially with ‘one shot’ (e.g. digit span) or timed tasks (e.g. oral versions of SDMT and TMT). Our experience to date has been that poor connections are often apparent from the outset, and in many instances can be remedied simply by re-establishing the call or asking other users on the network to minimise their own network usage (e.g. streaming). We have had one participant whose computer microphone proved to be faulty but were able to proceed by using a concurrent telephone call accompanying the computer’s video feed without appreciable lag (a solution, incidentally, that we have employed in clinical practice also). Another participant was unable to establish a videoconference link from home, despite repeated attempts, and ultimately completed their teleNP assessment onsite.

More difficult to administer are tasks that require physical interaction with stimuli provided by the examiner (e.g. paper forms, three dimensional blocks). While there is some evidence for administration of such tasks via TeleNP (e.g. Grooved Pegboard [57]; written version of the Symbol Digit Modalities Test [57,61], Complex Figure Copy and Recall Tests [62], Clock Drawing Test [47,48,62]) we have ultimately elected not to include them for a variety of reasons (impracticalities of providing materials to participants: Grooved Pegboard; poor sensitivity and reliability: Rey Complex Figure [31]; suitable oral version available: SDMT). In other instances, we opted to retain the conceptual element of a traditional pen-and-paper task, but change the mode of delivery and response (e.g. using the oral versions of the Trail Making Test and Symbol Digit Modalities Test [44,57,61,63,64]), though we note that this likely alters what the task is actually measuring (e.g. pen-and-paper versus oral Trail Making Test [44]).

TeleNP is not without its challenges. Familiarity with the required technology – on the part of both the examinee and clinician – can influence the degree of engagement with, and the flow and ease of, the interaction. Indeed, we have found it essential to factor in ∼15 min of initial set up time at the beginning of appointments to ensure participants are able to log onto the videoconference call and that their technology is functioning appropriately (assisting them via phone as necessary). A single neuropsychologist administered the TeleNP assessments for our protocol, with this individual completing multiple supervised practice administrations prior to commencing participant data collection (and reviewing the aforementioned TeleNP webinars provided by the Australian Psychological Association and the International Neuropsychological Society once these were available). These practice sessions were essential to ensuring familiarity with the testing technology and practicalities of administration via telehealth. We have also developed a set of Standard Operating Procedures for telehealth to facilitate the training of new staff as the project expands.

The suitability of TeleNP for specific patient groups is an important issue, such as paediatric populations, people with intellectual disability or severe cognitive compromise, and linguistically and culturally diverse groups. Indeed, many of these concerns are also relevant to traditional face-to-face consultations. This complexity has not yet been fully addressed by the field and remains a critical challenge to the broad application of clinical TeleNP. However, the acquisition of uniform test data for machine-learning analysis is a narrower problem, where these issues are partly avoided through assessment of a necessarily more targeted cohort in which TeleNP administration is appropriate.

Access to technology is another issue of concern, since not all individuals possess the hardware required to support videoconference-based TeleNP. The use of technology at a local facility (e.g. GP clinic or research site) can increase availability and address issues of social equity, while simultaneously ensuring the quality of technology and connectivity [47,48]. Indeed, the majority of evidence for telehealth administration comes from studies where examinees are tested via technology at a local research facility [47]. We have made this approach available to participants, in order to improve participant access to the study (we offer free parking for participants who are able to travel to the facility by car and offer taxi vouchers for those who require transport). To date, roughly 20 % of participants have opted to complete their TeleNP testing on-site at the research facility (at the time of their MRI scan); the remainder have completed the TeleNP assessment using their own technology at home.

Operational changes we implemented to enable TeleNP are outlined in Box 2 . TeleNP appointments are conducted via Zoom (using a HIPAA compliant Education account; zoom.us), with the ‘password’ and ‘waiting room’ features enabled. TeleNP sessions are delivered by a qualified clinical neuropsychologist. For tasks using oral or screenshare-based stimulus delivery, responses are recorded by the examiner on original test record forms. The web-delivered, computer-administered CANTAB tests are recorded and scored by the software itself. Throughout the TeleNP session (including CANTAB testing), the examinee remains in audiovisual contact with the examiner, enabling monitoring of behaviours and the occurrence of potential distractions. To date, our TeleNP participants have responded positively, reporting the experience to be smooth and efficient, and appreciating the opportunity to carry out the assessment without leaving their home. Those who have previously undergone face-to-face assessment have noted the telehealth experience to be similar.

Prior to their TeleNP appointment participants are:

Screened to check that they have adequate technology to support TeleNP, defined at minimum as:

Computer or iPad with web cam, microphone and internet connection (smartphones and non-iPad tablets are not suitable, given software requirements of the web-delivered computer-administered testing used in the AEP, see below).

Quiet, distraction free room in which to complete the TeleNP assessment

Emailed a link to a set of electronically-hosted surveys tapping elements of psychosocial functioning germane to epilepsy (e.g. Neurological Disorders Depression in Epilepsy [39]; Patient Health Questionnaire GAD-7 [40]; Epilepsy Anxiety Survey Instrument [43]; QoLiE-31 [65]; Liverpool Adverse Events Profile [[66], [67], [68]]; ABNAS [69]). The surveys are hosted on a REDCap [37,38] database server at our institution.

Emailed a telehealth information sheet, along with text describing an ‘agreement to telehealth’ whereby participant and researcher agree that they will not “record, reproduce, publish or make copies of the materials used during the neuropsychology telehealth session” [70]. Participants are advised that their TeleNP session cannot proceed until they confirm acceptance of this agreement.

At the beginning of the TeleNP session the neuropsychologist:

Verifies the participant’s identity

Checks the participant’s telehealth technology setup (e.g. microphone and webcam setup and testing; quality of connection; disabling of other apps and notifications; suitability of environment)

Explains what will happen in the event of lost connection (attempt to reconnect; if unable to, will call mobile phone; if no contact within 10 min, session considered aborted and will be rescheduled)

Confirms the participant’s current location and obtains additional contact information in event of emergency (e.g. seizure). We explain that in the event of a seizure, if we cannot reach one of the contacts provided, or if we feel a more urgent response is appropriate, that we will call an ambulance. This information is summarised in a teleNP information sheet provided to all participants in advance of their session. We are yet to have a participant experience a seizure during testing.

Re-iterates terms of agreement to participate in telehealth (e.g. participant will not record or reproduce any materials)

Neuropsychological testing is administered via the following methods [47,50,53,54,58,[70], [71], [72]]:

via ScreenShare linked to a high resolution document camera: Test of Premorbid Function [6]; Matrix Reasoning from WASI-II [9]; Boston Naming Test [18]; oral Symbol Digit Modalities Test [64].

via oral stimulus delivery: letter and category verbal fluency, reverse digit span, oral Trail Making Test [44], Rey Auditory Verbal Learning Test [27], Vocabulary from WASI-II [9].

via web-delivered computer-administered testing: using the CANTAB [25] web platform. While testing via this platform can be completed by the participant in a standalone manner, we have the examiner remain on the videocall throughout, to handle any unanticipated problems that might arise and also to monitor behaviour during the testing.

For each task, the neuropsychologist records observations of anything that might invalidate a test (e.g. temporary connection loss; distraction). All data is recorded on response forms coding using a random six digit participant identification code, and then transcribed onto a secure central database (REDCap).

Alt-text: Box 2

TeleNP reveals a typical pattern of impairments in epilepsy

Fig. 2 summarises the cognitive data we have acquired at the time of writing. Given the relatively small sample to date (n = 29), data have not been subgrouped according to AEP referral type (first unprovoked seizure: n = 17, new diagnosis epilepsy: n = 6, refractory epilepsy: n = 6), or into method of neuropsychological test administration (teleNP-home: n = 18, teleNP-onsite: n = 6; face-to-face: n = 5). Data are expressed as z-scores, calculated relative to normative data. The distributions shown in Fig. 2 are as expected for an epilepsy cohort, with reductions in the domains of processing speed, working memory, executive function, language, and anterograde memory [13,22].

Fig. 2

Boxplots summarising performance across the sample of participants collected to date, colour coded according to cognitive domain. ToPF = Test of Premorbid Functioning; FSIQ = WASI-II FSIQ 2; SDMT = Symbol Digit Modalities Test; RVP = CANTAB Rapid Visual Information Processing; DSB = Digit Span Backwards; SWM = CANTAB Spatial Working Memory; SWM: be468 = SWM between errors; SWM: Strategy = SWM strategy score; TMT = Trail Making Test; OTS = CANTAB One Touch Stockings of Cambridge; OTS: 1st Try = OTS problems solved on first choice; IED = CANTAB Intra/Extradimensional Set Shift; IED: yerta = IED total errors adjusted; COWAT = letter fluency; Animals = animal fluency; BNT = Boston Naming Test; RAVLT = Rey Auditory Verbal Learning Test; RAVLT: loss = words lost between trial 3 and delay; PAL = CANTAB Paired Associate Learning; PAL: tea28 = PAL total errors adjusted; PAL: fams28 = PAL first attempt memory score. RAVLT z-scores are derived from a local sample of 72 participants: mean age = 30.8 ± 10.9 years, total words recalled across trials A1-A3 = 26.8 ± 5.3; delay score = 9.0 ± 2.8; words lost across delay = 2.1 ± 1.6. Among the notably reduced (z < -4) BNT scores in four participants, one is from an individual in whom English is a second language, spoken for approximately 4 years; one is from an individual with a suspected left parietal focus; one is from an individual with a suspected reading difficulty; and one is from an individual with suspected left TLE.

Table 2 summarises the data for each cognitive task. Performance is considered for the sample as a whole (overall mean z-score, SD, n and result of one sample t-test/Wilcoxon signed rank test [relative to a mean/median z-score = 0, one-sided test], and percentage of cases performing >1.5 SDs below expectation), and also separately for each method of administration: teleNP-at home, teleNP-onsite, face-to-face (n for each method of administration, result of a one-way ANOVA/Kruskall-Wallis test).

Table 2

Neuropsychological test scores for the overall sample, and separately for each method of administration (at home TeleNP; onsite TeleNP, and face-to-face).

TEST	μ	SD	p (t/T)	n	n: z<-1.5 (%)	μ(home)	μ(onsite)	μ(f2f)	p (F/H)
ToPF	0.34	0.75	0.99	29	0 (0.0)	0.59 (18)	−0.19 (6)	0.11 (5)	0.07
FSIQ-2	−0.11	0.82	0.23	28	1 (3.6)	0.06 (17)	−0.57 (6)	−0.16 (5)	0.36
RVP: a’	−0.25	0.89	0.07	28	2 (7.1)	−0.08 (18)	−0.97 (5)	−0.16 (5)	0.47
Oral SDMT	−1.13	1.18	0.01	9	3 (33.3)	−0.93 (8)	−2.73 (1)	NaN (0)	0.16
^DSB	−0.10	0.97	0.07	24	0 (0.0)	−0.15 (18)	0.06 (6)	NaN (0)	0.71
^SWM: be468	−0.21	1.45	0.15	28	3 (10.7)	−0.35 (18)	0.04 (5)	0.02 (5)	0.68
^SWM: Strategy	−0.16	1.12	0.08	28	2 (7.1)	−0.15 (18)	0.02 (5)	−0.41 (5)	0.78
^Oral TMT B	−0.93	0.91	0.00	24	7 (29.2)	−0.75 (18)	−1.45 (6)	NaN (0)	0.11
OTS: 1st Try	0.24	0.99	0.89	28	0 (0.0)	0.31 (18)	0.27 (5)	−0.04 (5)	0.53
^IED: yerta	−0.14	0.94	0.29	28	2 (7.1)	−0.04 (18)	−0.27 (5)	−0.40 (5)	0.67
COWAT	−1.30	0.87	0.00	24	11 (45.8)	−1.11 (18)	−1.86 (6)	NaN (0)	0.07
Animals	−0.37	1.40	0.09	29	5 (17.2)	−0.26 (18)	−0.56 (6)	−0.52 (5)	0.65
^BNT	−1.77	1.65	0.00	28	13 (46.4)	−1.90 (18)	−1.76 (6)	−1.18 (4)	0.89
RAVLT: sum(A1-A3)	−0.92	1.05	0.00	29	8 (27.6)	−0.77 (18)	−1.22 (6)	−1.10 (5)	0.42
RAVLT: delay	−1.27	1.28	0.00	29	14 (48.3)	−1.02 (18)	−1.69 (6)	−1.66 (5)	0.23
RAVLT: loss	−1.07	1.27	0.00	29	12 (41.4)	−0.97 (18)	−1.29 (6)	−1.18 (5)	0.66
PAL: tea28	−0.12	1.17	0.29	28	3 (10.7)	−0.06 (18)	−0.54 (2)	0.05 (5)	0.95
PAL: fams28	−0.24	1.24	0.15	28	4 (14.3)	−0.18 (18)	−0.72 (2)	0.01 (5)	0.97

^ = nonparametric test (Wilcoxon, Kruskall-Wallis), used when Shapiro-Wilk test p < .05; “p (t/T)” column reports p-value from one sample t-test (T-statistic if nonparametric test used) on z-scores collapsed across administration method; “n z < -1.5 (%)” = number (and percentage) of participants with z < -1.5 (expected in 6.7 % of a normally distributed sample); “home” = teleNP with participant at home; “onsite” = teleNP with participant at research site; “f2f” = traditional face-to-face neuropsychological testing; parenthetic numbers in “μ(home)”,” μ(onsite)”, and “μ(f2f)” columns report n for those conditions; “p (F/H)” column reports the p-value from a one-way ANOVA (based on F; H-value if nonparametric test used) assessing effect of administration method; ToPF = Test of Premorbid Functioning; FSIQ-2 = WASI-II FSIQ 2; SDMT = Symbol Digit Modalities Test; RVP = CANTAB Rapid Visual Information Processing; DSB = Digit Span Backwards; SWM = CANTAB Spatial Working Memory; SWM: be468 = SWM between errors; SWM: Strategy = SWM strategy score; TMT = Trail Making Test; OTS = CANTAB One Touch Stockings of Cambridge; OTS: 1st Try = OTS problems solved on first choice; IED = CANTAB Intra/Extradimensional Set Shift; IED: yerta = IED total errors adjusted; COWAT = letter fluency; Animals = animal fluency; BNT = Boston Naming Test; RAVLT = Rey Auditory Verbal Learning Test; RAVLT: loss = words lost between trial 3 and delay; PAL = CANTAB Paired Associate Learning; PAL: tea28 = PAL total errors adjusted; PAL: fams28 = PAL first attempt memory score. RAVLT z-scores are derived from a local sample of 72 participants: mean age = 30.8 ± 10.9 years, total words recalled across trials A1-A3 = 26.8 ± 5.3; delay score = 9.0 ± 2.8; words lost across delay = 2.1 ± 1.6.

The ANOVA/Kruskall-Wallis tests indicate that no cognitive test shows a significant effect of administration method (p > .05, albeit with small group sizes). Collapsed across administration method, one sample t-tests/Wilcoxon signed rank tests confirm a pattern of impairments typical of those seen in epilepsy, with significant (p < 0.05) reductions in processing speed (oral SDMT), executive function (oral TMT B, COWAT), language (BNT, Animal fluency) and anterograde memory (RAVLT measures). Trends (p < 0.08) were also apparent for sustained rapid information processing (CANTAB RVP) and working memory (DSB, CANTAB SWM).

Screening of mood and anxiety also confirmed a relatively high proportion of individuals at risk for these disorders: 10 of 29 participants (34 %) were at high risk of mood disorder based on the NDDIE [total score > 15, see reference 39]; and between 8 and 11 (28 % and 38 %) were at high risk of anxiety disorder based on the brEASI [total score > 7, see reference 43] and GAD7 [total score > 7, see reference 42], respectively.

The future of TeleNP in epilepsy research and clinical trials

The social distancing requirements stemming from COVID-19 are likely to be with us for a long time. This highlights the importance of expanding the testing options available to neuropsychology, by developing assessment tools explicitly designed for use via telehealth. Such developments would be of great benefit even once the need for social distancing has passed, improving access to neuropsychological services. Existing, evidence-based screening tools already used in epilepsy (e.g. EpiTrack [13]) could be adapted and validated for delivery via online platforms. In the process, such tools could be extended, ensuring coverage of epilepsy relevant neuropsychological domains and exploiting the response sampling available via a computerised medium. The guiding principle should be to target those domains most affected or important to people with epilepsy [73], using measures sensitive to the lifetime variability of the condition, from disease onset through introduction of anti-seizure medications (ASMs) to chronic refractoriness and surgery. These domains include:

Anterograde memory: the most common cognitive complaint in epilepsy [74]; the majority of focal epilepsies affect the temporal lobe [75].

Executive functions/fluid intelligence: sensitive to ASM effects [13]; the frontal lobes are frequently involved in focal [75] and genetic [76] epilepsies.

Crystallized intelligence: considered less susceptible to ASM effects [77]; can be affected by age of epilepsy onset [78]; provides a measure of cognitive reserve [79].

Mood: frequently disturbed in people with epilepsy [80].

Adverse treatment side effects

Such screening assessments cannot replace comprehensive assessments and may miss subtle problems for some individuals. However, unlike other existing tools that have been developed for dementia screening, the tools would be validated for epilepsy, be age and education adjusted, address functions most often affected during the course of epilepsy and its treatment, and be useable for the remote assessment of patients unable to attend face-to-face assessment or who would otherwise be lost to follow-up. Ultimately this kind of approach will facilitate large-scale collection of data that would not otherwise be practical using traditional methods.

TeleNP in the clinical management of epilepsy

To this point we have emphasised the role of TeleNP in research. What of the role of TeleNP in clinical care? While COVID-19 strains the health systems of many countries around the world, individuals nonetheless continue to experience seizures and associated cognitive and psychological comorbidities. In response to COVID-19 and the rush towards telehealth, the ILAE Neuropsychology Task Force underscored that comprehensive telehealth neuropsychological assessments for epilepsy surgery candidates have not yet been carried out or validated [81]. The Task Force recommends that any surgical candidates proceed to surgery only after a comprehensive, face-to-face, neuropsychological work up, concluding that, “whilst compromise and new ways of working are necessary for urgent neurosurgical procedures, epilepsy surgery should not be conducted as an emergency procedure.” While this position aspires to an ideal, the effects of COVID-19 will in all likelihood be with us for a long time yet, and epilepsy surgery cannot simply cease to occur for the foreseeable future, especially not if the only barrier is failure to carry out a neuropsychological evaluation. In our view, a reasonable position would accommodate TeleNP using epilepsy-relevant, evidence-based instruments, augmented by shorter face-to-face assessments where required to address specific clinical issues.

The impetus to move towards telehealth stimulated by COVID-19 should be viewed as an opportunity to expand the reach and breadth of neuropsychology [47,54]. The clinical neuropsychologist’s role in an epilepsy surgery program extends beyond psychometric documentation. The delivery of counselling, psychoeducation, advocacy, and psychotherapy via telehealth has a solid evidence base [82], including in epilepsy specific contexts [[83], [84], [85], [86]], and is encouraged in recent set of epilepsy-specific consensus recommendations [87]. Speaking from the local perspective, within the Department of Clinical Neuropsychology at Austin Health we already employ telehealth (telephone, videoconferencing using the Coviu platform) routinely in the pre- and post-surgical counselling of patients [88]. Anecdotally, a number of our patients have commented that they feel more comfortable in their home environment, and find it easier to be open about their experiences when communicating through the intermediary of technology. The average age of our surgical cohort is in the mid-30′s representing a generation for whom technology is ‘second-nature’. Approximately 20 % of our patients live outside of metropolitan centres. Further expanding the role of TeleNP would be of great benefit to such a patient demographic, improving timely access to care. For example, it would enable important post-operative evaluations and follow-up without the burden of travel and the attendant costs and psychosocial disruption.

These anecdotal benefits are substantiated by our recent experience in Germany using phone or videoconference telemedicine in the counselling of people with epilepsy during the COVID pandemic [89]. Overall 82 % of the 239 adult epilepsy patients participating in the audit were satisfied with their telemedicine experience, with high rates of satisfaction especially for time, comprehensibility, and opportunity to get answers to current questions. The participants considered immediate convenience and shortfall of travel expenses as advantages of telemedicine. Approximately three quarters of participants reported that they would appreciate the opportunity for future telemedical counselling, but at the same consider telemedicine as an add-on service rather than a permanent substitute to visits onsite.

Conclusion

COVID-19 has abruptly and dramatically changed the way that society functions, including the operation of the health and medical research sector. Our experience shows that it is possible to continue to perform evidence-based, epilepsy-related neuropsychological research while at the same time fully supporting public health strategies aimed at containing and mitigating the effects of COVID-19. In the event that sustaining such policies into the medium or longer-term is necessary, the strategies adopted by the AEP have positioned it to continue to grow and expand, with no impact on the feasibility, integrity or safety of the project. Indeed, this model of telehealth-based operations provides a template for the healthcare of tomorrow, while decreasing the burden on traditional hospital systems. The challenges posed by COVID-19 are immense, and we must respond swiftly and creatively, where possible converting the adversity of the present into opportunities for the future.

Declaration of Competing Interest

Christoph Helmstaedter has received grants from the EU, travel support by Desitin, honoraries for talks, counselling, and advisory boards by GW pharmaceuticals, Eisai, UCB pharma, and Precisis, as well as license fees by EISAI, UCB pharma and Precisis. The remaining authors have no conflicts of interest.