Pediatric Teleneuropsychology: Feasibility and Recommendations.

OBJECTIVE: Teleneuropsychology (TeleNP) is a growing and promising practice within the telemedicine landscape that has been well established within the adult neuropsychology literature. This project aimed to demonstrate the feasibility of TeleNP in a pediatric clinical population and disseminate clinical decision-making procedures to guide best practices for pediatric TeleNP.
METHOD: This project conducted during the 2019 coronavirus (COVID-19) pandemic reflects the largest clinical cohort to date of TeleNP in a pediatric population (N = 129). Data were gathered retrospectively from patients who were rescheduled from in-person assessment to TeleNP between March to June 2020.
RESULTS: TeleNP was an accessible option for most patients and families, with no differences in demographic variables in patient appointment attendance, whether testing was conducted, and whether the patient was referred for face-to-face follow-up. Patients using laptops/desktops were more likely to undergo remote test administration in comparison with patients using phones/tablets (χ2 = 23.83, p < .002). Sixty-three percent of the sample were referred for a face-to-face follow-up assessment.
CONCLUSIONS: TeleNP is feasible in a pediatric clinical population ranging to begin the process of differential diagnosis and treatment planning. Pediatric TeleNP may be most efficacious as a screening procedure due to limited measures suited for remote administration and behavioral challenges interfering with testing requiring in-person follow-up. TeleNP screening as standard practice for patients who do not require a full, traditional neuropsychological battery may provide a more efficient care model, with more patients able to be seen using shorter batteries with less wait time. Published by Oxford University Press 2020.

Introduction

Telemedicine, or “medicine at a distance,” has been traditionally advocated within the medical field to extend healthcare reach to patients within rural settings or during natural disasters that preclude access to medical care. Evidence supports the use of telemedicine as a cost-effective alternative to traditional patient care, given face-to-face increased access to quality care, ecologically enhanced treatment, and reduction in travel burden for patients and their families (Hjelm, 2005). Telemedicine may include clinician-to-clinician consultation, remote patient monitoring, patient-to-clinician communication, telephone consultation, and videoconferencing between patient and clinician (Wosik ). The use and feasibility of videoconferencing methods, the focus of this paper, have increased due to multiple factors, including ongoing developments in technology, computing, and communication methods as well as an increase in the number of households with internet use (41.5% in 2000 vs. 85.3% in 2018, Statista, 2019).

Telemedicine is utilized by a broad range of medical subspecialties including, but not limited to, primary care (e.g., Mold, Hendy, Lai, & de Lusignan, 2019), palliative care (e.g., Calton, Abedini, & Fratkin, 2020), oncology (e.g., Sirintrapun & Lopez, 2018), neurology (e.g., Hatcher-Martin ), psychiatry (e.g., Kalin, Garlow, Thertus, & Peterson, 2020), psychology (e.g., Perrin ), and neuropsychology (e.g., Cullum, Hynan, Grosch, Parikh, & Weiner, 2014; Cullum, Weiner, Gehrmann, & Hynan, 2006; Grosch, Gottlieb, & Cullum, 2011). Telemedicine historically targeted adult patient populations and remains underutilized and understudied (Grosch ) within the field of pediatric neuropsychology.

Teleneuropsychology (TeleNP) is a growing and promising practice within the telemedicine landscape. Practice recommendations for TeleNP were first outlined by Grosch and coworkers (2011) and include guidelines for informed consent, privacy, and confidentiality, TeleNP competence, assessment-specific considerations (e.g., test validity and reliability, administration standardization, and test security), licensure and billing issues, and technical specifications. In addition to the benefits of telemedicine outlined above, TeleNP is generally well-accepted by consumers, with one study reporting a 98% satisfaction rate and 2/3 of older adult participants indicating no preference for traditional versus TeleNP assessment (Parikh ).

Much of the research within clinical and non-clinical adult populations highlights concordance between traditional face-to-face and TeleNP assessment, although subtle differences have been captured for specific assessment measures. The first meta-analysis of TeleNP with adults was conducted by Brearly and coworkers (2017). Results indicated that verbally mediated task performance was not affected by videoconference test administration, with comparable performance to on-site administration. Building on that work, Marra, Hamlet, Bauer, and Bowers (2020) conducted a meta-analysis that included 19 studies of neurocognitive testing in older adults. Consistent with earlier work by Brearly and coworkers (2017), results indicated that verbally mediated measures (e.g., digit span, verbal fluency, list learning) were not impacted by the assessment method, whereas there was less support for visually mediated, motor, and timed tasks (e.g., the Clock Drawing task, Oral Trail Mailing A & B, BVMT-R). In sum, research generally supports the feasibility, reliability, and validity of remote administration through secure videoconferencing methods (e.g., Brearly ; Cullum ; Cullum ).

Although TeleNP research with adults is growing, there is a dearth of research examining the feasibility of TeleNP within pediatric patient populations. Although not specific to TeleNP, Hodge and coworkers (2019a) compared the use of literacy measures (i.e., Woodcock Reading Mastery Tests-Third Edition, Test of Word Reading Efficiency-Second Edition, MultiLit, Dalwood Spelling Test, and parent-report questionnaires) across web-based and face-to-face methods in a sample of 37 Australian children with reading difficulties. Correlation analyses indicated strong agreement between delivery methods (Spearman‘s r = .79–.99). Furthermore, parents of participants expressed overall acceptance of remote assessment. In a second study by the same collaborators, participants completed an assessment of intellectual functioning via telehealth and face-to-face with strong correlations again identified between the two modalities. In the telehealth modality, test materials were scanned, digitized, and administered to participants via a “shared screen” allowing for the inclusion of subtests that were not verbally-mediated (e.g., Matrix Reasoning subtest of the Wechsler Intelligence Scale for Children—Fifth Edition; Hodge ). In both studies, a trained assistant was present on-site with participants to help with any technical difficulties, provide materials as needed, and co-score the assessments alongside a research assistant who was stationed remotely and provided the web-based administration. Most recently, the first investigation of in-home pediatric TeleNP showed good feasibility and reliability compared with in-person test results (Harder ).

Given requirements for neuropsychological assessment, models of TeleNP tend to be more complex than traditional telemedicine practices. Research studies comprise much of the existing TeleNP literature and to maintain a controlled research environment, participants are typically located in a separate room inside a clinic or hospital, rather than situated in their home (Marra ). Importantly, Wadsworth and coworkers (2018), in their validity study of TeleNP with older adults, determined that a local staff member was not needed during testing, but was available on-site should technical difficulties arise. In their study, even older adult participants with clinical diagnoses of mild cognitive impairment or Alzheimer’s disease reportedly did not require special assistance and acclimated well to the virtual testing environment.

The 2019 Severe Acute Respiratory Syndrome Coronavirus 2 (COVID-19) global pandemic presented a unique opportunity to utilize TeleNP within a large-scale children’s hospital setting, one that involves remote, in-home, videoconference-based assessment, without a trained, on-site assistant. Given quarantine restrictions, “safer at home” policies, and a need to reduce the transmission of COVID-19, conventional face-to-face neuropsychological assessments were strongly discouraged by some organizations and completely restricted by others across the USA. Within many large-scale healthcare centers, the waitlist for a neuropsychology appointment continued to grow, prior appointments were put on hold, institutions faced potential economic losses if assessments could not be completed, and job security for employed neuropsychologists and staff may have been threatened. Many medical subspecialties were forced to rapidly adopt new delivery methods of traditional face-to-face care and population-level interest in telemedicine steadily grew as COVID-19 cases increased (Bokolo, 2020; Hong, Lawrence, Williams, & Mainous III, 2020). The neuropsychology community quickly mobilized to provide justification for TeleNP based on prior research, as well as training resources, including courses and webinars to assist neuropsychologists in the use of TeleNP.

In March 2020, an advocacy team established within The Inter Organizational Practice Committee (IOPC) provided recommendations regarding the use of TeleNP during the COVID-19 pandemic (Bilder ). The IOPC is a team of practice chairs from the American Academic of Clinical Neuropsychology/American Board of Clinical Neuropsychology, the National Academy of Neuropsychology, Division 40 of the American Psychological Association, the American Board of Professional Neuropsychology, and the American Psychological Association Services, Inc. Furthermore, guidelines were provided by national organizations, including The American Psychological Association (APA), The Association of State and Provincial Psychology Boards, and The American Telemedicine Association. Historically, the ability to practice TeleNP was restricted by Medicare policy limiting reimbursement for telemedicine services solely to designated rural areas when patients were evaluated in a clinic, hospital, or other medical facility. Secondary to the COVID-19 pandemic and strong advocacy work by APA, Medicare, Medicaid, and many private insurance companies expanded telehealth coverage allowing for reimbursement of neuropsychology CPT codes via real-time, video telecommunication with no geographic restrictions. Taken together, this created an environment where TeleNP became both feasible and reimbursable. Most importantly, transitioning to TeleNP allowed neuropsychologists in this clinic to continue to meet the growing cognitive, emotional, and behavioral needs of patients and families heightened by the global COVID-19 pandemic.

Despite the growing body of literature documenting the feasibility, reliability, and validity of TeleNP in adults, few studies to date have been conducted using in-home evaluation procedures in a pediatric population to determine feasibility with different patient characteristics (Harder ). This project aimed to demonstrate the feasibility of TeleNP in a pediatric clinical population and disseminate clinical decision-making procedures to help guide pediatric TeleNP practice.

Method

Subjects

This project utilized a retrospective sample of clinical patients from a children’s hospital in the Southeastern USA who were initially scheduled for a face-to-face neuropsychological evaluation between 23 March and 30 June 2020 and addressed individually by each provider to determine appropriateness for telehealth. This sample included neuropsychological referrals from standard pediatric clinical service lines representing a range of complex medical and neurologic issues, including traumatic brain injury, low birth weight/prematurity, cancer survivorship, autoimmune conditions such as systemic lupus erythematosus (SLE), epilepsy, and cardiovascular concerns, among others, across the pediatric age spectrum (ages 1 to 21 years). Patients with hearing/vision impairment (n = 0), need for an interpreter (n = 2), or evidence of significant behavioral challenges (as noted in referral information or medical record review, n = 3) were not scheduled for a TeleNP evaluation due to the demands required in these patient populations and were rescheduled for face-to-face assessment when permitted by state mandate. Eight patients declined TeleNP in favor of rescheduling to a later date when in-person testing could take place. Thirteen patients were rescheduled for in-person testing with no additional documentation provided due to medical record limitations. All subjects spoke English as a primary language and testing was deferred in cases where English was not the primary language (n = 2). Our sample consisted of patients covered by a range of payors, with 65% insured by Medicaid within the state of Florida. Of note, our clinic does not have contracts for Medicaid providers with the neuropsychology service at this time. As a result, no pre-authorizations were required for our Medicaid patients, as all neuropsychology services are provided free of charge. All commercial insurance providers were billed as per current contracts, without the need for pre-authorization. No information regarding reimbursement was available at the time of this project.

In high-risk early childhood patients (< 3 years) referred for concerns with an early developmental delay following significant perinatal history (e.g., prematurity, low birth weight, etc.), TeleNP appointments were completed on all who were referred. These appointments included a clinical interview, parent ratings, and structured behavioral/play observation. Standard neuropsychological assessment procedures (clinical interview, behavioral observations, rating scales, and objective testing measures) were modified when appropriate for remote administration with school-aged children and adolescents. Each provider adapted a test list to accommodate the referral question and patient needs (see Table 1 for domains assessed and measures used). Measures with modified standardization, other than administration via TeleNP, are noted. This project was acknowledged as Quality Improvement and exempt by the Institutional Review Board. See Fig. 1 for decision points on patient appropriateness for TeleNP.

Table 1

Tests administered via teleneuropsychology by domain

		Standardization kept
Domain	Measure(s) Used	Yes	Yes, with modifications
Intellectual abilities	WASI-II vocabulary	X
	WASI-II similarities	X
	WASI-II matrix reasoning		X
	DAS-II		X
Developmental	Bayley-3		X
Learning and memory	ChaMP screener		X
Processing speed	Oral symbol digit modalities		X
Attention/executive functioning	WISC-V/WAIS-IV digit span forward	X
	DKEFS verbal fluency		X
	NEPSY-II word generation		X
	NEPSY-II inhibition		X
	DKEFS inhibition		X
	Oral trails		X
	DKEFS 20 questions		X
Academics	Bracken expressive form (SRC and subtest 7)		X
	CTOPP phonological awareness	X
	WIAT-III word reading	X
	WIAT-III spelling	X
	WIAT-III pseudoword decoding	X
Rating scales	BRIEF-2	X
	BASC-3	X
	PCEI	X
	CNMC	X
	DP-3	X
	MCHAT	X
	Vineland −3 Interview	X
	ABAS-3	X
Validity	MVP	X

Fig. 1

Teleneuropsychology decision tree.

TeleNP equipment and setup

The TeleNP videoconferencing equipment and procedures for this project included the Teladoc Health system, a HIPAA-compliant software with secure encryption of the communication transmission. The Teladoc Health system allows for connectivity between a provider using software via laptop, desktop, or tablet, to a patient using any device that has a camera and microphone. The HITRUST certified connection between the patient and provider runs through Teladoc Health’s global communications network. The network supports connectivity ranging between 300 kbps and 3 Mbps, depending on the technology or application used to make the connection. All traffic at rest and in transit are encrypted to the highest standards. The system leverages dynamic bandwidth management algorithms to maintain reliable audio/video interactions even in the event of varying bandwidth availability.

The Teladoc Health system was used to connect neuropsychology providers in their respective offices on the children’s hospital campus with patients and families quarantined at home. No TeleNP services were conducted by providers outside of the hospital setting. Subjects viewed the examiner through their personal electronic device (i.e., phone, tablet, laptop, or desktop). For the presentation of visual stimuli, the examiner utilized a screen sharing function and/or document camera to ensure proper stimuli orientation. Patients utilizing a handheld device were not administered visual stimuli due to variability in aspect ratio and examiner concerns for adherence to standardized test administration.

Procedures

Patient screening

TeleNP screening appointments were offered to appropriate patients/families to maintain distancing mandates due to COVID-19. Patients who declined TeleNP services were scheduled for face-to-face evaluation when non-emergent ambulatory care was permitted to resume in the state. Similarly, patients who met the ineligibility criteria described above were determined to not be appropriate for TeleNP service and were scheduled for a face-to-face evaluation when non-emergent ambulatory care was permitted to resume in the state.

Patients determined by the neuropsychologist to be appropriate for TeleNP were contacted in advance and offered a standard, face-to-face assessment within 90 days or TeleNP screening at the time of their previously scheduled appointment. Eight families declined telehealth at the initial screening in favor of deferring until face-to-face assessment was available. Families who opted to proceed with TeleNP were provided with a background history questionnaire and selected rating scales via email to be completed, returned, and scored before the TeleNP appointment.

Preparing for the TeleNP appointment

Providers were instructed to dress to their typical level of workplace professionalism and to utilize proper lighting to improve engagement. All patients were provided guidance to minimize technological issues, including updating device software to the latest version and ensuring reliable internet connection, proximity to internet source or wireless router, and compatible web browser (Google Chrome). Patients were not instructed on device type (e.g., smartphone/tablet versus laptop/desktop) to use prior to the evaluation to promote access to care.

Conducting the TeleNP evaluation

During the scheduled TeleNP evaluation, providers reviewed informed consent with parents/caregivers and acquired assent from patients before proceeding with a clinical interview. Parents/caregivers reporting longstanding developmental delays, concerns for Autism Spectrum Disorder, or significant behavioral concerns during the clinical interview portion of the TeleNP appointment were informed that TeleNP would not sufficiently meet patient needs and were provided with a face-to-face assessment appointment within 90 days (n = 7), comparable to exclusion criteria applied during the pre-appointment screening process. In the event that a patient-reported active self-harm, suicidal or homicidal ideation, providers were instructed to inform the patient and family to seek emergency care at the treatment facility identified during the informed consent process. Providers collected the patient’s physical address and phone number during the consenting process to ensure law enforcement could contact them and identify the location if needed (n = 0).

Patients determined by the neuropsychologist to continue to be appropriate for TeleNP based on information obtained during the clinical interview were provided information regarding general testing procedures. Prior to a beginning test administration, the ideal test environment was discussed with families, including the importance of maintaining a separate, quiet, distraction-free space with no interruptions, removal of cell phones, and no testing assistance provided to the examinee. Parents/caregivers were utilized as on-site assistants in case of technical difficulties but were otherwise instructed to leave the room. Exceptions were made in evaluating young children (<3 years) whose parents/caregivers provided testing assistance to support examiner ratings of developmental progress. In these instances, parents were given specific guidance on how to engage their young child in structured activities utilizing common household objects, unstructured play schemas utilizing the child’s preferred toys, and behavioral observation of typical daily life activities (e.g., eating, drawing, crawling, walking, etc.). Finally, parents/caregivers were instructed to not record the appointment through screen recording options featured in select software packages. See the Supplementary Appendix A for sample scripts to be read to patients and caregivers to facilitate TeleNP test administration and setting the home-based test environment. Providers selected test batteries based on clinical necessity and ability to administer via TeleNP. A list of selected assessment measures per domain is provided in Table 1.

Following the completion of test administration, patients were placed on hold while measures were scored and interpreted. The provider then gave feedback to the parents/caregivers and patient (when appropriate) regarding recommendations for treatment planning or face-to-face follow-up testing. TeleNP evaluation results were provided in the form of a written report. Sample report language detailing modifications for TeleNP is included in Supplementary Appendix B. Patients evaluated via TeleNP whose clinical needs were not fully met were scheduled for a face-to-face follow-up evaluation when non-emergent, ambulatory care was permitted to resume by the state (Fig. 1).

Data collection

After the conclusion of the time period (23 March 2020–30 June 2020), neuropsychologists and trainees reviewed the electronic medical record for patients scheduled for evaluation. Basic demographic information was collected from the medical record, as well as data from neuropsychological testing to address project aims for patients who arrived for their appointments.

Statistical analyses

Descriptive statistics were examined for all variables of interest in IBM SPSS Version 26. Independent samples t-tests and chi square analyses were conducted to assess the relationship among demographic, accessibility, and clinical variables to better understand the feasibility of TeleNP. Demographic variables included patient age, race/ethnicity, and parent education level. Accessibility variables included appointment attendance (yes/no), device type (i.e., phone/tablet or laptop/desktop computer), and whether testing procedures were initiated and/or completed. Clinical variables included evaluation focus (standard neuropsychological, focused psychological, or developmental), and clinical population.

To better understand provider decision-making to proceed with TeleNP prior to the appointment and following the initial meeting with the family, inter-rater reliability analyses were utilized and outlined in Fig. 1.

Results

In total, 129 patients were identified out of 155 in an initial review of patients scheduled, including 63 males and 66 females between the ages of 1 and 21 (M = 8.78 years, SD = 6.538). Eighty-one percent of patients (n = 105) who were scheduled for a TeleNP appointment arrived for their appointment and were seen by a provider. There were no differences among demographic variables and the ability to access TeleNP. Specifically, age (t (114) = 0.332, p = .74), race/ethnicity (χ2 = 7.56, p = .06), maternal education (χ2 = 4.48, p = .48), or paternal education (χ2 = 3.45, p = .63) did not significantly predict whether patients attended their scheduled TeleNP appointment. See Table 2 for full sample demographics and Table 3 for the demographics of patients who attended their scheduled appointment.

Table 2

Demographics of sample reviewed for teleneuropsychology consideration

		N	%	Minimum	Maximum	M	SD
Age		129	—	1	21	8.78	6.53
Gender	Female	66	51
Race	White	86	67
	Black	34	26
	Asian	2	2
	Other	7	5
Ethnicity	Non-Hispanic	114	88
	Hispanic	14	11
	Missing	1	<1
Focus of evaluation	Standard Neuropsychological	68	53
	Developmental	30	23
	Psychological	4	3
	Missing	15	12
	Other	12	9
Arrived for appointment	Yes	105	81

Table 3

Demographics of patients who attended their scheduled teleneuropsychology appointment

		n	%	Minimum	Maximum	M	SD
Age		105a		1	21	8.83	6.52
Gender	Female	54	51
Race	White	74	70
	Black	25	24
	Asian	2	2
	Other	4	4
Ethnicity	Non-Hispanic	94	90
	Hispanic	11	10
Maternal Education	Less than High School	2	2
	High School/GED	14	13
	Some College/AA	21	20
	College Graduate	15	14
	Post-College Education	10	10
	Not Reported	43	41
Paternal Education	Less than High School	3	3
	High School/GED	22	21
	Some College/AA	8	8
	College Graduate	15	14
	Post-College Education	8	8
	Not Reported	49	46
IQ scores	WASI-II Vocabulary T-Score	39	—	22	80	49.46	12.28
	WASI-II Matrix Reasoning T-Score	39	—	23	71	47.59	11.23
	WASI-II Similarities T-Score	7	—	20	62	45.71	14.14
	DAS Naming T-Score	8	—	41	52	45.25	4.33

aOne hundred five patients arrived for the appointment after 129 deemed appropriate.

Of the patients who attended their TeleNP appointment (n = 105), there were no significant differences in age (t (114) = 0.46, p = .80), race (χ2 = 4.64, p = .20), maternal education (χ2 = 0.78, p = .98), or paternal education (χ2 = 2.87, p = .72) and whether a provider proceeded with remote test administration during the TeleNP appointment. Patients scheduled for neurodevelopmental assessment in early childhood or focused psychological testing underwent testing significantly more often after the initial interview, with eight standard neuropsychology evaluations and one focused psychological evaluation deferred to face-to-face testing (χ2 = 17.59, p = .001) (Table 4).

Table 4

Teleneuropsychology evaluation characteristics and clinical decision-making

		n	%
Evaluation focus	Standard neuropsychological	59	56
	Focused psychological	4	4
	Developmental	30	29
	Other/missing	12	11
Device used	Phone/tablet	24	23
	Computer	52	50
	Unknown	29	27
Testing decision-making	Seen for testing	84	80
	Deferred testing	21	20
Clinical population	Traumatic brain injury	13	12
	Developmental	17	16
	Can survive screener	14	13
	SLE screener	6	6
	Cardiac	3	3
	CAR-T screener	2	2
	General intake	3	3
	Focused psychological	1	1
	General neuropsychological	29	28
	Missing	17	16
Referred for follow-up in 6 months	Yesa	66	63
	No	39	37

aOf the 66 referred for follow-up, 21 were deferred prior to initiating testing.

The patient device type was significantly related to whether remote testing was completed (χ2 = 23.83, p < .002). Patients using a laptop or desktop (n = 52) were more often tested through remote administration than patients using a phone/tablet (n = 24). Younger patients (0–2 years) referred for neurodevelopmental assessment exclusively used a phone or tablet for evaluation as these devices afforded the most mobility for the type of testing performed. Older children were more often evaluated when utilizing a laptop or desktop computer.

Of those tested utilizing the WASI-II (n = 39), 37 (95%) were considered standard neuropsychology evaluations and two (5%) were considered a focused psychological evaluation. Of those testing using the DAS-II, one (12%) was considered a standard neuropsychological evaluation, and seven (88%) were considered high-risk early childhood patients referred to neurodevelopmental testing due to significant perinatal concerns.

There was no significant relationship between age (t (96) = 0.31, p = .75), race (χ2 = 4.12, p = .24), maternal education (χ2 = 4.87, p = .43) or paternal education (χ2 = 4.27, p = .51) and provider decision to refer for face-to-face follow-up assessment. Of those patients who received testing via telehealth (n = 129), 63% were referred for face-to-face follow-up assessment, with the most common reason cited as the need for additional testing utilizing measures that could not be completed via the telehealth platform (Fig. 1). Forty-three percent of patients who underwent standard neuropsychological evaluations were referred for a face-to-face follow-up assessment in comparison to 25% of focused psychological evaluations and 7% of early childhood patients referred for neurodevelopmental assessment (χ2 = 21.48, p < .001). Fifty-eight percent of general neuropsychology patients (n = 17 of 29), all three cardiac patients, and 8 of 13 traumatic brain injury patients were referred for face-to-face follow-up. Patients referred for neuropsychological screening from clinical service lines as part of routine follow-up care were more likely to complete the evaluation process through TeleNP. Specifically, four of six patients with SLE and 16/18 (87%) of patients with a history of pediatric cancer had their needs met via TeleNP. The most common reasons that patients were referred for face-to-face follow-up evaluation following initial telehealth contact are outlined in Fig. 1.

Discussion

These data reflect the largest sample to date to support the use of TeleNP in a pediatric population. We found TeleNP to be feasible with children presenting with a broad range of clinical concerns across a wide age spectrum ranging from very early childhood to young adulthood. TeleNP allowed our team to continue to meet the clinical needs of many patients amid the global COVID-19 health pandemic when many other ambulatory service lines were shut down or operating at extremely limited capacity. Our ambulatory neuropsychology clinic typically operates on a 12–24-month waitlist, and the provision of service through TeleNP enabled waitlists to remain stable instead of continuing to increase due to ambulatory care shut down.

TeleNP was an accessible option for most patients and families in this project, with no differences in demographic variables on patient show rate, whether testing was conducted, and whether the patient was referred for face-to-face follow-up. Children ages 0–2 were more likely to be assessed using a phone or tablet, reflecting the flexible test environment permitted in neurodevelopmental assessment. TeleNP provided an optimal scenario for providers to observe target behaviors in early childhood and to assess development in a familiar home setting. Older children (ages 3+) were more often assessed when parents/caregivers accessed the TeleNP platform via laptop or desktop computer. This was thought to be due to increased demands for a standardized test environment associated with traditional neuropsychological testing conducted with older children in comparison to the permitted flexibility of early childhood testing. Patients and parents/caregivers who utilized handheld devices (i.e., phones/tablet) for the appointment were anecdotally observed to move around in space, limiting provider confidence in the test environment due to additional distractions. In addition, the aspect ratio of laptops/desktops are more comparable to face-to-face assessment for visual stimuli than handheld devices and increased provider confidence that results obtained through TeleNP were commensurate with presumed results that would have been obtained during face-to-face testing. When taken together, access to TeleNP was limited in this cohort by type of device utilized during the evaluation and suggests that patient resources may be a barrier to care.

Most patients (83%) scheduled for a face-to-face assessment before COVID-19 were determined by providers to be appropriate for a TeleNP evaluation. As previously noted, exclusion criteria included disruptive behaviors limiting participation in remote evaluation and need for autism spectrum disorder evaluation with components beyond the scope of TeleNP (i.e., face-to-face engagement required to comprehensively evaluate social cognition, including rapport building, pragmatic communication, and interactive play). Patients were evaluated via TeleNP for a range of cognitive, emotional, and behavioral needs stemming from medical morbidity, including the history of cardiac syndromes, traumatic brain injury, epilepsy, genetic syndromes, SLE, low birth weight/prematurity, and pediatric cancer, among others. All patients and families who received care via TeleNP were provided with a preliminary screening report and recommendations for initial treatment planning. Patients who were subsequently referred to services with historically lengthy wait lists, such as psychology and psychiatry, were able to be seen by these specialists via telehealth earlier than they would have if their neuropsychological evaluation was deferred until state mandates for ambulatory visits were lifted (Andrilla, Patterson, Garberson, Coulthard, & Larson, 2018). In addition, early consultation through TeleNP allowed providers to support families transitioning to online learning platforms in the acute stage of COVID-19. Parents often expressed relief during feedback for assistance in adjusting to their new expected role as “online learning teacher/facilitator” through recommendations aimed to support structuring daily routines, adjusting expectations for productivity, and in some cases, managing the educational needs of students with individual education needs typically provided through Section 504 Plans or Individual Education Plans.

Despite the benefits of the TeleNP screening, the majority of patients (63%) were determined to require face-to-face follow-up assessment to complete the evaluation. The most common reason cited by providers was the lack of measures conducive or available for remote administration across domains (e.g., continuous performance tests, visuomotor tasks, processing speed measures requiring a graphomotor response, full academic achievement screening, social cognition assessment, etc.). Given the need for in-person follow-up for the majority of patients in this sample, pediatric TeleNP may be most efficacious as a screening service to determine appropriate recommendations for intervention and/or further assessment. TeleNP can efficiently meet patient and family needs for pediatric patients who require non-traditional neuropsychological assessment, such as standard patient screening for patients at risk for developing cognitive, emotional, and/or behavioral sequelae related to medical/neurologic condition(s). In our sample, TeleNP met clinical needs without face-to-face follow-up for patients who required non-standard neuropsychological evaluation, including screening batteries for patients with SLE (75%), history of cancer survivorship (87%), and early developmental delay (93%). Continuing to utilize TeleNP as standard screening practice for patients who do not require a full, the traditional neuropsychological battery may provide a more efficient care model, with more patients able to be seen using shorter batteries with less wait time. If impairment is found in a screening battery, patients may then be referred for a full, face-to-face neuropsychological evaluation (Bilder ). Follow-up face-to-face assessment may be necessary for the majority of pediatric neuropsychology patients who require measures not suited for a remote administration, patients with behavioral challenges interfering with testing, or for children unable to access TeleNP services due to environmental or technological limitations. TeleNP provides a mechanism for families to begin the evaluation process and receive preliminary results and recommendations to assist in differential diagnosis and treatment planning.

Findings in this pediatric sample differ from results reported in adult TeleNP studies favoring the ability of remote services to meet adult patient needs (Cullum ; Cullum ; Grosch ). Specifically, pediatric patients in this cohort required in-person follow-up assessment more often than results reported in the adult TeleNP literature. Differences may be attributed to the breadth of the evaluation necessary in these populations. Older adults with concerns for dementia may be fully examined with a focused assessment in comparison to the more comprehensive demands inherent to pediatric neuropsychological assessments targeting developmental, cognitive, emotional, and behavioral concerns.

Anecdotally, many providers have stressed concern for the technical difficulties that may arise during a TeleNP evaluation. Consistent with findings in the adult TeleNP literature and Harder and coworkers (2020), our results demonstrated a limited frequency of interruptions related to technical difficulties, comparable to typical interferences that may occur during an-person evaluation (e.g., noise control outside of the testing room, etc.). To minimize technological interruptions, providers should ensure that they are using a reliable platform and that patients are instructed in reliable network connectivity in advance. Observations of behavior and process approach to testing should be conceptualized incorporating any interruptions that occur. Patients who experience technical difficulties may require face-to-face follow-up testing or a follow-up TeleNP appointment if issues persist and impede full assessment.

The second aim of this project was to disseminate clinical procedures to help guide pediatric TeleNP practice. Importantly, prior to implementing TeleNP provider considerations should begin with technological capability, including utilizing a HIPAA-compliant platform and ensuring the provider is comfortable with remote test administration. Patient population considerations should be made next and guided by provider expertise to determine appropriateness for TeleNP services. We recommend beginning with a well-known patient subspecialty so the neuropsychologist may better evaluate contributing factors to data obtained (i.e., testing environment versus condition-driven). Informed consent procedures should be determined and guided by references provided through APA and IOPC (American Psychology Association, 2013; Inter Organizational Practice Committee, 2020). Providers may need to rely more heavily on parents/caregivers for on-site support during the evaluation (i.e., obtaining consent/assent, ensuring appropriate test environment, securing patient privacy, managing technological issues). The test environment (both provider and patient) is highly impactful and a key component to obtaining reliable and valid data when evaluating a patient through TeleNP. The provider must be flexible and ready to refer to face-to-face assessment if the environment does not maintain standard testing conditions (i.e., quiet, distraction-free space with no family members, pets, or smartphones). We recommend a prospective, strategic plan for face-to-face service delivery if clinical needs are not able to be met through TeleNP. During the evaluation, measures relying on a document camera may need to be eliminated if the patient utilizes a handheld device due to concerns for aspect-ratio translation. Specific behavioral and process-based observations are important to notate during the evaluation. Finally, the clinical report should include reference to the testing medium and relevant limitations. Refer to the Supplementary Appendices A and B, for examples of scripts for setting test environment, noting behavioral observations, and language to include in the report.

Limitations of the current findings include a lack of face-to-face assessment to compare the reliability and validity of measurements. Due to state mandates limiting ambulatory care during the COVID-19 pandemic, face-to-face assessment was unable to be conducted. Therefore, we were unable to replicate findings in the adult TeleNP literature demonstrating similar test results across remote and face-to-face assessment conditions. However, Harder and coworkers (2020) have recently validated home-based TeleNP in a pediatric sample. Further investigation of pediatric TeleNP is needed to replicate these findings, expand the sample size, population diversity, and measures used with pediatric subjects.

Limitations to pediatric TeleNP extending beyond this feasibility project include a greater likelihood of behavioral dysregulation (age-appropriate or otherwise) interfering with test administration in comparison to adult TeleNP. Face-to-face assessment typically provides a unique, standardized environment where the examiner can directly address or modify challenging behaviors in vivo. TeleNP does not allow for the same behavioral scaffolding in children demonstrating impulsivity or inattention, limiting the utility of remote test administration and increasing the likelihood of a need to conduct or complete the evaluation in person. An additional limitation includes the narrowed scope of evaluation measures that may be administered via TeleNP. Specifically, most current visual motor, processing speed, and continuous performance tasks are not appropriate for remote administration. Future studies should address the paucity of assessment measures normed for remote evaluation across neuropsychological domains.

A final limitation highlighted by Scott Kruse and coworkers (2018) includes accessibility to video conferencing technology due to a range of factors, including socioeconomic limitations. Accessibility was rarely an issue in our sample derived from a socioeconomically diverse children’s hospital clinic (65% Medicaid coverage), suggesting that TeleNP may be a feasible accessible option for many patients, thereby limiting unintentional disproportionate care delivery. It is important to note that insurance coverage with Centers for Medicare & Medicaid Services and private insurers may be a barrier depending on institutional contracting. Providers and administrators should consider their own institutional contracts prior to implementing TeleNP to minimize patient financial burden and potential revenue impact for the department. As TeleNP usage increases, reimbursement rates should be examined on a state-by-state basis and at the national level. Due to limited access to interpreter services during the COVID-19 pandemic, the sample only included English-speaking patients. Future studies should explore the feasibility of TeleNP in a linguistically diverse population and other populations with limited access to TeleNP.

In summary, TeleNP is feasible in a broad pediatric clinical population aging from early childhood to young adulthood to begin the process of differential diagnosis and treatment planning. Clinical procedures and decision-making processes are provided to help guide practice procedures in the nascent area of pediatric TeleNP. Future studies should continue to address the reliability and validity of remote versus face-to-face assessment tools in pediatrics.

Click here for additional data file.