The Relationship between Health Literacy and Comfort with Teleneuropsychology in a Veteran Sample.

OBJECTIVE: Health literacy is a strong psychosocial determinant of health disparities and has been found to relate to various aspects of health-related technology use. With the increased implementation of neuropsychological services performed via telehealth during the coronavirus disease 2019 pandemic, the relationship between health literacy and comfort with teleneuropsychology warrants further investigation.
METHOD: The present study examined 77 Veterans participating in neuropsychological evaluations as a part of standard clinical care. The sample was diverse in terms of age ethnicity, and psychiatric and neurocognitive diagnoses. In addition to a fixed-flexible neuropsychological battery, the Brief Health Literacy Screening Tool (BRIEF) was administered to evaluate health literacy. Self-reported comfort with the teleneuropsychological evaluation was assessed using an informal 10-point scale, and qualitative comfort responses were also recorded.
RESULTS: Independent samples t-tests revealed older adults were more likely to participate in the evaluation via telephone than with VA Video Connect. Although health literacy was not related to telehealth modality, it was correlated with comfort with the teleneuropsychological evaluation (r = .34, p < .01), although it is notable that average comfort levels were high across modalities (M = 8.16, SD = 2.50).
CONCLUSIONS: Findings support the notion that teleneuropsychological services may feasibly be implemented with a diverse group of patients, although flexibility with modality may be necessary. Those performing these services should also be aware that patients with lower health literacy may feel less comfortable with teleneuropsychology as they seek to build rapport and optimize evaluation engagement.

Introduction

In February 2020, the Centers for Disease Control and Prevention (CDC) issued guidance recommending that healthcare facilities offer clinical services through virtual means when possible in order to minimize risk of coronavirus disease 2019 (COVID-19) exposure for patients and healthcare providers (Koonin et al., 2020). As over 77% of clinical neuropsychologists report spending professional time with older adults (i.e., >65 years old; Rabin, Paolillo, & Barr, 2016) at increased risk for complications from COVID-19 (CDC, 2020), many neuropsychology clinics have taken up this call to action and adopted telehealth practices in order to continue providing services to their patients during the pandemic (Guidotti et al., 2020; Hammers, Stolwyk, Harder, & Cullum, 2020; Marra, Hoelzle, Davis, & Schwartz, 2020a). These initiatives have been supported and bolstered by structural and legislative changes, including the relaxation of temporary psychology licensure and service provision across state borders (American Psychological Association, 2020), as well as expansion of financial coverage for teleneuropsychology services by federal (U.S. Congress, 2020; Coronavirus Preparedness and Response Supplemental Appropriations Act) and private (Inter Organizational Practice Committee, 2020) insurance.

Teleneuropsychology in practice can vary widely in types of services provided (e.g., only intake interview and/or feedbacks vs. comprehensive neuropsychological assessments; Marra et al., 2020a) as well as delivery format (e.g., trained technician, assistant proctored, in-clinic hybrid, and direct-to-home models; Stolwyk, Hammers, Harder, & Cullum, 2020). Research largely supports the utility of teleneuropsychology in assessing and treating older adults (see Marra, Hamlet, Bauer, & Bowers, 2020b for review); however, the majority of studies (pre-COVID-19) have had participants travel to local satellite clinics to complete the assessment where a technician configures the equipment for the patient and provides assistance during their televisit. Although prior neuropsychology protocols in practice primarily utilized this telehealth delivery format (e.g., Cullum, Hynan, Grosch, Parikh, & Weiner, 2014; Turner, Dorner, VanKirk, Myrick, & Tuerk, 2012), COVID-19 social distancing guidelines and stay-at-home orders have required neuropsychologists to rely more heavily on direct-to-home models of delivery (i.e., patients are assessed in their homes remotely by the clinician via telephone or televideo).

The COVID-19 pandemic required adaptability not only from providers, but also their patients, many of whom had little to no prior experience with receiving healthcare services via telehealth. The volume of reported telehealth service claims to private insurance companies during April 2020 increased by over 8000% compared with April 2019 (FAIR Health, 2020). Among Veterans treated through The Department of Veterans Affairs (VA), the largest healthcare system in the United States, there was 556% growth in telemental health encounters via video-conference in the first 6 weeks of the pandemic (March 11–April 22, 2020), and 77.5% of patients reported being first time users of the telemental health service platform (Connolly et al., in press).

Although there is limited data available on the utilization of teleneuropsychology by patients, a recent survey of clinical neuropsychologists conducted in April 2020 revealed a stepwise increase in percentage of respondents who indicated that they (1) used teleneuropsychology prior to the pandemic for clinical interviews (24%), testing (11%), feedback (26%), and intervention (25%); (2) used teleneuropsychology currently in their clinic (52%, 15%, 41%, and 36%); and (3) intended to continue to use teleneuropsychology after COVID-19 (90%, 59%, 88%, and 70%; Hammers et al., 2020). Taken together, these data suggest that the recent growth of teleneuropsychology services likely mirrors that of other telehealth services and will remain as a viable delivery format for neuropsychological services after the COVID-19 pandemic.

It is important to understand health literacy’s potential role in successful utilization of teleneuropsychology services by patients as it is apparent that virtually provided healthcare will continue post-pandemic. Health literacy has emerged over the past 30 years as one of the strongest psychosocial determinants of health disparities by age, race/ethnicity, and socioeconomic status (see Mantwill, Monestel-Umaña, & Schulz, 2015 for review). Health literacy is a multidimensional, dynamic construct, broadly defined as the degree to which individuals can obtain, process, understand, and communicate information about health-related information to make informed medical decisions (Berkman, David, & McCormick, 2010). Adult health literacy in the United States is below the international average for literacy, numeracy, and problem solving in technology-rich environments (Goodman, Finnegan, Mohadjer, Krenzke, & Hogan, 2013). Suboptimal health literacy is associated with poorer health outcomes (see Berkman, Sheridan, Donahue, Halpern, & Crotty, 2011 for review), including reduced utilization of preventative medication (Cho, Lee, Arozullah, & Crittenden, 2008; White, Chen, & Atchison, 2008), more emergency room visits (Baker, Gazmararian, & Williams, 2004; Cho, Lee, Arozullah, & Crittenden, 2008), hospitalizations (Baker, Parker, Williams, & Clark, 1998; Baker et al., 2002), and nonadherence to prescribed medications (Kripalani, Gatti,& Jacobsen, 2010; Murray et al., 2004).

Older age is well established as a risk factor for lower health literacy both in general populations (Gazmararian et al., 1999; Kutner, Greenburg, Jin, & Paulsen, 2006; Paasche-Orlow, Parker, Gazmararian, Nielsen‐Bohlman, & Rudd, 2005) and among older individuals (see Zamora & Clingerman, 2011 for review). Lower neurocognitive functioning has also been associated with inadequate health literacy, both in healthy older adult populations (Federman, Sano, Wolf, Siu, & Halm, 2009) and among persons with HIV (Morgan et al., 2015). As patients who present to neuropsychology clinics are commonly older adults and/or have neurocognitive complaints with medical co-morbidities, health literacy will be an important consideration in understanding patients’ experience with teleneuropsychology.

To date, there have been a handful of studies examining the role of health literacy in internet access and health-related online navigation skills. In a study of 131 adults with low income, persons with lower numeracy skills were less likely to have access to internet technology at small-to-medium effect sizes (Jensen, King, Davis, & Guntzviller, 2010). Another study of 14,102 participants with diabetes reported that persons with limited health literacy, as assessed and categorized by responses to the BRIEF Health Literacy Screening Tool, were less likely than those with adequate health literacy to register for access to an online health portal, log-on successfully to the portal, and carry out health activities on the portal (e.g., schedule an appointment, order refills, view labs, and send emails to providers; Sarkar et al., 2010). Studies have also suggested that health literacy is associated with health-related online navigation skills in both the setting of examiner-created health platforms (e.g., a mock online pharmacy and a health records navigation tool; Woods et al., 2016) and during naturalistic searches on the internet (e.g., Agree, King, Castro, Wiley, & Borzekowsk, 2015) at medium effect sizes, respectively. Taken together, these studies suggest that suboptimal health literacy may interfere with one’s ability to access, navigate, and engage with teleneuropsychology services.

Previous work has demonstrated the role of health literacy in internet access and navigation of health-related online tools; however, little is known about how health literacy may contribute to one’s ability to successfully utilize telehealth services (Gray, Joseph, & Olayiwola, 2020). Similarly, although low health literacy has been discussed qualitatively as a factor to consider when conducting teleneuropsychology (Sherwood & MacDonald, 2020), there have been no studies to date qualitatively examining the role of health literacy in patient experiences accessing and using teleneuropsychology services. Therefore, the aims of this study were threefold: (1) assess trends in the use of video versus telephone modalities for teleneuropsychology services; (2) evaluate the comfort level of patients using video or telephone modalities for teleneuropsychology services; and (3) gain greater understanding of the role of health literacy in patients’ access to and comfort levels with using teleneuropsychology services. We hypothesized that participants with lower levels of health literacy would report greater discomfort with using teleneuropsychology services, and that factors that have been established as relating to health literacy, including age, education level, and neurocognitive diagnosis, would influence this relationship.

Methods

Participants and Procedures

Data from this Institutional Review Board (IRB)-approved, cross-sectional study were gathered from 86 Veterans seen for standard outpatient neuropsychological telehealth evaluation at a large VA Medical Center between April and November 2020. Of those 86 Veterans, nine were excluded from the study because they did not complete all primary outcome measures, resulting in a final sample of 77 participants. A total of 40 participants (52%) were assessed using VA Video Connect (VVC), and 37 participants (48%) were assessed over the telephone. English was the dominant language for all patients, and each assessment was performed in English.

Demographically, the sample skewed male (n = 68, 88%) and White (n = 36, 47%), generally consistent with the broader demographic makeup of patients at the VA medical center where data was collected. Other races represented included African American (n = 27, 35%), Hispanic/Latinx (n = 10, 13%), and Native American (n = 1, 1%). One participant identified as “other” (1%) and two participants did not identify their race (3%). Participants’ ages ranged from 27 to 91 years old, with a mean age of 57.14 years (SD = 16.33). Education level ranged from 7 to 20 years, with a mean education level of 13.66 years (SD = 2.26). For other demographic details, including relationship status, living arrangement, and employment status, see Table 1a. Psychiatric and neurocognitive diagnoses can be found in Table 1b.

Table 1a

Participant demographics

	n	%
Relationship status
Married	41	53
Divorced	22	28
Widowed	6	8
Single	6	8
Partnered	2	3
Living arrangement
With another individual, not a paid caregiver	50	65
Alone	15	20
With a paid caregiver	8	10
Assisted living facility	3	4
Not reported	1	1
Employment status
Not working/retired	56	73
Employed full-time	15	19
Employed part-time	4	5
Volunteer work	2	3

Table 1b

Participant diagnoses

	n	%
Neurocognitive disorder (NCD)
Major/mild NCD due to vascular etiology	9	12
Major/mild NCD due to other unspecified etiology	7	9
Major/mild NCD due to mixed etiology	6	8
Major/mild NCD due to Alzheimer’s disease	3	4
Major/mild NCD due to Parkinson’s disease	3	4
Major/mild NCD due to frontotemporal dementia	3	4
Major/mild NCD due to Lewy body dementia	2	3
Psychiatric diagnoses
Depression	44	57
Post-traumatic stress disorder	36	47
Anxiety	20	26
Other (not specified)	11	14
Psychotic disorder	7	9
Bipolar disorder	3	4

Measures

Neuropsychological assessment battery

All participants were administered a 60–90 min neuropsychological assessment battery as part of standard care. Although a fixed-flexible approach was used depending on the individual clinic where the Veteran was seen, the majority of participants were administered a combination of the following cognitive measures: Controlled Oral Word Association Test (COWAT; Benton, Hamsher, & Sivan, 1994), semantic fluency (Animals), Oral Trail Making Test (Ricker & Axelrod, 1994), Hopkins Verbal Learning Test—Revised (Brandt & Benedict, 2001), Rey Auditory Verbal Learning Test (RAVLT; Rey, 1964), Wechsler Memory Scale—Fourth Edition (WMS-IV; Wechsler, 2009) select subtests (Logical Memory), and the Wechsler Adult Intelligence Scale—Fourth Edition (WAIS-IV; Wechsler, 2008) select subtests (Digit Span, Similarities, Information, Arithmetic). Additional measures used included the California Verbal Learning Test—Second Edition (CVLT-II; Delis, Kramer, Kaplan, & Ober, 2000), Verbal Naming Test, Multilingual Naming Test (Gollan, Weissberger, Runnqvist, Montoya, & Cera, 2012), Repeatable Battery for the Assessment of Neuropsychological Status (Randolph, 1998), Independent Living Scale (Loeb, 1996) select subtests (Money Management, Health and Safety), Wide Range Achievement Test—Fourth Edition (Wilkinson & Robertson, 2006) select subtests (Reading), Multilingual Aphasia Examination—Third Edition (Benton, Hamsher, & Sivan, 1989) select subtests (Sentence Repetition), Boston Diagnostic Aphasia Examination (Goodglass & Kaplan, 1983) select subtests (Complex Ideational Material), Rey Complex Figure Test (Osterrieth, 1944; Rey, 1941) Copy, Clock Drawing Test, Delis–Kaplan Executive Function System (Delis, Kaplan, & Kramer, 2001) select subtests (Verbal Fluency: Category Switching), and the Vocabulary subtest of the WAIS-IV.

In addition to cognitive testing, participants also completed at least one self-report screening measure of emotional functioning from the following: Geriatric Depression Scale (Yesavage et al., 1982), Geriatric Anxiety Inventory (Pachana et al., 2007), Generalized Anxiety Disorder-7 (Spitzer, Kroenke, Williams, & Löwe, 2006), Patient Health Questionnaire-9 (Kroenke, Spitzer, & Williams, 2001), Posttraumatic Stress Disorder (PTSD) Checklist for Diagnostic and Statistical Manual of Mental Disorders - Fifth Edition (DSM-5) (Weathers et al., 2013), and the Neurobehavioral Symptom Inventory (Cicerone & Kalmar, 1995).

Brief Health Literacy Screening Tool (BRIEF)

To broadly assess participants’ baseline health literacy, the Brief Health Literacy Screening Tool (BRIEF; Haun et al., 2009) was administered. The BRIEF was verbally administered and includes four questions: (1) “How often do you have someone help you read hospital materials”? (2) “How confident are you filling out medical forms by yourself”? (3) “How often do you have problems learning about your medical condition because of difficulty understanding written information”?, and (4) “How often do you have a problem understanding what is told to you about your medical condition”? For questions 1, 3, and 4, participants respond using the following 5-point Likert scale: 1 = always, 2 = often, 3 = sometimes, 4 = occasionally, and 5 = never. For question 2, the following 5-point Likert scale is used: 1 = not at all, 2 = a little bit, 3 = somewhat, 4 = quite a bit, and 5 = extremely.

BRIEF total scores range from 4 to 20. Scores of 4–12 are considered reflective of “inadequate” health literacy. Scores of 13–16 are considered “marginal,” and scores of 17–20 are considered “adequate.” In the BRIEF pilot study, the measure demonstrated 0.79 sensitivity (95% confidence interval [CI], 0.70–0.87) for detecting inadequate health literacy skills and 0.69 sensitivity (95% CI, 0.64–0.75) for detecting inadequate/marginal skills (Haun et al., 2009). Internal consistency in the present study was considered good (α = 0.82). In the current study, health literacy was examined as both a linear (i.e., BRIEF total score) and categorical variable (i.e., inadequate vs. marginal vs. adequate health literacy, meeting of statement).

Telehealth comfort questions

After completion of cognitive testing and the BRIEF, participants were also asked questions about their overall sense of comfort with engaging in telehealth services. Participants were asked, “On a scale from 1-10, how comfortable were you doing your appointment via phone/video”?, where 1 = not at all comfortable and 10 = completely comfortable. Notably, this rating scale was created for the present study as a simple tool to assess comfort level. It has not been formally validated, and thus cut scores are not available to divide responses into categories. Participants were also asked the following free response question: “Was there anything that could have made your phone/video experience better”?, and qualitative responses were recorded.

Statistical Analyses

All analyses were conducted using IBM SPSS Statistics (Version 23.0, released 2015). To address the first study aim, independent samples t-tests were conducted to compare (1) telehealth comfort ratings and (2) total BRIEF score in the VVC versus telephone assessment groups. To evaluate the second and third study aims, Pearson product–moment correlation coefficients were calculated to assess the relationships between participants’ age, education, BRIEF total score, and telehealth comfort ratings. Additional independent samples t-tests were conducted to assess whether comfort level differed between groups with and without examiner-recorded factors affecting the evaluation (i.e., hearing difficulty, poor internet connection, etc.), whether comfort level differed between those providing negative or neutral/positive qualitative feedback, and whether those with or without neurocognitive diagnoses evidenced differences in health literacy score or assessment modality. Finally, a hierarchical linear regression was performed to evaluate whether health literacy accounted for differences in comfort level above and beyond other variables. In this regression, comfort level was the independent variable, whereas the variables age, education, neurocognitive diagnosis, and assessment interference represented Model 1 predictors. Health literacy total score was added as Model 2.

Because it has been proposed that those with the lowest level of health literacy, commonly described as having “limited” or “inadequate” literacy, are often most negatively affected in their interactions with the healthcare system compared with individuals with marginal or adequate literacy levels (Sarkar et al., 2010), the data was then approached categorically, with total BRIEF scores divided into the three categories described previously (i.e., inadequate, marginal, and adequate). A “Welch’s” F test was performed to assess differences in comfort between these health literacy levels. A “Welch’s” F test was chosen as preferable to a traditional one-way analysis of variance based on a significant “Levene’s” F test result, F(2,74) = 5.86, p < .01, which indicated the homogeneity of variance assumption was not met. Effect size is reported as ω2 with the following descriptive levels: small = 0.01 ≤ ω2 < 0.06, medium = 0.06 ≤ ω2 < 0.14, and large = 0.14 ≤ ω2. A post hoc Games Howell procedure was then used to determine differences between the individual health literacy levels.

Results

Factors Affecting the Evaluation

External factors that may have affected the neuropsychological evaluation were recorded by each examiner. These factors included the patient having hearing difficulty (n = 8), reduced sound quality (n = 4), unstable internet connection (n = 3), the call being disconnected (n = 3), and difficulty understanding the patient (n = 5). As would be expected by some underlying commonalities between these factors, there was within-session overlap between these factors, such that 16 total patient assessments were affected by one or more factor. Those who did and did not have examiner-reported interference with the evaluation did not differ in terms of self-reported comfort level (t(75) = 0.64, p = .52, d = 0.17).

Comfort with the Teleneuropsychological Evaluation

Responses to a comfort item spanned the full scale from 1 to 10, but were on average moderately high (M = 8.16, SD = 2.50). Qualitative responses were also recorded. Although only 17 of 77 patients (22%) provided negatively valenced qualitative feedback, concerns about the evaluation included a poor signal/connection (n = 5), difficulty hearing (n = 4), preference for in-person appointment versus telephone or video appointment (n = 4), preference for video over phone appointment (n = 1), and concerns about the assessment length (n = 3). As would be expected, those who provided negative qualitative feedback about their assessment experience had lower comfort scores than those who provided no feedback or positive feedback (Comfort M = 6.88 and M = 8.47; t(75) = 2.35, p < .05, d = 0.63).

Differences in Telephone/VVC Modality

A total of 40 patients completed the neuropsychological evaluation via the VVC video platform, whereas 37 participated using a telephone. Demographically, those participating in the neuropsychological evaluation via telephone (Age M = 66.73, SD = 13.02) were significantly older than those participating via the VVC platform (Age M = 48.28, SD = 13.97) (t(75) = 5.98, p < .001, d = 1.37). They were also more likely to have a neurocognitive disorder (χ2 = 12.80, p < .001]. There was no difference in total BRIEF score between those participating via VVC or telephone (t(75) = 0.74, p = .46, d = 0.17). Furthermore, there was no significant difference in self-reported comfort level between the modalities (t(75) = 0.80, p = .43, d = 0.18].

Health Literacy and Teleneuropsychology Comfort

BRIEF total score was correlated with the teleneuropsychology comfort item response (r = .34, p < .01). Age, which was hypothesized to be a potential moderating factor, was neither significantly correlated with BRIEF score (r = .02, p = .89), nor was it correlated with the comfort item response (r = .04, p = .76). Similarly, education level was neither correlated with BRIEF score (r = .11, p = .34) nor comfort response (r = .04, p = .75). Health literacy score (t(73) = 1.91, p = .06, d = 0.50) and comfort rating (t(73) = 0.13, p = .21, d = 0.31) did not differ between those with and without a neurocognitive diagnosis.

To further explore the relationship between health literacy and comfort level, total scores were divided into established qualitative categories (i.e., inadequate, marginal, and adequate). In total, 22 participants (28.6%) endorsed inadequate health literacy, 16 (20.8%) indicated marginal health literacy, and 39 (50.6%) evidenced adequate health literacy. Comfort level response for each group is reported in Table 2. “Welch’s” F test revealed there was a significant effect of health literacy level on comfort response (F(2, 30.84) = 4.18, p < .05, ω2 = 0.07).

Table 2

Comfort level with teleneuropsychology by BRIEF Health Literacy Category

Health Literacy Category	n	Mean	SD
Inadequate	22	6.64	3.22
Marginal	16	9.00	2.28
Adequate	39	8.67	1.68

Post hoc analysis indicated significant differences in comfort level between those with inadequate and marginal health literacy and those with inadequate and adequate health literacy, all at the p < .05 level (d = 0.85 and d = 0.79, respectively). Comfort level did not differ between marginal and adequate health literacy groups (p = .86, d = 0.17).

Hierarchical linear regression predicting comfort level included age, education level, presence of factors affecting the evaluation, the presence of a neurocognitive disorder in Model 1 and the addition of health literacy (BRIEF Total Score) in Model 2. Model 1 did not contribute to a significant amount of variance in comfort level (R2 = .24); in addition, none of the individual factors were significantly contributory. The addition of health literacy in Model 2 contributed to an additional 10.4% of the variation, representing a significant change in R2 (F(1,74) = 8.19, p < .01).

Discussion

Although neuropsychological services performed via telehealth have increased over the past decade, significant variability has existed in which services are performed and how they are implemented (Marra et al., 2020a). Beginning in 2020, the COVID-19 pandemic has caused a sudden and vast increase in the need for neuropsychological evaluations to be implemented remotely, often directly into patients’ homes via telephone or video telehealth platforms (Guidotti et al., 2020; Hammers et al., 2020). Health literacy, which is a strong predictor of health disparities, has been associated with reduced access to internet technology and increased difficulty navigating online health portals (Jensen et al., 2010; Sarkar et al., 2010); however, it has not yet been examined in relation to teleneuropsychological evaluations. Given that neuropsychologists commonly work with patients at increased risk for inadequate health literacy (e.g., older adults and those with neurocognitive concerns or diagnoses), this area warrants further consideration.

The primary findings from the present study reveal that those who rate their health literacy as lower also tend to report lower comfort level with teleneuropsychology, irrespective of modality (e.g., phone or video) and that health literacy predicted user comfort above and beyond demographic or assessment-related factors. Although this linear relationship suggests that anyone with lower health literacy ratings may be at risk for lower evaluation comfort, further categorical analyses indicated that those with marginal and adequate health literacy were similarly comfortable with the evaluation, but these groups differed from those with inadequate health literacy at a large and medium effect size, respectively. This echoes the relationship seen between limited/inadequate health literacy and other negative telehealthcare outcomes, such as difficulty navigating online patient portals (Sarkar et al., 2010).

When considered in the context of the specific health literacy questions asked on the BRIEF health literacy screen, this suggests that those who have the most difficulty navigating areas of healthcare, such as reading hospital materials, filling out medical forms, understanding written information, and understanding verbally presented information, may also experience discomfort when participating in a teleneuropsychological evaluation. Because comfort may play a role in such crucial aspects of the evaluation as rapport building during the clinical interview, engagement with standardized testing and reception to feedback (AACN Board of Directors, 2007; Kapur & Kemp, 2016), understanding this relationship, along with ways to potentially intervene with those who have reduced health literacy to best promote comfort in the evaluation, may be important steps forward in the emerging field of teleneuropsychology.

Encouragingly, the present sample of Veterans evaluated by a neuropsychology outpatient clinic via telehealth was diverse in terms of ethnicity, age, and both neurocognitive and psychiatric diagnoses, suggesting that teleneuropsychological evaluations can be implemented across a range of individuals with unique presenting problems and needs. In fact, teleneuropsychology, like other forms of telehealth, may increase accessibility to services for individuals who may otherwise struggle to attend in-person evaluations due to physical/mobility issues, transportation issues, or other socioeconomic barriers (Elnitsky et al., 2013; Zulman et al., 2019). Along with usability, self-reported comfort level was also rated as generally high, which is an encouraging finding clinically. However, statistically it led to reduced variety in the sample, which somewhat limits the ability to understand individuals who have low levels of comfort. Qualitative responses revealed that lower comfort scores were associated with concerns about modality, assessment length, and technology issues. Although future studies evaluating individuals with greater levels of hesitation or discomfort in neuropsychological evaluations may be helpful to broaden the implication of results, these areas provide an early roadmap for possible evaluation pitfalls and areas for future intervention.

Although overall ability to participate in the evaluation and comfort in doing so was generally high, differences existed in the technology used for assessments, with older individuals more often assessed via telephone than video at a large effect size. This finding was consistent with literature on older adults’ access to telehealth technology (Lam, Lu, Shi, & Covinsky, 2020), which suggests that older adults have lower access to telehealth technology and more difficulty navigating telehealth, and has notable implications for implementation of teleneuropsychological services. Contrary to expectations, there were no differences in health literacy for those who preferred telephone versus video teleneuropsychology delivery format. Although there are several verbally mediated tasks of orientation (i.e., Mini Mental State Examination (MMSE) Orientation Items, Montreal Cognitive Assessment (MoCA) Blind), attention (i.e., WAIS-IV Digit Span, Arithmetic), language (i.e., COWAT, Animals, WAIS-IV Verbal Comprehension Index (VCI) Subtests), and memory (i.e., CVLT-II, RAVLT, WMS-IV Logical Memory) that can be performed via telephone, assessments of other domains, particularly visuospatial functioning or visual memory, are precluded from this form of assessment. Because deficits in these areas represent hallmark features of various syndromes and neurodegenerative disorders (e.g., vascular dementia, Lewy Body dementia), telephone assessments that omit these measures may not yield an accurate diagnostic impression of the patient. This is especially concerning in the context of a greater number of individuals with neurocognitive diagnoses in the present sample participating via telephone versus video. Given these limitations, future research may wish to further explore factors that contribute to older adults’ utilization of the telephone rather than video for teleneuropsychology services to improve access to video evaluations for older adults who are at the greatest risk of neurocognitive disorder and/or decline.

It is notable that, contrary to predictions, age, education level, and neurocognitive diagnosis were not associated with health literacy report or comfort using teleneuropsychology. Although this finding promotes the idea that those presenting across a range of demographic factors may all be able to participate effectively in a teleneuropsychological appointment and feel comfortable doing so, it does leave open questions as to what demographic or modifiable factors may play a role in the perhaps complex relationship between health literacy and teleneuropsychology comfort. In particular, future studies may wish to explore socioeconomic status, which was not assessed in the present study, but has been shown to be a strong predictor of health literacy (Stormacq, Van den Broucke, & Wosinski, 2019).

Although the present study offers important insight into the relationship between health literacy, technology usage, and comfort with teleneuropsychological assessment, there are factors that may somewhat limit generalizability of results. Due to variability in the modality of assessment between telephone and video evaluations, a self-report measure of health literacy was the most feasible option to evaluate this construct. Although this measure has been established as a valid assessment of health literacy (Haun et al., 2009), primary analyses ultimately compared two self-report responses (i.e., BRIEF and comfort questions), Thus, it is possible that a positive or negative response bias for some patients may have influenced results. Had a visual medium been present for all participants, an alternative measure of health literacy, such as the Test of Functional Health Literacy in Adults (Parker, Baker, Williams, and Nurss, 1995) or the Rapid Estimate of Adult Literacy in Medicine (Davis et al., 1991), may have been preferred to eliminate this potential bias, and thus future studies may consider implementing such a design.

An additional future direction involves the collection of collateral informant report. Collateral informants can provide important information about an individual’s level of cognitive and functional ability, especially in cases where patients are at risk for reduced insight, including in those individuals with neurocognitive disorders. It is notable that in the present study individuals with neurocognitive disorders were more likely to be evaluated via telephone. It is possible that these individuals also had lower insight into their health literacy levels, which could have tempered differences between telephone and video telehealth groups. To better account for patients with potentially reduced insight, future studies may wish to fortify self-reported health literacy measures with informant report.

Finally, data could only be collected from patients who were able to participate in teleneuropsychological evaluations, whether via phone or video. Thus, individuals with significant access limitations that precluded assessment altogether could not be characterized. Future studies may wish to explore what percentage of our patients may be affected in such a way, and how these severe access limitations may relate to constructs like health literacy.

In conclusion, findings from a diverse Veteran sample suggest individuals spanning different ages, educational backgrounds, and neurocognitive abilities may be able to effectively participate in neuropsychological evaluations performed via telephone or video, with a moderately high level of comfort. Older individuals may choose or only have access to telephone rather than video-based assessments, a finding worthy of further investigation and intervention, as visually based information can be a crucial component of neuropsychological batteries. Those with lower self-reported health literacy may feel less comfortable with video- and telephone-based neuropsychological assessments, which has implications for rapport and engagement, and also warrants further consideration as the field of teleneuropsychology continues to develop throughout and beyond the present COVID-19 pandemic.

Funding

None declared.

Conflict of Interest

None declared.