Development and implementation of an electronic Clinical Dementia Rating and Financial Capacity Instrument-Short Form.

Introduction: To address the need for remote assessments of cognitive decline and dementia, we developed and administered electronic versions of the Clinical Dementia Rating (CDR®) and the Financial Capacity Instrument-Short Form (FCI-SF) (F-CAP®), called the eCDR and eFCI, respectively.
Methods: The CDR and FCI-SF were adapted for remote, unsupervised, online use based on item response analysis of the standard instruments. Participants completed the eCDR and eFCI first in clinic, and then at home within 2 weeks.
Results: Of the 243 enrolled participants, 179 (73%) cognitively unimpaired (CU), 50 (21%) with mild cognitive impairment (MCI) or dementia, and 14 (6%) with an unknown diagnosis, 84% and 85% of them successfully completed the eCDR and eFCI, respectively, at home. Discussion: These results show initial feasibility in developing and administering online instruments to remotely assess and monitor cognitive decline along the CU to MCI/very mild dementia continuum. Validation is an important next step.

INTRODUCTION

The development of psychometrically validated assessments to identify older adults at risk for cognitive decline and dementia due to Alzheimer's disease (AD) is a critical need for clinical researchers. However, many assessments for clinical research, treatment trials, and clinical practice are conducted in person at a research clinic, which can be time consuming and expensive. , In addition to the increase in older adults using the internet for a wide variety of activities, , internet‐based patient communication, including cognitive assessments, is increasing in health‐care settings. There are currently over 40 different computerized neuropsychological tests, although most have not been clinically validated for unsupervised, at‐home use. , , , The COVID‐19 pandemic underscores a pressing need for the development and validation of inexpensive, scalable online methods to remotely assess and longitudinally monitor individuals, which may help to facilitate AD clinical trials and accelerate the development of new treatments more efficiently.

Measures of cognition and activities of daily living have been demonstrated to be highly valuable for clinical AD research, especially for detecting early changes that predict future decline and disease progression. One well‐validated measure of cognitive and functional decline is the Clinical Dementia Rating (CDR®), , , which is widely used to characterize progression of AD and related dementias. Semi‐structured interviews with a participant and a study partner are used to calculate cognitive and functional status scores across six domains, as well as a composite or global CDR score. Another well‐validated measure is the Financial Capacity Instrument‐Short Form (FCI‐SF) (F‐CAP®), , , , , which is used to detect decline in everyday financial skills in older adults. The FCI‐SF is a performance‐based assessment of four financial domains including monetary calculation skills, financial conceptual knowledge, understanding/using a checkbook and register, and understanding/using a bank statement. Items are scored and summed to calculate four component performance scores and a total score in addition to timing indexes of four tasks.

Although both the CDR and FCI‐SF are well validated in clinical settings, there are limitations that hinder their accessibility and scalability. Both require administration in a supervised setting with an experienced and certified rater, and therefore have high burden for administrators and participants. We developed online versions of both instruments, called the electronic CDR (eCDR) and the electronic Financial Capacity Instrument‐Short Form (eFCI) and assessed the feasibility of implementing these newly developed instruments through the Brain Health Registry (BHR) internet‐based platform. , The eCDR and eFCI, if validated, have the potential to facilitate clinical AD research and health care, by allowing remote assessment of cognition and function in an inexpensive, efficient, and scalable way.

METHODS

Description of the traditional instruments: CDR and FCI‐SF

Clinical Dementia Rating (CDR)

The CDR, developed at Washington University in St. Louis (WU), is used to detect the presence of dementia and when present, to determine the extent of cognitive and functional decline. , Using their best judgment, a clinician obtains the information necessary to assign a CDR score by conducting semi‐structured interviews with a participant and their study partner (typically, a spouse or adult child). Impairment is scored based on decline from a previously obtained level of function and is rated in six categories: Memory, Orientation, Judgment and Problem Solving, Community Affairs, Home and Hobbies, and Personal Care. Each category is rated independently across five levels of impairment: none (0), very mild (0.5), mild (1), moderate (2), and severe (3); an exception is the Personal Care domain, which is rated as 0, 1, 2, or 3. Scores in each category are synthesized to yield a global CDR score on an ordinal 5‐point scale, on which CDR 0 indicates normal cognition and CDR 0.5, 1, 2, and 3 designate very mild, mild, moderate, and severe dementia, respectively.

RESEARCH IN CONTEXT

Systematic Review: A PubMed search was conducted to review: (1) currently available computerized neuropsychological tests and remotely administered cognitive and functional assessments in older adults; (2) internet and technology use by older adults; (3) evidence for use of cognitive and functional measures in AD research; and (4) content, psychometric properties, and validity of the Clinical Dementia Rating (CDR) and Financial Capacity Instrument‐Short Form (FCI‐SF).

Interpretation: Our findings demonstrate the feasibility of implementing the electronic versions of the CDR and FCI (eCDR and eFCI) in a cohort of older adults with unimpaired cognition, mild cognitive impairment (MCI), and very mild dementia.

Future Directions: Ongoing analyses are now validating the eCDR and eFCI by: (1) comparing performance on e‐instruments to their in‐clinic counterparts, (2) estimating associations between e‐instrument performance and in‐clinic neuropsychological tests, and (3) assessing the sensitivity and specificity of the e‐instruments to detect MCI. Future studies will validate the instruments in more diverse populations.

Attributes of the CDR include multidimensional assessment of cognition, behavior, and function and linkage to validated diagnostic criteria for dementia due to AD. , The CDR is scored independently of neuropsychological test (NPT) performance, so NPT data can remain as independent variables in longitudinal studies. Because the instrument assesses decline from a person's previous level of function, it also lessens the influence of age, education, linguistic, and sociocultural confounders. The CDR has high inter‐rater reliability in multi‐center trials, , , , strong content and criterion validity, and internal consistency and internal responsiveness. There are well established, standard training and certification protocols to stably administer the CDR and assess cognitive impairment. Because the CDR captures clinically meaningful change, it has been proposed as the primary outcome measure in many AD clinical trials.

Financial Capacity Instrument (FCI‐SF)

The FCI‐SF, developed at the University of Alabama at Birmingham (UAB), is a 15‐minute measure that uses performance and time‐to‐completion variables to detect decline in everyday financial cognition in older adults. The measure uses financial concepts, coins/currency, documents, and other financially related stimuli relevant to the monetary system and financial practice of the US population. The FCI‐SF, derived from the FCI‐Long Form (FCI‐LF), comprises 37 performance items that evaluate four financial domains: monetary calculation, conceptual knowledge, using a checkbook/register, and using a bank statement. The FCI‐SF also includes six processing speed indexes, measuring the time‐to‐completion of four specific FCI tasks. Total performance scores range from 0 to 74 points and the six time‐to‐completion indices include a maximum composite time of 670 seconds. The FCI‐SF also includes four unscored yes/no questions concerning a participant's prior experience with specific financial tasks. The FCI‐SF requires a trained rater and has a detailed and well‐operationalized administration and scoring manual. Age‐ and education‐adjusted older adult norms for the FCI‐SF have been published. The FCI‐SF has been associated with dementia progression and has shown sensitivity to amyloid beta (Aβ) status in cognitively unimpaired (CU) older adults. The FCI‐SF has also been shown to have excellent internal reliability, interrater reliability, and concurrent validity with the FCI‐LF.

Statistical analysis of existing data: CDR and FCI‐SF

To inform item selection for the electronic versions of the CDR and FCI‐SF, item response theory (IRT) analysis of existing CDR and FCI‐SF clinical data was performed for this study. The IRT analysis approach was used to identify specific items that were most informative in estimating the underlying degree of cognitive/functional impairment; these items were kept in the electronic instrument. IRT analysis was also used to identify specific items that had less sensitivity, poor psychometric properties, and that contributed the least information in estimating the underlying degree of cognitive/functional impairment; these items were eliminated from the electronic instrument. Ordinary least squares linear regression analysis also was used to measure associations between individual items and overall instrument score.

IRT analysis for the CDR included item‐level data from WU's Memory and Aging Project. The WU dataset contained >3000 records with 65 items measuring the six subdomains of the CDR. IRT analysis of the CDR indicated that the majority of the items in the CDR discriminated well at mild and very mild levels of cognitive impairment. Of the original 65 items, 53 informative items that had demonstrated high discriminative power were kept in the final statistical model for estimation of overall and domain specific scores. ,

IRT analysis for the FCI‐SF used item‐level data from the Mayo Clinic Study of Aging (n = 1472) and the Cognitive Observations in Seniors (COINS) study at UAB (n = 144). Statistical IRT and differential item functioning (DIF) analysis of the FCI‐SF indicated that the majority of the items in the FCI‐SF discriminated well between individuals with varying levels of financial capacity and required either a higher or lower level of financial capacity to be answered correctly. All FCI‐SF items, including interrelated tasks, were retained in the eFCI.

Development of electronic instruments

Overall approach to adapt traditional instruments into online instruments

To adapt the CDR and the FCI‐SF for online administration, each item was reviewed to first determine whether it could be reasonably translated into an online format. The overall approaches used to adapt the CDR and the FCI‐SF for online use are described in Table 1. Items with discrete response options (e.g., yes/no) were adapted verbatim in a simple survey format (Table 1, Row 1). Items with open‐ended responses were altered to have multiple‐choice options (Table 1, Row 2) or dropdown menus. To preserve the integrity of online data collection for these instruments, multiple choice and drop‐down menu options were generated that contained a list of the most frequent responses. Investigators used their extensive experience with the CDR and the FCI‐SF to identify and develop a list of the most frequent responses to open‐ended questions. For items likely to show practice effects, such as the short memory test within the CDR and the items requiring specific calculations in the FCI‐SF, alternative versions of items were developed. All potential items were evaluated by the study team, including clinicians familiar with the in‐clinic administration of the instruments, to determine item content and wording, as well as the importance of clinical judgment. The instruments were adapted for online use via Qualtrics software (https://www.qualtrics.com), a survey software tool for data collection. The eCDR can be administered on smartphones, tablets, and computers, whereas the eFCI is optimally administered on tablets or computers due to screen size requirements for the tasks.

TABLE 1

Approaches used to adapt traditional CDR and FCI‐SF online use

Test	Type of response/test materials	Adaptation	Example figure(s)
CDR	Discrete response options (e.g., yes/no)	Simple survey format
CDR	Open‐ended response field for written text	Multiple choice answer options or dropdown menus
FCI‐SF	Open‐ended response field for written text	Multiple choice answer options
CDR	Open‐ended response field for numerical response	“Slider” question type for numerical questions
CDR	Verbal reminders with extra details frequently provided by assessors during in‐clinic assessments	Extra details were included as supplemental text
FCI‐SF	Physical stimuli provided to participant (examples: check book, blank checks and check register)	Images of physical stimuli were displayed on device screen
FCI‐SF	Physical stimuli provided to participant to make out a check	Images of a check with clearly highlighted fields to identify different parts of the check that were specific to the question being asked
FCI‐SF	Verbal instructions to have scratch paper, a pencil, and a calculator readily available	Written instructions were displayed on device screen
FCI‐SF	Verbal reminders follow along with the administrator as they read the question aloud	Written reminders to scroll down to read and answer the question were displayed on device screen

Abbreviations: CDR, Clinical Dementia Rating; eCDR, electronic Clinical Dementia Rating; eFCI, electronic Financial Capacity Instrument; FCI‐SF, Financial Capacity Instrument – Short Form.

eCDR development

The eCDR was developed collaboratively between the University of California San Francisco (UCSF) and WU. Additional methods were used to address challenges specific to adapting the CDR instrument. Some items, such as the autobiographical recall question, in the CDR that could not be reasonably translated into an online format due to their open‐ended nature (10% of CDR questions) were eliminated from the eCDR (Table 2) or were altered to have a wide range of answer options. For example, the “slider” Qualtrics response type was used for numerical questions (Table 1, Row 4). Another challenge was administering and scoring the adapted eCDR without the ability to ask follow‐up questions and without the judgment of an experienced assessor. To address this, extra details that assessors most frequently provided during past in‐clinic assessments were included as supplemental text in the eCDR. For example, because the CDR was intended to assess decline from a person's previous level of function, reminders for the study partner were incorporated throughout the eCDR to only endorse an answer if the participant's observed behavior was: consistent, a change compared to the participant's longstanding behavior, due to memory and thinking problems, and interfered with the participant's daily function (Table 1, Row 5). A related challenge in adapting the CDR for online use was replicating the clinician's judgement used to assign a CDR score based on a participant's or study partner's responses. To address this, a standard, automated scoring algorithm was developed based on the IRT model to remotely score the eCDR without clinician judgment. , Li et al. described further details on the IRT analysis and the development of the automated scoring algorithm for the eCDR. The algorithm generates multiple outcome measures: categorical global and box scores (like the CDR), and novel, continuous global and box scores. The eCDR scoring algorithm has more granular scoring than the CDR, giving it the potential for higher sensitivity to identify subtle cognitive and functional changes in largely CU individuals with a CDR = 0. Last, the CDR relies heavily on the accuracy of study partner responses, and the accuracy of study partner responses is judged by the assessor based on the dyad relationship. To address this, the eCDR asks both the participant and the study partner questions such as how long they have known each other, whether they currently live together, and how many hours per week they spend together. The participant portion of the eCDR was expected to take a total of 15 minutes, and the study partner portion of the eCDR was expected to take a total of 20 minutes. Pilot data and user feedback from internal alpha testing were used to make minor changes to optimize the eCDR.

TABLE 2

Items eliminated from the CDR to adapt for online use in eCDR

Type	Question
SP	Is he/she taken to social functions outside a family home? If no, why not?
SP	If in nursing home, does he/she participate well in social functions (thinking)?
SP	If in nursing home, what can he/she no longer do well (H and H)?
SP	Ability to perform household tasks: Please describe (Blessed)
SP	Tell me about some recent event in his/her life within 1 week (autobiographical)
SP	Tell me about some recent event in his/her life within 1 month (autobiographical)
P	A few moments ago within 1 week (autobiographical)
P	A few moments ago within 1 month (autobiographical)
P	What is the name of this place?
P	Upon arriving in a strange city, how would you locate a friend you wanted to see?

Abbreviations: CDR, Clinical Dementia Rating; eCDR, electronic Clinical Dementia Rating; P, participant; SP, study partner.

eFCI development

The eFCI was developed in collaboration between UCSF and UAB and in consultation with co‐author Dr. Marson, the creator of the FCI‐SF. Additional methods were used to address the challenges specific to adapting the FCI‐SF for online use. An alternative version of the FCI‐SF was created for development of the eFCI: 100% of the alternative FCI‐SF items were adaptable for online use. The eFCI captured timing information for each item and was expected to take a total of 20 minutes to complete. Similar to the issue for the CDR, the FCI‐SF required a trained rater to administer the FCI‐SF in clinic. To address this, we used multiple‐choice answers or drop‐down menus that contained the most frequent responses and intentional foil responses, as well as additional text within the electronic instrument instructions, to create an eFCI that could be scored automatically without a rater. We also included instructions for participants at the beginning and throughout the eFCI to have scratch paper, a pencil, and a calculator readily available for use (Table 1, Row 8). Another challenge was adapting the physical stimuli, such as a checkbook and bank statement, used in the FCI‐SF. We created images of the physical stimuli used in clinic that display on the participant's device screen, including a bank statement, checkbook, check register, and a blank check with clearly highlighted fields to identify different parts of the check specific to the question being asked (Table 1, Row 7). An automatic eFCI scoring algorithm was developed. Pilot data and user feedback from internal alpha testing were used to make minor changes to optimize the eFCI.

Feasibility of implementing electronic instruments

Participants

Participants and study partners were recruited into the Electronic Validation of Online Methods to Predict and Monitor Cognitive Decline (eVAL) study from two sources. First, we referred current BHR participants to the study using automated e‐mail invitations with the following inclusion criteria: agreed to be contacted about future studies, 55 years of age and older, residing within 50 miles of the UCSF site. Second, participants were recruited from existing clinical cohorts from three Alzheimer's Disease Research Center (ADRC) sites: UAB, Mayo Clinic (Mayo), and WU. All participants signed consent during their in‐clinic visit, and all activities were performed under institutional review board approval at the local site.

Electronic instrument implementation

In‐clinic visits occurred annually during which a trained study staff member administered the CDR, FCI‐SF, eCDR, and eFCI in a supervised testing session. Participants were assisted at the beginning of the electronic instrument (e‐instrument) and had the option to get help from the study staff member as needed. Within 2 weeks of the supervised visit, participants returned home, completed registration on the BHR platform by following instructions provided in the invitation e‐mail and completed the eCDR and eFCI in an unsupervised setting at home without a study staff member. The supervised in‐clinic e‐instrument version was taken before the unsupervised at‐home e‐instrument version in all cases, which may represent a confound. This is a limitation that will be addressed in the Discussion section.

The eCDR and eFCI included instructions emphasizing that the participant should complete the instrument “entirely on his/her own, without any assistance.” After the baseline in‐clinic visit, participants were asked to complete the eCDR and eFCI every 6 months at home in an unsupervised setting. Study partners were also asked to complete the eCDR in an unsupervised setting through the BHR platform on the same timeline. Using the BHR platform, participants and study partners registered by creating a username and password, agreed to an online informed consent, and completed the electronic instruments. Participants were also asked to self‐report “How confident are you with computers?”, with the option to endorse “Not confident at all,” “I usually need help,” “It depends on the task,” or “Confident.” Automatic e‐mails were sent to participants and study partners on 1, 3, 5, 11, and 15 days after their at‐home visit due date and phone calls were made to participants and study partners on approximately 8 and 18 days after their at‐home visit due date to remind them of their at‐home and in‐clinic visits and study tasks.

Clinical diagnoses

The Uniform Data Set, Version 3 (UDS) was administered to all participants. Clinical diagnosis of CU, mild cognitive impairment (MCI), or mild dementia were obtained from the UDS section D1.

Statistical analysis

To assess the reliability between the unsupervised at‐home eCDR and the supervised in‐clinic eCDR, McNemar's Chi‐square test was performed on a 2 × 2 table of at‐home and in‐clinic eCDR results of 0 or 0.5 (n = 189). Because one of the cell counts was small (<5), continuity correction was applied. We also calculated the mean and median total time it took both participants and study partners to complete their portion of the eCDR (n = 207 pairs). For the eCDR, we removed outliers based on the 1.5× interquartile range (IQR) rule. This was necessary because the initial total time included the sum of any pauses and re‐starts taken resulting in highly right‐skewed values.

A Pearson correlation coefficient was calculated to assess the reliability between the unsupervised at‐home eFCI and the supervised in‐clinic eFCI (n = 209). We also calculated the mean total time to complete the eFCI (n = 209).

RESULTS

Feasibility of electronic instrument implementation

Between January 17, 2020 and January 5, 2022, there were 243 participants enrolled across all sites. Enrollment and task completion rates are shown in Table 3. Out of all 3192 participants invited to enroll in the study, 250 (176 invited from BHR and 74 invited from ADRCs) declined consent or withdrew from the study. Most common specified reasons for declining consent or withdrawing from the study are included in Table 4. Seventy‐seven participants were determined ineligible and excluded.

TABLE 3

Completion of electronic instruments

Variable	UCSF	ADRC	Total
Invited	2915	277	3192
Enrolled	140 (5%)	103 (37%)	243 (8%)
Completed P eCDR in‐clinic	136 (97%)	100 (36%)	236 (74%)
Completed P eCDR at home	135 (96%)	72 (70%)	207 (85%)
Completed SP eCDR at home	124 (89%)	68 (66%)	192 (79%)
Completed eFCI in‐clinic	136 (97%)	101 (98%)	237 (98%)
Completed eFCI at‐home	134 (96%)	71 (69%)	205 (84%)

Notes: Enrolled percentages indicate number enrolled out of invited participants; completed percentages indicate number that completed task out of number enrolled. Data presented as number as of January 5, 2022.

Abbreviations: ADRC, Alzheimer's Disease Research Center; eCDR, electronic Clinical Dementia Rating; eFCI, electronic Financial Capacity Instrument; P, participant; SP, study partner; UCSF, University of California San Francisco.

TABLE 4

Summary of declined reasons and exclusion reasons

Declined or exclusion reason	Total
Reason for declining consent	250 a
Participant does not specify reason	120 (48%)
Overall not interested	70 (28%)
Too much time	18 (7%)
Inability to use a computer	16 (6%)
Unable to come into clinic	13 (5%)
No available study partner	11 (4%)
Study partner doesn't want the participant to be involved	10 (4%)
Medical reason	9 (4%)
COVID‐19 related concerns	7 (3%)
Not a good time for participant	7 (3%)
Not willing to complete assessments	3 (1%)
Involved in other studies	3 (1%)
Death	2 (1%)
Lack of access to a computer or internet at home	1 (0.4%)
Reason for exclusion	77 a
No available study partner	22 (29%)
Inability to complete computer tasks at home	17 (22%)
Medical reason	10 (13%)
Does not specify reason	10 (13%)
Unable to come into clinic	7 (9%)
Participates in other research activities	5 (7%)
Study partner doesn't want the participant to be involved	4 (5%)
Lost to follow up	1 (1%)
COVID‐19 related concern	1 (1%)
Participant not able to complete assessments	1 (1%)
Study Partner doesn't speak English	1 (1%)

Note: Data presented as number (%) as of January 5, 2022.

Unique count of participants as more than one reason can be endorsed.

Characteristics of enrolled participants

Table 5 summarizes the demographic profile, clinical diagnoses, and computer confidence of participants. Of all enrolled participants, 179 (73%) were CU, 49 (20%) had MCI, 1 (1%) had dementia, and 14 (6%) had an unknown diagnosis.

TABLE 5

Characteristics of participants

	Enrolled at UCSF N = 140	Enrolled at ADRCs N = 103	Total enrolled at all sites N = 243
Demographics
Age years, median (IQR)	73 (67–79)	74 (67–78)	74 (67–79)
Female (%)	82 (59%)	43 (42%)	125 (51%)
Education
High school	2 (1%)	8 (8%)	10 (4%)
Some college	8 (6%)	9 (9%)	17 (7%)
Two‐year degree	5 (4%)	5 (5%)	10 (4%)
Four‐year degree	40 (29%)	31 (30%)	71 (29%)
Master's degree	55 (39%)	30 (29%)	85 (35%)
Doctoral degree	12 (8%)	2 (2%)	14 (6%)
Professional degree	18 (13%)	6 (6%)	24 (10%)
Unknown	0 (0%)	12 (11%)	12 (5%)
Race
White (%)	115 (82%)	90 (87%)	205 (84%)
Black or African American (%)	1 (1%)	11 (11%)	12 (5%)
Asian (%)	13 (9%)	0 (0%)	13 (5%)
More than one race (%)	10 (7%)	0 (0%)	10 (4%)
American Indian or Alaskan Native (%)	0 (0%)	0 (0%)	0 (0%)
Native Hawaiian or Other Pacific Islander (%)	0 (0%)	0 (0%)	0 (0%)
Decline to State (%)	1 (1%)	1 (1%)	2 (1%)
Unknown (%)	0 (0%)	1 (1%)	1 (1%)
Ethnicity
Hispanic or Latino (%)	8 (6%)	0 (0%)	8 (3%)
Not Hispanic or Latino (%)	132 (94%)	98 (95%)	230 (95%)
Declined to state (%)	0 (0%)	3 (3%)	3 (1%)
Unknown (%)	0 (0%)	2 (2%)	2 (1%)
Diagnosis
Cognitively unimpaired (%)	110 (79%)	69 (67%)	179 (73%)
Mild cognitive impairment (%)	30 (21%)	19 (18%)	49 (20%)
Dementia (%)	0 (0%)	1 (1%)	1 (1%)
Unknown (%)	0 (0%)	14 (14%)	14 (6%)
Computer confidence
Usually needs help (%)	1 (1%)	2 (2%)	3 (1%)
Depends on task (%)	42 (30%)	30 (29%)	72 (30%)
Confident (%)	94 (67%)	45 (44%)	139 (57%)
Unknown (%)	3 (2%)	26 (25%)	29 (12%)

Abbreviations: ADRC, Alzheimer's Disease Research Center; IQR, interquartile range; UCSF, University of California San Francisco.

Note: Data presented as median (IQR) or number (%) as of January 5, 2022.

Reliability between in‐clinic and at‐home administration and completion times

The reliability between the supervised in‐clinic eCDR and unsupervised at‐home eCDR was estimated with a McNemar's Chi‐squared test statistic of 0.364 with degrees of freedom equal to one and associated P‐value of .547 (with continuity correction). This supports high test–retest reliability of the administration setting of the eCDR, meaning there is not a significant difference between eCDR scores taken in clinic from those taken at home. The mean total time to complete the participant portion of the eCDR was 12 minutes and 24 seconds (median: 12 minutes and 20 seconds). The mean total time to complete the study partner portion of the eCDR was 16 minutes and 28 seconds (median: 16 minutes and 3 seconds).

The Pearson correlation coefficient between the supervised in‐clinic eFCI and the unsupervised at‐home eFCI was 0.731 (P < .001). This supports a high correlation between the eFCI completed in clinic and at home. The mean total time to complete the eFCI was 13 minutes and 38 seconds (±4 minutes and 39 seconds).

DISCUSSION

The development of online unsupervised cognitive assessments can improve the accessibility of existing instruments used to identify, diagnose, and monitor individuals at risk for cognitive decline. We adapted the traditional in‐clinic CDR and FCI‐SF by developing the online unsupervised eCDR and eFCI instruments and automated scoring algorithms for each instrument. Our preliminary results demonstrated feasibility (the ability of a participant to complete all aspects of the electronic instrument) of our approach. We are now validating the e‐instruments against traditional instrument counterparts, clinical diagnosis of CU or MCI/very mild dementia, and exploring the relationship between e‐instrument performance and other important outcomes, such as clinical diagnosis and in‐clinic neuropsychological test performance. If validated, these novel e‐instruments would help facilitate screening and assessment of older adults for AD clinical trials, AD clinical research, and clinical care. Further, they can be used for population‐based screening for early detection of cognitive impairment, and for epidemiological studies.

A significant component of e‐instrument adaption was developing an algorithm‐based instrument that would not require a clinician to score responses using clinical judgment. To do so, we included items that were identified by clinicians as having significant influence on clinician judgment and score assignment. In addition, to best mimic in‐clinic administration, we incorporated in the e‐instruments specific examples and content for answer choices, as well as extra details and prompts that clinicians most frequently used such as instructions to have scratch paper, a pencil, and a calculator readily available and reminders to scroll down to view the entire question in the eFCI (Table 1, Rows 8 and 9). Additionally, we used multiple‐choice answers rather than have open‐ended response fields. However, presenting a list of multiple‐choice answers may have inherently prompted participants, diminished the use of a participant's abstract thinking to craft a response, and possibly changed the construct of the question. To reconcile this in the development of the eCDR and the eFCI, multiple‐choice options and dropdown menus were created to present and capture a wide range of answer responses. Adaptations to existing scoring algorithms were made to incorporate the addition of multiple‐choice answers.

We have implemented the eCDR and eFCI using the BHR platform and have collected data from participants across four clinical sites (UCSF, WU, UAB, and Mayo Clinic). This supports previous studies that have demonstrated the feasibility and benefit of leveraging unsupervised, online measures as a scalable approach to assess cognitive and functional decline and to improve the efficiency of screening and recruitment for clinical trials. , ,

While our results demonstrate that development and implementation of these new online measures is feasible, we acknowledge limitations in our study. First, we acknowledge selection biases that may limit generalizability of our results. Our sample includes only individuals who have access to the internet, are computer literate, and can successfully complete tasks online. In addition, most of the sample self‐identified as White and had high educational attainment. A crucial next step is to assess feasibility and validity of the online instruments in diverse populations. We are currently using culturally tailored participant communications and digital advertising to increase the enrollment of traditionally underrepresented populations (URPs) in the study. These include recruitment strategies that adapt best practices generated from other projects, rooted in community engaged research methods, focused on enrolling diverse populations in BHR. If validated in diverse populations, the electronic instruments have the potential to expand accessibility to these valuable measures of cognitive and functional status, particularly for individuals who do not have access to a research clinic due to where they live or other logistical constraints that are known to unduly burden URPs. , , , , , , , , , , , , As digital technology and internet use among older adults, including URPs, continues to grow, online assessment approaches show promise as a feasible, valid approach. , , , , , , , , However, lack of access to technology remains a concern for making these assessments available to those with lower socioeconomic status. Further, because 30% of participants expressed that their confidence using a computer “depends on the task,” it will be important to explore the contribution of computer confidence to e‐instrument performance in future analyses. The current sample includes 73% CU individuals and 20% with MCI. The current study focuses on the potential of the e‐instruments to detect/screen for early stages of cognitive impairment. It will be important to address whether the e‐instruments are feasible and valid in a sample with a broader range of cognitive impairment in future studies, and to identify the level of cognitive impairment at which participants are unable to reliably complete e‐instruments.

Finally, in our study, we did not counterbalance the order of e‐instrument administration (in clinic vs. at home), due to current clinical protocols at the sites. Therefore, we could not address the degree to which the initial, supervised administration influenced the subsequent, unsupervised e‐instrument performance. Further, although we have included clear instructions for participants to complete the eCDR and eFCI without any assistance, it is possible that the study partner or someone else may have helped the participant during unsupervised visits.

The goal of this article is to describe the methods used to develop the eCDR and eFCI, and to present evidence for feasibility of their remote administration in a cohort of older adults. Ongoing analyses are now validating the eCDR and eFCI as novel measures to detect and monitor cognitive and functional decline. Future analyses will: (1) compare performance on e‐instruments to their in‐clinic counterparts, (2) estimate associations between e‐instrument performance and both baseline and longitudinal change in in‐clinic neuropsychological tests from the UDS, and (3) assess the sensitivity and specificity of the e‐instruments to detect MCI. We also plan to investigate the association between e‐instrument performance and AD biomarkers, including amyloid and tau. If validated, the eCDR and eFCI will have many, high‐impact applications in the Alzheimer's disease and related dementias and public health fields. They can be used to efficiently screen participants for preclinical and prodromal AD clinical trials and observational studies, and to identify suitable candidates for further in‐clinic assessment and treatment of cognitive impairment. This approach can ultimately have practical clinical applications in health care, epidemiology, and public health settings, to identify individuals at risk for AD and cognitive decline.

CONFLICTS OF INTEREST

Taylor Howell, Shilpa Gummadi, Chau Bui, Jessica Santhakumar, Kristen Knight, Carol Chambless, Adam Gersteneker, Roy Martin, Connie Mayo, Krista L. Moulder, Maria Carroll, and Yan Li have no interests to declare. Rachel L. Nosheny received support for her work from the NIH; received funding for academic travel from the Mild Cognitive Impairment (MCI) 2020 Symposium; and served on the International Society to Advance Alzheimer's Research and Treatment (ISTAART) Subjective Cognitive Decline (SCD) Professional Interest Area (PIA) “Dyadic Patterns of Subjective Report” workgroup. R. Scott Mackin has received research support from the National Institute of Mental Health and Johnson and Johnson. Nikki H. Stricker has received research support outside the scope of this work from the NIA, Alzheimer's Association, and Biogen; a Mayo Clinic invention disclosure has been submitted for remote cognitive assessment tools outside the scope of this work (the Stricker Learning Span and the Mayo Test Drive platform). Ronald C. Petersen receives funding from the National Institute on Aging: P30 AG062677, U01 AG006786, U01 AG024904, U24 AG057437 and National Institute for Neurological Disorders and Stroke: Uf1 NS125417. He consults for Roche, Inc., Merck, Inc, Biogen, Inc., Genentech, Inc., Eisai, Inc. and Nestle, Inc. He receives royalties from Oxford University Press and UpToDate. John C. Morris is funded by NIH grants # P30 AG066444; P01AG003991; P01AG026276; U19 AG032438; and U19 AG024904. Neither his nor his family owns stock or has equity interest (outside of mutual funds or other externally directed accounts) in any pharmaceutical or biotechnology company. Daniel Marson and UAB Research Foundation (UABRF) have previously received royalty income and other compensation for the use of the FCI‐SF in a clinical trial. He receives support from the NIH and is also the inventor of the FCI‐SF, which is owned by the UABRF. He and his company receive compensation for sales and services related to the FCI‐SF. He serves as a paid consultant to UCSD in connection to the e‐VAL research project and the present paper. Richard Kennedy and Yue Zhang receive support from NIH grants R01 AG057684 and R01AG059009. Erik D. Roberson receives support for his work from the following funding sources: NIH, Bluefield Project to Cure Frontotemporal Dementia, Alzheimer's Drug Discovery Foundation. He has also received support from the Alzheimer's Association, Alector, the Weston Brain Institute, and BrightFocus Foundation. He has served on advisory boards for Biogen, AGTC, and AVROBIO, and on a DSMB for Lilly. Michael W. Weiner receives support for his work from the following funding sources: NIH: 5U19AG024904‐14; 1R01AG053798‐01A1; R01 MH098062; U24 AG057437‐01; 1U2CA060426‐01; 1R01AG058676‐01A1; and 1RF1AG059009‐01, DOD: W81XWH‐15‐2‐0070; 0W81XWH‐12‐2‐0012; W81XWH‐14‐1‐0462; W81XWH‐13‐1‐0259, PCORI: PPRN‐1501‐26817, California Dept. of Public Health: 16–10054, U. Michigan: 18‐PAF01312, Siemens: 444951–54249, Biogen: 174552, Hillblom Foundation: 2015‐A‐011‐NET, Alzheimer's Association: BHR‐16‐459161; The State of California: 18–109929. He also receives support from Johnson & Johnson, Kevin and Connie Shanahan, GE, VUmc, Australian Catholic University (HBI‐BHR), The Stroke Foundation, and the Veterans Administration. He has served on Advisory Boards for Eli Lilly, Cerecin/Accera, Roche, Alzheon, Inc., Merck Sharp & Dohme Corp., Nestle/Nestec, PCORI/PPRN, Dolby Family Ventures, National Institute on Aging (NIA), Brain Health Registry and ADNI. He serves on the Editorial Boards for Alzheimer's & Dementia, TMRI, and MRI. He has provided consulting and/or acted as a speaker/lecturer to Cerecin/Accera, Inc., Alzheimer's Drug Discovery Foundation (ADDF), Merck, BioClinica, Eli Lilly, Indiana University, Howard University, Nestle/Nestec, Roche, Genentech, NIH, Lynch Group GLC, Health & Wellness Partners, Bionest Partners, American Academy of Neurology (AAN), NYU, Japanese Government Alliance, National Center for Geriatrics and Gerontology (Japan), US Against Alzheimer's, Society for Nuclear Medicine and Molecular Imaging (SNMMI), The Buck Institute for Research on Aging, FUJIFILM‐Toyama Chemical (Japan), Garfield Weston, Baird Equity Capital, and T3D Therapeutics. He holds stock options with Alzheon, Inc., Alzeca, and Anven. The following entities have provided funding for academic travel; Kenes, Intl., Merck, ADCS, ATRI, Eli Lilly, The Alzheimer's Association, Merck, Tokyo University, Kyoto University, AAN, AC Immune, CHU Toulouse, St. George Hospital University, Indiana U., U. Melbourne, Australian Catholic University, Japanese Government Alliance, National Center for Geriatrics and Gerontology (Japan), US Against Alzheimer's, NYU, USC, and SNMMI.