A Validation Study of the Inbrain CST: a Tablet Computer-based Cognitive Screening Test for Elderly People with Cognitive Impairment.

BACKGROUND: Computerized versions of cognitive screening test could have advantages over pencil-and-paper versions by eliminating rater-dependent factors and saving the time required to score the tests and report the results. We developed a computerized cognitive screening test (Inbrain Cognitive Screening Test [Inbrain CST]) that takes about 30 minutes to administer on a touchscreen computer and is composed of neuropsychological tests already shown to be sensitive in detecting early cognitive decline in Alzheimer's disease (AD). The aims of this study were to 1) introduce normative data for Inbrain CST, 2) verify its reliability and validity, 3) assess clinical usefulness, and 4) identify neuroanatomical correlates of Inbrain CST.
METHODS: The Inbrain CST runs on the Microsoft Windows 10 operating system and comprises 7 subtests that encompass 5 cognitive domains: attention, language, visuospatial, memory, and executive functions. First, we recruited 480 cognitively normal elderly people (age 50-90) from communities nationwide to establish normative data for Inbrain CST. Second, we enrolled 97 patients from our dementia clinic (26 with subjective cognitive decline [SCD], 42 with amnestic mild cognitive impairment [aMCI], and 29 with dementia due to AD) and investigated sensitivity and specificity of Inbrain CST for discriminating cognitively impaired patients from those with SCD using receiver operating characteristic (ROC) curve analyses. Third, we compared the Inbrain CST scores with those from another neuropsychological test battery to obtain concurrent validity and assessed test-retest reliability. Finally, magnetic resonance imaging (MRI)-based cortical thickness analyses were performed to provide anatomical substrates for performances on the Inbrain CST.
RESULTS: First, in the normative sample, the total score on the Inbrain CST was significantly affected by age, years of education, and gender. Second, Inbrain CST scores among the three patient groups decreased in the order of SCD, aMCI, and AD dementia, and the ROC curve analysis revealed that Inbrain CST had good discriminative power for differentiating cognitively impaired patients from those with SCD. Third, the Inbrain CST subtests had high concurrent validity and test-retest reliability. Finally, in the cortical thickness analysis, each cognitive domain score and the total score of Inbrain CST showed distinct patterns of anatomical correlates that fit into the previously known brain-behavior relationship.
CONCLUSION: Inbrain CST had good validity, reliability, and clinical usefulness in detecting cognitive impairment in the elderly. Furthermore, it showed neuroanatomical validity through MRI cortical thinning patterns. These results suggest that Inbrain CST is a useful cognitive screening tool with efficiency and validity to detect mild impairments in cognition in clinical settings.

INTRODUCTION

As human life expectancy increases, the number of patients with dementia continues to increase. According to the World Alzheimer Report 2015 in Alzheimer Disease International, about 46.8 million people worldwide have dementia, and the number of patients with dementia will increase to 74.7 million by 2030.1 Individuals with dementia need to be detected at an early stage to enable effective prevention and early intervention to reduce the effects of dementia on current and future society.

Because cognitive decline is an early clinical symptom of dementia, various cognitive screening tests, such as the Mini Mental State Examination (MMSE)2 and the Montreal Cognitive Assessment (MoCA),3 are commonly used to identify cognitive impairment in clinical and community settings. However, the MMSE is not sensitive enough to detect subtle cognitive deficits, and although MoCA is known to be useful for detecting mild cognitive impairment (MCI),34 both MMSE and MoCA only provide a total or global score but not individual cognitive domain scores. If a cognitive screening test involves an in-depth examination of multiple cognitive domains and yields cognitive domain sub-scores as well as a global score, it should be useful to differentiate amnestic from non-amnestic type of MCI or single from multi-domain type of MCI.5 In addition, cognitive screening tests should be easy to administer and score and also allow clinicians to easily interpret the results.

Technological advances have facilitated a transition from conventional paper-and-pencil neuropsychological tests to computerized testing methods. In fact, many conventional neuropsychological tests have been digitally adapted6 and shown to have similar ability to differentiate cognitive impairment in older adults.789 Computerized screening tests have several advantages over older paper-and-pencil versions. First, computer-based tests, including tablet computer-based cognitive tests, can improve the scientific value of the measurements by eliminating common sources of external errors, such as the examiner's bias in test administration, and providing a consistent testing environment.61011 Second, digital devices can collect not only the final results of a test but also information about the testing process, including reaction times and sequences of responses. Third, automated scoring systems eliminate the time required for healthcare practitioners to score the tests. Therefore, tablet- or computer-based testing devices can improve the sensitivity and feasibility of cognitive tests intended to detect and monitor cognitive changes and disease progression, which would also enable the large-scale administration of cognitive testing in clinical trials.

We developed a new tablet-based cognitive screening test, named Inbrain CST, to efficiently and accurately detect cognitive decline in older adults. It was designed to encompass five cognitive domains: attention, language, visuospatial, memory, and executive function. Therefore, the test provides not only the total score and individual test scores but also five cognitive domain scores that can be used practically to diagnose mild cognitive impairment. Inbrain CST is administered on a tablet-PC, requires minimal assistance, and takes approximately 30 minutes to complete, minimizing the fatigue from prolonged testing that can occur in elderly people during comprehensive neuropsychological testing. The first purpose of this study was to introduce the standardization process and patterns of norms for the Inbrain CST. Second, we investigated its clinical usefulness in discriminating patients with cognitive impairment from those with normal cognition. Third, we compared the Inbrain CST scores with the results from another comprehensive neuropsychological test battery to establish concurrent validity. Finally, to examine the clinical meaningfulness of this new test, we confirmed the neuroanatomical substrates of Inbrain CST by conducting a correlation analysis between cortical thickness and Inbrain CST scores.

METHODS

Participants

To develop normative data for the Inbrain CST, we recruited 496 cognitively normal individuals who dwell in communities nationwide, using statistics about the elderly populations in different regions of Korea.12 We included 230 participants from Seoul and Gyeonggi-do (46.4%), 20 from Gangwon-do (4%), 18 from Chungcheong-do (3.6%), 59 from Jeolla-do (11.9%), and 169 from Gyeongsang-do (34.1%). Their ages ranged from 50 to 90, and they all had at least 1 year of formal education. The cognitive functions of these participants were considered intact because their scores on the Korean version of the MMSE (K-MMSE)13 were above the age/education-matched mean minus 1 standard deviation. Participants with various medical conditions that could cause cognitive impairment were excluded from this normative sample, based on Christensen's health screening criteria.14 Sixteen participants out of a total 496 were excluded, because 14 participants reported as having subtle decline in the activities of daily living on the Korean Instrumental Activities of Daily Living15 and 2 participants were identified as outlier based on the inter-quantile range methods.16

To validate the Inbrain CST, we recruited another group of participants: 97 patients with subjective cognitive decline (SCD, n=26), amnestic mild cognitive impairment (aMCI, n=42), or Alzheimer's disease (AD) dementia (n=29) who visited the Memory Disorder Clinic at Samsung Medical Center from August 2018 to August 2019. All these patients had been assessed using comprehensive neuropsychological tests, the Seoul Neuropsychological Screening Battery 2nd edition (SNSB-II),17 within one year before the Inbrain CST was administered. An experienced neurologist reviewed all clinical information about the patients and diagnosed them with SCD, aMCI, or AD dementia according to the relevant diagnostic criteria. The criteria for SCD, based on the recommendation of Molinuevo et al.,18 are as follows: 1) self-experienced persistent decline in cognitive capacity in comparison with a previously normal status, 2) normal performance on all neuropsychological tests, and 3) cannot be explained by other psychiatric or neurologic disease. The criteria for aMCI, based on Petersen's criteria,19 are as follows: 1) cognitive complaint, preferably corroborated by an informant, 2) objective cognitive impairment for age and educational level, 3) relatively preserved general cognition, 4) intact activities of daily living, and 5) not demented. The diagnosis of AD dementia is based on the criteria proposed by the National Institute of Neurological and Communicative Disorders and Stroke and the Alzheimer's Disease and Related Disorders Association.20

Inbrain CST

The Inbrain CST was administered on a 12-inch tablet PC (resolution 1,920 × 1,280 pixels) running Microsoft Windows 10. It consists of 7 neuropsychological tests assessing 5 cognitive functions: The Visual Span Test (VST): forward and backward tasks were used to assess attention; the Difficult Naming Test (DNT) and semantic (fruits)/phonemic (Korean alphabet digeut) word fluency test assessed language; the Block Design Test assessed visuospatial ability; the time orientation questions and Word Place Association Test (WPAT) assessed memory; and the Korean-Trail Making Test-Elderly version (K-TMT-E)21 was applied to evaluate executive function. The details of each test and its measurement variables are explained in Table 1. Each subtest contained both verbal and written instructions, and an examiner monitored the entire testing process and made behavioral observations during testing. Scores were automatically recorded, but in some tests (e.g., the word fluency test) the examiner had to write down participants' responses, double-checking the accuracy later by replaying the voice-recorded answers.

Table 1

Detailed explanation of each test in the Inbrain CST

Cognitive domain	Test	Description	Variables (maximum score)
Attention	VST	Nine squares are scattered on a screen. Certain squares turn blue briefly in variable sequence, and the participant should tap the squares that changed color in the same order (forward task) or in reverse order (backward task). The number of squares turning blue increases from two to eight, and two performance chances are given for each trial. One point is given for correctly tapping the order. Possible range of scores for both tasks is 1 to 14. The test stops when the participant fails twice in a trial.	VST: forward (14), VST: backward (14)
Language	DNT	Confrontational naming test with 15 line-drawn items that are relatively difficult to name because of their low usage frequency.	Number of correct responses
	Semantic & phonemic word fluency test	In the semantic generative naming trial, a participant should list as many fruits as possible in one minute.	Number of correct words for each trial
		In the phonemic generative naming trial, a participant should list as many words as possible beginning with a certain Korean alphabet for one minute.
Visuospatial function	Block Design Test	A pattern is presented on the screen using six types of squares that are all blue or all white or a combination. A participant should reproduce the same pattern by dragging one of the six squares. A total of 10 patterns are shown sequentially, and the difficulty of the pattern depends on the number of squares to be used and the time limitation. The scores for the items differ based on difficulty.	Total score (40)
Memory	Time orientation	A participant states the current year, month, date, day of the week, and season.	Total score (5)
	WPAT	Nine words are presented one by one in a 3 × 3 grid in a particular sequence over 3 trials. A participant should memorize the words and their location in the grid. After each trial, the participants recall the words both immediately and after a 10-minute delay. Then a recognition task asks the participants to recognize both the words and their location.	Correct number of responses for immediate recall (27), delayed recall (9), word recognition (18), place recognition (9)
Executive function	K-TMT-E	Part A: a participant connects numbers from 1 to 15 in ascending order as quickly as possible, using a digital pen.	Part A time to complete, Part B time to complete
		Part B: a participant connects the numbers and the day of the week alternatively in order as quickly as possible using a digital pen.

CST = Cognitive Screening Test, VST = Visual Span Test, DNT = Difficult Naming Test, WPAT = Word Place Association Test, K-TMT-E = Korean-Trail Making Test-Elderly version.

Formulation of the total score of Inbrain CST

To formulate the total score of the Inbrain CST, a multiple factor analysis (MFA)22 was conducted using 13 measurement variables (the scores of VST: forward; VST: backward; DNT; semantic word fluency test; phonemic word fluency test; Block Design Test; time orientation; WPAT: immediate recall, delayed recall, word recognition, and place recognition; K-TMT-E Part A time and K-TMT-E Part B time). The different combinations of variables were combined into five groups representing the cognitive domains: attention, language, visuospatial function, memory, and executive function. The MFA was then generated to identify similarities within the groups of variables. The results indicate that the 13 variables can be summarized by two factors, which were adequate to explain our data set. Different weights (factor loadings) were assigned to each variable. The cognitive domain scores were estimated using different combinations of the weighted measurement variables. The summation of the five domain scores was used to construct the total score. The total score of the Inbrain CST was standardized to the range from 0 to 100: attention (0–20), language (0–23), visuospatial function (0–10), memory (0–26), and executive function (0–21).

Development of norms for Inbrain CST

A Bayesian analysis23 was applied to the normative sample to obtain the distribution and predictive interval for the standardized scores of Inbrain CST based on age, education, and gender. Although the choice of prior settings can be quite important, we did not have any previous knowledge about the measurement features of the Inbrain CST scores, which corresponded to the uniform distribution. The posterior distribution was then obtained using the No-U-Turn Sampler, an extension of the Hamiltonian Monte Carlo engine.24 A regression line with a linear trend was generated as the simplest predictive model, using age and education as covariates.

To use the posterior distribution for reference, we needed to check the reliability of the Markov Chain Monte Carlo (MCMC) by inspecting the convergence of the Markov chain and the malfunction of the MCMC from the MCMC summary table and trace plot. Based on the numerical summary of MCMC sampling, the potential scale reduction factor, Rhat,25 was less than 1.1, and the effective number of samples was more than 1,000, indicating that the MCMC chains had converged.26 A well-mixed chain indicated that the MCMC samples did not correlate with one another within the parameter. Because the trace plot also showed well-mixed MCMC chains, the credible interval was estimated from the posterior probability. The constructed credible interval was then used as the standard interval of the Inbrain CST for age and education.

Comparison of Inbrain CST with other neuropsychological tests for concurrent validity

For the validation study with 97 individuals already diagnosed with cognitive impairment, the SNSB-II17 had been administered to all participants within one year before they completed the Inbrain CST. The following tests in the SNSB-II were used for the concurrent validity study: Digit Span test forward and backward for attention; semantic and phonemic Controlled Oral Word Association Test and the Korean version of the Boston Naming Test for language; the Rey-Osterrieth Complex Figure Test (RCFT) for visuospatial function; Seoul Verbal Learning Test-Elderly version (SVLT-E) for memory; and the K-TMT-E for executive function. The K-MMSE was also administered to assess patients' general mental state.13

Brain magnetic resonance imaging (MRI) and cortical thickness

In addition to conventional MRI sequences, we obtained 3-dimensional (3D) T1 turbo field echo MR images from 84 patients (21 patients with SCD, 37 patients with aMCI, and 26 patients with AD). The 3D images were acquired using a 3.0T MRI scanner (Philips 3.0T Achieva; Philips Medical Systems, Best, The Netherlands) with the following imaging parameters: sagittal slice thickness, 1.0 mm with 50% overlap; no gap; repetition time of 9.9 ms; echo time of 4.6 ms; flip angle of 8°; and matrix size of 240 × 240 pixels reconstructed to 480 × 480 over a field view of 240 mm.

We carried out image preprocessing for each participant using FreeSurfer 6.0 (http://surfer.nmr.mgh.harvard.edu/) and then constructed outer and inner cortical surface meshes from the MR volume of each subject. The outer surface was constructed by deforming the inner surface, resulting in two meshes that were isomorphic with the same vertices and connectivity. Using a previously proposed method,27 we transformed each subject's cortical surface to 40,962 vertices from each hemisphere, which served as the inter-subject correspondence. To eliminate noise in the cortical thickness data, the manifold harmonic transform (MHT) was used to map the cortical thickness from the surface onto the frequency domain.2829 This MHT discarded high-frequency components in the transformed cortical thickness data as noise to reduce the dimensionality of the cortical thickness data.27

To investigate neuroanatomical substrate related to the Inbrain CST scores, we performed a correlation analysis between the Inbrain CST scores (the total and five domain scores) and cortical thickness using the Freesurfer group analysis. Because we expected to find relationships of age, gender, education, and intracranial volume (ICV) with cortical thickness, we obtained the partial correlation coefficient between the Inbrain CST scores and cortical thickness after controlling for age, gender, education, and ICV as covariates. To control for the type I error of multiple comparison, the P values (−log10p−val) of the correlation coefficients were transformed into q values, which take the false discovery rate into account. The q values were coded in color and projected onto the 3D group-averaged surface model using the Surfstat MATLAB toolbox (http://www.math.mcgill.ca/keith/surfstat).

Statistical analysis

The statistics for the normative sample involving 480 cognitively normal, community-dwelling, older adults have already been described above (“Formulation of the total score of Inbrain CST” and “Development of norms for Inbrain CST”). To examine the influence of demographic variables on the total score of Inbrain CST, gender differences were investigated using the independent t-test, and the effects of age and education level were confirmed through a multiple regression analysis.

In the validation study, analysis of variance (ANOVA) and Pearson's χ2 test were used to analyze the demographic characteristics of the three patient groups. To examine the performance differences on individual tests, cognitive domain scores, and total score of Inbrain CST among the three patient groups, analysis of covariance was used while adjusting for the demographic variables that differed statistically among the three groups. Concurrent validity was assessed using a correlation analysis between the Inbrain CST individual test scores and the subtests of the SNSB-II described above. Test–retest reliability was evaluated using an interclass correlation analysis. We conducted receiver operating characteristic (ROC) curve analyses and obtained the sensitivity and specificity of Inbrain CST to ensure that it can be a useful instrument for differentiating aMCI and AD dementia from cognitively normal SCD. Statistics identifying anatomical correlations between cortical thickness and Inbrain CST were described above (“Brain magnetic resonance imaging (MRI) and cortical thickness”).

Ethics statement

This study protocol was reviewed and approved by the Institutional Review Board (IRB) of Samsung Medical Center (IRB approval No. 2017-04-058). All participants, community-dwelling, cognitively normal, older adults and the members of all three patient groups, signed IRB-approved informed consent prior to participation.

RESULTS

Normative data for the Inbrain CST

Of the 480 participants, 192 (40%) were men and 288 (60%) were women. The mean age was 67.58 ± 9.89 (men, 68.2 ± 9.15; women, 67.15 ± 10.34), and the mean years of education was 11 ± 3.86 (men, 12.21 ± 3.48; women, 10.19 ± 3.89). The mean Inbrain CST total score among the 480 normative subjects was 54.48 ± 12.58 (range, 24–81). The Inbrain CST total score was significantly affected by age, years of education, and gender. Women's total score of Inbrain CST was lower than men's (men, 56.57 ± 10.76; women, 53.09 ± 13.50; t = 3.00; P = 0.003). In the multiple regression analysis, the Inbrain CST total score became lower as age increased (t = −19.44, P < 0.001) and years of education decreased (t = 16.00, P < 0.001). The mean and standard deviation of the Inbrain CST total score in this normative sample are presented in Table 2, stratified according to age, educational level, and gender. The norms of the other cognitive domain scores and individual subtest scores will be provided by the Inbrain CST software, which can be purchased for commercial use from the Microsoft Store.

Table 2

Summary of normative data for the Inbrain CST total score according to age, education and gender

Age, yr	Years of education
	Men				Women
	1–6	7–9	10–12	≥ 13	1–6	7–9	10–12	≥ 13
50–59	54.40 ± 7.93	60.10 ± 6.83	64.00 ± 6.82	69.80 ± 7.76	52.00 ± 7.93	59.00 ± 6.60	63.60 ± 6.50	70.50 ± 7.80
60–69	47.80 ± 7.89	53.60 ± 6.77	57.40 ± 6.77	63.10 ± 7.79	44.80 ± 7.74	51.70 ± 6.58	56.20 ± 6.54	63.20 ± 7.78
70–79	41.40 ± 7.86	47.10 ± 6.78	51.10 ± 6.76	56.60 ± 7.70	37.50 ± 7.78	44.40 ± 6.57	48.90 ± 6.58	55.90 ± 7.76
80–90	34.80 ± 7.86	40.60 ± 6.81	44.40 ± 6.85	50.10 ± 7.82	30.10 ± 7.81	37.00 ± 6.60	41.70 ± 6.60	48.50 ± 7.73

CST = Cognitive Screening Test.

Performances on the Inbrain CST among the three patient groups

The ANOVA revealed that the three groups (SCD, aMCI, and AD) differed significantly in age but not education. In the post hoc analysis, the individuals with SCD were significantly younger than the AD patients (P = 0.034). The Pearson's χ2 test showed a difference in gender, such that women were overrepresented in the SCD group. As expected, an ANOVA revealed significant differences in K-MMSE scores, with the SCD group scoring the highest, followed by the aMCI and AD patients. The demographic characteristics of the patient groups are presented in Table 3.

Table 3

Clinical and demographic characteristics of participants

Variables	Group			P value	Post hoc
	SCD (n = 26)	aMCI (n = 42)	AD (n = 29)
Age in years	68.46 ± 6.28	71.69 ± 7.30	73.62 ± 8.74	0.042	SCD < AD
Years of education	12.62 ± 3.68	12.57 ± 3.89	12.17 ± 4.48	NS
Gender, M/W	3/23	18/24	14/15	0.009
K-MMSE	29.04 ± 1.00	26.33 ± 2.35	20.45 ± 3.56	< 0.001	SCD > aMCI > AD

SCD = subjective cognitive decline, aMCI = amnestic mild cognitive impairment, AD = Alzheimer's disease, K-MMSE = Korean-Mini Mental State Examation, NS = not significant.

Overall, performances differed significantly among the three patient groups in all the individual subtests of the Inbrain CST. The post hoc analysis revealed that the SCD group scored higher than the aMCI and AD groups in most tests. The exception was the phonemic word fluency test, in which the aMCI group performed as well as the SCD group. The aMCI patients outperformed AD patients in all the individual subtests of the Inbrain CST. In all the cognitive domain scores and the total score of Inbrain CST, the group scores descended in the order of SCD, aMCI, and AD; these results are presented in Table 4.

Table 4

Inbrain CST subtest scores, cognitive domain scores, and total scores in the three patient groups

Variables	Test (maximum score)	Group			P value	Post hoc
		SCD (n = 26)	aMCI (n = 42)	AD (n = 29)
Attention	VST: forward (14)	5.58 ± 3.36	4.69 ± 2.42	2.48 ± 2.43	< 0.001	SCD > aMCI > AD
	VST: backward (14)	5.65 ± 2.15	4.98 ± 2.23	2.76 ± 2.44	< 0.001	SCD > aMCI > AD
Language	DNT (15)	10.50 ± 3.35	8.90 ± 2.84	5.21 ± 3.99	< 0.001	SCD > aMCI > AD
	Semantic word fluency test	11.42 ± 2.62	9.26 ± 3.24	6.07 ± 2.89	< 0.001	SCD > aMCI > AD
	Phonemic word fluency test	8.08 ± 3.72	8.50 ± 4.00	5.45 ± 3.96	< 0.005	SCD = aMCI > AD
Visuospatial function	Block Design Test (40)	29.62 ± 11.78	19.29 ± 13.87	9.93 ± 11.56	< 0.001	SCD > aMCI > AD
Memory	Time orientation (5)	4.54 ± 0.71	3.83 ± 1.31	1.90 ± 1.47	< 0.001	SCD > aMCI > AD
	WPAT: IR (27)	17.77 ± 2.78	13.95 ± 3.96	9.03 ± 3.68	< 0.001	SCD > aMCI > AD
	WPAT: DR (9)	6.27 ± 1.54	2.93 ± 2.51	0.14 ± 0.44	< 0.001	SCD > aMCI > AD
	WPAT: WR (18)	16.92 ± 1.65	15.33 ± 2.23	11.45 ± 2.84	< 0.001	SCD > aMCI > AD
	WPAT: PR (9)	5.54 ± 2.67	2.60 ± 1.96	1.17 ± 1.23	< 0.001	SCD > aMCI > AD
Frontal/executive function	K-TMT-E-A time (sec)	21.54 ± 6.80	27.84 ± 7.46	51.71 ± 28.42	< 0.001	SCD > aMCI > AD
	K-TMT-E-B time (sec)	36.93 ± 20.42	60.74 ± 33.51	157.40 ± 10.51	< 0.001	SCD > aMCI > AD
Cognitive domain scores	Attention (20)	7.87 ± 3.13	6.80 ± 2.38	3.70 ± 3.15	< 0.001	SCD > aMCI > AD
	Language (23)	13.63 ± 3.02	12.02 ± 3.51	7.52 ± 3.77	< 0.001	SCD > aMCI > AD
	Visuospatial function (10)	7.40 ± 2.95	4.82 ± 3.47	2.48 ± 2.89	< 0.001	SCD > aMCI > AD
	Memory (26)	20.57 ± 2.67	15.89 ± 3.43	9.83 ± 2.73	< 0.001	SCD > aMCI > AD
	Executive function (21)	10.00 ± 2.69	7.07 ± 2.02	4.41 ± 2.48	< 0.001	SCD > aMCI > AD
Total score (100)		59.47 ± 10.70	46.60 ± 9.70	27.94 ± 11.15	< 0.001	SCD > aMCI > AD

CST = Cognitive Screening Test, SCD = subjective cognitive decline, aMCI = amnestic mild cognitive impairment, AD = Alzheimer's disease, VST = Visual Span Test, DNT = Difficult Naming Test, WPAT: IR = Word Place Association Test: Immediate Recall, WPAT: DR = Word Place Association Test: Delayed Recall, WPAT: WR = Word Place Association Test: Word Recognition, WPAT: PR = Word Place Association Test: Place Recognition, K-TMT-E-A = Korean-Trail Making Test-Elderly version Part A, K-TMT-E-B = Korean-Trail Making Test-Elderly version Part B.

Correlation between Inbrain CST and SNSB-II in the three patient groups

To investigate concurrent validity, a correlation analysis was performed between the individual subtests of the Inbrain CST and conventional neuropsychological tests from the SNSB-II, and the results are presented in Table 5. The correlations between the individual subtests of Inbrain CST and the relevant neuropsychological tests that were supposed to measure the same cognitive constructs were mostly significant. However, correlation coefficients between the two attention tests, the VST of Inbrain CST and the Digit Span test of SNSB-II, ranged from 0.140 to 0.499, which was lower than expected. The correlation between the VST-forward of Inbrain CST and the Digit Span forward test was not statistically significant. In the language domain, the correlation coefficient between the two confrontational naming tasks was 0.811, and correlation coefficient between the word fluency tests of Inbrain CST and the SNSB-II ranged from 0.654 to 0.709, indicating strong correlation. In the visuospatial domain, the Block Design Test of Inbrain CST and the RCFT copy of SNSB-II showed a significant but relatively weak correlation (r = 0.480). The memory domain yielded strong correlation; the correlation coefficients for the immediate recall, delayed recall, and recognition tasks between the WPAS of Inbrain CST and the SVLT of SNSB-II ranged from 0.506 to 0.861. In the executive function domain, the computerized version of the K-TMT-E in Inbrain CST and the conventional K-TMT-E showed strong correlation, ranging from 0.653 to 0.858. The total score of Inbrain CST correlated strongly with the K-MMSE (r = 0.852), and the correlations between the total score of Inbrain CST and the other neuropsychological tests from the SNSB-II were also significant (r = 0.437–0.764).

Table 5

Correlation coefficients between Inbrain CST and various neuropsychological tests (n = 97)

Variables	VST: forward	VST: backward	Category fluency: fruits	Phonemic fluency: digeut	DNT	Block Design Test	Time orientation	WPAT: IR	WPAT: DR	WPAT: WR	WPAT: PR	K-TMT-E-A time	K-TMT-E-B time	Inbrain CST total score
Digit Span: forward	0.140	0.293**	0.270**	0.379***	0.408***	0.387***	0.162	0.295**	0.209*	0.267**	0.280**	−0.289**	−0.339**	0.437***
Digit Span: backward	0.352***	0.499***	0.393***	0.442***	0.438***	0.440***	0.258*	0.311**	0.325**	0.325**	0.320**	−0.422***	−0.489***	0.571***
Semantic fluency: animal	0.288**	0.424***	0.654***	0.489***	0.530***	0.507***	0.409***	0.478***	0.545***	0.476***	0.471***	−0.524***	−0.533***	0.690***
Phonemic fluency: total	0.201*	0.383***	0.466***	0.709***	0.529***	0.554***	0.273**	0.307**	0.386***	0.355***	0.343**	−0.503***	−0.484***	0.630***
K-BNT	0.320**	0.383***	0.552***	0.471***	0.811***	0.484***	0.434***	0.416***	0.499***	0.541***	0.450***	−0.531***	−0.591***	0.708***
RCFT: copy	0.410***	0.452***	0.401***	0.396***	0.348***	0.480***	0.504***	0.424***	0.419***	0.442***	0.313**	−0.778***	−0.688***	0.618***
Time orientation	0.396***	0.528***	0.464***	0.375***	0.426***	0.562***	0.818***	0.506***	0.684***	0.739***	0.554***	−0.631***	−0.666***	0.764***
SVLT: IR	0.333**	0.376***	0.670***	0.395***	0.505***	0.540***	0.533***	0.677***	0.758***	0.559***	0.597***	−0.446***	−0.572***	0.742***
SVLT: DR	0.305**	0.384***	0.551***	0.276**	0.427***	0.544***	0.612***	0.628***	0.861***	0.620***	0.688***	−0.456***	−0.506***	0.736***
SVLT: recognition	0.318**	0.313**	0.453***	0.185	0.452***	0.493***	0.591***	0.525***	0.672***	0.692***	0.571***	−0.455***	−0.466***	0.650***
K-TMT-E-A-time	−0.319**	−0.377***	−0.358***	−0.408***	−0.319**	−0.378***	−0.407***	−0.319**	−0.310**	−0.395***	−0.273**	0.683***	0.653***	−0.526***
K-TMT-E-B-time	−0.357***	−0.578***	−0.549***	−0.445***	−0.500***	−0.522***	−0.511***	−0.483***	−0.501***	−0.579***	−0.454***	0.677***	0.858***	−0.733***
K-MMSE	0.469***	0.626***	0.589***	0.399***	0.504***	0.634***	0.753***	0.597***	0.712***	0.745***	0.608***	−0.701***	−0.808***	0.852***

CST = Cognitive Screening Test, K-BNT = Korean Boston Naming Test, RCFT = Rey Complex Figure Test, SVLT = Seoul Verbal Learning Test, SVLT: IR = Seoul Verbal Learning Test: Immediate Recall, SVLT: DR = Seoul Verbal Learning Test: Delayed Recall, K-TMT-E-A = Korean-Trail Making Test-Elderly version Part A, K-TMT-E-B = Korean-Trail Making Test-Elderly version Part B, K-MMSE = Korean-Mini Mental State Examination, VST = Visual Span Test, DNT = Difficult Naming Test, WPAT: IR = Ward Place Association Test: Immediate Recall, WPAT: DR = Ward Place Association Test: Delayed Recall, WPAT: WR = Ward Place Association Test: Word Recognition, WPAT: PR = Ward Place Association Test: Place Recognition.

*P < 0.05; **P < 0.01; ***P < 0.001.

Test–retest reliability of Inbrain CST

To examine the test–retest reliability of Inbrain CST, we re-administered it to 16 cognitively normal participants (5 men, 11 women) with a median interval of 175 days (range, 105–287). We performed an intraclass correlation (ICC) analysis for individual subtests of Inbrain CST, and the results are presented in Table 6. Although the ICC coefficients were statistically significant in most individual subtests, ranging from 0.492 to 0.914, no significant correlation was found for the VST-backward and K-TMT-E.

Table 6

Intraclass correlation coefficients for individual parameters of the Inbrain CST for test–retest reliability

Test parameters	Mean (SD) of the 1st test	Mean (SD) of the 2nd test	ICC
VST: forward	5.63 (2.70)	4.75 (3.26)	0.614*
VST: backward	4.88 (1.86)	5.38 (2.45)	0.518
Semantic word fluency test	9.63 (2.42)	10.56 (2.85)	0.631*
Phonemic word fluency test	8.81 (4.26)	8.50 (4.24)	0.912***
DNT	8.37 (2.28)	8.37 (2.58)	0.880***
Block Design Test	22.94 (12.50)	25.44 (14.69)	0.900***
Time orientation	4.25 (1.18)	3.88 (0.96)	0.707*
WPAT: IR	15.56 (4.29)	16.31 (3.79)	0.730**
WPAT: DR	4.19 (2.81)	4.50 (3.35)	0.914***
WPAT: WR	15.56 (2.48)	15.69 (1.85)	0.790**
WPAT: PR	3.12 (2.56)	3.69 (3.52)	0.878***
K-TMT-E A time	26.19 (71.97)	25.89 (64.31)	0.492
K-TMT-E B time	62.64 (41.88)	45.42 (15.72)	0.506

CST = Cognitive Screening Test, ICC = Intraclass correlation coefficient, VST = Visual Span Test, DNT = Difficult Naming Test, WPAT: IR = Word Place Association Test: Immediate Recall, WPAT: DR = Word Place Association Test: Delayed Recall, WPAT: WR = Word Place Association Test: Word Recognition, WPAT: PR = Word Place Association Test: Place Recognition, K-TMT-E-A = Korean-Trail Making Test-Elderly version Part A, K-TMT-E-B = Korean-Trail Making Test-Elderly version Part B, SD = standard deviation.

*P < 0.05; **P < 0.01; ***P < 0.001.

ROC curve analysis of Inbrain CST

SCD (n = 26) vs. aMCI + AD (n = 71)

We conducted an ROC curve analysis and calculated the area under the curve (AUC) to confirm the power of the Inbrain CST total score to differentiate cognitively impaired patients (aMCI and AD dementia groups) from cognitively normal people (SCD group). Inbrain CST had a sensitivity of 0.846 and a specificity of 0.845, with a cut-off total score of 50.4. The AUC of the Inbrain CST was 0.879, indicating that the power of discrimination for the cognitive impairment group was good (Fig. 1A).

Fig. 1

ROC curves for the Inbrain CST total score in the comparison between (A) the SCD group and the cognitively impaired group (aMCI + AD dementia), (B) SCD and aMCI, (C) the non-dementia group (SCD + aMCI) and the AD dementia group.

ROC = receiver operating characteristic, CST = Cognitive Screening Test, K-MMSE = Korean-Mini Mental State Examination, SCD = subjective cognitive decline, aMCI = amnestic mild cognitive impairment, AD = Alzheimer's disease.

SCD (n = 26) vs. aMCI (n = 42)

In discriminating the aMCI group from the cognitively normal SCD group, the AUC of the Inbrain CST total score was 0.812. The Inbrain CST total score had a sensitivity of 0.808 and a specificity of 0.762 when the cut-off score of 51.9 was applied (Fig. 1B).

SCD + aMCI (n = 68) vs. AD (n = 29)

When AD dementia patients were compared with individuals without dementia (the aMCI and SCD groups), a cut-off score of 39.1 produced an AUC of 0.930 for the Inbrain CST total score, with a sensitivity of 0.824 and specificity of 0.828 (Fig. 1C).

Cortical thinning patterns related to Inbrain CST

Inbrain CST attention domain score

The statistical map showed a significant correlation between cortical thickness and the Inbrain CST attention domain score in the precuneus, posterior and inferior areas of the temporal cortex, and small areas of the dorsolateral prefrontal, parietal and cingulate regions of the left hemisphere. In the right hemisphere, significant correlations were shown in the precuneus, temporal and parietal cortices, anterior cingulate gyrus and small areas of the orbitofrontal region (Fig. 2A).

Fig. 2

Cortical thinning pattern correlated with the cognitive domain scores of Inbrain CST. The q value denotes the FDR-corrected P value. (A) Attention domain score. (B) Language domain score. (C) Visuospatial function domain score. (D) Memory domain score. (E) Executive function domain score. (F) Total score.

CST = Cognitive Screening Test, FDR = false discovery rate.

Inbrain CST language domain score

Significant correlations between the language domain score and cortical thickness existed in the left temporal cortex, small areas of the left dorsolateral prefrontal and parietal cortex, and a very small area in the right middle temporal gyrus (Fig. 2B).

Inbrain CST visuospatial function domain score

The correlation between the visuospatial domain score and cortical thickness was observed only in a very small area of the superior temporal gyrus of the right hemisphere (Fig. 2C).

Inbrain CST memory domain score

The areas that represented a significant correlation between the memory domain score and cortical thickness were widespread compared with the other cognitive domains, encompassing the bilateral interior temporal regions, temporal and parietal cortices, precuneus, small areas of the dorsolateral prefrontal cortex and right anterior cingulate region (Fig. 2D).

Inbrain CST executive function domain score

The right anterior cingulate region, small areas of the bilateral parietal cortex, precuneus, and inferior temporal regions presented significant correlations between the executive function domain score and cortical thickness (Fig. 2E).

Inbrain CST total score

The cortical areas significantly correlated with the Inbrain CST total score were the bilateral inferior temporal regions, temporal and parietal cortices, precuneus, small areas in the dorsolateral prefrontal lobes and right anterior cingulate region (Fig. 2F).

DISCUSSION

In this study, we have introduced normative data and investigated the reliability, validity, and clinical usefulness of our newly developed, tablet PC-based, cognitive screening test, Inbrain CST. The main results are as follows: 1) Inbrain CST scores from the 480-person normative sample were influenced by age, educational level, and gender. 2) Patient groups with SCD, aMCI, and AD dementia showed significant differences in all subtests, the five domain scores, and the total score of Inbrain CST, indicating that Inbrain CST is sensitive enough to reflect the degree of cognitive decline. The ROC curve analysis yielded decent sensitivity and specificity in discriminating among the three patients groups (aMCI and AD from SCD, aMCI from SCD, AD from SCD and aMCI). In particular, the Inbrain CST was quite sensitive in discriminating patients with aMCI from those with SCD, suggesting its effectiveness for screening at the early stage of neurodegenerative diseases. 3) Inbrain CST showed significant correlations with widely used conventional neuropsychological tests, demonstrating high validity. The test–retest reliability analysis showed a moderately significant ICC in all individual subtests except the VST and K-TMT-E. 4) In the cortical thickness analysis, each cognitive domain score and the total score of Inbrain CST showed quite distinct patterns of anatomical correlates that fit into previously known brain–behavior relationships.

Although an increasing number of computerized and touchscreen-based cognitive screening tests is becoming available, Inbrain CST has several advantages. First, as a screening test, Inbrain CST can provide both a total score and 5 cognitive domain scores. The total score of Inbrain CST represents individuals' global cognitive function, and the domain scores help differentiate between the amnestic and the non-amnestic types among patients with MCI. The MoCA, one of the most commonly used cognitive screening tests, is also composed of sub-test items related to multiple cognitive domains and known to be useful for detecting MCI.34 However, MoCA is composed of very short and brief items of various cognitive tests that may not be sensitive enough for in-depth evaluation of each cognitive domain. In contrast, Inbrain CST employs neuropsychological tests that are known to represent each cognitive domain. Furthermore, Inbrain CST's cognitive domain scores and total score can provide more accurate information about the participant's actual cognitive characteristics because they are composite scores obtained through multi factor analysis, rather than simple summation scores of test items. In addition, Inbrain CST can also provide standard scores of a patient's current cognitive state, based on the standardized data from 480 cognitively normal adults. Second, the cognitive domain scores can specify what type of cognitive training would most benefit a patient, offering useful information about a patient's cognitive profile. Third, the face validity of Inbrain CST was high because the individual subtests were adapted from conventional neuropsychological tests already used widely in clinical settings. Furthermore, the memory test of Inbrain CST focuses particularly on associative memory by binding verbal items with their location. Neuroanatomically, associative memory is known to be related to the hippocampus proper and medial temporal lobes,3031 and binding memory tests that evaluate associative memory can be particularly sensitive in detecting memory decline in preclinical AD.3233343536 Therefore, Inbrain CST might be a useful screening tool for patients in the early stages of neurodegenerative disease, such as the preclinical or prodromal stage of AD. Fourth, Inbrain CST is easily accessible and convenient to use because it runs on the Windows operating system. Also, because both oral and written instructions are provided, patients with auditory disabilities can be tested as well. Self-administration could be possible in individuals comfortable with operating touchscreen devices, but even our participants who had little experience with a touchscreen tablet needed minimal supervision. Furthermore, Inbrain CST is time-efficient, taking less time to perform than other comprehensive neuropsychological batteries while providing rich information about a patient's cognitive profile.

As hypothesized, the cortical thickness analyses provided neuroanatomical validation of Inbrain CST. The language domain score showed relationships predominantly with the left temporal and parietal regions, and the visuospatial domain score was correlated with a very small area of the right temporal cortex, confirming material specificity. The memory domain score showed positive correlations with cortical thinning in brain regions that mediate memory functions, such as interior temporal regions that involve the parahippocampal gyrus.373839 These areas also showed a strong correlation with the total score, suggesting that memory scores could have a greater effect on the total score than the other cognitive domain scores. The VST (attention) and K-TMT-E (executive function) tests showed sporadic correlations with multiple brain areas, suggesting that these tests need not only attention and executive function but also more various kinds of cognitive functions, such as visuospatial searching and manual performance, because they require much interaction with the touchscreen device.

The VST showed lower validity and reliability than the other individual subtests. It specifically showed a low correlation with the Digit Span test, although both are designed to measure attentional capacity. This could be due to a different type of material modality; the VST of Inbrain CST uses visual stimuli, whereas the Digit Span test uses auditory stimuli. Moreover, the ICC coefficient for the test–retest reliability of the VST-backward test was not significant. Because the VST is one of the early assignments in the Inbrain CST, participants unskilled with the touchscreen device might have had difficulty in performing it. Some studies reported challenges when using digital devices for cognitive tests, suggesting that older adults could be at a disadvantage due to limited personal experience with digital devices and age-related physical changes such as dry skin or arthritis that could potentially affect hand use.4041 The test–retest reliability of K-TMT-E was also not significant, which can be understood partially as a result of the limited technology experience of the participants.

This study has several limitations. First, although Inbrain CST was standardized with cognitively normal older adults, those normative data should be expanded into diverse groups, including people with little education and illiterate people. Second, Inbrain CST might still have some barriers to use, such as variability in patients' prior experience with digital devices, low familiarity with a touchscreen, and the possibility that data could be obstructed by software failure or network problems. Because of these limitations, self-administration of Inbrain CST might be inapplicable to those with low technology literacy. Therefore, further work is necessary to make Inbrain CST more customized to those who are illiterate or those who have low familiarity with digital technology. Finally, we cannot completely exclude the possibility that our results on the correlation between cortical thickness and scores of Inbrain CST may represent brain regions associated with AD rather than brain regions associated with specific cognitive function. Therefore, future study is warranted to perform the same analysis involving not only a larger sample of normal individuals but also a variety of neurodegenerative disorders including Alzheimer's dementia.