Visual-spatial perception: a comparison between instruments frequently used in the primary care setting and a computerized cognitive assessment battery.

BACKGROUND: The development of screening instruments will help the primary care team to determine when further comprehensive cognitive assessment is necessary.
DESIGN: A retrospective analysis based on medical records. PATIENTS AND
SETTING: Patients referred to a comprehensive geriatric assessment unit. ANALYSIS: Cognitive screening and assessment included visual-spatial components: the Mini Mental State Examination, the Clock Drawing Test, the Montreal Cognitive Assessment Test, and the Neurotrax (Mindstreams) computerized cognitive assessment battery.
RESULTS: The average age of the 190 eligible patients was 81.09±5.42 years. Comparing the individual tests with that of the visual-spatial index of Neurotrax, we found the Trail Making B test to be most sensitive (72.4%) and the Cube Test to have the highest specificity (72.8%). A combination of tests resulted in higher sensitivity and lower specificity.
CONCLUSION: The use of a combination of visual-spatial tests for screening in neurocognitive disorders should be evaluated in further prospective studies.

Introduction

Cognitive decline associated with advancing age is one of the most important challenges currently facing health care providers, both in the spheres of diagnosis and management and in the delivery of supportive services to the patient and caregivers.1–3 The spectrum of neurocognitive decline from subjective cognitive complaints to mild cognitive impairment (MCI) and ultimately to a progressive dementia syndrome places the primary care physician at the forefront in providing care for a growing number of cognitively impaired older patients, the majority of whom reside in the community. While systematic population screening for cognitive impairment has not been shown to be effective,4,5 the assessment of patients who are referred to the clinic for cognitive symptoms and decreased cognitive function is recommended.6–8

MCI is a clinical syndrome that has been clearly defined, both in the clinical situation and in research settings.7,9,10 MCI is prevalent in the older population, with some 15% of those older than 65 years having this condition, with a range of 3%–42% reported in community-based studies.11–13 The presence of predominant memory complaints and impairment alongside the preservation of function in instrumental daily activities suggests amnestic MCI, and this is generally considered to represent an early stage of Alzheimer’s disease.9,10,14 Approximately 10% of patients with MCI will progress to dementia annually.15,16

In a longitudinal archival study,17 it was shown that changes in visual-spatial perception sometimes appear in the preclinical phase of dementia as early as 3 years before the diagnosis of dementia, while memory decline occurs some 2 years before diagnosis.

The purpose of this study was to examine whether the use of visual-spatial perception testing (as a component of existing screening tools for the early detection of cognitive impairment) is of value for the determination of cognitive impairment, as compared to the visual-spatial component of a more comprehensive computerized neurocognitive battery used for cognitive assessment.

Methods

We performed a retrospective analysis based on medical records from the years 2005 to 2010 of patients seen at the comprehensive geriatric assessment (CGA) unit of the Clalit Health Services in Beer Sheva, Israel. The multidisciplinary staff of the CGA unit include specialist geriatricians, geriatric nurses, a dietician, an occupational therapist, a physical therapist, and a social worker.

The study population included patients older than 65 years referred by the primary care physician for geriatric assessment. The assessment battery included the following components: 1) Demographics, including age, gender, and education level. 2) History and examination, with emphasis on medical history and physical examination, drug therapy, laboratory and other investigations (available via the computerized medical record system), psychosocial assessment, and nutritional status. 3) Depression screening by Patient Health Questionnaire.18 4) Basic Activity of Daily Living.19,20 5) Instrumental Activity of Daily Living (the Older Americans Resources and Service Instrumental Activity of Daily Living).21 6) Comorbidity status (Charlson comorbidity index22 and cumulative illness rating scale – geriatric version).23 7) Cognitive screening was performed using the following instruments: a) Mini Mental State Examination, rated on a scale of 0–30;24 b) Clock Drawing Test (CDT) rated on a scale of 0–2;25,26 c) Montreal Cognitive Assessment (MoCA) test rated on a scale of 0–30;27 d) Neurotrax (Mindstreams) computerized cognitive assessment battery, validated for use in the elderly.28–30

The visual-spatial components of each of the aforementioned instruments were compared. A dichotomy score of either normal or abnormal was allocated for each of the tests as follows:

1) Mini Mental State Examination: The intersecting pentagons were rated as normal (score 1) or abnormal (score 0). 2) CDT as normal (score 2) or abnormal (1 or 0). Scores were allocated as follows: 0 – neither numbers nor hands are present, or they have been placed incorrectly; 1 – the hands are present and they have been placed correctly, and numbers are either missing or incorrectly placed; 2 – both hands and numbers are present and have been placed correctly. 3) Cube Drawing Test component of the MoCA test as normal (score 1) or abnormal (score 0). 4) Trail Making B test component of MoCA test as normal, where all points (letters and numbers) are connected in the correct order (score 1), or at least one point is missed or not in the correct sequence (score 0). 5) Visual-spatial index of the Neurotrax (Mindstreams) computerized cognitive assessment battery (a score of ≤85, representing one standard deviation [SD] below normal was scored as 0, and >85 as 1).

A diagnosis of dementia was made according to DSM-IV criteria31 and of MCI according to consensus criteria for MCI.7,9,10

The study was approved by the Ethics Committee of Clalit Health Services at the Meir Medical Center.

Statistical analysis

We calculated sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for each of the measures for determining cognitive impairment.

Sensitivity = true positive/gold standard positive

Specificity = true negative/gold standard negative

PPV = true positive/test outcome positive

NPV = true negative/test outcome negative

Using the visual-spatial index of the Neurotrax (Mind-streams) assessment battery as the reference,28–30 the other tests were compared both individually and in varied combinations, in order to determine sensitivity and specificity, PPV, and NPV. Chi-square analysis was performed with P<0.05 considered to be statistically significant.

Results

Study population

A total of 190 patients assessed at the CGA unit over the years 2005–2010 with data available for all the study instruments were eligible for inclusion in the study (Table 1).

Table 1

Characteristics of study participations (N=190)

Baseline variables	N	%
Sex
Male	79	41.6
Female	111	58.4
Age (years)
Range	67–94
Mean ± SD	81.09±5.42
Cognitive diagnosis
Normal	62	32.6
MCI	90	47.4
Dementia, total	38	20
Alzheimer type	27	71
Vascular type	9	23.8
Lewy body type	1	2.6
Others	1	2.6
Years of education
Range	0–25
Mean ± SD	13.83±10.02
Depression screening (by PHQ-9)
Total	127 (63 missing data)
Range	1–27
Mean ± SD	12.9±6.6
Basic Activity of Daily Living (BADL)
Total	154 (36 missing data)
Range	35–100
Mean ± SD	92.06±11.89
Instrumental Activity of Daily Living (IADL)
Total	146 (44 missing data)
Range	0–14
Mean ± SD	9.60±3.55
Charlson combined score (with age)
Range	3–11
Mean ± SD	5.28±1.65
Total CIRS-G score
Range	4–19
Mean ± SD	10.91±4.66
Severity index (total CIRS-G score/total number of categories endorsed)
Range	1.3–2.6
Mean ± SD	1.99±0.27

Abbreviations: SD, standard deviation; MCI, mild cognitive impairment; PH Q-9, Patient Health Questionnaire-9; CIRS-G, cumulative illness rating scale – geriatric version.

The average age and SD was 81.09±5.42 years (range: 67–94), and 111 (58.4%) were females. The mean education level was 13.83 years (SD ±10.02). Based on clinical and cognitive assessment, 62 (32.6%) of the patients were determined to be cognitively normal, 90 (47.4%) were diagnosed as having MCI, and 38 (20%) were diagnosed as having dementia. Of those with dementia, 27 (71%) patients were diagnosed with Alzheimer’s disease, nine (23.8%) with vascular dementia, one (2.6%) with diffuse Lewy body dementia, and one (2.6%) with other dementia.

Cognitive evaluations

The results of the cognitive assessments are presented in Table 2.

Table 2

Selected outcomes cognitive assessment of study population

Cognitive assessment	N	%
Neurotrax
Global score
Normal	83	43.7
Abnormal	107	56.3
Mean ± SD	84.75±12.34
Range	55.9–112.1
Visual-spatial index
Normal	92	48.4
Abnormal	98	51.6
Mean ± SD	87.43±13.07
Range	62.5–128.4
Cube Test
Normal	126	66.3
Abnormal	64	33.7
Trail Making B test
Normal	80	42.1
Abnormal	110	57.9
Pentagon Copying
Normal	73	38.4
Abnormal	117	61.6
Clock Drawing Test
Normal	98	51.6
Abnormal	92	48.4

Abbreviation: SD, standard deviation.

With regard to the Neurotrax computerized cognitive assessment, a total of 107 (56.3%) subjects scored below one SD from the normal (<85%) on the global score index. For the visual-spatial index, 98 subjects (51.6%) scored below one SD.

With regard to abnormal results for the other cognitive tests, the Cube Test was abnormal in 64 subjects (33.7%), the Trail Making B test in 110 (57.9%), the Intersecting Pentagon Copying (IPC) test in 117 (61.6%) subjects, and the CDT was abnormal in 92 (48.4%).

Comparing the individual cognitive tests with that of the visual-spatial index of Neurotrax, we found the Trail Making B test to be most sensitive (72.4%) and the Cube Test to have the highest specificity (72.8%) (Table 3).

Table 3

Comparison between single tools and visual score of Neurotrax test for evaluation of visual-spatial ability

Type of tests	Neurotrax visual-spatial index					Sensitivity(%)	Specificity (%)	PPV (%)	NPV (%)
	Abnormal (≤85), N=98		Normal (>85), N=92		P-value
	N	%	N	%
Cube Test
Abnormal	39	39.8	25	27.2	0.091	39.8	72.8	60.9	53.2
Normal	59	60.2	67	72.8
Trail Making B test
Abnormal	71	72.4	39	42.4	<0.0001	72.4	57.6	64.5	66.3
Normal	27	27.6	53	57.6
Pentagon Copying
Abnormal	67	68.4	50	54.3	0.053	68.4	45.7	57.3	57.5
Normal	31	31.6	42	45.7
Clock Drawing Test
Abnormal	64	65.3	34	37.0	<0.0001	65.3	63.0	65.3	63.0
Normal	34	34.7	58	63.0

Abbreviations: PPV, positive predictive value; NPV, negative predictive value.

The effect of combining two different tests, Trail Making B and IPC, was a higher sensitivity of 87.8%. The highest specificity of combined tests was for Trail Making B test and the Cube Test (54.3%) (Table 4).

Table 4

Comparison between two integrated single tools and visual score of Neurotrax test for evaluation of visual-spatial ability

Combinations of tests	Neurotrax visual-spatial index					Sensitivity(%)	Specificity (%)	PPV (%)	NPV (%)
	Abnormal (≤85), N=98		Normal (≤85), N=92		P-value
	N	%	N	%
Trail B + Cube Test
Abnormal	73	74.5	42	45.7	<0.0001	74.5	54.3	63.5	66.7
Normal	25	25.5	50	54.3
CDT + Cube Test
Abnormal	72	73.5	43	46.7	<0.0001	73.5	53.3	62.6	65.3
Normal	26	26.5	49	53.3
CDT + Trail B
Abnormal	83	84.7	51	55.4	<0.0001	84.7	44.6	61.9	73.2
Normal	15	15.3	41	44.6
Trail B + Pentagon Copying
Abnormal	86	87.8	61	66.3	<0.0001	87.8	33.7	58.5	72.1
Normal	12	12.2	31	33.7
CDT + Pentagon Copying
Abnormal	81	82.7	59	64.1	<0.001	82.7	35.9	57.9	66.0
Normal	17	17.3	33	35.9
Cube + Pentagon Copying
Abnormal	74	75.5	52	56.5	<0.001	75.5	43.5	58.7	62.5
Normal	24	24.5	40	43.5

Abbreviations: PPV, positive predictive value; NPV, negative predictive value; CDT, Clock Drawing Test.

When looking at the combined effect of three tests compared to the visual-spatial index, the highest sensitivity was 88.8% for a combination of Trail Making B test, Cube Test, and IPC. This combination also had the highest specificity of 33.7% compared to other multiple combinations of tests. When looking at the combination of all four tests, the sensitivity was 93.9%, and specificity was 27.2% (Table 5).

Table 5

Comparison between three and four integrated tools and visual score of Neurotrax test for evaluation of visual-spatial ability

Type of tests	Neurotrax visual-spatial index					Sensitivity (%)	Specificity (%)	PPV (%)	NPV (%)
	Abnormal (≤85), N=98		Normal (≤85), N=98		P-value
	N	%	N	%
Cube + Pentagon + Trail B
Abnormal	87	88.8	61	66.3	<0.0001	88.8	33.7	58.8	73.8
Normal	11	11.2	31	33.7
CDT + Cube + Pentagon	+ Trail B
Abnormal	92	93.9	67	72.8	<0.0001	93.9	27.2	57.9	80.6
Normal	6	6.1	25	27.2

Abbreviations: PPV, positive predictive value; NPV, negative predictive value; CDT, Clock Drawing Test.

Discussion

Our results show that tests of visual-spatial function are useful for screening patients for impaired cognitive function. While most screening instruments currently in use include an evaluation of visual-spatial function as one component of the test,24–29 no screening tool is based solely on this aspect of cognitive function. The high sensitivity found for the combination of all four tests that were administered was associated with a low specificity. Our results suggest that when at least one of the tests is abnormal, further cognitive assessment is indicated.

The high sensitivity of the combined tests may eliminate the need for further cognitive screening. The time required to administer these four tests varies from 3 to 5 minutes, which is certainly acceptable for screening. These tests are all simple paper-based tests that can be easily administered in the clinic or home setting by nonprofessional staff who have received the necessary training (Table 5).

To the best of our knowledge, this is the first such study to evaluate the combined value of screening tests for visual-spatial function in cognitive assessment. The use in this study of the Neurotrax visual-spatial index based on the Block Design of the Wechsler Adult Intelligence Scale-332 as the reference test, as well as the use of a cognitive diagnosis based on a CGA by a multidisciplinary team, strengthens the value of our findings.

Our study has a number of limitations. We performed a retrospective analysis based on the charts of patients referred to a single CGA unit in the southern district of Israel. Our patients spoke either Hebrew or Russian as their primary language and were all of Jewish ethnicity. Thus, the generalizability of our results is limited. The choice of a computerized cognitive assessment battery as the gold standard rather than formal neuropsychological assessment is a possible limitation. However, the Neurotrax battery has been well validated in numerous studies.28–30 We realize that many of the tests that we used are not limited to visual-spatial function but that they assess other aspects of cognitive function as well, particularly executive function. The overlap with other cognitive functions may clearly have a confounding effect on our results.

We also acknowledge that most conditions causing the dementia syndrome present primarily with memory, language, and executive function deficits, at least initially, and that impaired visual-spatial function in itself does not constitute one of the usual diagnostic criteria (Table 6).

Table 6

Sensitivity and specificity of individual tests compared with clinical diagnosis

Type of tests	Diagnosis					Sensitivity (%)	Specificity (%)	PPV (%)	NPV (%)
	MCI/dementia (N=128)		Normal (N=62)		P-value
	N	%	N	%
Cube Test
Abnormal	49	38.3	15	24.2	0.07	38.3	75.8	76.6	37.3
Normal	79	61.7	47	75.8
	128		62
Trail B test
Abnormal	80	62.5	30	48.4	0.08	62.5	51.6	72.7	40.0
Normal	48	37.5	32	51.6
	128		62
Pentagon
Abnormal	79	61.7	38	61.3	1.0	61.7	38.7	67.5	32.9
Normal	49	38.3	24	38.7
	128		62
CDT total
Abnormal	74	57.8	24	38.7	0.02	57.8	61.3	75.5	41.3
Normal	54	42.2	38	61.3
	128		62
Neurotrax exam total score
Abnormal	84	65.6	23	37.1	<0.01	65.6	62.9	78.5	47.0
Normal	44	34.4	39	62.9
	128		62

Abbreviations: PPV, positive predictive value; NPV, negative predictive value; MCI, mild cognitive impairment; CDT, Clock Drawing Test.

Conclusion

The combination of four short, paper-based tests, namely the Trail Making B test, the Cube Test, the IPC test, and the CDT, was found to be highly sensitive (93.9%) for identifying subjects with cognitive impairment, when compared to the Neurotrax (Mindstreams) computerized cognitive assessment battery. The use of a combination of visual-spatial tests for screening in neurocognitive disorders should be evaluated in further prospective studies.