Accuracy of the Cognitive Assessment Battery in a Primary Care Population.

BACKGROUND: There are several cognitive assessment tools used in primary care, e.g., the Mini-Mental State Examination (MMSE) and the Montreal Cognitive Assessment. The Cognitive Assessment Battery (CAB) was introduced as a sensitive tool to detect cognitive decline in primary care. However, primary care validation is lacking. Therefore, we investigated the accuracy of the CAB in a primary care population.
OBJECTIVE: To investigate the accuracy of the CAB in a primary care population.
METHODS: Data from 46 individuals with cognitive impairment and 33 individuals who visited the primary care with somatic noncognitive symptoms were analyzed. They were investigated with the MMSE, the CAB, and a battery of neuropsychological tests; they also underwent consultation with a geriatric specialist. The accuracy of the CAB was assessed using c-statistics and the area under the receiver operating characteristic curve (AUC) was used to quantify the binary outcomes ("no cognitive impairment" or "cognitive impairment").
RESULTS: The "cognitive impairment" group was significantly different from the unimpaired group for all the subtests of the CAB. When accuracy was based on binary significant reduction or not in one or several domains of the CAB, the AUC varied between 0.685 and 0.772. However, when a summation or logistic regression of several subcategories was performed, using the numerical values for each subcategory, the AUC was >0.9. For comparison, the AUC for the MMSE was 0.849.
CONCLUSIONS: The accuracy of the CAB in a primary care population is poor to good when using binary cutoffs. Accuracy can be improved to high when using a summation or logistic regression of the numerical data of the subcategories. Considering CAB time, lack of adequate age norms, and a good accuracy for the MMSE, implementation of the CAB in primary care is not recommended at present based on the results of this study.

Introduction

Mental disorders, including dementia, are prevalent conditions in primary care populations, and primary care is perhaps the most important care setting for their detection [1]. Despite great interest and attempts to find medications that can delay or prevent dementia, there has been little success [2, 3]. Complex social and medical care situations often create burden throughout the disease process, not only for patients and family, but also for primary care [4]. The diagnosis of dementia is described by general practitioners as a difficult task requiring careful medical and social considerations, accurate diagnostic instruments are needed, and interaction with patients and relatives affects the timing to diagnosis. Cognitive testing is an important part of clinical dementia investigation, apart from case history, physical examination, and neuroimaging [5]. In primary care, where large numbers of patients are attending, the tools used for cognitive testing must be both accurate (in their detection of cognitive decline) and practical regarding time for investigation and reliability between different investigators.

The Mini-Mental State Examination (MMSE) has been widely used and is recommended in guidelines as an instrument for cognitive evaluation in the first, basic investigation when dementia is suspected [6, 7]. In recent years, additional cognitive tools for the detection of cognitive deterioration have been introduced: a quick test for cognitive function (AQT) [8, 9], the Neurobehavioral Cognitive Status Examination [10, 11], the Montreal Cognitive Assessment (MoCA) [12, 13], and different short versions of the MoCA [14]. Even though more complex neuropsychological assessment has been used in the investigation of cognitive function in the elderly [15], those resources are seldom available in primary care. The Cognitive Assessment Battery (CAB) has been another tool suggested for use to detect cognitive decline, after validation on a specialist-clinic population [16]. The CAB was also created to detect minor declines in cognitive function and is therefore constructed to cover several cognitive domains in more depth than the aforementioned tests. The subtests of CAB cover learning and episodic memory (delayed recall of text), speed and attention (Symbol Digit Modalities Test), visuospatial function (Clock Drawing Test and cube drawing), language (parts of the Token Test and naming 30 items), and executive function (Stroop) [16]. The CAB was concluded to be a cognitive test tool that would be positioned between tests such as the MMSE and a full neuropsychological evaluation and which could also be used in primary care [16]. Diagnostics in the earlier stage of dementia and mild cognitive impairment help clarify the need for more extensive cognitive tests adapted for primary care to evaluate patients and possible disease progression [17]. In contrast to, for example, the MMSE, AQT, and MoCA, the CAB usually takes around 30 min to perform in patients with relatively intact cognitive function.

This study aimed to evaluate the qualities of the CAB in a primary care population. Based on routine clinical cases in four separate primary health care centers, patients with cognitive symptoms as well as patients with good brain health visiting their general practitioner for other symptoms were invited to participate in a dementia investigation study, where the CAB was one of the cognitive tests evaluated. The purpose of the study was to evaluate the accuracy of the CAB in a primary care population.

Subjects and Methods

A total of 81 participants (aged ≥65 years and living at home) were recruited from four primary health care centers in Sweden between December 2007 and May 2009 (Table 1). After the evaluation (see details below), 46 individuals were classified as having “cognitive impairment” and 33 as having “no cognitive impairment” (see Johansson et al. [11]). The four primary care centers served a total of 49,800 people; 11,200 were ≥65 years old.

Table 1

Demographic and cognitive data (MMSE, CABsum) for patients with no cognitive impairment and patients who were cognitively impaired

	No cognitive impairment (n = 33)	Cognitively impaired (n = 46)	p value
Age, years	75 (66–87) [30]	79 (65–88) [26]	0.002a
Education, years	12 (6–16) [15]	9 (5–16) [13]	0.026a
Sex (m/f)	25/8	2/24	0.013a
MMSE	29 (25–30) [2]	26 (15–30) [13]	0.000
Delayed recall	7 (0–13) [11]	2 (0–9) [6]	0.000
SDMT	32 (17–51) [77]	19 (0–38) [88]	0.000
Naming	27 (17–30 [7]	22 (0–29) [38]	0.000
CDT/cube drawing	12 (4–12) [2]	10 (0–12) [8]	0.001
Token Test	6 (3–6) [0.5]	6 (1–6) [2.3]	0.011
Stroop 3	32 (17–85) [148]	54 (25–656) [656]	0.000
CABsumb	86 (54–106) [139]	58 (15–85) [285]	0.000

Data are expressed as median (range) [variance]. CABsum, numerical sum from the CAB subtests; CDT, Clock Drawing Test; MMSE, Mini-Mental State Examination; SDMT, Symbol Digit Modalities Test.

Mann-Whitney U test. Differences between groups for the cognitive tests were analyzed using univariate analysis of variance where the model was corrected for age and education.

Sum of delayed recall, SDMT, naming, CDT/cube drawing, and Token Test.

The exclusion criteria consisted of cerebral infection, brain tumor, ongoing verified psychiatric illness at study start, and stroke or head trauma within the 4 weeks before inclusion. However, patients with a previous psychiatric diagnosis but in a clinically stable condition (determined by the general practitioner) and unmodified antidepressant medication for the last 6 months were included. All 81 patients underwent physical examination by a general practitioner, routine laboratory examinations, and ECG evaluation. Patients with cognitive symptoms also had a CT brain scan performed as part of the clinical workup.

All patients underwent the MMSE and the Clock Drawing Test during the same appointment with an occupational therapist in primary care. The CAB was performed by another occupational therapist (M.M.J.) or a trained nurse at the memory clinic 3–4 weeks later. The CAB was performed blindly without knowledge of whether the patient was cognitively impaired.

The patients then underwent a third stage of more extensive testing in a standardized order by a neuropsychologist using the Alzheimer's Disease Assessment Scale-cognitive memory test, the letter Verbal Fluency Test, immediate and delayed story recall, the Trail Making Test A and B, and language test (WAISS-III in Swedish translation). In addition, the patients were examined by a geriatric specialist (J.M.). A final consensus decision of “no cognitive impairment” or “cognitive impairment” was made by the geriatric specialist and the neuropsychologist using the results of the neuropsychological tests (CAB data excluded), the laboratory results, and the CT scan. “Cognitive impairment” represented a >2 SD decline in one or more domains and thus contained both mild cognitive impairment and dementia cases [18]. The CAB results were also blindly (in absence on final evaluation data) categorized into either “no cognitive impairment” or “cognitive impairment” with >2 SD of the mean from the “no cognitive impairment” group as a cutoff.

Statistics

The descriptive data for the two groups are presented as median (range) values, and nonparametric tests were used for group comparisons. Several variables had data that were not normally distributed and were not homogenous, with large difference in variance between the groups (Table 1).

Overall discrimination between “no cognitive impairment” and “cognitive impairment” was assessed using c-statistics, a measure of goodness of fit for binary outcomes. The area under the receiver operating characteristic curve (AUC) was used to quantify the binary outcomes. The receiver operating characteristic curve continually plots every ideally possible sensitivity versus specificity across all threshold cutoff points. The AUC reflects the accuracy of the prediction and can be compared among the different models. There is a general criterion that an AUC level <0.5–0.7 is of rather low accuracy, between 0.7 and 0.9 it is considered good accuracy, and >0.9 it is considered high accuracy [19]. We also used a logistic binary regression for the whole set of CAB variables and tested the subsequent model in a receiver operating characteristic curve analysis.

We used >2 SD of the mean from “no cognitive impairment” as the cutoff to define cognitive decline [16].

Results

The cognitively impaired group was different from the unimpaired group regarding age and education (Table 1). Differences between the groups for the cognitive tests were analyzed using a univariate analysis of variance where the model was corrected for age and education. With this correction, the cognitively impaired group was still significantly altered in all tests performed.

The accuracy for different subtests was then analyzed with reference to the diagnostic outcome (“no cognitive impairment” or “cognitive impairment”) made by the expert panel. We first tested the accuracy for 1, 2, or 3 subtests with >2 SD decline in cognitive function (Table 2). The accuracy expressed as AUC was 0.685–0.772. We also tested the separate subtests using the numerical score for each test, obtaining AUC values from 0.643 (Token Test) to 0.884 (Stroop 3) (Table 3).

Table 2

Receiver operating characteristics for subtests of significant deficits (−2 SD) in 1, 2, or 3 cognitive domains of the CAB

Domains with significant cognitive deficit	AUC	95% CI
1	0.772	0.661–0.883
2	0.748	0.639–0.857
3	0.685	0.570–0.800

AUC, area under the receiver operating characteristic curve; CAB, Cognitive Assessment Battery.

Table 3

Receiver operating characteristics for subtests of the CAB

Test	AUC	95% CI
Delayed recall	0.859	0.774–0.944
SDMT	0.862	0.784–0.940
Naming	0.837	0.784–0.925
CDT/cube drawing	0.695	0.579–0.811
Token Test	0.643	0.523–0.763
Stroop 3	0.884	0.807–0.960

AUC, area under the receiver operating characteristic curve; CAB, Cognitive Assessment Battery; CDT, Clock Drawing Test; SDMT, Symbol Digit Modalities Test.

We then used the numerical sum from the CAB subtests (designated CABsum) that would result in lower scores in case of deterioration (delayed recall, Symbol Digit Modalities Test, naming, Clock Drawing Test/cube drawing, and Token Test). This resulted in a better AUC accuracy for the CABsum of 0.916 (95% CI 0.857–0.975). For comparison we used the scale data for MMSE with AUC 0.849 (95% CI 0.766–0.931) (Fig. 1). We also analyzed the same CAB variables in a logistic regression model, but the AUC remained largely the same (AUC 0.912, 95% CI 0.851–0.974). The range for the CABsum score was 15–106 out of 179 maximum points (Table 1). When selecting 80% specificity and sensitivity, the cutoff score was 72.

Fig. 1

ROC curve for the sum of data for delayed recall, Symbol Digit Modalities Test, naming, Clock Drawing Test/cube drawing, and Token Test (CABsum) in comparison to the reference line and the MMSE. CABsum AUC = 0.916 (95% CI 0.857–975); MMSE AUC = 0.849 (95% CI 0.766–0.931). AUC, area under the ROC curve; CABsum, numerical sum from the Cognitive Assessment Battery subtests; MMSE, Mini-Mental State Examination; ROC, receiver operating characteristic.

Discussion and Conclusions

Cognitive testing in primary care is an essential part of dementia investigation, as the outcome may provide early indication of whether further investigations or referrals should take place. As described in the introduction, several cognitive test batteries have been suggested for use in the primary care investigation of possible dementia. To our knowledge, this is the first report on the CAB in a primary care population.

The CAB has been presented as a more sensitive test to detect early or slight cognitive decline, but only one study has validated the instrument against dementia and subtypes of cognitive decline [16]. The CAB is followed by an instruction that each tested cognitive domain should be analyzed separately (decline or not), after which a total interpretation of all domains should lead to a statement of whether there is evidence for mild cognitive decline or decline that is consistent with dementia. As reported by Nordlund et al. [16], we used the −2 SD criterion for cognitive impairment as an operational criterion for each category. When testing the accuracy of decline in 1, 2, or 3 cognitive domains, the outcome was around 0.7, i.e., poor to good. We therefore chose to use the sum of numerical data (CABsum) for all the cognitive subtests that would give a lower test value in case of cognitive impairment. We also performed a receiver operating characteristic test for the outcome of logistic binary regression of all variables in the CAB. Both methods resulted in high accuracy levels (AUC >0.9). Since the numerical summation easily transformed into clinical practice, we suggest that this method may be superior to the use of categorical subtest CAB data to categorize brain function as normal or in decline. In this respect, as shown in Figure 1, depending on cutoff values, the sensitivity and specificity can be set to e.g. 80% and the corresponding cutoff value (72) be used to detect possible cognitive decline.

Like the previous CAB paper by Nordlund et al. [16], the present study has some limitations. One is the limited sample size. Despite a long inclusion period, it was difficult to recruit participants, and therefore the study group was small. However, the age distribution (66–88, mean 75 and 79 for the two groups, respectively) of our study is more adjusted to the prevalent age groups for dementia, in comparison to Nordlund et al.'s [16] study with a reported mean of 67 and 64/71. Even with a limited number of samples, we believe that the accuracy of the CAB and the comparison with the accuracy of the MMSE is valid.

Meta-analysis of cognitive screening for mild cognitive impairment has reported an AUC of 0.664 for the MMSE and an AUC of 0.811 for the MoCA [20], which can be compared to the MMSE data here (AUC of 0.849). Differences in sample selection and sample size may significantly contribute to differences in accuracy of the instruments tested, but clearly, using the recommended guidelines for the CAB using separate domains has lower accuracy than using the numerical data for the whole CAB set. However, there is also a risk of losing the qualitative aspect of looking at each domain separately when considering the use of CABsum.

In Sweden, there is an increasing number of individuals who present to primary care with symptoms of stress and exhaustion and who fulfil the diagnostic criteria for exhaustion disorder [21]. These patients can show cognitive impairment in executive tests with speed and working memory and attention tests as well as delayed recall. There is a clinical challenge in the diagnosis between exhaustion disorder and mild cognitive impairment or early dementia, especially in younger patients. In one recent study, the CAB was suggested to be a useful instrument for cognitive screening for exhaustion disorder in primary care [22]. To what extent the CAB has the ability to distinguish between exhaustion disorder and early dementia needs further study.

The time needed for the test is an important factor to consider in primary care. The CAB takes around 20–30 min to perform. The additional value of the CAB in dementia investigations can be questioned when considering not only accuracy, but also the time taken to perform the test. In addition, the lack of large normative data sets for the subtests of CAB is a weakness that must be addressed before the test is introduced in primary care.

Statement of Ethics

This study was conducted according to the provisions of the Helsinki Declaration and was approved by the Ethical Review Board of Linköping University, Sweden (DNR M137-07). Written informed consent was obtained from all study participants.

Disclosure Statement

The authors have no conflicts of interest to declare.

Funding Sources

This study was founded by the “Clinical Research Funds” provided by Linköping University and the County Council of Östergötland.

Author Contributions

A.S. Kvitting: study concept and design, analysis and interpretation of data, drafting/revising the manuscript for intellectual content, study supervision and coordination, obtaining funding. M.M. Johansson: study concept and design, collection of data, analysis and interpretation of data, revising the manuscript, study supervision and coordination. J. Marcusson: study concept and design, medical examination of participants, analysis and interpretation of data, revising the manuscript, study supervision and coordination, obtaining funding. All authors reviewed the final manuscript.