The Cognitive Telephone Screening Instrument (COGTEL): A Brief, Reliable, and Valid Tool for Capturing Interindividual Differences in Cognitive Functioning in Epidemiological and Aging Studies.

AIMS: The present study set out to evaluate the psychometric properties of the Cognitive Telephone Screening Instrument (COGTEL) in 2 different samples of older adults.
METHODS: We assessed COGTEL in 116 older adults, with retest after 7 days to evaluate the test-retest reliability. Moreover, we assessed COGTEL in 868 older adults to evaluate convergent validity to the Mini-Mental State Examination (MMSE).
RESULTS: Test-retest reliability of the COGTEL total score was good at 0.85 (p < 0.001). Latent variable analyses revealed that COGTEL and MMSE correlated by 0.93 (p < 0.001), indicating convergent validity of the COGTEL.
CONCLUSION: The present analyses suggest COGTEL as a brief, reliable, and valid instrument for capturing interindividual differences in cognitive functioning in epidemiological and aging studies, with the advantage of covering more cognitive domains than traditional screening tools such as the MMSE, as well as differentiating between individual performance levels, in healthy older adults.

Introduction

It is a major target of many epidemiological and aging studies to better understand the pathways through which interindividual differences in vulnerability to cognitive decline emerge, which has relevance for the health and well-being of older adults [1, 2]. However, study protocols in most studies often only allow the application of a brief measure to screen for cognitive functioning. That is why only the Mini-Mental State Examination (MMSE [3]) has been included in many epidemiological and aging studies [4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14]. However, focusing only on the MMSE may present a limitation for studies addressing healthy aging in cognitive functioning, since within the healthy range of cognitive functioning the MMSE cannot differentiate well between individual performance levels, leading to ceiling effects, which makes it difficult to evaluate interindividual differences in cognitive functioning [15, 16].

The Cognitive Telephone Screening Instrument (COGTEL) is a test battery that may be a useful alternative to the MMSE as it allows the detailed assessment of performance in 6 cognitive domains (prospective, short-term, long-term, and working memory, verbal fluency, and inductive reasoning – using tests adopted from well-established neuropsychological instruments such as the Wechsler scales), including an additional total score that is indicative of overall cognitive functioning [17]. Importantly, the COGTEL can be widely applied since it differentiates between individuals on the full performance range of adult cognitive functioning. It is not limited to cognitive impairments but is also indicative of interindividual differences in cognitive functioning within the healthy performance range [17, 18]. Moreover, the COGTEL is flexible in its administration since it can be applied in a face-to-face session as well as over the phone, and takes only about 10–15 min. With these advantages, the COGTEL has been optimized to be a detailed measure of interindividual differences in cognitive functioning that can be included in epidemiological and aging studies [2, 17, 18, 19].

In terms of psychometric properties, the validity of phone-based versus face-to-face application of the COGTEL subtests has been demonstrated [17]. However, to date, an evaluation of the reliability and validity of the COGTEL to external measures of global cognitive functioning (such as the MMSE) is missing. This would be important in order to investigate the suitability of the COGTEL as an instrument for capturing interindividual differences in cognitive functioning in epidemiological and aging studies. Therefore, the present study set out to evaluate the psychometric properties of the COGTEL in terms of its test-retest reliability and convergent validity to the MMSE in 2 different samples of older adults.

Methods

Participants

We analyzed data from 2 different (nonoverlapping) samples of older adults. First, we evaluated the test-retest reliability of the COGTEL in a sample of 116 community-dwelling older adults from Apuí, Fonte Boa, Manaus, and Tonantis, Brazil (45 men and 71 women). The mean age was 68.3 years (SD = 7.1, range: 60–88). The interval between test and retest was 7 days. Second, we evaluated the convergent validity of the COGTEL to the MMSE in a sample of 868 community-dwelling older adults from the same cities (361 men and 507 women). The mean age was 70.1 years (SD = 6.8, range: 57–92). In both samples, the participants were volunteers recruited via advertisements distributed through newspapers, local radio, churches, and senior centers. As we explicitly aimed to take the full range of interindividual differences available in the cognitive status of participants into account, no inclusion/exclusion criteria based on MMSE scores were applied. All participants gave informed consent. The present study was conducted in accordance with the Declaration of Helsinki and was approved by the local ethics commission.

Materials

Cognitive Telephone Screening Instrument

We used the COGTEL to assess cognitive functioning. The COGTEL consists of 6 subtests covering prospective memory (0 or 1 point), verbal short- and long-term memory (0–8 points each), working memory (0–12 points), verbal fluency (0 to unlimited; as many words as the participant can name within 1 min), and inductive reasoning (0–8 points; see Kliegel et al. [17] for a more detailed description). The scores of the 6 subtests can be analyzed individually or combined into a weighted total score (7.2 × prospective memory + 1.0 × verbal short-term memory + 0.9 × verbal long-term memory + 0.8 × working memory + 0.2 × verbal fluency + 1.7 × inductive reasoning score) [17].

Mini-Mental State Examination

We used the MMSE [3] as an indicator of the amount of cognitive impairment. The MMSE consists of 5 subsections covering orientation (0–10 points), immediate and delayed free recall (0–3 points each), counting backwards (0–5 points), and language (0–9 points). A total score can be derived by adding the 5 subsection scores.

Procedure

Participants were individually tested in face-to-face sessions by study personnel specifically trained in the application of the COGTEL and the MMSE.

Statistical Analyses

We evaluated the reliability and validity of the COGTEL as follows. First, we inspected the test-retest reliability of the COGTEL total score (as well as that of the 6 COGTEL subtests separately) in the sample of 116 older adults. Second, we evaluated the convergent validity by inspecting the relation of the COGTEL total score to the MMSE total score in the sample of 868 older adults. Third, using latent variable analysis, we investigated this relation in the latter sample in more detail. Specifically, we modeled a latent COGTEL variable based on the 6 COGTEL subtests as indicators and a latent MMSE variable based on the 5 MMSE subsections as indicators and investigated the correlation between the latent COGTEL and the latent MMSE variable.

Results

Descriptive Statistics

Means and standard deviations in the sample of 116 participants for the 6 COGTEL subtests and the COGTEL total score are displayed in Table 1. Means and standard deviations in the sample of 868 participants for the 6 COGTEL subtests and the COGTEL total score, as well as for the 5 MMSE subsections and the MMSE total score, are displayed in Table 2.

Table 1

Descriptive statistics and test-retest reliability in the sample of 116 older adults

Variable	Test		Retest		Reliability
	mean	SD	mean	SD	r
COGTEL total score	13.56	7.51	15.28	8.95	0.85***
Prospective memory	0.25	0.43	0.26	0.44	0.75***
Short-term memory	2.91	1.64	3.68	2.10	0.71***
Long-term memory	3.32	2.08	3.80	2.43	0.77***
Working memory	2.48	1.94	2.67	2.06	0.81***
Verbal fluency	11.95	8.24	13.09	8.64	0.90***
Reasoning	0.87	1.23	0.91	1.23	0.60***

Means and standard deviations as well as test-retest reliability for Cognitive Telephone Screening Instrument (COGTEL) total score and the 6 COGTEL subtests (raw scores) in the sample of 116 older adults.

p < 0.001.

Table 2

Descriptive statistics in the sample of 868 older adults

Variable	Mean	SD
COGTEL
Prospective memory	0.28	0.45
Short-term memory	3.74	2.03
Long-term memory	3.97	2.36
Working memory	3.15	2.61
Verbal fluency	12.71	7.92
Reasoning	1.87	1.82
COGTEL total score	17.59	9.59
MMSE
Orientation	8.27	1.90
Immediate recall	2.94	0.29
Delayed recall	2.10	1.03
Counting backwards	2.86	2.16
Language	7.45	1.23
MMSE total score	23.62	4.58

Means and standard deviations for the 6 Cognitive Telephone Screening Instrument (COGTEL) subtests (raw scores) and COGTEL total score as well as for the 5 Mini-Mental State Examination (MMSE) subsections (raw scores) and MMSE total score in the sample of 868 older adults.

Test-Retest Reliability of the COGTEL Instrument

We observed good test-retest reliability for the COGTEL total score. For the 6 COGTEL subtests, the test-retest reliability ranged from acceptable to good (see Table 1).

Convergent Validity of the COGTEL Instrument

There was a substantial correlation of the COGTEL total score with the MMSE total score (r = 0.65, p < 0.001). To evaluate this relation in more detail, we investigated the correlation between a latent COGTEL and a latent MMSE variable. First, we applied a model in which the latent COGTEL variable and the latent MMSE variable were allowed to correlate freely (see Fig. 1). This model provided a good account of the data (χ = 22.14, df = 24, p = 0.571, comparative fit index >0.99, incremental fit index >0.99, root mean square error of approximation <0.001, standardized root mean square residual = 0.02). In this model, the latent COGTEL and MMSE variables were significantly correlated (r = 0.93, p < 0.001).

Fig. 1.

Correlation between the latent Cognitive Telephone Screening Instrument (COGTEL) and the latent Mini-Mental State Examination (MMSE) variables. All coefficients are standardized. *** p < 0.001.

In a next step, we contrasted this model with 2 alternative models and statistically tested the change in model fit (Δχ2 and its significance). First, we applied an alternative model in which the correlation between the latent COGTEL and MMSE variables was constrained to be equal to zero, therefore representing 2 completely independent variables. Compared to the initial model, this alternative model provided a significantly worse model fit (Δχ2 = 601.71, Δdf = 1, p < 0.001), indicating that in the initial model the latent COGTEL and MMSE variables shared a substantial part of their interindividual variance.

Second, we applied an alternative model in which the correlation between the latent COGTEL and MMSE variables was constrained to be equal to 1, therefore sharing all of their interindividual variance. Compared to the initial model, this alternative model provided a significantly worse model fit (Δχ2 = 4.06, Δdf = 1, p = 0.044), indicating that in the initial model there was a substantial part of interindividual variance that the latent COGTEL and MMSE variables did not have in common.

Discussion

The present study set out to evaluate the reliability and validity of the COGTEL in 2 different samples of older adults. First, the test-retest reliability was of good size for the COGTEL total score and ranged from acceptable to good size for the 6 COGTEL subtests. Thus, the observed COGTEL test-retest reliability was comparable to that of the adopted Wechsler scales (ranging from 0.38 to 0.87, e.g., as reported for young and middle-aged adults [20, 21]). The test-retest reliability of the COGTEL total score was also within the range of the test-retest reliability of the MMSE total score (commonly between 0.80 and 0.95 [22]). This indicates that the COGTEL total score and the COGTEL subtests can be reliably used for the assessment of cognitive functioning.

Second, we observed a substantial correlation of the COGTEL total score with the MMSE total score, indicating convergent validity of the COGTEL instrument. Specifically, both instruments shared about half of their interindividual variance, with the rest being residual variance. The latter suggests that, compared to the MMSE, the COGTEL may provide additional information on interindividual differences in cognitive functioning.

Third, we evaluated this issue in latent variable analyses in more detail. Model comparisons indicated that COGTEL and MMSE share a substantial part of interindividual variance (again indicating convergent validity of the COGTEL instrument at a latent level), but that there is a substantial part of interindividual variance that COGTEL and MMSE do not have in common. The latter may be due to the fact that the COGTEL allows a more fine-grained differentiation between individuals within the healthy range of functioning in a broad set of cognitive domains [17, 18]. This may be because the COGTEL subtests place a much greater demand on memory and executive control [17], for which a large performance range can be captured in healthy aging using COGTEL [2, 18, 19]. For example, the COGTEL requires a greater amount of information that needs to be retained in short- and long-term memory (recall of 8 word pairs, of which 4 are semantically unrelated, in COGTEL vs. recall of 3 words in MMSE), it places a greater load on working memory (recall 12 series of up to 7 digits in reverse order in COGTEL vs. count 5 times backwards in MMSE), and it involves a much larger demand on executive control in the applied language tests (generating words according to prespecified rules in COGTEL vs. basic evaluation of understanding, reading, and writing in MMSE) [17, 18]. Besides that, the COGTEL assesses 2 abilities that are not covered in the MMSE, namely, prospective memory (remembering to say one's year of birth later when encountering the verbal fluency test) and inductive reasoning (completing 8 number sequences according to a mathematical rule that the participant needs to detect him-/herself), which place a great demand on executive control [17, 23, 24]. This further underlines that the COGTEL can add valuable information regarding interindividual differences in cognitive functioning, which is the key outcome variable in many epidemiological and aging studies with the aim of better understanding the pathways through which interindividual differences in vulnerability to cognitive decline emerge. Thereby, particularly in studies addressing healthy aging in cognitive functioning, using the COGTEL could overcome the limitation of using only the MMSE, which cannot differentiate well between individual performance levels in healthy aging and leads to ceiling effects [15, 16].

Thus, given that the COGTEL differentiates between individuals on the full performance range of functioning in a broad set of cognitive domains [17, 18], and given its quick and flexible assessment (in a face-to-face session or alternatively over the phone), we propose the COGTEL as a brief, reliable, and valid instrument for capturing interindividual differences in cognitive functioning in epidemiological and aging studies.

Disclosure Statement

The authors declare that there is no conflict of interest.