How predictive is the MMSE for cognitive performance after stroke?

Cognitive deficits are commonly observed in stroke patients. Neuropsychological testing is time-consuming and not easy to administer after hospital discharge. Standardised screening measures are desirable. The Mini-Mental State Examination (MMSE) is the test most widely applied to screen for cognitive deficits. Despite its broad use, its predictive characteristics after stroke have not been exhaustively investigated. The aim of this study was to determine whether the MMSE is able to adequately screen for cognitive impairment and dementia after stroke and whether or not the MMSE can predict further deterioration or recovery in cognitive function over time. To this end, we studied 194 first-ever stroke patients without pre-stroke cognitive deterioration who underwent MMSEs and neuropsychological test batteries at 1, 6, 12, and 24 months after stroke. The MMSE score 1 month after stroke predicted cognitive functioning at later follow-up visits. It could not predict deterioration or improvement in cognitive functioning over time. The cut-off score in the screening for 1 cognitive disturbed domain was 27/28 with a sensitivity of 0.72. The cut-off score in the screening for at least 4 impaired domains and dementia were 26/27 and 23/24 with a sensitivity of 0.82 and 0.96, respectively. The results indicated that the MMSE has modest qualities in screening for mild cognitive disturbances and is adequate in screening for moderate cognitive deficits or dementia in stroke patients 1 month after stroke. Poor performance on the MMSE is predictive for cognitive impairment in the long term. However, it cannot be used to predict further cognitive deterioration or improvement over time.

Introduction

Over half of stroke patients suffer from disturbances in cognition such as memory problems and mental slowness. In the acute phase after stroke, these cognitive deficits often remain unrecognised because, at that time, a patient frequently is not fully aware of possible cognitive dysfunction. Furthermore, a thorough neuropsychological evaluation is not routinely administered. After a longer period after stroke, however, cognitive impairment is considerably prevalent [1-3]. These so called ‘invisible’ consequences can have a great negative impact on rehabilitation and outcome [4, 5].

Over the last decade, stroke care has become much more efficient, costs have lowered and hospital stays shortened. Most functional recovery occurs within the first months after stroke, and starting a rehabilitation course in an early stage is beneficial for a patient’s outcome. Also, cognitively impaired elders benefit from admission to acute stroke and rehabilitation units [6, 7]. Usually, patients undergo physical and cognitive screening in an early stage to arrange the most effective rehabilitation program after hospital discharge and to arrange appropriate home adjustments. Therefore, valid screening measurements are warranted, as is screening for cognitive problems later in stroke recovery. At that time most stroke care often is provided by a general practitioner.

The Mini-Mental Stage Examination (MMSE) is the instrument most widely used in screening for cognitive problems in hospitalised patients and in outpatient settings. It comprises thirty items providing information about orientation, attention, learning, calculation, delayed recall, and construction [8]. Several studies report acceptable validity of the MMSE as a screening instrument and its relationship to functional outcome in stroke patients [9-12]. Others conclude that it is not an accurate screening tool for cognitive deficits in stroke patients, as it cannot differentiate between focal and diffuse lesions, it is language-, age-, and education-dependent, and it is insensitive to right-sided lesions [13-15].

Although the value of the MMSE in the screening for cognitive dysfunction in stroke patients is still under debate, it is widely used and it has become a clinical standard. It is brief and easily applicable, and has a low interrater variability [13, 16].

The aims of this study were to see whether the MMSE is sufficiently accurate in screening for mild and moderate cognitive disturbances and to determine whether it can be used in a two-step approach, selecting patients with no cognitive deficits from those would need thorough cognitive testing by a neuropsychological test battery. It is conceivable that patients with a poor score on the MMSE are prone to problems with rehabilitation and further deterioration and so would need closer follow-up. Therefore we studied whether the MMSE in an early phase could predict cognitive performance and cognitive deterioration over time.

Materials and methods

Patients

Between January 2000 and August 2001, 194 consecutive patients admitted with supratentorial stroke to the Neurology Department of the University Hospital Maastricht were enrolled in this study. Study and patient characteristics have been described previously [17]. Patients were asked within 48 h after stroke to participate in the study.

All participants underwent a structured medical and neurological assessment including medical history, physical examination, laboratory studies, and brain CT. CT was performed on the day of admission or the day after. These data were registered in the Maastricht Stroke Register (MSR).

Stroke was diagnosed by a neurologist by clinical evaluation and a brain CT scan. Inclusion criteria were a first hemispheral stroke, age over 40 years, adequate post-stroke fluency in Dutch, and an initial post-stroke MMSE score >15. Exclusion criteria were severe aphasia, pre-stroke dementia, and other major neurological or psychiatric disorders that could interfere with the neuropsychological testing.

Patients were followed-up 1, 6, 12, and 24 months after the event for clinical and neuropsychological evaluation.

The study was based on informed consent and was approved by the local medical ethics committee.

Assessment of cognitive functioning

Pre-stroke cognitive functioning was assessed by means of a semi-structured interview with a patients’ caregiver based on the DSM-IV diagnostic criteria for dementia. The caregiver was asked whether the patient had memory problems or other cognitive impairments before stroke and how much these impairments interfered with daily living. This interview was repeated at each assessment. However, after the first assessment, the focus shifted from comparison with pre-stroke functioning towards cognitive functioning compared to the previous assessment.

Within 1 month all participants were assessed by a well-trained neuropsychologist (SR) using the standardised Dutch translation of the MMSE and a neuropsychological test battery consisting of the following tests: CAMCOG, Concept Shifting Test, Stroop Colour Word Test, Auditory Verbal Learning test and the Groninger Intelligence Test. This battery assesses memory, language, mental speed, orientation, attention, praxis, executive functioning, calculation, and visuospatial functioning. Cognitive functioning was compared with that of a norm group from the Maastricht Aging Study (MAAS), which investigates normal cognitive ageing in healthy older volunteers. Norm tables are stratified according to age, sex, and educational level [18].

A score lower than the 10th percentile of the score of the norm group defined a deficit on a cognitive domain [19]. Table 1 represents the ten cognitive domains tested. Patients were screened for disturbances in 1, 2, and at least 4 domains. These categories were chosen somewhat arbitrarily. Literature reports studies screening for at least 1 disturbed domain, and in another study performed by Tatemichi et al. cognitive impairment was defined as a disturbance in at least 4 domains. [2, 20, 21].

Table 1

Cognitive domains

Domain	Tests
Memory	AVLT, total words direct recall and delayed recall
Mental speed	SCWT I, CST 0,I, and II
Executive functioning	Mean interference score SCWT, CST
Calculation	GIT, sums
Visuospatial	GIT, mental rotation
Orientation	CAMCOG items 139–148 (place, person, time)
Attention	CAMCOG items 178–179 (serial 7’s, counting backward)
Praxis	CAMCOG items 183–186/188–189/191–193 (copying, ideational, ideomotor)
Reasoning	CAMCOG items 197–200, 201–203 (similarities, perception)
Language	CAMCOG items 149–163/181–182 (understanding, expression, writing)

AVLT Auditory Verbal Learning Test, SCWT Stroop Colour Word Test, CST Concept Shifting Test, GIT Groninger Intelligence Test

Diagnosis of dementia was based on data including medical history, test performance, structured information from a patient’s informant about daily life functioning, and clinical observation. Two experienced clinicians independently made the diagnosis according to the DSM-IV criteria for dementia, blind to MMSE data. The diagnosis of Vascular Dementia (VaD) was based on the NINDS-AIREN criteria. Agreement between the two raters was κ = 0.88; p = 0.01. If there was a discrepancy in diagnosis, a consensus meeting was arranged. If consensus was not reached, the patient was not considered to be demented.

A change in cognitive performance was defined as an improvement or deterioration in the number of cognitive domains that were impaired.

Statistical analyses

For patient characteristics, descriptive statistics were administered. To determine the prevalence of patients who improved or deteriorated cognitively during the follow-up period, only patients with complete follow-up data were studied.

Pearson’s correlation coefficients were calculated to study a relationship between baseline MMSE scores and MMSE scores at later follow-up visits, and between baseline MMSE scores and the number of cognitive domains disturbed. Spearman’s Rho was calculated to study the relationship between baseline MMSE score and the course of cognitive functioning at later follow-up visits.

Multivariate linear and logistic regression analyses adjusted for age, sex, and educational level were administered to see whether baseline MMSE scores could predict a disturbance in cognitive domains tested, whether the MMSE could predict dementia, and whether baseline MMSE could predict the course of cognitive functioning at later follow-ups.

To assess a cut-off score of the MMSE in order to screen for impairment in at least one, two and four out of ten cognitive domains and to screen for dementia, Receiver Operating Characteristics (ROC) curves were obtained. The Area Under the Curve (AUC) was calculated as a measure of the accuracy of the scale. The closer its AUC value is to 1.0, the more accurate a test is in determining impaired patients and non-impaired patients. Sensitivity, specificity, and the positive predictive value were calculated at the cut-off points.

Missing data were imputed by logistic regression. Data imputation was performed if there was another test covering the same cognitive domain available and if the missing test was administered previously or at a later follow-up.

All analyses were performed with the statistical Package for Social Sciences version 15 (SPSS-15).

Results

Between January 2000 and June 2001, 194 first-ever stroke patients were included in this study. In this period, 592 patients were diagnosed as suffering from a stroke. Of these patients, 80 died within 1 month after stroke and 363 were excluded (89 were not first-ever strokes, 57 had a stroke located in the brainstem or cerebellum, 46 MMSE < 15, 34 had severe aphasia, 22 had comorbid neurological or psychiatric disorders, 6 were in a coma, 5 were not native Dutch speaking, 9 were younger than 40 years, 9 lived too far from the hospital, 6 were admitted too long after their stroke and 35 declined participation) [17].

Patient characteristics at baseline (1 month after the event) are presented in Table 2. During the follow-up, 54 patients dropped out (27 died, 24 patients refused further participation, 1 patient was too ill at the 24-month follow-up, and 2 were untraceable).

Table 2

Demographic characteristics at baseline (1 month)

	Total cohort	Follow-up completed	Drop-outs	p*
N	194	140	54
Education (low) %	56.2	50.7	70.4	0.019
Sex (F) %	44.8	45.7	42.6
Age mean (SD)	68.3 (12.5)	65.8 (12)	74.6 (11.5)	<0.000
Cortical lesion %	37.6	38.2	50.0
Left sided lesion %	42.1	39.6	52.8
Rankin mean (SD)	3.1 (1.4)	3.0 (1.3)	3.2 (1.4)
MMSE mean (SD)	25.5 (3.5)	26 (3.3)	24.1 (3.7)	0.002
Diagnosis of dementia at baseline %	11.4	7.9	20.4	0.02

Rankin: no handicaps: 0, bedridden: 5

SD standard deviation, MMSE Mini-Mental State Examination (minimum score: 0, maximum score: 30)

* p values for significant differences in the comparison between patients who completed the follow-up and drop-outs are presented

Drop-outs were older than non-drop-outs (75.4 and 65.8 years, respectively, p < 0.001), had lower baseline MMSE scores (24.1 and 26.0, respectively, p < 0.001), and were less educated (70.4 and 50.7% of patients were less educated, respectively, p < 0.001).

Table 3 shows the percentage of patients who deteriorated or improved at least in one domain of the cognitive domains tested, compared to baseline at 12 and 24 month follow-ups. Patients were categorised into baseline MMSE scores over 27, between 27 and 24, or lower than 24. Patients with MMSE scores below 24 were older than patients with scores higher than 27 (70.5 (SD 11.8) and 64.0 (SD 11.9) years, respectively, p = 0.04) and more often less educated (70 and 39.4%, respectively, χ 2 = 10.0, df = 4, p = 0.04). After 12 months, most patients improved or remained stable in number of cognitive domains disturbed in each category of baseline MMSE score. Patients with baseline MMSE scores lower than 24 more often deteriorated. Patients with higher baseline MMSE scores more often were stable in the number of cognitive domains disturbed (χ 2 = 13.6, df = 4, p = 0.009). After 24 months, measures were comparable. Chi square tests between categories were not significant.

Table 3

Patients who improved or deteriorated in cognitive functioning according to baseline MMSE score

	Improved	No change	Deteriorated
12 months
Baseline MMSE ≥ 27	40.8	40.8	18.4
Baseline MMSE < 27 and ≥24	65.6	18.8	15.6
Baseline MMSE < 24	58.6	10.3	31.0
24 months
Baseline MMSE ≥ 27	41.0	43.6	15.4
Baseline MMSE < 27 and ≥24	52.9	32.4	14.7
Baseline MMSE < 24	51.7	17.2	31.0

Numbers represent percentages

Change in cognitive functioning means an improvement or deterioration of 1 in 10 previously described cognitive domains compared to baseline

At 24 month follow-up, the number of patients followed -up completely with baseline MMSE ≥ 27 was 78, MMSE < 27 and ≥24 was 34, and for MMSE < 24 was 29

There was no significant difference in mean baseline MMSE score, age, or educational level between patients who deteriorated cognitively during follow-up and patients who did not. These findings were comparable in patients with left-sided lesions and right-sided lesions and in demented and non-demented patients.

Baseline MMSE correlated well to scores on the MMSE at later follow-up visits 6, 12, and 24 months: r = 0.77, r = 0.76, and r = 0.73, respectively, (p < 0.01).

The MMSE score at baseline related well to the number of cognitive domains disturbed at baseline and at later follow-up visits at 6, 12, and 24 months: r = −0.68, r = −0.70, r = −0.62, and r = −0.69, respectively (p < 0.01).

Table 4 shows the prevalence of patients suffering from at least 4 disturbances in cognitive domains and the prevalence of patients with a maximum of 1 disturbance based on their baseline MMSE score.

Table 4

Percentages of patients with at least 4 cognitive domains and 1 or fewer domains disturbed

	Baseline	6 months	12 months	24 months
≥4 domains disturbed
Baseline MMSE
<24	80	80.0	75.9	79.3
24 ≤ MMSE < 27	53.1	28.1	25.0	29.0
≥27	14.1	4.0	10.5	7.7
≤1 domains disturbed
Baseline MMSE
<24	6.7	3.3	17.2	10.3
24 ≤ MMSE < 27	9.4	25.0	46.9	29.0
≥27	56.4	72.0	65.8	69.2

At baseline, the number of patients with an MMSE score <24 was 30, with an MMSE score between 24 and 27 was 32, and with an MMSE score ≥27 was 78

Only patients with complete follow-up (N = 140) were studied

To study whether baseline MMSE could predict the number of disturbed cognitive domains, multivariate regression analysis was performed with adjustment for age, sex and level of education. Baseline MMSE was a significant predictor for the number of disturbed cognitive domains at baseline and later follow-up visits at 6, 12, and 24 months after the event (p < 0.05). Furthermore, the MMSE was a significant predictor for the diagnosis of dementia in the multivariate model (p < 0.05).

Figure 1 shows the ROC of the MMSE in the screening for a disturbance in cognitive domains and for the diagnosis of dementia.

Fig. 1

Receiver Operating Curves of the MMSE in the screening for impaired cognitive domains and dementia

At baseline, 163 patients suffered from at least 1 disturbed domain out of 10 cognitive domains, 137 patients from at least 2 and 85 patients suffered from at least 4 disturbed domains out of 10 cognitive domains. Table 5 shows optimum cut-off scores of the MMSE and their sensitivity and specificity in the screening for cognitive impairments and dementia.

Table 5

Screening abilities of the MMSE at optimum cut-off points in the screening for impaired cognitive domains and dementia

Impaired domains	Cut-off	Sensitivity	Specificity	PPV	AUC
≥1	27/28	0.72	0.71	0.93	0.79
≥2	27/28	0.80	0.70	0.86	0.86
≥4	26/27	0.82	0.75	0.72	0.88
Dementia	23/24	0.96	0.83	0.41	0.94

PPV positive predictive value, AUC area under the curve

In the screening for 1 impaired cognitive domain, we found a moderate sensitivity of 0.72 at a cut-off score of 27/28. At a cut-off of 28/29, sensitivity increased to 0.87 but the specificity decreased to 0.42. These values were comparable in patients with left-sided lesions and patients with right-sided lesions.

To study whether baseline MMSE scores could predict further deterioration or improvement in cognitive functioning, correlations and regression analyses were performed using the MMSE as a continuous scale, and after categorising patients into baseline MMSE scores over 27, between 27 and 24, or lower than 24. There was no correlation between the baseline MMSE score and an improvement or deterioration in cognitive functioning. Neither could baseline MMSE scores predict an improvement or deterioration in cognitive functioning in a multivariate regression model adjusted for age, sex and education. A deterioration in MMSE score correlated with deterioration in cognitive functioning over time.

Discussion

The aim of this study was to investigate whether, 1 month after stroke, the MMSE could predict cognitive impairment and its course over 2 years. We found significant correlations between the MMSE score 1 month after the event and cognitive deficits at each follow-up visit 6, 12, and 24 months after. Furthermore, the MMSE predicted cognitive impairment and dementia in a multivariate regression model adjusted for age and education. These findings are comparable to a study by Tatemichi and co-workers, who found that poor performance on the MMSE after 1 week after stroke was an important indicator for poor performance in cognitive functioning after 3 months [2].

After 1 year, about 60% of patients with baseline MMSE scores less than 27 and 40% of those with MMSE scores of at least 27 showed an improvement in their cognitive performance. Patients with low MMSE scores more often deteriorated, whereas patients with higher scores more often were stable in the number of disturbed cognitive domains. However, baseline MMSE scores were not sensitive in screening for a change in cognitive performance. Deterioration or improvement in the number of disturbed cognitive domains at later follow-up visits was not predicted by the MMSE 1 month after the event. A deterioration in MMSE score correlated with deterioration in cognitive functioning in time.

We found MMSE cut-off scores of 27/28 showed a good sensitivity in screening for at least 2 disturbed domains, as did a cut-off of 26/27 in screening for at least 4 disturbed domains. In the diagnosis for dementia, we found a cut-off score of 23/24 with a sensitivity and specificity of 0.96 and 0.83, respectively. However, in screening for 1 disturbed domain we found a cut-off score of 27/28 with a moderate sensitivity and specificity of 0.72 and 0.71. The cut-off scores in our study are comparable to those in the literature [8, 22, 23]. Our results suggest that an MMSE score of less than 27 possibly indicates substantial cognitive disturbances and thorough screening for cognitive decline is warranted. The MMSE is moderately sensitive in the mildly cognitively affected patients.

The validity of the MMSE in screening for cognitive decline and dementia in stroke is still debated in the literature [13, 20, 22]. It is said to be sensitive only when a patient is already severely impaired [16]. In screening for dementia in general, a cut-off of 23/24 is administered. However, the literature advises adjusting this cut-off to an individual patient’s confounders [24] or increasing the cut-off value [23]. In our study, based on sensitivity at the optimum cut-off points, the MMSE showed moderate properties in screening for mild cognitive disturbances, but was a sensitive tool in screening for moderate cognitive deterioration and dementia.

Nys and co-workers studied the validity of the MMSE in the first week after stroke in screening for any disturbance in 1 of 6 cognitive domains in 34 stroke patients and 34 age- and education-matched controls [20]. They could not find an optimum cut-off point in the screening for at least 1 out of 6 disturbed domains. Our study included a larger sample and contained a comparable amount of subcortical and cortical lesions. We did not examine the patients cognitively in the first days after stroke. The MMSE was performed in the same session as the neuropsychological test battery.

In another study comprising 112 stroke patients, the MMSE had a moderate sensitivity (0.62) and specificity (0.88) at a cut-off point of 24 in screening for cognitive impairment [25]. The authors concluded that the MMSE is not sensitive to memory complaints. Time between the MMSE (first week after stroke) and cognitive testing (within 3 months after stroke) varied substantially. Therefore, the relationship between the two may have been weakened, since spontaneous recovery may have occurred between tests.

Several studies report limitations of the MMSE in screening for dementia and cognitive impairment, especially in subcortical infarctions and small vessel disease, where it would not differentiate between focal and diffuse lesions. Furthermore, it would be insensitive to right-sided lesions [13-15]. We found no differences in screening properties between patients with left-sided lesions and those with right-sided lesions.

Other studies report the MMSE to be sufficiently accurate as a screening instrument for cognitive impairment and dementia in stroke patients in the clinical situation, and support the ability of the MMSE to follow-up cognitive performance over time [7, 9, 22, 26, 27]. Other studies found the MMSE to be a useful screening tool when taking confounders into account such as age and education [10, 12, 24, 28], or when another additional screening test is performed [14, 29] and state that the difference in screening ability between left and right-sided lesions is exaggerated [12]. We found the MMSE to be an independent predictor for cognitive functioning.

The standard criteria and the ease of use make the test attractive [7, 30]. Furthermore, all short screening tests share deficits in sensitivity [31].

This study has some limitations to be mentioned. We excluded a substantial number of patients with our exclusion criteria. Our study was comprised of the relatively cognitively better part of stroke patients, since patients with severe aphasia and an MMSE score <15 were excluded. However, this generally applies to follow-up studies that require long-term cooperation. Patients are required to understand the tasks to measure their cognitive performance accurately. Including these patients would have increased the relationship between the MMSE and cognitive dysfunction, and would have improved the sensitivity values in our study and the accuracy of the MMSE in screening for cognitive impairment and dementia. It is not conceivable that the exclusion of the other patients influenced our results.

The 54 patients lost to follow-up were older, less educated, and had more cognitive deficits and lower MMSE scores. This may have rather weakened our results.

In the cross-sectional analyses we found that most patients improved in cognitive functioning, and that patients with low baseline MMSE scores deteriorated more often than patients with high scores. These findings indicate that there are some differences in cognitive functioning over time between patients with high and low MMSE scores. However, neither in patients with high baseline MMSE scores nor in patients with low scores is this a linear effect predicted by baseline MMSE scores. Patients with high baseline MMSE scores more often were stable in their cognitive performance, whereas patients with low baseline MMSE scores were less likely to stabilize and showed a more dynamic course in improvement or deterioration of cognitive performance. Moreover, a change in cognitive functioning could not be predicted by the MMSE scores in a regression model adjusted for age and education. As is shown in this and other studies, it is conceivable that a number of patients with cognitive impairment will deteriorate and a number of patients will improve [32, 33]. Most improvement is seen in the first months after stroke. The cross-sectional measurements after 24 months will partially overlap with those found at the 12-month visit. Thus, although we analysed only patients from whom we had test results at each follow-up visit, we cannot be precise in determining the temporal effect on cognitive functioning in an individual patient and the relationship with the baseline MMSE from our study design. Alternatively, linear mixed models analyses could be applied to analyse the course of cognitive functioning in time. However, this was problematic since drop-outs differed substantially in demographics and in MMSE scores from patients who completed the follow-up. A study of the risk factors for decline was beyond our goals. Furthermore, our study was focussed on validating the screening ability of the MMSE. For this purpose, the utilisation of a fully completed data set was preferred. Although we assessed pre-stroke cognitive functioning in a retrospective manner by means of a structured interview based on the DSM-IV criteria for dementia with the patients’ caregiver, and not by means of a measurement scale, we cannot precisely distinguish how pre-stroke cognitive performance influences post-stroke cognitive performance, cognitive recovery, or how stroke influences pre-stroke cognition. To rule out such an effect, for instance, the change in MMSE scores between 6- and 1-month visits could be used to predict further cognitive change. However, this was beyond the goals of our study. Studying the effects between pre-stroke cognitive performance, stroke and post-stroke cognitive functioning would require a different study design.

We did not adjust for age and education in calculating the optimum cut-off. However, defining confounding factors for the MMSE was beyond the aim of this study. In the multivariable model, age and educational level did not predict cognitive functioning, whereas the baseline MMSE did.

Despite its limitations reported in the literature, we found that the MMSE, which is easy to use, is a useful instrument to screen for moderate to severe cognitive deficits in stroke patients and to screen patients for more detailed neuropsychological assessment. A score on the MMSE of <27 1 month after stroke is also related to poor cognitive performance in the longer term. However, it cannot be used to predict whether an individual patient will recover or deteriorate.