The Agreement between the MMSE and IQCODE Tests in a Community-Based Sample of Subjects Aged 70 Years or Older Receiving In-Home Nursing: An Explorative Study.

AIM: It was the aim of this study to compare the Mini-Mental State Examination (MMSE) with the Informant Questionnaire for Cognitive Decline in the Elderly (IQCODE) and to explore the characteristics of subjects with possible dementia with only one of the two tools.
METHODS: We used a random sample of patients aged 70+ receiving social service or in-home nursing. The patients were tested with the MMSE, and the next of kin was interviewed using the following: the IQCODE, the Cornell Scale for Depression in Dementia (CSDD), the Neuropsychiatric Inventory (NPI), instrumental activities of daily living (IADL), personal ADL (PADL) and the General Medical Health Rating (GMHR).
RESULTS: Subjects with dementia defined only according to the MMSE showed a pattern of scores on IADL, PADL, CSDD, NPI-10 and GMHR similar to the no-dementia group according to both the MMSE and the IQCODE. Those with dementia defined only according to the IQCODE showed a pattern of scores similar to the possible dementia group according to both the MMSE and the IQCODE.

Introduction

In Norway, there are more than 70,000 people with dementia, and about half of them are living in their own homes. The number is expected to double by the year 2040, and studies indicate that less than half of these people with dementia actually have a diagnosis of dementia [1]. The prevalence of dementia in the population aged ≥75 years and living at home is 10% [2]. The prevalence is even higher among those who need help with their activities of daily living (ADL) [3].

Before appropriate treatment and care can be provided, patients need to be adequately evaluated. A proper diagnosis gives the patients, their families and the municipal care services a realistic chance to plan and implement necessary measures [1,4]. The staff of the municipal health and social services is in direct contact with elderly people in the municipalities applying for services, either by themselves or through others (e.g. family members). Thus, the municipal health and social care staff will meet many patients who may have dementia but lack a diagnosis. To provide services of good quality and to detect conditions influencing the mental capacity other than dementia, efficient tools for assessing this population are needed. Such tools are not only needed to diagnose dementia but may identify those subjects who require further examination.

Different kinds of screening tools for detecting dementia are also used in research projects. These screening tools are mainly adopted to include (or exclude) persons with dementia in (or from) a project or to compare subjects with dementia with those without dementia [5,6,7,8,9].

There are several screening tools for dementia. Two of the most common instruments are the Informant Questionnaire for Cognitive Decline in the Elderly (IQCODE) long form [10] or short form [11] and the Mini-Mental State Examination (MMSE) [12].

A meta-analysis comparing the IQCODE and the MMSE showed that both tools have a similar effectiveness in detecting dementia [13]. However, even though both tools seem to be valid and sensitive in detecting dementia [14,15] and share a similar sensitivity and specificity [4], some differences exist. The MMSE is a cognitive assessment of patients, whereas the IQCODE is a proxy-rated questionnaire in which an informant, who knows the patient in question well, rates the change in function over the last 10 years. It is claimed that the MMSE is a sensitive tool for measuring education while the IQCODE is not [15]. Despite the fact that both tools are rated as good screening tools for detecting possible dementia, several studies have shown that the correlation between the two instruments is not impressive [correlation coefficient (r) from −0.41 to −0.78 [15]; r = −0.42 [16], and r = −0.37 [17]]. Thus, it is obvious that the MMSE and the IQCODE identify different patients with different patterns of decline, but we have not found any studies that analyse these differences in depth.

A recent literature review from Lin et al. [18] shows that the MMSE has a better sensitivity (85.4-88.3%, depending on the threshold value) and specificity (86.2-90.2%) compared with the IQCODE that had a sensitivity of 75-81% and a specificity of 68-80%.

Both the MMSE and the IQCODE have more than one recommended cut-off value for reasonably suspecting dementia. In this paper, we use the recommendation of Jorm [15] from 2004 that suggests a cut-off value of ≤24 for the MMSE and a cut-off value of ≥3.44 for the IQCODE as a mental decline corresponding to dementia, and we will use the term ‘possible dementia’ for these scores.

To get a deeper understanding of the strengths and weaknesses of the MMSE and the IQCODE, we designed a study aiming to compare the MMSE with the IQCODE and to explore the characteristics of people with possible dementia in only one of the two tools.

Subjects and Methods

In Norway, the care of the elderly is within the jurisdiction of the local authorities (municipalities) that provide social services (housing and home care), in-home nursing and institutional care (e.g. nursing homes). In 2009, nearly 175,000 people received municipal social service or in-home nursing, and about 130,000 (74%) of them were ≥75 years old (Statistics Norway http://ssb.no/).

The study included participants from 19 municipalities. The sample was formed from a selection of large and small, urban and rural municipalities. In each of the participating municipalities, a random sample was selected from subjects aged ≥70 years, receiving practical social service (home care) or municipal in-home nursing. In total, 1001 patients were included, but 3 were excluded due to incomplete data. Thus, the study sample comprised 998 patients, including 680 (68.1%) women and 318 (31.9%) men. The women were older than the men (mean age 83.7 ± 5.7 vs. 82.6 ± 5.6 years, respectively; p = 0.007).

Staff from the municipal services, including registered nurses, occupational therapists, social nurses and auxiliary nurses, collected the data. All those who participated in the data collection went through a standardized 1-day training course in the use of the different assessment scales. Some data were collected by interview or testing of the person in question (for example, using the MMSE), but most of the data came from an interview with a family member who knew the patient well.

Scales

The MMSE is an assessment scale measuring cognitive function, such as memory, orientation, attention, calculation and visuospatial performance (construction) [12]. The scores on the MMSE range from 0 to 30, where lower values indicate poorer cognitive function. The MMSE is frequently used both in clinical practice and in research [14,18,19].

The IQCODE is an informant (proxy)-rated questionnaire that rates the change in function over the last 10 years. It is a widely used screening instrument for dementia that originally included 26 questions [20] but was later developed into a short form with 16 questions [11]. The correlation between the long and the short form is high, and the validity of the short form is identical to that of the long form. Thus, there is little to be gained by using the 26-question version [15]. Each question is rated from 1 (‘much improved’) to 5 (‘much worse’).

The Cornell Scale for Depression in Dementia (CSDD) [21] is scored based on an interview with both the patient and a caregiver who has observed the patient over time. This instrument is suitable for use among patients with and without dementia [22] and consists of 19 variables with a possible score of 0-2, giving a sum score ranging from 0 to 38.

The Neuropsychiatric Inventory (NPI) assesses neuropsychiatric symptoms. It originally consisted of 10 items but now also includes 2 neurovegetative items. The frequency score (0-4) and the intensity score (0-3) are multiplied, producing an item score ranging from 0 to 12 and a sum score ranging from 0 to 144 [23]. Since the proxy often did not live with the patient, there were a lot of missing scores for the neurovegetative items (sleep disturbance and meals). Thus, we used the 10-item scale in the present study, with a possible sum score of 0-120.

To assess the function in ADL, two scales were used, both developed by Lawton and Brody [24]. The first scale assesses instrumental ADL (IADL) and consists of 8 items. The sum score ranges from 8 to 31, with a higher score indicating lower performance [25]. The second scale rates physical self-maintenance, or personal ADL (PADL), and consists of 6 items. The sum score ranges from 6 to 30, with a higher score indicating lower performance.

The General Medical Health Rating (GMHR) was developed in 1999 [26]. The patient's current and past medical history is taken from the patient, a family member or a health worker who knows the patient well. After having collected the information about the patient's health, GMHR is rated as 1 (poor), 2 (fair), 3 (good) or 4 (excellent).

Statistics

All the analyses were performed with the IBM SPSS statistics software version 21 (IBM Corporation, Armonk, N.Y., USA). According to the suggestions of Jorm [15], the cut-off for possible dementia was set to ≤24 for the MMSE and to ≥3.44 for the IQCODE. The agreement of the MMSE and the IQCODE was tested with correlations (the actual scores) and with κ statistics (dichotomized scales). The differences between the MMSE and the IQCODE regarding the scores of IADL, PADL, CSDD, NPI and GMHR were analysed with t test statistics. The associations between the actual scores (no cut-off) of the MMS, the IQCODE and other patient characteristics were tested with linear regression models.

Ethics

The participants received information about the study and gave their written consent. If the patient did not have the capacity to give consent, a family member was informed and could consent on behalf of the patient. This procedure was approved by the Regional Ethics Committee for Medical Research in Southern Norway, the Data Inspectorate and the Directorate for Health and Social Affairs.

Results

Table 1 shows the patient characteristics of the sample. Pearson's correlation coefficient between the MMSE and the IQCODE was r = −0.61 and, thus, r2 = 0.37. The distribution of the MMSE and the IQCODE is shown in a scatter plot in figure 1.

Table 1

Patient characteristics

Women/men	680 (68)/318 (32)
Age, years	83.4 ± 5.6
GMHR score	1.6 ± 0.9
IADL sum score	15.3 ± 6.6
PADL sum score	9.1 ± 3.5
CCSD sum score	4.7 ± 5.3
NPI-10 sum score	4.9 ± 9.2
MMSE total score	24.5 ± 4.8
IQCODE total score	3.5 ± 0.6

Data are given as the mean ± SD or n (%).

Fig. 1

Scatter plot of the correlation between the MMSE and the IQCODE sum scores. The cut-off value of ≤24 for the MMSE is shown left of the vertical line. The cut-off value of ≥3.44 for the IQCODE is shown above the horizontal line. Thus, the cases with possible dementia according to both tests are given in the upper left square. The lower right square shows the cases without dementia according to both tests. The lower left square shows the cases with possible dementia according to the MMSE but not the IQCODE, whereas the upper right square shows the cases with possible dementia according to the IQCODE but not the MMSE.

Information on diagnoses was available from 975 patients, of whom 84 patients (9%) had a diagnosis of dementia that was known to themselves, their caregivers or the home care system. Possible dementia according to both the MMSE and the IQCODE was present in 271 patients (28%), of whom 73 (27%) had a diagnosis of dementia. Possible dementia only according to the IQCODE was present in 111 patients (11%), of whom 5 (5%) had a diagnosis, and possible dementia only according to the MMSE was present in 132 patients (14%), of whom 3 patients (2%) had a diagnosis. Three patients had a diagnosis of dementia but scored as not having dementia with both tools. The weak association between the scores on the two scales and the diagnosis of dementia made it impossible to use the diagnosis as a ‘gold standard’ and we did not pursue this further.

After dichotomizing the MMSE and the IQCODE, 4 groups were formed: group 1, no dementia according to both tests (n = 466); group 2, possible dementia according to the MMSE but not the IQCODE (n = 136); group 3, possible dementia according to the IQCODE but not the MMSE (n = 118), and group 4, possible dementia according to both tests (n = 272). These numbers give a prevalence of possible dementia of 41% for the MMSE and of 39% for the IQCODE in this population. There was a disagreement between the MMSE and the IQCODE in more than 1 of the 4 groups of patients included in this study. The agreement between the MMSE and the IQCODE was κ = 0.47 (p < 0.001). Regarding IADL, PADL, CSDD, NPI and GMHR, those subjects with no dementia according to both tests had the lowest decline on all these scales, followed by those who had possible dementia only according to the MMSE. The clearest decline was between those who had possible dementia only according to the MMSE compared with those who had possible dementia only according to the IQCODE. The differences were statistically significant for all the tests except for PADL and GMHR (table 2).

Table 2

Characteristics of subjects with no dementia or possible dementia according to the MMSE and/or the IQCODE

	No dementia (n = 466)		Possible dementia according to the MMSE (n = 136)		Possible dementia according to the IQCODE (n = 118)		Possible dementia according to both the MMSE and the IQCODE (n = 272)
	n	mean ± SD	n	mean ± SD	n	mean ± SD	n	mean ± SD
IADL score	464	11.1 ± 3.8	136	14.3 ± 5.7	117	17.4 ± 5.8	271	20.1 ± 6.7
PADL score	464	7.7 ± 2.4	136	9.2 ± 3.2	117	10.0 ± 3.4	271	11.2 ± 4.6
CSDD score	463	2.8 ± 3.4	136	4.0 ± 4.7	118	7.9 ± 7.1	270	6.8 ± 6.0
NPI-10 score	403	2.6 ± 5.4	113	3.3 ± 6.2	87	11.2 ± 13.4	213	12.0 ± 14.2
GMHR score	465	1.7 ± 0.9	138	1.6 ± 0.9	115	1.4 ± 0.8	279	1.4 ± 0.9

The p values (Student's t test) for the differences between the scores for ‘possible dementia according to the MMSE’ and ‘possible dementia according to the IQCODE’ were all <0.001 except for PADL where the p value was 0.07 and for GMHR where it was 0.28.

To get a more complete picture of the pattern, we wanted to test the difference between the MMSE and the IQCODE when using the actual scores (i.e. not dichotomized). Thus, we performed a multiple linear regression with the MMSE total score and the IQCODE total score as dependent variables and with the other patient characteristics described in table 1 as independent variables. The IADL and PADL were strongly correlated (Pearson's r = 0.760), but IADL had a stronger association with both the MMSE and the IQCODE than PADL in the bivariate analysis (table 3). Thus, we kept IADL in the multivariate analyses and left PADL out of the models. We also found a strong correlation between NPI-10 and CSDD (Pearson's r = 0.627), and NPI-10 had the strongest association with the outcomes (MMSE and IQCODE) in the bivariate analysis (table 3), so we kept NPI-10 in the multivariate models and left CSDD out. As shown in table 3, the IQCODE has a stronger association with decline in IADL, NPI-10 and GMHR compared with the MMSE, and the MMSE is more strongly associated with age. The models explained 56% of the variation in the IQCODE and 35% of the variation in the MMSE.

Table 3

Multiple linear regression for the MMSE and the IQCODE

	MMSE				IQCODE
	bivariate linear regression		multivariate linear regression		bivariate linear regression		multivariate linear regression
	standardized coefficients	p value	standardized coefficients	p value	standardized coefficients	p value	standardized coefficients	p value
Age	−0.183	<0.001	−0.085	0.005	0.109	0.001	0.003	n.s.
Gender	−0.68	0.032	0.004	n.s.	0.041	0.200	−0.044	n.s.
IADL score	−0.580	<0.001	−0.559	<0.001	0.700	<0.001	0.643	<0.001
PADL score	−0.403	<0.001			0.441	<0.001
CSDD score	−0.196	<0.001			0.357	<0.001
NPI-10 score	−0.257	<0.001	−0.078	0.014	0.461	<0.001	0.258	<0.001
GMHR score	0.130	<0.001	−0.077	0.012	−0.148	<0.001	0.100	<0.001
Adjusted r2			0.354			0.561

n.s. = Not significant.

Discussion

The present study shows that people with possible dementia according to only the MMSE had a pattern of scores on IADL, PADL, CSDD, NPI-10 and GMHR similar to those who ended up in the no-dementia group according to both the MMSE and the IQCODE. Those who had possible dementia according to only the IQCODE had a pattern of scores similar to those who had possible dementia according to both the MMSE and the IQCODE (table 2). Figure 1 shows that there was considerable variance between the MMSE and ICODE scores also within the 4 groups, hence the low r2 value.

The modest correlation and low κ between the IQCODE and the MMSE are in line with other studies described in the Introduction. Even though the prevalence of dementia according to these instruments is quite similar, the present study shows that the two instruments do not identify the same patients as having possible dementia. The patients identified as having possible dementia according to the IQCODE, but not according to the MMSE, had significantly lower function in IADL and more severe neuropsychiatric symptoms (CSDD and NPI-10) compared with the patients identified as having dementia according to the MMSE, but not according to the IQCODE. It is surprising that, even though the low κ and low r2 are well known, we have not found any studies exploring this disagreement in depth.

IADL, PADL, NPI-10 and CCSD are all based on information from a proxy, a family caregiver or a community nurse who know the patient well. The IQCODE is also based on information from a proxy, while the MMSE is a test performed directly with the patient. Thus, it might be that the IQCODE is coloured by the results of the other tools.

It may seem odd that PADL and GMHR did not have the same association with the IQCODE score as IADL, CSDD and NPI-10 (table 2). This is probably due to the characteristics of the included subjects. All participants in this study were included because they received municipal in-home nursing services, which are provided to people who need help with ADL tasks and often have reduced health.

The MMSE is not independent of educational level, ethnicity or mental capacity earlier in life [27,28], and, thus, it may be that the MMSE wrongly identifies people who always have shown a low performance in mental capacity or who are not used to performing theoretical tasks as possibly having dementia. People with a high capacity earlier in life may have a reduced mental capacity that is not identified by the MMSE. Even though memory is a core issue in dementia, other functions like change in personality, behaviour and ADL performance are also part of the definition. Thus, it may be that people with a relatively well-preserved memory, but deficiencies in other areas, are overlooked by the MMSE and identified by the IQCODE.

There seems to be an agreement that early detection of dementia is important for several reasons. Early detection of dementia will improve the opportunity to make tailored care plans that maintain independence, give affected patients and their families a chance to make plans for the future and improve medical care [29]. However, it is not obvious whether the use of screening instruments is the best way to detect dementia.

In the International Classification of Diseases established by the World Health Organization, dementia is defined as including both cognitive symptoms (‘disturbance of multiple higher cortical functions, including memory, thinking, orientation, comprehension, calculation, learning capacity, language, and judgement’) and psychological and behavioural symptoms (‘deterioration in emotional control, social behaviour, or motivation’) [30]. Most of the research that gives advice in this field recommends both a test of the person in question and information from a proxy [18,29] in order to identify all possible cases. This recommendation will indeed raise the sensitivity of a screening but also reduce the specificity. This study has shown that there was an agreement in only 272 of the 526 subjects who had possible dementia according to either the IQCODE, the MMSE or both.

What are the benefits or drawbacks of a high sensitivity and a low specificity? The arguments for screening are similar to the argument for early diagnosis outlined above [29,31]. The arguments against screening are 2-fold. The first is the lack of evidence that an earlier diagnosis has an influence on decision making by the clinician, patient or family [32]. The other is the possibility that screening for dementia may cause direct or indirect harm because of the diagnostic inaccuracy of screening (false-positives and false-negatives). However, a recent literature review [18] found no trials that directly assessed the effect of screening for cognitive impairment on decision making, health or social outcome. Lin et al. [18] found only one study directly commenting on the potential harm of screening. Iliffe [33] came to a similar conclusion: ‘there are no experimental studies to refute or confirm harms of screening’ (for dementia). Thus, we are left with arguments and not empirical evidence.

The sample in the present study is not representative of the entire population of people ≥70 years but covers subjects who receive social support or in-home nursing. The prevalence of dementia in this group is higher than that in the remaining population [3]. The high number of subjects with an MMSE score around 24 and an IQCODE score around 3.44 places a great proportion of the group in a grey area where a proper clinical assessment is more relevant than the use of screening tools. On the other hand, these subjects are already in contact with the health or social services, and a very low proportion of those who scored as having dementia in one or both tools had a diagnosis. In order to detect dementia in elderly people who receive municipal care services, better tools and routines are obviously needed. Thus, it is reasonable to suggest a screening test to identify those who are in need of further investigation.

The results of the present study indicate that using only the MMSE or only the IQCODE as an indicator of dementia in research projects may result in serious bias of the results.

It is important to reiterate that it was not possible to test the two screening tools against a ‘gold standard’ in the present study. Thus, it has not been possible to evaluate which of the tools is best, but it is clear that using only one of the tools results in systematic bias. Thus, further research is needed in different populations and with a certain clinical diagnosis as the ‘gold standard’.

Conclusion

Even though the MMSE and the IQCODE had a similar prevalence of possible dementia, the agreement on which patients had possible dementia was low. Thus, we need studies focusing on how to combine tests that include both information from patients (tests) and information from proxy (questionnaires) in a more efficient way. Alternatively, we must develop new screening tools that combine the tests performed with patients and the questionnaires given to a proxy.