Reliability and Validity of the Clinical Dementia Rating for Community-Living Elderly Subjects without an Informant.

BACKGROUND: The Clinical Dementia Rating (CDR) scale is widely used to assess cognitive impairment in Alzheimer's disease. It requires collateral information from a reliable informant who is not available in many instances. We adapted the original CDR scale for use with elderly subjects without an informant (CDR-NI) and evaluated its reliability and validity for assessing mild cognitive impairment (MCI) and dementia among community-dwelling elderly subjects.
METHOD: At two consecutive visits 1 week apart, nurses trained in CDR assessment interviewed, observed and rated cognitive and functional performance according to a protocol in 90 elderly subjects with suboptimal cognitive performance [Mini-Mental State Examination (MMSE) <26 and/or Montreal Cognitive Assessment (MOCA) <26] and without informants according to a protocol. CDR domains and global scores were assigned after the second visit based upon corroborative information from the subjects' responses to questions, role-play, and observed performance in specifically assigned tasks at home and within the community.
RESULTS: The CDR-NI scores (0, 0.5, 1) showed good internal consistency (Crohnbach's α 0.83-0.84), inter-rater reliability (κ 0.77-1.00 for six domains and 0.95 for global rating) and test-retest reliability (κ 0.75-1.00 for six domains and 0.80 for global rating), good agreement (κ 0.79) with the clinical assessment status of MCI (n = 37) and dementia (n = 4) and significant differences in the mean scores for MMSE, MOCA and Instrumental Activities of Daily Living (ANOVA global p < 0.001).
CONCLUSION: Owing to the protocol of the interviews, assessments and structured observations gathered during the two visits, CDR-NI provides valid and reliable assessment of MCI and dementia in community-living elderly subjects without an informant.

Introduction

Given the rapid aging of the population worldwide, global estimates of the prevalence of Alzheimer's disease are expected to increase from 25 million currently to 63 million in 2030 and to a staggering 114 million by 2050 [1]. Over the last two decades, there have been modest but significant advances in the pharmacological treatment of this devastating condition. However, effective management and treatment to delay disease progression requires early and accurate diagnosis of dementia at its earliest stages [2,3].

The Clinical Dementia Rating (CDR) [4,5] scale is extensively used to assess cognitive and functional impairment in Alzheimer's disease. The CDR scale has established inter-rater reliability, can be administered by any trained personnel (usually a clinician or trained nurse) and employs a semi-structured interview with both the patient and a reliable informant to rate performance on six domains of functioning (memory, orientation, judgment and problem solving, community affairs, home and leisure activities, and personal care). The validity and reliability of the CDR scale was evaluated in many studies [5,6,7], and the scale as a whole was translated into 60 languages and dialects.

The transcultural feasibility of the CDR assessment across three different races (Chinese, Malay and Indian) in Singapore was demonstrated in a validation study involving heterogeneous, mixed groups of outpatients in a memory clinic sample. The CDR scale differentiates stages of disease severity congruent with the DSM-IV criteria for dementia [8] and has a good correlation with the Mini-Mental State Examination (MMSE) and neuropsychological tests of verbal memory and categorical fluency [9]. Notably, the CDR scale was found to make diagnostic distinctions between nondemented individuals and subjects with dementia even in the mildest stages (i.e. CDR 0.5) [10,11].

Although being a useful screening and severity-rating scale for dementia, CDR requires collateral information from a reliable informant to assess changing patterns of the subject's cognitive and functional performances. Hence, the utility in its current form poses a difficulty in two groups of individuals living independently in the community: elderly subjects without an informant (i.e. those living alone without a caregiver) and those who are existentially alone (i.e. although sharing a flat, they have limited communication with their roommates). In such situations, it is not known whether collateral information on the subject's changes in cognitive and functional status are equally reliable and valid when obtained by a surrogate informant in the person of the interviewer who is both observer and assessor. This study intended to adapt the original CDR scale for use with elderly subjects without an informant (CDR-NI) and evaluate its reliability and validity as an instrument to identify and assess early dementia among community-dwelling elderly individuals.

Methods

Study Design and Participants

This study is part of a currently ongoing second-wave cohort of the Singapore Longitudinal Aging Study (SLAS-2), which has enrolled 3,100 participants since 2009. The population sampling and measurement methodology is identical to that described for the first-wave cohort (SLAS-1), details of which have been described in prior publications [12]. Briefly, a total population sample of all elderly subjects aged 55 or older who were residents in contiguous study localities in the South East (SLAS-1) and South Central/South West regions of Singapore (SLAS-2) were identified from door-to-door census and invited to participate in the study. Residents who were mentally or physically unable to give informed consent or to participate in the study were excluded. The response rate was 78.5%, and the study was approved by the National University of Singapore Institutional Review Board. All participants signed written informed consent for study participation before completing extensive baseline interviews, physical examination, and testing for an extensive range of demographic, psychosocial, health, behavioral, lifestyle, and biomedical variables including neuropsychological and functional assessments for the screening and diagnosis of dementia.

The participants were screened for cognitive impairment using the locally validated Singaporean, translated, and modified versions of the MMSE [13] and the Montreal Cognitive Assessment (MOCA) [manuscript in preparation]. Participants with performance scores below the threshold values (MMSE <26 and/or MOCA <26) for determining cognitive impairment were subsequently invited for further clinical assessment for dementia (CAD) using an established protocol including CDR, clinical and laboratory tests, and final panel reviews for diagnosis of AD and other dementias. Functional status was assessed using the Instrumental Activities of Daily Living (IADL) [14] and the Basic Activities of Daily Living (BADL) Scales, which have been validated for use in the Singaporean population [15,16].

Clinical Assessment of Dementia

A three-member consensus panel consisting of 2 geriatricians and 1 neuropsychiatrist reviewed the results of the clinical history, physical examination, laboratory investigations, and CT scans and made a consensus diagnosis of mild cognitive impairment (MCI) and dementia according to published criteria [17].

Modified CDR without an Informant

Research interviewers who had received prior certified training to perform the CDR assessment were blinded to the cognitive screening results of the study participants. They visited the participants at their homes on two separate occasions 1 week apart. On the first visit, the interviewer explained the purpose of the visit to the subject and interrogated his/her present and past memory by discussing about current events addressed in newspapers, magazines, and television programs. The participant's orientation to time, place, and person was also evaluated. To assess his/her ability to perform personal care, the interviewer observed the participant's skills in selecting clothes, getting dressed, cleaning the floor, the kitchen and the toilet as well as in arranging furniture and using household appliances, the telephone, and the television remote control (see Appendix).

The interviewer also observed the participant's activities outside the home, such as getting around in the neighborhood, moving in traffic, and personal safety. The category of judgment and problem solving was assessed by a shopping role-play, observing how the participant handled small sums of money, and by asking how he/she managed with household problems such as a lost house key, leaking water, a blocked kitchen drain, and changing a light bulb.

On the second visit 1 week later, the interviewer talked with the participant about the first visit and explained the purpose of the interview, which was to assess his/her autobiographical memory of recent events. The subject's performance in the home domain (preparing meals or snacks, performing other household chores, handling eating utensils properly, etc.) and in the leisure activities domain was observed. The interviewer reviewed the notes on the home interviews and observations made during the two visits, completed the questions in the informant section of the CDR, and finally scored the six domains of the CDR scale.

Reliability and Validity

Four pairs (2 raters in each pair) of trained research staff were assessed for inter-rater reliability. Each pair made a conjoint interview with the same subject and independently assigned CDRs. In 39 of the subjects, a rater repeated the interview with and rating of the same participant after an interval of around 1 month (±3 days) for a test-retest reliability assessment. The accuracy of the CDRs was evaluated in reference to the clinical diagnoses of MCI or dementia as determined by the consensus panel.

Data Analysis

The statistical analyses performed using SPSS Statistical Package version 18 (SPSS Corp., Chicago, Ill., USA) included descriptive statistics and χ2 tests for categorical variables (MCI and dementia). Reliability analyses included Crohnbach's α for internal consistency of the CDR-NI scale, and the κ statistic for inter-rater and test-retest reliability of domain and global CDR scores. The validity of the CDR-NI scores was determined in three ways. Firstly, concurrent validity was assessed by the agreement between the global CDR score of each rater and clinical assessment according to the DSM-IV criteria. Secondly, we ascertained convergent validity via significance testing with one-way between-groups ANOVA to compare mean scores of MMSE, MOCA, ADL and IADL across the CDR stages. Bonferroni correction for multiple pairwise comparisons was used in post hoc tests of significant differences between the groups.

Results

Up to the present study, 400 out of the 789 elderly subjects screened were identified to have low cognitive test results (total scores of MMSE <26 or MOCA <26). The tests included 97 individuals without informants. After excluding 6 individuals who refused to accept assessments with CDR and CAD and 1 subject who died, 90 eligible persons participated in the CDR-NI validation study. Among them, CAD detected 37 (41%) clinically diagnosed cases of MCI, 4 (4.4%) of mild dementia, and 49 (54.6%) with no dementia.

The average age of the study participants was 71.3 years (SD 7.8). The majority were Chinese (93.3%), 59% were female, 26% were single, 56% were living alone, 56% had at least primary education, 81% were retired or unemployed, and 67% lived in low-end public housing apartments. The total BADL score ranged from 18 to 20 (mean 19.9 ± 0.24) and the total IADL score ranged from 17 to 24 (mean 23.72 ± 0.98). The MMSE scores ranged from 12 to 30, (mean 24.6 ± 3.7) and the MOCA scores ranged from 8 to 28 (mean 19.0 ± 5.3) (table 1).

Table 1

Characteristics of elderly subjects without an informant (n = 90)

Variables	Means ± SD or n (%)
Socio-demographics
Age, years	71.3 ± 7.8
Male gender	37 (41.1)
Chinese ethnicity	84 (93.3)
No formal education	40 (44.4)
Single, divorced, widowed	24 (26.7)
One-/two-room public housing apartments	60 (66.7)
Living alone	50 (55.6)
Retired or unemployed	73 (81.1)
Physical function
BADL score (range: 18 – 20)	19.9 ± 0.24
IADL score (range: 17 – 24)	23.7 ± 0.98
Cognitive function
MMSE (range: 12 – 30)	24.56 ± 3.7
MOCA (range: 8 – 28)	18.97 ± 5.3
Clinical assessment of dementia by DSM-IV
No dementia	49 (54.4)
MCI	37 (41.1)
Mild dementia	4 (4.4)

Reliability

Internal consistency (Crohnbach's α) for each of the 2 raters was 0.83-0.84. The κ statistics of the inter-rater reliability was 0.95 for the global score and ranged from 0.77 to 1.00 for the six domains, scoring lowest in the categories of judgment and problem solving and community affairs (0.77 and 0.79, respectively). The test-retest reliability κ was 0.80 for the global score and ranged from 0.75 to 1.00 for the six domains (table 2). The domains with the lowest test-retest reliability were memory, judgment and problem solving and community affairs (0.75, 0.78, and 0.79, respectively).

Table 2

Inter-rater and test-retest reliability of CDR

Domains	Inter-rater reliability (n = 90), κ	Test-retest reliability (n = 39), κ
Memory	0.95*	0.75*
Orientation	0.93*	0.85*
Judgment and problem solving	0.77*	0.78*
Community affairs	0.79*	0.79*
Household and leisure activities	1.00*	1.00*
Personal care	1.00*	1.00*
Global CDR	0.95*	0.80*

p < 0.001.

Validity

The degree of agreement between CDR-NI assessment and clinical diagnosis by DSM-IV is shown in table 3. CDR-NI determined 1 in 10 (4/37) cases of MCI as no dementia and 1 in 10 (5/49) cases of no dementia as MCI. The κ statistic (0.79, p < 0.001) indicated good agreement between the CDR-NI rating and clinical assessment status of MCI dementia.

Table 3

Agreement between the global CDR-NI score and clinical assessment (n = 90)

Clinical assessment	Global CDR-NI score
	CDR = 0	CDR = 0.5	CDR = 1	Total
No dementia	44 (89.8)	5 (10.2)	0 (0)	49 (100)
MCI	4 (l0.8)	33 (89.2)	0 (0)	37 (100)
Mild dementia	0 (0)	1 (25)	3 (75)	4 (100)
Total	48 (53.3)	39 (43.3)	3 (3.3)	90 (100)
κ	0.79*

Values are n (%).

p < 0.001.

As shown in table 4, different stages of the CDR scale (0, 0.5, and 1) were associated with significant differences in the mean scores for MMSE, MOCA and IADL, but not for BADL. In post hoc tests with Bonferroni correction, significant differences were found for comparisons of no dementia versus mild dementia and MCI versus mild dementia, but there were no significant differences for the mean scores of MMSE, MOCA, IADL and ADL when comparing beween no dementia and MCI.

Table 4

Cognitive and functional scores by CDR-NI status (n = 90)

Variables	CDR-NI			Global p	Bonferroni p value
	CDR = 0 (a)	CDR = 0.5 (b)	CDR = 1 (c)		(a) vs. (b)	(a) vs. (c)	(b) vs. (c)
MMSE	25.5 (2.9)	23.8 (4.0)	19.2 (4.5)	0.001*	0.07	0.002*	0.043*
MOCA	20.3 (4.3)	18.0 (5.6)	9.3 (1.2)	0.001*	0.14	0.001*	0.01*
IADL	23.8 (0.6)	23.8 (0.8)	22.0 (3.4)	0.001*	1.00	0.001*	0.001*
ADL	20.0 (0.3)	20.0 (0.0)	19.7 (0.5)	0.12	1.00	0.26	0.13

One-way ANOVA (Bonferroni post hoc test).

Significant p value at <0.05.

Discussion

The CDR scale is a reliable and valid informant-based global assessment method that has been successfully employed in many dementia studies. In prior research, CDR displayed a good inter-rater reliability and was highly correlated with objective measures of cognitive performance on the MMSE, the Abbreviated Mental Test, and detailed psychometric tests [9,18]. A valid CDR assessment relies on the availability of reliable collateral sources providing information about the patient's current level of cognitive functioning relative to previous levels. Usually a spouse, adult children, or other close relatives such as a daughter-in-law assume a caregiving role or act as domestic helpers who look after the infirmed elderly subjects. This is becoming more and more difficult given the increasing proportion of elderly individuals who live alone, which is high (over 40%) in most countries in Europe and the USA [19]. Although the percentages are presently lower in most Asian countries [20], they will become a major issue as traditional multitier families are becoming less common. In Singapore (resident population, 5 million), the proportion of elderly subjects living alone increased from 8.2% in 2000 to 12.2% in 2010 (Department of Statistics) and is expected to be more than doubled by 2030. In this study, approximately one quarter of the elderly subjects with low cognitive functioning were without an informant for CDR assessment.

The research literature is very limited on available resources for cognitive assessment using CDR when there is no reliable informant. In this study, we applied the novel approach of obtaining collateral information for CDR assessment through the interviewer, who was surrogate informant, observer, and rater at once. Our results indicated that CDR-NI displayed a good inter-rater, test-retest, and internal consistency. It also showed a good clinical validity in reference to the clinical assessment of dementia using the DSM-IV criteria and for determining the severity of cognitive and functional impairment. The inter-rater reliability was lowest in the domain categories of judgment and problem solving and community affairs (0.77 and 0.79, respectively). This was not surprising, since these domains test higher cognitive function and necessitate greater judgmental skills on the part of the assessor.

The quality and accuracy of collateral source information from traditional informants as the basis for CDR is open to question, as it may be affected by various informant characteristics. Studies suggest that factors such as the type of relationship to the patient, frequency of contact, common/separate domicile, age, education, the informant's mental health, and the quality of the informant-patient relationship predict the quality of collateral source information [1]. In our study, it was common that the subjects' informants, who were mostly their spouses and close friends, expressed little confidence in their ability to recall events, behavior patterns, and provide accurate information. A semi-structured interview format in the CDR assessment permits flexibility in assessing changes and takes the subjects' prior and accustomed level of functioning in daily tasks into account. A major caveat is that clinical judgment is required for the interviewer to critically appraise the informant account and assign CDR scores. The determination of impaired functioning needs to consider prevailing societal norms and perception of normality. For example, the CDR rater needs to be mindful of the tendency in many traditional societies to downplay cognitive symptoms out of respect for the elderly or the perception of forgetfulness as a reflection of normal aging. This may contribute to response bias and inaccurate information on the part of the informant. For these reasons, some authors have taken the approach of corroborating collateral information with direct observations of the individuals' performance in their everyday activities [21]. The CDR-NI approach is quite similar to this as clinically trained nurses assess the subjects by observing their daily living activities at home and in their neighborhood. Without recourse to an informant's reports, CDR-NI could arguably assess impaired cognitive functioning in some cases with higher accuracy. However, the possible limitations are that the frequency of nurse-informant contacts is limited compared to the traditional approach, and the reference period for assessment is only 1 week. On the other hand, this may be compensated by the fact that given their prior training and experience in CDR assessment, nurses are likely to generate more meaningful and accurate information when these are corroborated by their personal observations. Compared to the established CDR assessment with an informant, the CDR-NI entails a larger expense of time and resources to obtain a cognitive assessment, but this should be weighed against the loss of information and missing data whenever CDR is not performed because no traditional informant is available.

A few limitations of this study should be noted. The number of dementia cases was small, and they were all mild cases identified in the community setting. Hence, the sample size was inadequate for demonstrating statistical significance for the observed differences between the MCI and mild dementia groups. The validity and reliability of the CDR-NI scale in clinic and institution samples of subjects with dementia is unknown and has to be evaluated in future studies.

In conclusion, we found that for community-living elderly subjects without an informant, the CDR scale can be used to assess cognitive and functional impairment through a modified protocol of interviews, assessments, and structured observations gathered by trained nurse interviewers during two visits. The CDR-NI scale provided valid and reliable assessment of MCI and dementia in this study. Future research should be conducted in other community, clinic, and institutional populations to further substantiate the use of the CDR-NI scale for dementia assessment in elderly subjects without an informant.

Disclosure Statement

The authors declare no conflicts of interest in relation to the current study.

Domains	Informant-based CDR	Modified CDR without informant
Memory	A series of questions are asked to the informant about his/her awareness of the subject's memory impairment. Personal information on the subject such as place and date of birth, name and place of school, type of last job or spouse's last job is collected by the informant. To assess the subject's past and present memory recall, the informant is asked about details of recent events or social activities he/she performed together with the subject within the past week and within 1 month. All information collected by the informant is used to rate the severity of the subject's memory decline after the interview.	First visit 1 Ask memory questions  Where and when was he/she born?  Which was the last school he/she attended?  What was his/her or the spouse's last job?  When did he/she retire and why? 2 Memory of recent events  Talk about newspaper articles/magazines/television programs and dramas. Talk about current events such as gathering, outing, meeting with friends and relatives, or visit places like the market, food court, senior activity center, etc.  Talk about social events such as weddings, funerals, birthdays, recent journeys, or religious occasions.  Ask for details such as the location, date, time, and duration of events, how the subject got there, whom he/she met, what it was like, etc. 3 Observe behavior and attention  Assess the behavior of the subject, that is whether he/she looks confused, irritated, depressed, and anxious or whether he/she pays attention to the interview and understands the conversation. Second visit (1 week later) 1 Check recent memory Ask him/her to recall the name and institution of the interviewer and the purpose of the interview. Ask about the events and social activities that were discussed the week before.
Orientation	The informant is asked to rate the orientation of the subject to date, month, year, and day of the week as well as his/her familiarity with his/her home and places in the neighborhood. The subject is asked questions about his/her orientation to date, day of the week, month, time, place, and person during the interview.	First visit 1 Check his/her orientation by asking about the date, day of the week, time, and location. 2 Ask for the name of the most prominent person of the country. 3 Ask for places outside the neighborhood where he/she can go alone with public transportation and ask him/her to describe how he/she gets there.
Personal care	The informant is asked to rate the subject's mental ability to perform/maintain personal care such as dressing, washing, grooming, eating habits, and sphincter control (urination and defecation).	First visit 1 Observe whether the subject is dressed appropriately. 2 Observe the appearance of the subject (clean and tidy clothes). 3 Check whether there is any evidence of incontinence (urine, feces). Second visit 1 Observe how the subject uses utensils for meals and whether he/she is messy while eating.
Home care	The changes of the subject's performance in household chores and leisure activities are assessed by the informant. The informant is also asked to rate the level of the subject's ability to perform everyday activities at the usual level.	First visit 1 Inspect the house (living room, toilet, and kitchen) for cleanliness and arrangement of furniture. 2 Pay attention to the proper organization of household items. 3 Observe the safety in the house (use of gas stove, electric power plugs, burned pots and pans, etc.). 4 Observe whether he/she remembers to lock the main door and bring the keys whenever he/she goes out. Second visit 1 Observe the way the subject prepares meals (special meals or simple meals).
Social and community level	Information related to the subject's social functioning in the community such as mobility outside the home and shopping independently, attending social activities, and cultivating social relationships with friends is collected from the informant.	Second visit 1 Ask the subject to make a phone call to a friend or a relative. 2 Ask the subject to tell you about the current news or stories of television dramas. 3 Specify three items and ask the subject to take you to a nearby shop or market to buy them. 4 Assess whether he/she recalls the items. 5 Ask the subject to take you to the nearest bus stop to get back to his/her house. 6 Pay attention to the subject's awareness of road safety and observe whether he/she respects the traffic lights to cross the road.
Judgment and problem solving	The informant is asked to rate the ability of the subject to solve general problems, handle money, business transactions, and urgent household problems, and to understand situations and explanations. During the interview with the subject, questions related to similarities and differences between two objects are tested to assess the understanding of the subject.	Second visit 1 Observe whether the subject knows the exact amount of money change when he/she buys the things you asked him/her to buy. 2 Ask the subject what he/she would do if he/she lost the key.
Scoring	The subject is asked to perform simple calculations and problemsolving tasks to judge his/her ability to handle money and household problems. Based on the informant's information, judgment, subjective response as well as the subject's performance of memory, orientation, and judgment tasks, all six domains are scored by the CDR assessor.	Second visit 1 Based on the subject's response to memory, his/her orientation, judgment, accomplishment of basic everyday activities at home/in the neighborhood, road and home safety, the interviewer assigns scores for the six domains.