Evaluation of Activities of Daily Living/Instrumental Activities of Daily Living to Accurately Determine Severity of Moderate and Severe Alzheimer's Disease: Comparison of Assessments by Receiver Operating Characteristic Curve and Discriminant Analyses.

AIM: The aim of this study was to determine Activities of Daily Living/Instrumental Activities of Daily Living (ADL/IADL) evaluations that will enable better understanding of the severity of Alzheimer's disease (AD).
METHODS: AD patients were evaluated by Functional Independence Measure (FIM), Hyogo Activities of Daily Living Scale (HADLS), and Assessment of Motor and Process Skills (AMPS) to identify the assessments that will enable highly precise discrimination of AD Clinical Dementia Rating (CDR) 2 (moderate) and CDR3 (severe) using receiver operating characteristic (ROC) curve and discriminant analyses.
RESULTS: The participants were 75 AD patients (CDR2 = 50, mean age = 80.3 ± 5.9 years; CDR3 = 25, mean age = 78.3 ± 9.0 years). The evaluation methods consisted of FIM, HADLS, and AMPS. The results were divided into FIM-M, FIM-C, HADLS-ADL, HADLS-IADL, AMPS-motor skills, and AMPS-process skills. The values for the area under the curve (AUC) were compared by ROC curve and discriminant analyses. AUC values for FIM-C and AMPS-process skills were 0.956 and 0.947, respectively. With these two evaluations only, values ≥0.9 were shown. Moreover, the AUC of the discrimination score (combination of the FIM-C and AMPS-process skills) was significantly higher than those for FIM-M, FIM-C, HADLS-ADL, HADLS-IADL, and AMPS-motor skills.
CONCLUSIONS: The results demonstrated that evaluation by FIM-C and AMPS-process skills methods was useful for discriminating between CDR2 (moderate) and CDR3 (severe) AD. Moreover, the results indicated that these two evaluation methods enabled more accurate determination of severity and the spared capabilities of AD patients.

Introduction

For effective occupational therapy of Alzheimer's disease (AD) patients, abilities to perform Activities of Daily Living (ADL), Instrumental Activities of Daily Living (IADL), and work performance need to be evaluated and the resulting information directly connected to an occupational therapy plan. Because patients with moderate and severe AD are assisted through many ADL and IADL programs, ADL and IADL evaluations that enable proper discrimination of the severity of AD are important.

Clinical Dementia Rating (CDR) is an evaluation method commonly used worldwide [1]. Beyond using CDR to obtain information linked to an AD occupational therapy program, ADL/IADL performance and work performing skills of AD patients need to be directly evaluated.

In CDR, the “home and hobbies” section best shows the difference between moderate and severe AD [1]. A CDR2 (moderate) rating of this section indicates AD patients who can perform simple household chores, whereas a CDR3 (severe) rating indicates those who can hardly perform household chores [1, 2]. However, among AD patients who require assistance with ADL, such as dressing and toileting, some can perform some IADL. For example, we have worked with many severe AD patients who found it difficult to understand movement directives and required guidance for all ADL but could slowly fold a towel or apron and place it properly without guidance. Although a lack of memory regarding when, where, and what is common in AD patients [3], their procedural memory is often relatively spared [4]. Even severe CDR3 patients are known to retain the ability to perform activities they were engaged in for many years as lifestyle habits. Understanding this reality is important for an occupational therapist so that this spared capability is not overlooked.

Methods used to evaluate ADL and IADL performance include Functional Independence Measure (FIM) [5], Hyogo Activities of Daily Living Scale (HADLS) [6], and Assessment of Motor and Process Skills (AMPS) [7, 8]. However, these methods have various advantages and disadvantages when evaluating patients with AD. With FIM, the level of self-reliance for individual ADL items (such as eating and grooming) can be determined quantitatively, based on the amount of assistance required. Moreover, it can be used to evaluate cognitive functions, including communication ability. However, FIM is not effective in evaluating a patient's ability to perform IADL, even for severe AD patients, or the range of skills that enable independent performance of ADL. HADLS was developed as an evaluation method specifically for AD patients, and when used with ADL, even IADL can be evaluated. It is also clinically effective in severe AD patients [9]. However, as this system is usually performed via an interview, providers of reliable information are required, which can be a severe disadvantage in the case of AD patients. Fortunately, IADL that cannot be addressed by FIM can be evaluated with AMPS, wherein common ADL and IADL problems in daily living can be selected and evaluated; therefore, the spared skills of the patients can be evaluated in detail by procedural memory. Unfortunately, AMPS cannot evaluate communication performance or the level of independence and the amount of assistance required for various ADL and IADL. As such, we aimed to clarify which method or combination of methods using FIM, HADLS, and AMPS would best enable the discrimination between CDR2 (moderate) and CDR3 (severe) AD.

To determine whether the Five Cognitive Tests (memory, attention, language, visuospatial function, and reasoning) can be applied for screening mild dementia, Sugiyama et al. [10] evaluated CDR0 (none) and CDR0.5 (questionable) elderly patients using the Five Cognitive Tests with the Mini-Mental State Examination (MMSE). They also calculated the area under the curve (AUC), cutoff values, sensitivity, and specificity using a receiver operating characteristic (ROC) curve analysis. The results indicated that, compared with MMSE, CDR0 (none) and CDR0.5 (questionable) groups could be discriminated with high precision using the Five Cognitive Tests.

Based on these previous findings, we used the method by Sugiyama et al. [10] as reference and aimed to determine evaluations or evaluation combinations of FIM, HADLS, and AMPS that will enable precise discrimination between CDR2 and CDR3 AD using ROC curve and discriminant analyses. A successful evaluation should measure the severity of CDR2 and CDR3 AD and the differences in the spared capabilities of the patients. To proceed with this mission, we categorized the evaluation methods into FIM-motor skills (FIM-M), FIM-cognitive skills (FIM-C), HADLS-ADL, HADLS-IADL, AMPS-motor skills, and AMPS-process skills. Finding the most effective combination of these methods would better enable occupational therapists to evaluate ADL and IADL in detail and therefore better design highly productive occupational therapy programs.

Materials and Methods

Participants

The participants were 75 AD patients currently hospitalized in a dementia ward. The participants were patients diagnosed with AD by a physician based on the tenth edition of the International Classification of Diseases (ICD-10). Of the participants, 50 were CDR2 (moderate) and 25 were CDR3 (severe). Patients who could not follow verbal instructions, with clearly severe physical illness or unstable chronic disease, and with serious visual disorders were excluded (Table 1).

Table 1

Participants' characteristics and evaluation results

n (CDR2/CDR3)	75 (50/25)
Men/women,n	38/37
Age, years (range)	79.6±7.1 (60–96)
Years of education after graduation from elementary school (range) Complications including overlapping,n	5.7±3.2 (0–11)
Cerebrovascular diseases	57
Diseases of internal medicine	60
Diseases of orthopedics	21
Parkinson's syndrome	7
MMSE, score	10.1±5.9
FIM-Total, score	81.1±21.3
FIM-M, score	64.1±17.6
FIM-C, score	17.0±5.3
HADLS-Total, score	54.91±19.77
HADLS-ADL, score	23.09±15.09
HADLS-IADL, score	31.83±6.27
AMPS-motor skills, logits	1.64±0.68
AMPS-process skills, logits	0.30±0.72

This study was approved by the ethics committee of Juntendo Tokyo Koto Geriatric Medical Center (approval number 75-1) and the ethical review board of the Faculty of Health Sciences, Kyorin University (approval number 26-49). Written informed consent was obtained from all subjects or their families for publishing the data used in this study.

Evaluation

The evaluation items consisted of CDR [2], FIM [5], HADLS [6], and AMPS [7, 8].

CDR [2] was assessed by evaluating memory, orientation, judgment and problem solving, community affairs, home and hobbies, and personal care, each on a 5-point scale and based on the clinical information and behavioral observation of the patients. CDR consisted of a simple set of criteria for comprehensive evaluation of severity of dementia. The levels of severity were CDR0 (none), CDR0.5 (questionable), CDR1 (mild), CDR2 (moderate), and CDR3 (severe).

The evaluation items in FIM [5] consisted of two major items, namely, FIM-M (13 items) and FIM-C (5 items). Depending on the level of independence and level of assistance needed, each item was scored on a 7-point scale. The lower the score was, the larger was the amount of assistance required, thus indicating a low level of independence. Maximum and minimum scores in FIM-M were 91 and 13 points, respectively, while those in FIM-C were 35 and 5 points, respectively. The maximum and minimum scores for FIM-Total were 126 and 18 points, respectively.

HADLS [6] is the overall impairment scale for ADL and IADL in AD patients. In the evaluation based on caretaker interview, 18 of the items on daily living activities were evaluated on a 3- to 7-point scale. In addition, of the lower-ranking items of HADLS, the total score for the 8 items on ADL (toileting, eating, dressing, grooming, wash, tooth brushing/wash dentures, bathing, and locomotion) was set as “HADLS-ADL,” and that for the 10 items on IADL (using a telephone, shopping, cooking, cleaning up, bed making, cleaning up after eating, laundry, using fire, using switches, and financial management) was set as “HADLS-IADL”; both were used as objects of the analysis.

AMPS [7, 8] measures the minimum units of behavior, namely, motor skills and process skills, and is said to evaluate the qualitative aspects of ADL. Motor skills are required to perform specific body movements in order to accomplish a task. Process skills are required to adapt one's actions according to the situation when performing a task. In the evaluation, participants select two tasks that match their interest and ability from a list of 125 tasks standardized by their level of difficulty. The participants are then observed as they perform those tasks. The 16 items on motor skills and 20 items on process skills are each evaluated using a 4-point scale in accordance with the capability of the patient. The measured values for motor and process skills are calculated from the evaluation results by interval scale (logits).

Procedure

All evaluations by CDR, FIM, HADLS, and AMPS were performed within 1 week of hospitalization for each patient.

CDR was evaluated after the first author had a discussion with the attending nurse. FIM was evaluated by the first author based on observations of the patients' behavior in the ward and information on their living conditions from the attending nurse. HADLS was evaluated by the first author based on an interview with live-in relatives or persons familiar with the patients' living conditions immediately before hospitalization. AMPS was also evaluated by the first author.

Data Analysis

ROC Curves for FIM, HADLS, and AMPS

The correlations between CDR and FIM-M, FIM-C, HADLS-ADL, HADLS-IADL, AMPS-motor skills, and AMPS-process skills were calculated; a significant correlation was revealed for all (rs: −0.235 to −0.855). The ROC curves were prepared for all six evaluation methods that correlated significantly with CDR in order to discriminate CDR2 and CDR3 AD, and AUC values were calculated to investigate the cutoff values, sensitivity, and specificity.

Investigation of Discrimination Precision between CDR2 and CDR3

To determine the combinations of the ADL and IADL evaluation methods (FIM-M, FIM-C, HADLS-ADL, HADLS-IADL, AMPS-motor skills, and AMPS-process skills) that enable a precise discernment between severities of CDR2 and CDR3 AD, discriminant analysis (stepwise method) was performed using discrimination of CDR2 and CDR3 as the dependent variable, and FIM-M, FIM-C, HADLS-ADL, HADLS-IADL, AMPS-motor skills, and AMPS-process skills as the independent variables. Subsequently, ROC curves were constructed to confirm the precision of the evaluations that were selected based on linear discrimination and stepwise method in discriminating CDR2 and CDR3. The significant differences in the calculated AUC were subsequently investigated.

AUC comparisons were performed using EZR version 1.32 (Saitama Medical Center, Jichi Medical University, Saitama, Japan) [11], which is a graphical user interface for the R Foundation for Statistical Computing (R version 2.13.0). SPSS Statistics (Ver. 21.0, IBM Corporation, Armonk, USA) was used for all the other analyses. All p values were two sided and p values less than 0.05 were considered statistically significant.

Results

ROC Curves of FIM, HADLS, and AMPS

AUC values were estimated from the ROC curves that were constructed for the discrimination of CDR2 and CDR3 AD. With FIM-M, AUC was 0.783 (95% confidence interval [CI]: 0.627–0.893, p = 0.000); at the cutoff value of 62.0 points, sensitivity and specificity were 72.0 and 68.0%, respectively. With FIM-C, AUC was 0.956 (95% CI: 0.909–1.000, p = 0.000); at the cutoff value of 16.0 points, sensitivity and specificity were 88.0 and 88.0%, respectively. With HADLS-ADL, AUC was 0.846 (95% CI: 0.749–0.943, p = 0.000); at the cutoff value of 22.40 points, sensitivity and specificity were 74.0 and 80.0%, respectively. With HADLS-IADL, AUC was 0.820 (95% CI: 0.719–0.922, p = 0.000); at the cutoff value of 35.00 points, sensitivity and specificity were 74.0 and 84.0%, respectively. With AMPS-motor skills, AUC was 0.574 (95% CI: 0.438–0.711, p = 0.149); at the cutoff value of 1.43 logits, sensitivity and specificity were 74.0 and 48.0%, respectively. With AMPS-process skills, AUC was 0.947 (95% CI: 0.900–0.994, p = 0.000); at the cutoff value of 0.04 logits, sensitivity and specificity were 94.0 and 88.0%, respectively (Table 2, Fig. 1).

Table 2

AUC, cutoff value, sensitivity, and specificity in the various evaluations

	AUC	p	95% confidence interval		Cutoff value	Sensitivity, %	Specificity, %
			lower limit	upper limit
FIM-M	0.783	0.000	0.627	0.893	62.0	72.0	68.0
FIM-C	0.956	0.000	0.909	1.000	16.0	88.0	88.0
HADLS-ADL	0.846	0.000	0.749	0.943	22.40	74.0	80.0
HADLS-IADL	0.820	0.000	0.719	0.922	35.00	74.0	84.0
AMPS-motor skills	0.574	0.149	0.438	0.711	1.43	74.0	48.0
AMPS-process skills	0.947	0.000	0.900	0.994	0.04	94.0	88.0

AMPS, Assessment of Motor and Process Skills; AUC, area under the curve; FIM-C, Functional Independence Measure Cognition; FIM-M, Functional Independence Measure Motor; HADLS-ADL, Hyogo Activities of Daily Living Scale Activities of Daily Living; HADLS-IADL, Hyogo Activities of Daily Living Scale Instrumental Activities of Daily Living.

Fig. 1

Receiver operating characteristic curves of FIM-M, FIM-C, HADLS-ADL, HADLS-IADL, AMPS-motor skills, AMPS-process skills, and discrimination score. FIM-M (p = 0.000), FIM-C (p = 0.049), HADLS-ADL (p = 0.004), HADLS-IADL (p = 0.001), and AMPS-motor skills (p = 0.000) were significantly different from the discrimination score. AMPS, Assessment of Motor and Process Skills; FIM-C, Functional Independence Measure Cognition; FIM-M, Functional Independence Measure Motor; HADLS-ADL, Hyogo Activities of Daily Living Scale Activities of Daily Living; HADLS-IADL, Hyogo Activities of Daily Living Scale Instrumental Activities of Daily Living.

Discrimination Precision for CDR2 and CDR3

Discriminant Analysis and ROC Curves of the Discrimination Scores

Discriminant analysis was performed with discrimination of CDR2 and CDR3 as the dependent variables, and FIM-M, FIM-C, HADLS-ADL, HADLS-IADL, AMPS-motor skills, and AMPS-process skills as the independent variables. The results revealed that when FIM-C and AMPS-process skills were selected, the linear discriminant was 0.213 × FIM-C + 0.959 × AMPS-process skills − 3.911. Wilk's λ (random) was 0.343 (p = 0.000) and the true discrimination rate was 90.7% (Table 3).

Table 3

Results of the discrimination analysis

	Standardized canonical discriminant function coefficient	Canonical discriminant function coefficient	p value
FIM-C	0.723	0.213	0.000
AMPS-process skills	0.486	0.959	0.000

At Wilks's λ of 0.343 (p = 0.000), canonical rate was 90.7%, linear discriminant (0.213 × FIM-C + 0.959 × AMPS-process skills − 3.911). FIM-C, Functional Independence Measure Cognition; AMPS, Assessment of Motor and Process Skills.

When the ROC curve for discriminating CDR2 and CDR3 was constructed using the discriminant scores obtained from the linear discriminant (0.213 × FIM-C + 0.959 × AMPS-process skills − 3.911), AUC was 0.985 (95% CI: 0.963–1.000, p = 0.000), and sensitivity and specificity were 96.0 and 92.0%, respectively, at the cutoff value of −0.604 (Fig. 1).

Comparison of AUC Values

Comparison of AUC values of two items revealed that FIM-M (p = 0.002), HADLS-ADL (p = 0.019), HADLS-IADL (p = 0.007), and AMPS-motor skills (p = 0.000) were significantly different from FIM-C (AUC = 0.956). In addition, FIM-M (p = 0.004), HADLS-ADL (p = 0.029), HADLS-IADL (p = 0.008), and AMPS-motor skills (p = 0.000) were significantly different from AMPS-process skills (AUC = 0.947).

There was no significant difference between FIM-C (AUC = 0.956) and AMPS-process skills (AUC = 0.947, p = 0.803) and between the discrimination score (AUC = 0.985) and AMPS-process skills (AUC = 0.947, p = 0.119).

FIM-M (p = 0.000), FIM-C (p = 0.049), HADLS-ADL (p = 0.004), HADLS-IADL (p = 0.001), and AMPS-motor skills (p = 0.000) were significantly different from the discrimination score (AUC = 0.992) (Table 4, Fig. 1).

Table 4

Comparison of AUC values in the various evaluations

	FIM-C	AMPS-process skills	Discrimination score
FIM-M	0.0021	0.004**	0.000**
FIM-C	−	0.803	0.0491
HADLS-ADL	0.019*	0.029*	0.004**
HADLS-IADL	0.007*	0.008**	0.001**
AMPS-motor skills	0.000**	0.000**	0.000**
AMPS-process skills	0.803	−	0.119

p < 0.05

p < 0.01.

AMPS, Assessment of Motor and Process Skills; AUC, area under the curve; FIM-C, Functional Independence Measure Cognition; FIM-M, Functional Independence Measure Motor; HADLS-ADL, Hyogo Activities of Daily Living Scale Activities of Daily Living; HADLS-IADL, Hyogo Activities of Daily Living Scale Instrumental Activities of Daily Living.

Discussion

To determine which evaluation will enable the most precise and robust discrimination between CDR2 and CDR3 AD, we plotted ROC curves of these evaluations to calculate AUC values and investigate the related cutoff values, sensitivity, and specificity.

We found that AUC values for process skills determined by FIM-C and AMPS were 0.956 and 0.947, respectively, indicating values of ≥0.9 only for these two evaluations. Swets [12] previously reported prediction rates obtained using AUC; the accuracy was judged as high, moderate, and low for rates ≥0.9, 0.7–0.9, and 0.5–0.7, respectively. Based on these findings, our AUC results indicated that only FIM-C and AMPS-process skills had a high accuracy in effectively discriminating between CDR2 and CDR3 AD. In other words, it became evident that this differentiation can be accurately determined using appropriate methods. To our knowledge, this is the first study to report such findings.

Based on the results, FIM-C, which is strongly affected by cognitive functions, was strongly correlated with MMSE (rs = 0.843, p < 0.01) and CDR (rs = −0.909, p < 0.01), suggesting severe dementia. CDR evaluates memory, orientation, judgment and problem solving, community affairs, home and hobbies, and personal care, each on a 5-point scale and based on clinical information and patients' observed behavior. Severity of dementia is comprehensively judged and strongly reflects diminished cognitive function. These findings suggested that FIM-C enables accurate determination of differences in severity between CDR2 and CDR3 AD.

Moreover, communication ability, i.e., understanding based on hearing, sight, and verbal and nonverbal expressions, is included in the FIM-C evaluation. Decreased communication ability in patients with AD manifests consistently as AD advances [13], and the decrease becomes profound from CDR2 onwards [4]. However, in a clinical setting, even among CDR2 and CDR3 AD patients, many can still effectively express their intentions, desires, and feelings and some can understand the nonverbal expressions of the occupational therapist. Indeed, in clinical settings, the range of individual differences in communication abilities across CDR2 and CDR3 AD patients can be quite remarkable.

Although communication ability is not among the CDR evaluation items, it was involved in the results of evaluation items such as judgment power, problem solving, social adjustment, and understanding family situations. Therefore, high AUC value of FIM-C for discriminating between CDR2 and CDR3 AD was likely a major contributing factor in enabling the evaluation of communication ability by FIM-C.

Investigating the Precision of Discriminating between CDR2 and CDR3

Discriminant analysis was performed with CDR2 and CDR3 as dependent variables and FIM-M, FIM-C, HADLS-ADL, HADLS-IADL, AMPS-motor skills, and AMPS-process skills as independent variables. Based on the results, FIM-C and AMPS-process skills were selected as evaluations. Moreover, comparison of the AUC values of both methods revealed that FIM-C differed significantly from FIM-M, HADLS-ADL, HADLS-IADL, and AMPS-motor skills. In addition, AMPS-process skills differed significantly from FIM-M, HADLS-ADL, HADLS-IADL, and AMPS-motor skills. This demonstrated that FIM-C and AMPS-process skills are highly precise ADL/IADL evaluations for discriminating between CDR2 and CDR3 AD.

Comparison of AUC values also revealed significant differences in the discrimination score (FIM-C and AMPS-process skills combined) among FIM-M, FIM-C, HADLS-ADL, HADLS-IADL, and AMPS-motor skills. This confirmed that FIM-C and AMPS-process skills combination is a highly precise approach for discriminating between CDR2 and CDR3 AD. In particular, as there were significant differences between the discrimination score and FIM-C, the FIM-C plus AMPS-process skills combination, compared with FIM-C alone, could increase the discrimination score and improve the possibility of better discrimination between the severity of CDR2 and CDR3 AD. These results indicate that even with a marked decrease in cognitive function, as evidenced by FIM-C results, the evaluation of ADL/IADL by AMPS-process skills enables the identification of various spared abilities of the patients. Thus, evaluation of patients with CDR2 and CDR3 AD using FIM-C and AMPS-process skills combination provides useful information for establishing occupational therapy programs that maximize the use of intact skills in daily living activities in AD patients.

AMPS-process skills enables the identification of current capabilities that may become problems in daily living, as well as intact skills that are strongly influenced by cognitive capabilities. However, AMPS-process skills do not address communication ability, which is markedly decreased in patients with moderate to advanced AD. In contrast, FIM-C assesses both cognitive function and communication ability. Thus, the combination of these two methods enables more accurate discrimination between CDR2 and CDR3 AD.

Disclosure Statement

The authors declare no conflict of interest.