A Direct Performance Test for Assessing Activities of Daily Living in Patients with Mild Degenerative Dementia: The Development of the ETAM and Preliminary Results.

BACKGROUND: There are currently only a few performance tests that assess the capacity to perform activities of daily living. These measures frequently require a long time to administer, are strongly cognition oriented, or have not been adequately validated.
METHODS: The Erlangen Test of Activities of Daily Living in Mild Dementia (ETAM) was developed in a 4-phase process that was based on the International Classification of Functioning, Disability, and Health (ICF). A pilot study was conducted on 30 subjects with mild dementia with a mean age of 80 years. The subjects' mean score on the MMSE was 21.5. Twenty-one of the 30 subjects were women.
RESULTS: Ten items were developed and tested in the pilot study. The mean time required to complete the test was 26 min. The item analysis showed difficulties that ranged primarily from r = 0.28 to r = 0.79. The ETAM had a moderate correlation with the MMSE (rMMSE = 0.310) and a low correlation with the Geriatric Depression Scale-15 (GDS-15; rGDS-15 = 0.149).
CONCLUSION: The preliminary version of the ETAM is quick and easy to use and has predominantly satisfactory item characteristics. There still is the need to revise the items 'giving directions' and 'making tea' with regard to standardisation.

Introduction

There is ample scientific evidence that people with mild dementia have limited abilities to perform complex activities of daily living. Thus, Pérès et al. [1] demonstrated in their prospective study that initial impairments in the performance of more demanding activities of daily living are detectable as early as 10 years before the first clinical diagnosis of dementia. According to Muò et al. [2], the International Classification of Functioning, Disability, and Health (ICF) domains of mobility, handling of finances (economic life), domestic life, communication, and self-care are especially important for enabling elderly persons to maintain their autonomy when living at home. Handling finances and taking medication, in particular, already show deficits in the initial stages of dementia. Due to their sensitivity, they are important indicators of the capacity to perform activities of daily living [3]. The behavioural domain of mobility has also proven to be susceptible to impairment due to its dependency on the quality of executive functions in the early stages of dementia. Mobility has been shown to be an important predictor of not only the autonomy of dementia sufferers [4] but also their risk of comorbid physical conditions such as obesity and diabetes. However, for the family caregivers of persons with mild dementia, the greatest strain arises from impairments in the area of communication [5]. Caregivers experience deficits in this domain as a personal loss [5]. Contrary to widespread beliefs, Jefferson et al. [6] reported that the domain of domestic life is affected early in the development of dementia. This is mainly due to the fact that complex executive functions are required to perform activities in the household, and these are already impaired in the early stages of dementia. This domain is also particularly important because elderly persons spend roughly one third of their day performing household tasks [7].

Lawton and Brody [8] were the first to operationalise abilities to perform activities of daily living in dementia in 1969. In addition to identifying the basic activities of daily living (ADL) such as personal hygiene and the preparation of food, they also postulated the existence of more complex instrumental activities of daily living (IADL). The tasks falling into this category, such as taking medication and using public transport, vary widely and differ from the ADLs with respect to the high demand they place on cognitive abilities. However, an increasing number of authors are calling for the dualism of the ADL and IADL concepts to be dropped as they see activities of daily living as falling along a continuum [9,10]. This is supported by the results of a study by Spector and Fleishman [9], who applied factor analysis to demonstrate that the great majority of the activities included in the ADL and IADL items load on the same construct.

In 2006, Attix and Welsh-Bohmer [10] reported the existence of at least 40 different tests for assessing activities of daily living in patients with dementia, most of which were self- and informant-rating measures. The wide use of self- and informant-based measures is mainly due to the fact that they are easy and quick to administer [10]. Rating scales also have the advantage that respondents can base their ratings on a broad spectrum of experiences. The assessments therefore usually remain more stable over time and across situations than the results of performance tests. However, persons with dementia themselves tend to underestimate their own deficits in accomplishing the tasks of daily living [11,12,13]. The use of self-rating assessment measures to assess abilities to perform activities of daily living is therefore problematic and associated with a low level of validity. Similar to the persons with dementia themselves, their relatives tend to overestimate their capacity to perform the tasks of daily living, particularly in mild dementia [14,15]. However, this phenomenon is strongly dependent on the complexity [16] and observability [17,18] of the respective ability. In addition, of the numerous available informant-based measures, only the Bayer Activities of Daily Living Scale (B-ADL) [19] is specifically tailored towards identifying deficits in daily living activities in individuals with mild dementia.

The greatest disadvantage of informant-based assessment measures is that they require an informant who is well acquainted with the person in question as well as with his/her daily life. This informant will therefore also be aware of the treatment that his or her relative is undergoing. As a result, interventions that cannot be blinded – as is true of most non-pharmacological therapies – are associated with a considerable amount of assessment bias, which affects the validity of the results. But with the use of performance-based assessment procedures, an independent external rater who does not know the patient or his/her life can easily be blinded to the study conditions.

However, the few performance tests available to date have major limitations with regard to the time required for and the ease of administration. One recently developed measure in this field is the Functional Living Skills Assessment (FLSA) [20]. This test is specifically designed for outpatients who are in mild-to-moderate phases of dementia. The FLSA measures the 8 behavioural domains of resources, consumer skills, public transportation, time management, money management, leisure, telephone skills, and self-care and health; its psychometric properties – sensitivity and specificity – can be considered satisfactory [20]. However, healthy subjects take a mean of 45 min to complete the test, whereas dementia patients take over 1 h. Similar problems are also associated with other performance tests such as the Direct Assessment of Functional Abilities (DAFA) [13], which takes 1.5 h to complete, and the Independent Living Scales (ILS) [21], which take 45 min.

The results of the validation studies are also usually based on very small and selective samples, e.g. 18 dementia patients for the Structured Assessment of Independent Living Skills (SAILS) [22] and 28 dementia patients for the DAFA. One exception is the Test of Everyday Functional Abilities (TEFA) [23], which is easy and quick enough to administer but shows a correlation of r = 0.9 with the MMSE. Therefore, it does not discriminate sufficiently between abilities to perform activities of daily living and cognitive abilities. Some other measures such as the Direct Assessment of Functional Status (DAFS) [24] and the TEFA have been found to have ceiling effects and thus do not adequately differentiate the performance of patients with mild dementia from that of normal controls. Therefore, they are not suitable for use in this area [25,26]. A further weakness of existing performance tests is that they frequently fail to cover all relevant domains of activities of daily living. An example of this is the DAFS, which does not include the taking of medication. Other performance tests are tailored specifically to the measurement of only individual activities of daily living, e.g. the Financial Capacity Instrument (FCI) [27], which assesses only the handling of finances. This explains why these measures are seldom employed in everyday clinical routines [26]. And yet it would be of great therapeutic relevance to be able to diagnose deficits in activities of daily living early in mild dementia, since the greatest treatment effects can be achieved at this early stage of the disease [28,29] and institutionalisation could be postponed by reducing the caregiver burden [30].

In response to this situation, Graessel et al. [29] developed a performance test for measuring abilities in daily living activities in dementia patients, i.e. the Erlangen Test of Activities of Daily Living (E-ADL). With an average administration time of 8 min, the E-ADL is fast and easy to administer. It consists of 5 items that cover the behavioural domains of preparing food, personal hygiene, and other areas. The items are scored for each step of an action. If the task is completed, the subject receives 6 points per item. If the performance is deficient or absent, the score is correspondingly reduced. The maximum possible score on the E-ADL is therefore 30 points. Overall, the psychometric properties of the measure proved to be good in both the first and the revalidation [31] studies, with Cronbach's α between 0.68 and 0.77 and a 14-day test-retest reliability of rs = 0.73. The construct validity of the measure can also be considered good. It was computed as the correlations of the E-ADL score with both the score on the IADL subscale of the Nurses' Observation Scale for Geriatric Patients (NOSGER) [32] (r = −0.57) and the orientation/practice subscale of the Alzheimer's Disease Assessment Scale-cognitive subscale (ADAS cog) [33] (r = 0.64) [31]. However, in the revalidation study of 130 patients with dementia of all degrees of severity, the E-ADL also proved to be too simple for the domain of mild dementia [31]. The E-ADL is thus a valid and easy-to-administer instrument for measuring moderate and severe dementia, but it is not sensitive enough to measure mild dementia. Like the DAFS and TEFA, it failed to differentiate between the performance of persons who were minimally impaired and that of normal controls. It is thus necessary to develop more difficult items.

The aim of the present study was therefore to develop a performance test for abilities to accomplish tasks of daily living for patients with mild dementia and to present the initial results of the pilot study. The goal was to construct a valid test that would be faster to administer than existing instruments for measuring ADL abilities. The Erlangen Test of Activities of Daily Living in Mild Dementia (ETAM) was to be designed specifically for persons with mild dementia. Additional issues investigated in the pilot study were the gender specificity of the measure and the influence of respondents' educational levels on the results. With regard to the construct validity of the ETAM, interest was focussed on the correlation between performance on the ETAM and the results for the domains of cognition and depressiveness.

The theoretical basis for the development of the ETAM was the ICF, which was published by the World Health Organisation (WHO) in 2001 [34]. In addition to bodily functions and structures, the ICF also includes and systematically classifies contextual factors as well as the activities and participation of an individual. The WHO classifies activity as ‘the execution of a task or action by an individual’ (p. 123) and participation as ‘involvement in a life situation’ (p. 123). The domain entitled ‘activation and participation’ is differentiated into 9 main categories: (1) learning and applying knowledge, (2) general tasks and demands, (3) communication, (4) mobility, (5) self-care, (6) domestic life, (7) interpersonal interactions and relationships, (8) major life areas, and (9) community, social, and civic life. Of these 9 categories, communication, mobility, self-care, and domestic life are particularly relevant for the independent living of persons with dementia [2]; therefore, we operationalised each of these categories with at least 2 items.

Methods

Item Development

We employed a multi-method procedure to generate suitable items (fig. 1). In the first step, existing performance tests on abilities to carry out activities of daily living were sought with the aid of the literature databases PubMed and PsyCONTENT. The search results were analysed and evaluated on the basis of relevance, administrability, and the time required for administration.

Fig. 1

The 4-phase development process of the ETAM.

In the second step, 18 family caregivers of persons with dementia were interviewed in November/December 2011. In an open unstructured interview, the caregivers were asked to describe the problems that they had observed in their relatives with dementia in the execution of activities of daily living.

In the third step, 11 ICF-oriented items were developed on the basis of the first 2 steps. These were then presented to experts from dementia research (e.g. the head of an outpatient memory clinic) and care provision (e.g. the manager of a care institution), who were requested to make a written evaluation. The categories for the evaluation were ease and speed of administration, practical relevance, gender neutrality, logical order of the items, item difficulty, and feasibility of standardisation.

In the fourth step, the items were tested on 6 persons with dementia in order to develop graded assessment criteria for the individual items on the basis of different kinds of performance deficits (‘errors’).

Testing of the Items in a Pilot Study

The pilot study was carried out on 30 patients from 5 day care centres in Bavaria and Thüringen (Germany). The criteria for inclusion were the presence of mild dementia, operationalised by an MMSE score of between 18 and 23 or an MMSE score of >23 in combination with a score of <9 on the Demenz Detection Test (DemTect), and consent to participate in the study [35]. The criteria for exclusion were: (1) paralysis of the upper limbs, (2) a diagnosis that could explain the person's cognitive deficits as due to a disorder other than dementia (e.g. Korsakoff's syndrome), and (3) the existence of a legal guardian. Persons were recruited from 5 day care centres in the Southern and Eastern parts of Germany. To do so, a psychologist from the central study site visited the care centres for 3 days per centre and administered the MMSE to all persons who did not meet the exclusion criteria and who gave their consent to participate. After the inclusion of 30 persons, the recruitment was stopped. Of all the 40 persons screened, 4 persons could not be included in the study because their performance was above and 6 persons because their performance was below the inclusion criterion.

Suitable persons were informed in detail about the test and gave their written consent. The Ethics Committee of the Medical Faculty of the Friedrich-Alexander-Universität Erlangen-Nürnberg examined and approved the procedure (approval No. 233_13B9).

To calculate discriminant validity, the subjects' depressiveness was measured with the Geriatric Depression Scale-15 (GDS-15) [36]. To determine inter-rater reliability for about one third of the cases (n = 9), two investigators assessed the subjects' performances independently of each other. These 9 people attended the same day centre, which was chosen for the inter-rater tests due to its proximity to our research centre.

Data Analysis

Means and standard deviations (SDs) were calculated. The mean time required to administer the whole test was determined. Since the ETAM items have 7- and 4-point response formats (0-6 or 0-3), the ratio of the subject's sum of squared points to the sum of the squared item maximum

[37] was used as the difficulty index. The discriminatory power was calculated by determining the correlation of the item with the scale as a whole without the item in question (corrected item-total correlation) [38]. A Mann-Whitney U test was used to detect differences in the ETAM score according to the variables sex and education (high: >9 years; low: <10 years). As the sample contained significantly more women (n = 21) than men (n = 9), women of a comparable age were assigned to the 9 men. The resulting subsample was tested for age-adjusted sex differences with the aid of Mann-Whitney U tests. We used Spearman's rank-order correlation coefficient to calculate the association between the assessments as a measure of inter-rater reliability. We computed Cronbach's α as a measure of the internal consistency of the test. We tested the discriminant validity of the ETAM items by calculating their Pearson correlations with the MMSE and GDS-15. The model diagnostics revealed no deviation from a normal distribution (Kolmogorov-Smirnov test: p = 0.475). The data analysis was carried out with the Statistical Package for the Social Sciences (SPSS) version 19.

Results

Sample Statistics

The 30 elderly persons included in the study had a mean age of 80 years (SD ±8 years). The majority of the subjects (70%) were women. Their mean duration of schooling was 9 years (73%). The subjects had a mean score of 21 on the MMSE. The characteristics of the sample are summarised in table 1.

Table 1

Characteristics of the study sample (n = 30)

Sociodemographic variables
Age, years	80 ± 8
Female	21 (70)
Formal schooling
9 years	22 (73)
10 years	5 (17)
13 years	3 (10)
MMSE	21 ± 2
GDS	4 ± 1

Values denote means ± SD or n (%).

Item Set

Of the domains and items collected by means of a literature search, we eliminated all those that (a) took >30 min to carry out, (b) could be performed only at home, and (c) were clearly gender specific. This applied, for example, to going shopping accompanied by someone else (a), using a washing machine (b), or ironing/sewing (c). Of the resulting items, caregivers especially stressed handling domestic appliances, orientation, and taking medication as important.

As a result of the feedback we received from the experts, the following modifications were made: one item that had been intended to measure the handling of domestic appliances using a hand-held vacuum cleaner had to be modified as it showed a gender-specific bias towards women, showed poor standardisation, and was too time-consuming. Instead, the handling of appliances was operationalised by an item for which an alarm clock had to be set. In 6 persons with mild dementia who participated in the pilot study, the item for measuring the preparation of food proved to be too time-consuming to perform and therefore had to be deleted.

The 10 items selected for the pilot study are presented in table 2. Eight of these items consist of 6 sub-steps, whereas the 2 items ‘giving directions’ and ‘alarm clock’ were differentiated into only 3 sub-steps. One point was given per sub-step when the subject performed correctly, so that a maximum overall score of 54 points could be achieved.

Table 2

Item description

ICF domain ‘activities and participation’	Item	Item Domains
Chapter 3: communication	Letter	Addressing an envelope; choosing between stamps with different values
	Phone call	Finding a telephone number in the telephone book; making a phone call with a mobile phone for elderly people; listening to and reporting a voicemail message
Chapter 4: mobility	Traffic situations	Following basic rules in traffic situations on the basis of example situations given in pictures, e.g. traffic lights
	Giving directions	Orientation to time and place when describing a route
Chapter 5: self-care	Medications-indication	Assignment of a concrete medication to a group of medications (e.g. throat lozenges)
	Medications-expiry	Checking how long a medication will be usable by looking at the expiry date
	Medications-box	Placing medications in a medicine box according to a predefined regime
Chapter 6: domestic life	Making tea	Making 2 cups of tea, including selecting the desired kind of tea
	Alarm clock	Orientation in time; reading and setting times on a clock
	Finances	Comparing offers; adding up sums of money; counting out money

Results of the Pilot Study

A mean of 38.1 with an SD of 6.6 was calculated for the total score. The ETAM took an average of 26 min to administer. All items were found to have item difficulties of 0.27 ≤ pi ≤ 0.79. Three of the items (deciding on the indication for medication, appraisal of traffic situations, and spatial orientation) showed discriminatory powers of rit ≤ 0.20. All of the other discriminatory powers were 0.28 < rit < 0.59 (see table 3 for the discriminatory powers). The result of the Mann-Whitney U test on the influence of sex was not significant (p = 0.894), whereas level of education had a significant effect on the results of the ETAM (p = 0.018), i.e. the higher the subjects' level of education, the better their results on the ETAM.

Table 3

Item characteristics

Item name	Time, min	Mean	SD	Min.	Max.	Item difficulty	Item discriminatory power	Correlation with the MMSE	Correlation with the GDS-15
ETAM sum	26.00	38.10	6.62	23	50			0.31	0.15
Letter	5.07	3.97	1.40	2	6	0.49	0.28	0.21	0.16
Phone call	3.50	3.87	1.17	2	6	0.45	0.37	0.26	0.13
Traffic situation	3.77	2.77	1.52	1	6	0.28	0.03	0.09	-0.04
Giving directions	<1	0.53	0.51	0	1a	0.53	-0.01	0.14	-0.1
Medicines-indication	1.97	5.17	1.39	0	6	0.79	0.20	0.37	0.04
Medicines-expiry	2.70	4.83	1.62	0	6	0.72	0.44	0.36	0.35
Medicines-box	2.83	4.60	2.09	0	6	0.71	0.59	0.04	0.12
Making tea	*	5.07	1.20	1	6	0.75	0.37	-0.07	0.19
Alarm clock	1.87	2.33	0.96	0	3b	0.70	0.39	0.08	-0.22
Finances	4.60	4.97	0.93	3	6	0.71	0.59	-0.1	0.11

No timing possible, since the ‘making tea’ item is interrupted in the middle and continued later.

Maximum of 1 point scorable on ‘giving directions’.

Maximum of 3 points scorable on the ‘alarm clock’ item.

A Spearman's rank-order correlation coefficient of rs = 0.996 was found for the inter-rater reliability. The ETAM was found to have an internal consistency of α = 0.65 in the study sample. Moderate correlations of r = 0.36 and r = 0.37 were found between the MMSE test of cognitive abilities and 2 items from the ETAM (indication for medication and expiration date). All other items were only weakly correlated with the MMSE (0.09 ≤ r ≤ 0.21). Apart from the item ‘expiry date’ (r = 0.35), the remaining items showed a correlation of r ≤ 0.22 with the GDS-15. The correlations of the ETAM with the MMSE and GDS-15 are also presented in table 3.

Discussion

The present paper describes the development of a performance test to measure abilities to accomplish activities of daily living in persons with mild dementia. The test was validated in a pilot study that included 30 subjects. The resulting psychometric properties showed the measure to be feasible and indicated that the abilities measured by the test are relatively independent of a subject's level of cognitive performance and depressiveness.

The ETAM is the only performance test for measuring abilities of daily living in patients with mild dementia. It has its scientific basis in the ICF international standard [39]. According to Muò et al. [2], the ICF domains communication, mobility, self-care, and domestic life are particularly relevant for persons with dementia. We developed 10 items that cover these relevant domains. A wide range of methods were employed in the development of the measure in order to cover as adequately as possible all areas of activities of daily living that are relevant for people with mild dementia. Existing measures were analysed, family members were interviewed, and feedback was obtained from experts. This procedure differs widely from those used in the design of other performance tests, as only one method was employed to design most other tests – e.g. interviews with clinicians were used for the DAFS [24], and the modification of an existing questionnaire was used for the DAFA [13].

The average time required to complete the ETAM is 26 min, thus making it, along with the TEFA (completion time 15 min), less time-consuming than other performance tests. However, unlike the TEFA, the ETAM proved to be relatively independent of cognitive capacities in the pilot study. This is a prerequisite for substantiating and establishing the measurement of ADL abilities in addition to cognitive abilities as the cardinal symptoms of dementia. The psychometric properties of the ETAM were found to be satisfactory. In view of the complexity of the construct, the internal consistency of α = 0.65 can also be considered satisfactory. According to Schnell et al. [40], internal consistencies of α > 0.60 are commensurate with performance tests. The high inter-rater reliability is an index of the successful development of objective and practicable assessment criteria.

A further unique characteristic of the ETAM is that it is specialised for persons with mild dementia, and this predestines it for use as a diagnostic measure for the detection of deficits in activities of daily living in patients in the early stages of dementia. Its results can be taken as a basis for instituting important therapeutic interventions that are particularly efficient in the early stages of the disease [29].

The item difficulties of the preliminary version were all within the range of 0.2 ≤ pi ≤ 0.8 suggested by Lienert and Raatz [41]. However, 3 items (indication for medication, appraisal of traffic situations, and spatial orientation) had to be deleted because their discriminatory powers were below the lower limit of rit = 0.3 recommended by Bortz and Döring [38].

A high level of standardisation was attained for 8 of the 10 items. The exceptions were the items that measured making tea and giving directions. The standardisation of the tea item can easily be solved on a future version of the test by using a cord extension set, so that every participant can prepare the tea remaining seated. The directions item asked the respondents to describe how to get to the bathroom, and, obviously, the directions to the bathroom could not be standardised across centres. In our view, there currently is no solution for this standardisation problem, and it was therefore decided that other aspects of mobility such as the use of public transport should also be included. All other items can be measured both in the domestic setting and in other settings that are not familiar to the subjects. Thus, the measurement of the ICF mobility domain proved to be difficult, particularly in terms of standardisation, discriminatory power, and the time required for administration. Other authors have reported similar difficulties with measuring mobility [42].

The relations between activities of daily living and level of education found in our study are consistent with the current state of research. Hampel et al. [43] reported better coping strategies for persons with dementia who had attained higher levels of education, and this may explain why people who had completed ≥10 years of formal schooling had significantly better results.

We did not find a relation between test performance and sex in our study. The reports in the literature regarding this vary considerably. Sato et al. [44] reported no differences between the sexes with regard to performance for the 568 senior citizens they investigated. In the 90 elderly subjects studied by Bürge et al. [45], the men achieved significantly better performances particularly in the domain of mobility, whereas, by contrast, the women had better results in the domains of hygiene and personal care. The authors suggested that the men's superior performance in the domain of mobility could possibly be explained by the fact that the two sexes have different physical attributes. In our own pilot study, the men also obtained slightly better scores in the domain of mobility than the women, whereas the women showed a trend towards better scores on the other items. However, the differences were not significant overall.

One strength of the present study is the wide range of methods employed to develop the items. A complex 4-stage development process was used so that we could take into account as broad a range of aspects of activities of daily living as possible. The greatest weakness of our study is the size of the sample. With 30 subjects with mild dementia, the validation sample was larger than those used to develop many of the measures described in the literature. However, it was still too small to be able to make generalisable statements about the quality of its psychometric properties.

Future research is especially needed to find a way to operationalise the ICF mobility category with discriminatory power. Moreover, in the current version of the test, a ceiling effect was observed, since even the person with the lowest score (23 points) obtained almost half of the total number of possible points. It is therefore urgent that we modify the items and tighten the assessment criteria. In addition to modifying the content, the next validation should be carried out on a larger sample and should include additional reliability and validity parameters such as test-retest reliability, sensitivity to change, and convergent validity.

With the development of the ETAM in its current version, we have been successful in developing a performance test that is quick to complete and score, covers a broad range of activities of daily living, and is nonetheless easy to administer. More extensive validation studies with a bigger sample size are required for a final examination of the test's psychometric properties.

Disclosure Statement

The authors have no conflicts of interest to disclose.