Performance-based IADL evaluation of older adults with cognitive impairment within a smart home: A feasibility study.

INTRODUCTION: Mild cognitive impairment (MCI) is characterized by subtle deficits that functional assessment via informant-report measures may not detect. Sensors can potentially detect deficits in everyday functioning in MCI. This study aims to establish feasibility and acceptability of using sensors in a smart home for performance-based assessments of two instrumental activities of daily living (IADLs).
METHODS: Thirty-five older adults (>65 years) performed two IADL tasks in a smart home laboratory equipped with sensors and a web camera. Participants' cognitive states were determined using published criteria including measures of global cognition and comprehensive neuropsychological test batteries. Selected subtasks of the IADL assessment were autonomously captured by the sensors. Total time taken for each task and subtask were computed. A point scoring system captured accuracy and number of attempts. Acceptability of the smart home setup was assessed.
RESULTS: Participants with MCI (n = 21) took longer to complete both tasks than participants with healthy cognition (HC; n = 14), with significant time differences observed only in "Cost calculation." Completion time for IADL tasks and scores correlated in the expected direction with global cognition. Over 95% of the participants found the smart home assessment acceptable and a positive experience. DISCUSSION: We demonstrated the feasibility and acceptability of the use of unobtrusive commercially available sensors in a smart home for facilitating parts of the objective assessment of IADL in older adults. Future studies need to identify more IADLs that are suitable for semi-automated or automated assessments through the use of simple, low-cost sensors.

INTRODUCTION

Dementia is a pressing public health challenge with significant socioeconomic impact on individuals, caregivers, and the community. , Globally the number of people projected to be diagnosed with cognitive deficits is expected to increase drastically over the next 20 to 30 years. The Well‐Being of the Singapore Elderly (WiSE) study found that one in 10 seniors 60 years of age or older has dementia. From 80,000 Singaporeans with dementia in 2017, the numbers are expected to more than double to 150,000 in 2030. , There is an increasing focus on early detection and prevention of dementia. Mild cognitive impairment (MCI) is an at‐risk stage for neurodegenerative diseases such as Alzheimer's Dementia (AD) , and presents a window of opportunity for intervention. Recent studies indicate an annual conversion rate of 9.6% from MCI to dementia, , with up to 50% converting to dementia within 3 years. American practice guidelines have emphasized the importance of early detection of MCI to allow for lifestyle modifications, intervention, and forward planning.

Functional impairments are a diagnostic criterion for dementia. At the MCI stage, individuals retain their abilities to perform basic activities of daily living (BADLs) but often have subtle functional deficits. These subtle deficits can present as changes in the ability to perform day to day tasks or instrumental activities of daily living (IADLs) such as handling money, doing simple household chores, telephone use, meal preparation, medication management, or traveling on public transport. Functional deficits can manifest as slowness in speed of execution of common daily tasks and may be detectable at an early stage of cognitive decline. It is important to note that IADL impairment confers an almost 4 times higher rate of conversion to dementia. , As such, detecting functional deficits early will be useful for MCI or dementia‐related clinical assessments. ,

RESEARCH IN CONTEXT

Systematic review: Functional status or impairments are important for the diagnosis of mild cognitive impairment (MCI) and dementia. Traditional self or informant report measures are prone to inaccuracies and gold standard occupational therapist led assessments are limited by finite resources. There is a lack of evidence on sensor‐enabled functional screening or assessments.

Interpretation: We conducted a feasibility study to evaluate if an assessment of two instrumental activity of daily living (IADL) tasks could be done in a smart home equipped with sensors and a web camera. Time taken to complete tasks as well as accuracy were compared between participants with MCI and those with healthy cognition. Participants from both groups could successfully complete the tasks, with the MCI group taking longer to complete the more cognitively demanding task.

Future directions: Our results demonstrate feasibility and future research can identify more IADL tasks suitable for semi‐automated or automated assessment through the use of accessible, commercially available sensors.

Traditional functional assessment tools such as the Lawton IADL Scale have been useful for assessing periodic functional changes in older adults within the community or hospital setting but are not sufficiently sensitive to pick up subtle changes, such as reduction in execution speed of tasks. Many functional assessment tools were designed originally for detection of dementia rather than MCI. Moreover, tools that rely on caregivers/proxies to report functional deficits are prone to inaccuracies such as over‐ or under‐reporting. Functional assessments that are clinically useful typically entail direct observation by trained professionals such as occupational therapists. These labor‐intensive assessments are usually performed within the patient's residence, a mock home setting, or in the community. However, population aging and resource scarcity makes low maintenance and unobtrusive sensor‐based evaluation an attractive complement to more laborious and costly assessment methods.

Increasingly smart technologies are being introduced into living spaces (sometimes referred to as "smart homes") to assist older adults in their everyday activities for facilitating aging‐in‐place within the community. These smart technologies can be broadly classified into five categories: passive infra‐red (PIR) motion sensors, body‐worn sensors, pressure sensors, video monitoring, and sound technologies. These technologies traditionally serve to monitor health and act as systems to detect sudden changes, such as falls and slips. More recently, simple and low‐cost sensors that do not intrude on the privacy of their users, such as contact sensors, have been used to monitor frequency of activities within the older adult's home. , Similarly, PIR sensors have been used successfully to track the movement and presence of persons in the home setting. ,

In this cross‐sectional study, we aimed to evaluate the feasibility of using commercially available sensors in a smart home to perform functional assessments of older adults to evaluate their ability to perform two IADL tasks: "Making a phone call" and "Cost calculation." We hypothesized that (1) participants with MCI require more time to complete these two tasks and (2) the ability score to complete these two tasks would correlate with neurocognitive scores.

MATERIALS AND METHODS

Study design and participant recruitment

This cross‐sectional feasibility study was conducted at the Geriatric Education and Research Institute (GERI), Singapore, over 6 months from June 12, 2019 to December 20, 2019. The study was approved by the National Healthcare Group Domain Specific Review Board (DSRB—2018/01165) and all participants gave informed consent before study procedures. Participants were recruited from existing cohort studies (The Yishun Study (DSRB 2017/00212) and Singapore Longitudinal Ageing Study (NUS‐IRB 04‐140), and from the community. Participants were included if they were (1) 65 years of age or older, (2) were able to walk at least 10 minutes, (3) required no mobility aid, (4) were able to perform all BADLs independently, and (5) scored 24 or more on the Mini‐Mental State Examination (MMSE). Participants were excluded if they had a history of any neurocognitive or movement disorders or scored below 24 on the MMSE.

Data collection

Basic sociodemographic information and the medical history of each participant was collected. Barthel's ADL questionnaire, Lawton's IADL questionnaire, Geriatric Depression Scale (15‐item), MMSE, Montreal Cognitive Assessment (MoCA), Digit Symbol Substitution Test (DSST), and the Clinical Dementia Rating Scale (CDR) were performed. The Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) was re‐administered for participants who had not performed these tests within the past 1 year.

Cognitive measures

The MMSE and MoCA assess various cognitive domains including executive function, visuospatial skills, orientation, attention, recall, as well as comprehension. Both tools have been validated in the local population. , The DSST assesses attention, visioperceptual ability, motor speed, and writing operations that are reflective of several higher cognitive functions like perception, encoding and retrieval processes, transformation of information stored in active memory, and decision‐making. The CDR consists of semi‐structured interviews, and scores are calculated based on six domains: memory, orientation, judgment and problem solving, community affairs, home and hobbies performance, and personal care. It has been used extensively in the Asian population to stage severity of neurocognitive disorders.

Participants’ cognitive status of HC versus MCI was established by experienced clinicians with the MMSE, MoCA, CDR, and neuropsychological test performance, with MCI operationally defined according to published criteria : (1) subjective memory and cognitive difficulties; (2) objective cognitive impairment in one or more domains: MMSE global score ≥24, and at least one neurocognitive domain score 1 to 2 SD less than the age and education‐adjusted mean values; (3) CDR global score = 0.5; (4) essentially independent in performing BADL (Barthel's Index); and (5) not diagnosed with dementia. It is worth noting that researchers who performed the sensor‐based functional assessments were blinded to participants’ cognitive status.

Smart home and living laboratory

Functional assessments were conducted in the GERI smart home lab modeled like a two‐bedroom apartment in Singapore. The lab was equipped with a one‐way mirror to allow for direct observation from the adjoining technical room (Figure 1 and Figure 2). Within the smart home participants were tasked to complete two IADLs, namely, making a telephone call and calculating cost while seated at a table. A web cam was installed with a limited view of the table to ensure that no identifiable data/feature were recorded, thereby preserving the privacy of each participant. One contact sensor (Door/Window Sensor Gen5, Aeotec, USA) was mounted on the telephone handset (for participant's use) within the smart home and another contact sensor was mounted on a the assessor's telephone handset. A telephone list was placed on the table. A contact sensor–equipped coin box was placed on the table for the calculating cost task. A large 55‐inch television screen was used to show participant's a digital shopping inventory (Figure S1) for the calculating cost task.

FIGURE 1

Model (not to scale) of the smart home in Geriatric Education and Research Institute (GERI). Location of the seated participant, land phone, telephone list, coin‐box, television, and the web camera are shown

FIGURE 2

Figure showing a seated participant watching television. Note the one‐way mirror to the left of the participant

Functional assessment

After a brief orientation of the smart home by the assessor, participants were given up to 30 minutes of "free and easy" time to acclimatize to the smart home environment alone. They were aware a web cam was part of the setup. The assessor remained in the technical room and delivered instructions for the two IADL tasks: "Making a phone call" and "Cost calculation." These tasks were identified from a validated performance‐based measure of IADLs, the Sydney Test of ADL in Memory Disorders (STAM). The assessor would also be making notes of any performance features such as hesitation.

Making a phone call task

Participants were instructed to perform the following steps (Figure S2):

Search an alphabetically ordered telephone list of 515 clinics for a particular clinic and read out the phone number of the clinic [Subtask “Search”]

Write the identified phone number onto a piece of paper [Subtask “Write”]

Correctly dial the written number using the telephone provided [Subtask “Call”]

If the task was completed correctly, the phone in the technical room would ring. Through the use of the contact sensors, the time it took for the participant to make the call was quantified, that is, from the time the caller lifted his handset to dial the number, until the assessor in the technical room lifted his handset (standardized as after three rings).

Cost calculation task

Participants were instructed to perform the following steps:

Identify four items (butter, dark chocolate, six eggs, and sugar) from the shopping list, note down the cost of each of these items on the paper provided, and calculate the total cost of these four items ($12.50). [Subtask “Identify & Add.”]

Take out $12.50 from a coin‐box which contained notes and coins of varying denominations and close the box immediately [Subtask “Coin‐box.”] The time duration for this task was quantified through the use of a single contact sensor, which was mounted on the cover of the coin box.

Next, the participant had to calculate the change he/she would receive for the four items above if he/she had only a $20 note with them [Subtask “Subtract.”]

Finally, the participant had to identify two more items—orange juice and milk—and add the cost of these two items to the earlier four items and calculate the total cost of all six items [Subtask “Add.”]

For all calculation tasks the participants were provided with paper to fill‐in their answers in pre‐defined spaces.

Scoring

For all functional subtasks, participants were provided with a maximum of three trials to successfully complete them. If the participant, voluntarily or when prompted (after 2 minutes of inactivity), informed the assessor that she/he had forgotten the instructions, the instructions for that task were repeated and the attempt was considered as a new trial. After three failed attempts, the assessor would move onto instructions for the next task. If a participant needed more than one attempt per subtask, the total time was calculated by adding the duration for all attempts. Web‐cam recordings were subsequently reviewed for scoring of the tasks.

In the original STAM scoring, there were nine tasks and each task was scored on a 4‐point scale. Participants received 1 point for each component completed correctly (maximum 4 points per task). Cognizant of subtle differences between HC and MCI, we took into account total time taken per task, time taken per subtask, number of errors, and number of attempts in scoring performance. We formulated a cumulative point score based on the following criteria: (a) a point (score of "1") if the duration to complete the subtask was greater than the mean ± SD of the whole group, (b) a point if they made at least 1 error during the subtask regardless of the number of attempts, or (c) a point for each re‐attempt (maximum of two re‐attempts), with a higher cumulative point score indicating a poorer performance.

Feedback

A feedback questionnaire was used to capture participants’ experience of performing the two functional tasks within the smart home. The four questions asked in the questionnaire were (a) “How was your experience with a functional assessment based in a smart home?” (b) “Is it acceptable to you to not have any contact with another person while in the smart home for assessment?” (c) “Do you find the use of a camera within the smart home acceptable?” (d) “Would you consider having one in your home for monitoring?” In addition, we also surveyed if the participant had wireless connectivity in their residence and if they owned a smart phone.

Statistical analysis

Data analysis was performed using R (Version 4.0.0) under RStudio (Version 1.2.5042) environment. Means and SDs were calculated for variables of interest. The Shapiro normality test were used to check for normality of the data. Unpaired Welch two‐sample t test was used to compare data that were normally distributed; otherwise the Wilcoxon rank‐sum test with continuity correction was used. All data for the two functional assessments were checked for outliers. Participant timing for any subtask that was greater than mean ± 3 SD for that subtask was considered as an outlier and was substituted with the maximum timing remaining in that list. Pearson's correlation was used to investigate association between time taken for functional tasks, and pen and paper cognitive assessments, and ADL questionnaires.

RESULTS

Participant characteristics and cognitive profile

Thirty‐five participants (20 women and 15 men) were enrolled with no dropout. The mean age of the participants was 71.7 ± 4.6 years with a mean 8.7 ± 4.3 years of education. There were 14 participants with HC and 21 with MCI.

Table 1 shows the characteristics of the participants with HC and MCI. There were no significant differences between the two groups with respect to age, gender, and years of education. As expected, there were significant differences in the MMSE, MoCA, and CDR scores between the groups. We found no difference in the DSST between the two groups.

TABLE 1

Characteristics of participants

	MCI groups (n = 21)	HC groups (n = 14)	P
Age (y)	71.6 ± 5.0	71.8 ± 4.2	.774
Female, n (%)	12 (57.1%)	8 (57.1%)	1
Education (y)	8.6 ± 4.3	9.0 ± 4.4	.778
MMSE a	27.4 ± 2.1	29.1 ± 1.6	.016
MoCA a	23.0 ± 3.3	26.1 ± 3.0	.008
CDR‐SOB a	0.64 ± 0.48	0.25 ± 0.33	.013
CDR‐Global a	0.26 ± 0.26	0.07 ± 0.18	.026
GDS	1.81 ± 2.25	1.93 ± 1.33	.860
DSST	32.0 ± 10.8	33.9 ± 8.9	.559

Denotes a significant P value. Unpaired welch two‐sample t test was used to compare data that were normally distributed; otherwise Wilcoxon rank‐sum test with continuity correction was used. Data were mean ± SD unless otherwise indicated.

Functional assessment

There were no significant differences in total time taken to perform the phone call task between the HC and MCI groups. Although the MCI group needed more time for correct identification of phone number subtask (90.48 s vs 72.64 s), this difference did not achieve statistical significance.

The total time taken for the "Cost calculation" task was significantly longer in MCI compared than the HC group (271.4 s vs 204.9 s, P = .015). Although not attaining statistical significance, the MCI group required more time for retention, identification, and cost summation of the four items to purchase (185.14 s vs 149.00 s, P = .069). The time needed for both groups to retrieve the right amount of money from the coin box was comparable. The MCI group took more time (23.24 s vs 10.43 s, P = .036) to perform subtraction/calculate of the change they would receive and also needed more time to add two more items to their calculations (47.29 s vs 30.43 s, P = .009) (Table 2).

TABLE 2

Time taken to complete functional tasks (in seconds)

Task	MCI groups (n = 21)	HC groups (n = 14)	P
Phone tasks
Search	90.48 ± 81.03	72.64 ± 58.44	.337
Write	15.86 ± 7.06	18.71 ± 6.49	.950
Call	27.33 ± 9.10	25.50 ± 5.79	.433
Overall	133.67 ± 82.50	116.86 ± 58.10	.337
Calculation tasks
Identify and add	185.14 ± 74.93	149.00 ± 69.69	.069
Coinbox	15.76 ± 8.82	15.00 ± 8.34	.349
Subtract a	23.24 ± 25.27	10.43 ± 11.82	.036
Add a	47.29 ± 19.69	30.43 ± 19.27	.009
Overall a	271.43 ± 96.66	204.86 ± 75.56	.015

Denotes a significant P value. Unpaired Welch two‐sample t test was used to compare data that were normally distributed; otherwise Wilcoxon rank‐sum test with continuity correction was used. Data were mean ± SD.

The functional task scores computed based on STAM criteria revealed no significant differences between HC and MCI groups for both tasks (Table S1). The cumulative task scores revealed no significant differences between groups for both tasks (Table 3). However, for both scoring methods, there was a trend for MCI participants to perform worse in the calculation of cost task.

TABLE 3

Functional tasks scores (cumulative point score)

Task	MCI groups (n = 21)	HC groups (n = 14)	P
Phone tasks
Search	0.33 ± 0.80	0.57 ± 0.76	.916
Write	0.10 ± 0.30	0.36 ± 0.50	.971
Call	0.24 ± 0.54	0.07 ± 0.27	.166
Overall	0.67 ± 0.97	1.00 ± 0.88	.894
Calculation tasks
Identify and add b	1.56 ± 1.26	1.17 ± 0.83	.230
Coinbox	0.24 ± 0.62	0.29 ± 0.61	.687
Subtract	0.43 ± 0.68	0.14 ± 0.53	.052
Add c	0.70 ± 1.13	0.23 ± 0.44	.167
Overall	2.88 ± 2.85	1.75 ± 1.42	.160

aDenotes a significant P value. Unpaired Welch two‐sample t test (one‐sided) was used to compare data that were normally distributed, otherwise Wilcoxon rank‐sum test with continuity correction was used.

‐5 MCI and 2 HC participants did not successfully complete this sub task.

‐1 MCI and 1 HC participants did not successfully complete this sub task.

Data were mean ± SD.

Correlation between functional tasks and cognitive assessments

Total time to perform both tasks was significantly negatively correlated with MMSE, MoCA, and DSST scores, respectively. Total cumulative functional task scoring was similarly negatively correlated with MMSE, MoCA, and DSST, with the contribution to this correlation predominantly coming from the calculating money task (Table 4). A significant positive correlation was seen between timing and scores with CDR sum of boxes scores; this is to be expected as a lower CDR‐SOB score is indicative of milder cognitive impairment. There was no significant correlation between the timings and scores with IADL scores.

TABLE 4

Correlational (pearson) relationship between functional tasks and physical and cognitive assessments

	Timing			Scores
	Phone	Money	Overall	Phone	Money	Overall
MMSE	‐0.32	‐0.62	‐0.57	‐0.16	‐0.62	‐0.66
MoCA	‐0.36	‐0.53	‐0.53	‐0.28	‐0.52	‐0.60
CDR‐SOB	0.36	0.59	0.57	0.23	0.66	0.66
DSST	‐0.37	‐0.26	‐0.36	‐0.26	‐0.49	‐0.47
Lawton	0.01	‐0.25	‐0.16	‐0.22	‐0.05	‐0.17

Boldness indicates statistical significance at P < .05.

Feedback questionnaire

Overall, the study methods including the assessment within the smart home, and use of a web camera within a research laboratory setting was acceptable to >95% of study participants. Because we were interested in developing smart homes in the community, we elicited if cameras would be acceptable in participant's own homes and the level of digital connectivity. Fifteen participants (42%) were not comfortable having monitoring web cameras in their home and highlighted the issue of privacy as their main concern. A higher percentage from the HC (57% vs 33% from MCI) declined the use of home monitoring web cameras. Thirty participants (85%) had wireless connectivity in their homes and 22 (63%) owned a smart phone.

DISCUSSION

In this study we investigated the feasibility of using a smart home equipped with sensors for a functional assessment of older adults performing two IADL tasks. Our findings indicate that it was feasible; the MCI group needed more time to complete the cost calculation task. Within the calculation task, the between group time difference was significant for the last two steps. For the phone call task, the difference between the MCI group and HC group did not achieve statistical significance. A likely explanation for these findings is that the calculation task is more cognitively demanding. This is also consistent with earlier research on MCI and performance of IADL, with reports of phone‐related tasks or finance‐/shopping‐related tasks being differentially affected in MCI. , , , Moreover, a recent comprehensive systematic review on MCI and IADL deficits found that the capacity for finance‐related tasks, which includes monetary skills, was affected in the majority of studies with large effect sizes.

In a recent review on MCI and deficits in IADLs, it was suggested that functional performance tools should investigate measures such as accuracy and speed of processing, in addition to executive functions, in order to distinguish between those with MCI and the cognitively healthy group. Prior studies similarly emphasized the importance of investigating speed of processing of functional tasks in participants with functional deficits. , , That being said, when both accuracy as well as speed of processing are considered, existing evidence has been inconclusive as to which IADL tasks are most sensitive to cognitive decline. Our scoring method accounted for both speed and accuracy for task performance. Although not achieving statistical significance for both tasks, the calculation task yielded pattern differences. There was a higher overall score in the MCI group compared to the HC group, and this was consistent in three of the four subtasks (“identify and add,” “subtract,” and “add”)

Correlational analyses findings were not surprising; time taken for functional tasks and scores correlated in the expected direction with global cognitive scales. There was no correlation between the timing taken for tasks and Lawton's IADL scores; all participants had rated themselves as fully competent in their IADLs. This finding underscores the unreliability of self‐reported measures of function. Objectively, MCI participants did have difficulty with calculating costs and money, but they had rated themselves as fully independent in managing financial matters. Subtle deficits are also often overlooked by caregivers, with inaccurate reports of functional impairment due to limited contact time. Direct observation of performance using standardized tools such as the Assessment of Motor and Process Skills is labor intensive and costly; therefore it is done selectively for individuals undergoing diagnostic assessments. Our findings indicate that this technology is feasible for performance‐based assessment semi‐independently. In settings in which highly trained therapists are a finite resource, this method of assessment can complement current self/informant report tools, allow for a brief functional assessment facilitated by other personnel, and open up the potential for full automation.

We were able to establish that this mode of smart home assessment, with a web camera in the system, was acceptable to over 95% of study participants. This finding contrasted with almost half of participants not being comfortable with having web cameras for assessment or monitoring being deployed in their homes. This highlights the importance of limiting the use of intrusive modes of technology like cameras to a controlled lab‐based environment, and avoiding use of cameras in residential homes as we refine sensor‐based assessment and home‐monitoring systems for the purpose of diagnostics and digital phenotyping.

One of the main strengths of the study is the smart home set up itself. Closely modeled after a typical 2‐ to 3‐room apartment in Singapore, it provided a natural environment for performance‐based assessment of IADLs that would improve ecological validity. With sensors and a web‐camera we were able to partially automate the process to evaluate speed and accuracy of task completion. We were mindful to ensure that the study was as unobtrusive as possible and preserved privacy by having the web camera angled only slightly at the table to capture only the task being performed.

Limitations include the small sample size, a clinical diagnosis of cognitive state without supporting neuroimaging, and the lack of differentiation between MCI subtypes. Moreover, we selected only two tasks for our IADL assessment, as we were constrained by the ability of the types of sensors to detect granular behavior and were disinclined to use a web camera to observe all activities within the smart home. Although we built on a validated STAM scoring system by including time taken, errors, and repetitions, the cumulative point score we computed was not validated. Given resource constraints at the time, we could not fully automate the process, with only one subtask from each IADL being autonomously captured by the sensors. There was still a need for an observer to deliver instructions and score performance, albeit from behind a one‐way mirror. The steps that required an observer and paper‐based recording of answers are currently being planned for digitalization in subsequent studies. Finally, the majority of participants were from existing community studies and an element of selection bias cannot be entirely discounted. These individuals may represent a more motivated or engaged group and may have skewed our acceptability findings.

There is a need to identify more IADL tasks suitable for semi‐automated or automated assessments through the use of a mix of technology including simple, low‐cost sensors. The configuration of the technology setup should be optimized to allow for fully automated instructions to be delivered via the use of a mobile or tablet‐based application. Because participants were generally amenable to web‐camera use within the lab setting, we can further explore the use of artificial intelligence–assisted video analytics of behavior for performance‐based assessment of function, which may provide the granular assessment of the other subtasks that still required human interface in this present study, to discriminate between individuals with HC and MCI.

CONFLICTS OF INTEREST

None declared.

Supplementary information

Click here for additional data file.

Supplementary information

Click here for additional data file.

Supplementary information

Click here for additional data file.