Multidimensional assessment of challenging behaviors in advanced stages of dementia in nursing homes-The insideDEM framework.

INTRODUCTION: Assessment of challenging behaviors in dementia is important for intervention selection. Here, we describe the technical and experimental setup and the feasibility of long-term multidimensional behavior assessment of people with dementia living in nursing homes.
METHODS: We conducted 4 weeks of multimodal sensor assessment together with real-time observation of 17 residents with moderate to very severe dementia in two nursing care units. Nursing staff received extensive training on device handling and measurement procedures. Behavior of a subsample of eight participants was further recorded by videotaping during 4 weeks during day hours. Sensors were mounted on the participants' wrist and ankle and measured motion, rotation, as well as surrounding loudness level, light level, and air pressure.
RESULTS: Participants were in moderate to severe stages of dementia. Almost 100% of participants exhibited relevant levels of challenging behaviors. Automated quality control detected 155 potential issues. But only 11% of the recordings have been influenced by noncompliance of the participants. Qualitative debriefing of staff members suggested that implementation of the technology and observation platform in the routine procedures of the nursing home units was feasible and identified a range of user- and hardware-related implementation and handling challenges. DISCUSSION: Our results indicate that high-quality behavior data from real-world environments can be made available for the development of intelligent assistive systems and that the problem of noncompliance seems to be manageable. Currently, we train machine-learning algorithms to detect episodes of challenging behaviors in the recorded sensor data.

Introduction

Behavioral disturbances occur frequently in advanced stages of Alzheimer's disease and other dementias [1], [2], [3], [4]. Such disturbances include night-day shift, agitation, aggression, vocalizations, pacing, repeated movements, and mannerisms [5], as well as depression, anxiety, or psychosis [6]. These symptoms are often summarized under the umbrella term of “challenging behaviors” [7], indicating that these behaviors challenge or even exhaust the coping abilities of family and professional caregivers [8]. Challenging behaviors are key determinants for the institutionalization of people with dementia into a nursing home and for health care resource utilization [9], [10].

Treatment for challenging behaviors is a challenge in itself. Psychotropic medication is frequently used, but the efficacy is low and adverse effects are common [11], [12]. Therefore, behavioral and person-centered interventions have been developed and applied in nursing science and medical research [13], [14] and are recommended as first-line interventions in current guidelines [15], [16]. Such interventions focus on the context of a behavior and integrate the cognitive and health status of the patient with knowledge about personal traits and the patient's current situation, including interpersonal interaction and medication [17]. Instruments exist to systematically assess behavioral features of people with dementia [18], [19], but instruments that combine assessment of behaviors with individualized intervention selection are still rare [20]. Such instruments require expert knowledge; therefore, their use is currently limited to professional care. Most challenging behaviors, however, occur at the patients' homes and involve the family caregivers.

The project insideDem aims to develop a computerized expert system to support assessment of challenging behaviors and individualized intervention selection by family and professional caregivers. The input to such system integrates a knowledge base on behavioral symptoms of dementia from the systematic literature and expert reviews with automated detection of behavioral features from unannotated multimodal sensor data.

A range of previous studies has used actimetry to detect features of behavioral changes in dementia, including day-night shift and agitation [21], [22]. We have previously found that accelerometric motion features allow detection of agitation levels in people in mild to moderate stages of dementia at their homes [23].

In the present study, we aimed to extend these previous studies by including the whole range of behavioral symptoms that are typically found in patients with moderate to severe stages of dementia. We used multimodal sensor assessment together with real-time observation and video recording of 17 residents with moderate to severe dementia in specialized care units of two nursing homes. Primary outcome was feasibility, including integration of the assessments into the routine procedures of the nursing homes. In addition, we report the demographic and behavioral characteristics of the study participants. The buildup of the technology platform integrated into the routine workflow of two nursing homes has been described in a recent conference paper [24].

Subjects and methods

Subjects

Participants were recruited from the residents of dementia care units of two nursing homes in the Krefelder Städtische Seniorenheime, Krefeld, Germany. We recruited nine participants in the first unit (unit-1) and eight participants in the second unit (unit-2). The demographic characteristics are shown in Table 1.

Table 1

Subjects' demographics

Number of subjects (male/female)	Age (years) mean (SD) (min–max)	MMSE median (min–max)	GDS median (min–max)	TFDD depression median (min–max)
17 (6/11)	81 (6) (73–94)	8 (5–18)	5 (4–7)	5 (1–9)

Abbreviations: GDS, Global Deterioration Scale; MMSE, Mini–Mental State Examination.

NOTE. Single item for evaluation of depressive symptoms evaluated by others taken from the TFDD (test for early diagnosis of dementia with differentiation from depression), [25].

All participants were in moderate to very severe stages of dementia with a Global Deterioration Scale [26] score of >3. Antidepressants were given in 11, atypical antipsychotics in 12, low-potential antipsychotics in six, nonbenzodiazepine hypnotics in five, benzodiazepines in one, and antidementia drugs in eight participants. Assessment of cognitive functions with the Mini–Mental State Examination [27] was not possible in nine persons; four residents refused the examination and five residents had a degree of cognitive dysfunction that precluded testing. All except one participant were no more able to give informed consent. Therefore, in addition to the people with dementia, the holders of their durable power of attorney gave written informed consent for the participation in the study.

In both units, behavioral observation by an expert observer was conducted in the unit's sitting room, which was open for all residents, their visitors, and staff. To obtain informed consent for the video recording in the sitting room of unit-2, the holders of the durable power of attorney of the residents that were not the target of the study as well as all staff members and regular visitors were also asked for their written informed consent. The study was approved by the institutional review board of the German Society of Nursing Science (No. 16-007).

Baseline testing

All residents included in the study were in advanced stages of dementia. Therefore, direct psychometric testing of the participants was restricted to the Mini–Mental State Examination [27], and severity was assessed according to the Global Deterioration Scale [26]. In addition, we obtained the Neuropsychiatric Inventory (NPI) [18] and the Cohen-Mansfield Agitation Inventory (CMAI) [19] indexes through the professional caregivers. The professional caregivers were also asked for the presence of depression in the participants using the depression item from the test for early diagnosis of dementia with differentiation from depression [25]. The syndrome of dementia and the type of dementia were determined according to International Classification of Diseases, 10th Revision (ICD-10) codes in 15 residents by a neurologist and in two cases by the resident's general practitioner.

Behavior annotation

Behavior annotations were conducted in unit-1 between April 20, 2016 and May 13, 2016 and in unit-2 between June 15, 2016 and July 13, 2016. For the nine cases in unit-1, we used the dementia care mapping (DCM) [28] conducted by certified dementia care mappers. DCM is a multicomponent observation method and is based on the standardized observation of residents' well-being. DCM was conducted every 5 minutes and included formal coding and free-text entries for each of the observed subjects. Two codes were recorded: Behavioral Category Code and Well/Ill-Being (WIB). Behavioral Category Codes are descriptions of activities and behaviors, and WIB represents the affective state of the residents [29]. Because trained dementia care mappers were not available for observations in unit-2, the DCM was only conducted in unit-1.

Parallel to the DCM in unit-1 and as only observation in unit-2, behavior annotations were conducted according to a basic annotation scheme. This scheme was based on the frequencies of NPI- and CMAI-derived abnormal behaviors in both units before the start of the observation period. The basic annotation scheme consisted of the behaviors of aggression, performing repetitious mannerisms, pacing, apathy, general restlessness, and trying to get to a different place. In analogy to the WIB dimension of the DCM, these basic behaviors were further qualified along a scale from emotional negative to emotional positive valences of the behavior. The basic annotations were performed every 5 minutes in each participant together with the DCM or alone.

Observations (and video recordings, see Section 2.5) took place in the sitting rooms of the care units, which were open for all residents, visitors, and staff. Thus, we had to obtain informed consent not only from the residents who were actually participants of the study but also from all other people who eventually would enter the sitting room during the 4-week observation period. In addition, we had to retrieve the consent of the works council to conduct the study. During two afternoon meetings, we introduced the project to family caregivers and staff members. In addition, two coworkers of the study in the nursing homes provided information in direct contact with all people who would be affected by the study. The family members were primarily concerned about the privacy of the data, particularly of video recordings. The staff members were concerned that the material could be used to control the quality of their work. In our experience, the two coworkers of the study who were also staff members of one of the nursing homes were particularly able to communicate the intention of the study to staff and family members of the participants and to explain the measures that were taken to ensure data safety and privacy.

Sensor-based recordings

All participants were mounted with three sensor bracelets each (one was worn on the wrist of the dominant hand during the day, second was worn on the ankle during the day, and third on the ankle at night time between around 6 PM and 8 AM). The bracelets included sensors for acceleration, movement, and rotation. In addition, the sensor bracelets recorded the ambient loudness levels (but not spoken language), the ambient light level, and air pressure. Together with the staff members, we defined standard operating procedures to embed the device handling into the care routines. Staff members were trained in device handling.

Video recording

Video recording was conducted to allow later for an accurate and reliable behavior annotation as ground truth for sensor data training. Video recordings were only performed in unit-2. We were not allowed to take video recordings in unit-1 because one caregiver denied the consent. The video recordings were realized using a network-attached storage system with a built-in video surveillance solution. The nurses were equipped with a remote control to stop the recordings, if necessary. Videos were recorded between 9 AM and 1:30 PM and between 3 PM and 8 PM. To ensure the highest level of data protection, video recordings were stored on a closed system, which at the end of the study was transported to the Computer Science Department of the University of Rostock; from there, the data were transferred to a safe partition of the central server of the German Center for Neurodegenerative Diseases, which was only accessible to two coworkers of the German Center for Neurodegenerative Diseases who were named to the data protection authorities and the ethical board.

Data retrieval and storage platform

The data retrieval and storage system had to meet the requirements of the data security and safety boards of the involved institutions. For this, all data needed to be encrypted, signed, and stored redundantly on at least two locations. The technical system was designed to allow remote control of the sensors and the video equipment as well as all the local computers and server setup. All systems had been set up to work completely autonomous. This included automatic checks, self-monitoring of all subsystems, systematic checks of connections between the devices, comparison of achieved data volumes with the expected amount of data, and safety measures.

The overall data flow is depicted in Fig. 1. We targeted at a maximum time span of 24 hours between recording the data on the bracelet and a first analysis to detect problems during the recording. A more detailed technical description of the setup can be found in [24].

Fig. 1

Data retrieval and storage solution. After offloading the data from the bracelets (1), all data files are signed and encrypted (2), a backup on the NAS is created (3), on which a second backup to an external hard disk is made (4), the data are transferred securely to servers at the Computer Science Department of the University of Rostock (5), and finally data are analyzed (6). To provide the level of data protection required for meeting the privacy standards, only on these machines the data could be decrypted and analyzed. Abbreviation: NAS, network-attached storage.

Results

Demographic characteristics

The demographic characteristics of the 17 cases included in our study are shown in Table 1. From the 17 cases, eight cases had a diagnosis of Alzheimer's disease dementia, two cases had a diagnosis of vascular dementia, one case had a diagnosis of alcohol-related dementia, and six cases had a diagnosis of dementia of unknown origin. Almost 100% of participants received a total NPI score of 4 or higher, indicating a frequency of symptoms of at least once a week and at least moderate severity, as used in previous studies [30], [31]. The most frequent behavioral domains in the NPI in decreasing order of frequency were agitation, irritability, and aberrant motor behavior.

Proportion of complete data sets

Overall, we retrieved 414.25 hours of the DCM and 3460.58 hours of the basic behavior annotation. In reference to the annotated time periods, we had 71% of complete sensor data in unit-1 and 75% in unit-2. In unit-2, 1038 hours of video recordings were available with an overlap of 76% with annotations. In addition, we had 4258 hours of ankle sensor bracelet recording during the night hours across both units.

Frequency of abnormal behavioral categories

The distribution of the behavioral categories according to the DCM and the basic behavior annotations are shown in Fig. 2, Fig. 3.

Fig. 2

Frequencies of not-overlapping DCM behavioral categories in the unit-1 sample. Frequencies are plotted as the percentage of the entire sample. Abbreviation: DCM, dementia care mapping.

Fig. 3

Frequencies of not-overlapping basic behavioral categories in the unit-1 and unit-2 samples. Frequencies are plotted as the percentage of the entire sample.

Qualitative information on recording feasibility

We systematically debriefed the two nurses who supervised the handling of the sensor-based recordings. The reported challenges fell into two categories, recording failures due to user challenges or technical challenges. They are listed in Table 2. Table 3 lists all recording errors by category, documented parallel to the recording in wearing protocols.

Table 2

Qualitative information on recording feasibility

User challenges:• No general rejection of sensor bracelets by any participant. • Most participants seemed to regard the bracelets as something special. Their attitude toward the bracelet was positive, so that many participants showed the bracelets to their family caregivers. • Compliance fluctuated in up to 4 of 17 participants over the day. • Putting on the devices was sometimes challenging—on a few days, 5–10 attempts were needed until the participant agreed to wear the device. • Sometimes, nurses had tosearch for bracelets that the participants had intentionally removed. • The bracelets displayed the time on pushing a button. This was intended to provide an additional functionality to the participants. Some participants were no more able to push the button to display the time, so that permanent visibility would have been more useful. • In a few cases, nurses forgot to start the recording or attached the devices upside down. On the basis of this experience, in unit-2, a sound that signaled the start of the recording helped to reduce this recording failure. • The status screen of the bracelet was sometimes reported as being unintelligible and symbols were reported as being misleading. Technical troubleshooting had to be supported by external technical help. • After consultation symbols could be better explained and in some cases troubleshooting could be done by the nurses themselves. • For future studies, the nurses suggested a very well-defined documentation and explanations of symbols that represent the current state of the device and instructions for troubleshooting. • For the handling of the bracelets, the display was somewhat difficult to read as the font was very small and recognition of the device number was sometimes difficult. • Cable connections were partially instable, which complicated the download of the data and the recharging of the bracelet. • The clasp was easy to open, so that participants could easily take off the bracelet during the day. • Batteries of the bracelets had to be controlled and sometimes ran out of power during the recording.

Table 3

Types and occurrences of recording problems

Types of recording problems	Number of occurrences
Device worn upside down	19
Forgot to start recording	32
Offload/charging too late	22
Device soiled	1
Device damaged	4
Device deactivated too early	13
Device taken off by participants	32
Device interchanged by participants	1
Device put on too late	14
Device rotated in-between recording	9
Double recording	16
Device taken off too early	2
No recording (unknown reason)	32

NOTE. One occurrence represents one single event during the overall recording period for all study participants.

Reliability of 5-minute annotation segments

With respect to the DCM, we found a significant difference (Fisher's exact test, P = .001) between the aggregated mappings of the observers across the entire observation period. Cohen's κ (all behavior classes as an outcome) for the 41 available 5-minute segments where two certified dementia care mappers rated the same study participant during the same time period with the DCM was 0.28, consistent with a low reliability of the DCM. The analysis of the aggregated data of the 2178 available segments where two raters rated the same participants during the same time period with the basic behavior annotation revealed no significant differences (chi-squared = 56, df = 49, P = .23). Despite this consistency of the aggregated data, multiclass Cohen's κ [32] (all behavior classes as an outcome) for the 2178 available segments where two raters rated the same participants during the same time period with the basic behavior annotation was only 0.45.

Discussion

We reported the design, setup, feasibility, and the sample characteristics of a complex multisensory data recording field study in two care units for people with dementia. The field study was embedded in the routine care procedures and required the collaboration of nursing home staff, caregivers, and residents.

The large majority of participants were in severe stages of dementia. Accordingly, the prevalence of neuropsychiatric symptoms as averaged across the preceding 4 weeks using the NPI was high, with almost 100% of participants exhibiting at least moderately severe symptoms in at least one behavioral domain at least once a week. This high frequency is similar to the frequency of neuropsychiatric symptoms in other nursing home samples [33], and much higher than the prevalence of 53% to 78% in community-dwelling older people with dementia [4], [31], [34], consistent with the observation that more pronounced neuropsychiatric symptoms increase the likelihood for a person to enter into institutionalized care [8], [35].

Studies using continuous long-term observations or video recordings of behaviors of people with dementia in nursing homes are still rare. One study used a total of 5 hours for behavior annotations per resident and showed a moderate association between aggregated staff rating and direct observation of behaviors [36]. One previous study used videotaping of social behaviors in people with advanced stages of dementia [37]. Different to our study, this previous study divided the participants into groups of four to six people for observation of social interactions but did not monitor spontaneous behaviors in the normal setting of the care unit. Videotaping has also been used to monitor caregiver resident interaction [38], [39]. One study determined intrinsic and extrinsic factors associated with problematic vocalizations as detected on almost 5 hours of videotaping per participant in 138 people with dementia in nursing homes to determine the effect of proximity variables [40].

The design of our study posed several challenges. Different to previous studies that focused on specific interactions, for example, between participants of groups of people that were formed for the experiment [37] or selected scenes of caregiver resident interactions [38], [39] and limited time intervals with one-to-one observation and videotaping [40], we aimed to collect sensor and video data and annotations that reflected the naturally occurring activities of the target group of people with advanced dementia, requiring involvement of participating and nonparticipating residents, staff members, and caregivers in the informed consent process.

In addition to classical actigraphy [41], [42], the sensors monitored rotation, air pressure, loudness, and light level. This allowed gathering additional context information for understanding behaviors but reduced battery life so that devices had to be recharged every 12 hours. In addition, during day hours, sensors were mounted on the dominant hand and one ankle per resident to allow detection of more complex behaviors. These requirements posed additional burden on staff members to mount and unmount, recharge, and read out the devices every 12 hours. Considering the complexity of the tasks to be performed, the overall rate of about 73% complete data sets, including sensor data, observations, and videotaping in the unit-2 cohort, appears high and illustrates that such complex experiments can be conducted in the setting of routine care of nursing homes.

The selection of the real-time behavior annotation instruments was guided by the following considerations:

The DCM [28] was already established at both nursing homes, so that it would not interfere with normal procedures in the facilities and be already known to the patients.

The DCM provides a comprehensive annotation of behaviors, which would also provide a rich data set for further qualitative analysis.

The basic behavior annotation was a compromise between available resources and a minimum of information required. The selection of the dimensions to be annotated was based on a previous survey of the occurring behavior abnormalities in the residents of both nursing homes based on NPI [18] and CMAI [19] ratings. The annotation scheme was designed to be very similar to the DCM including 5 minutes of interval in number and letter coding as well as free-text notes.

The reliability of our DCM annotation was lower compared with the findings from one previous report with Cohen's κ for the behavioral categories of 0.54 [29]. However, the number of 41 segments of overlapping ratings by two observers was very low in our study compared with the previous report with 1683 paired observations [29]; our study was not primarily designed as a reliability study. One previous study achieved high inter-rater reliability for 5 minutes of observation of behaviors including pacing and verbal and physical aggressive behaviors [36] where one observer annotated one single resident based on annotation categories that were similar to the categories in our basic annotation. With our basic behavior annotation, however, we followed the DCM standard that one observer rated about five to eight residents in parallel.

Overall, our approach of using highly structured passive observation [43] for behavior annotations provides a rich insight into the full extent of behaviors and context factors and has high face validity. However, because of the complexity of behaviors and the large amount of information to be coded within a short time interval, as well as exhaustion even of experienced observers during observation sessions, the reliability of this approach was limited. We would likely have been able to increase the reliability by using shorter observation periods per observer and less people to be observed by one observer. This would have required a higher number of trained observers and would have interfered more strongly with the routines in the nursing homes' sitting rooms. In our study, the number of experienced observers was limited, and the integration as far as possible of all research processes into the naturalistic setting of the nursing homes was required to reach the goal of the study to identify unrestricted behaviors. These logistic problems indicate that obtaining a reliable estimate of the temporal distribution of challenging behaviors that are relevant for dementia care is expensive. Thus, a sensing device that could provide a mechanical detection of the presence of such behaviors would be an interesting instrument for supporting evidence-based care.

In contrast to direct observations, video recordings can be annotated offline, and scenes can be repeated as often as necessary to reach high agreement. At the same time, video recordings do not provide the same richness of data as direct observation. Our original plan was to combine both methods, so that the direct observation protocols could be used to enrich the highly reliable information from the video recordings. Because we were not allowed to perform video recordings in one of the two nursing care units, we will be able to apply this approach only in a subset of cases.

In summary, our study is one of only few studies that used a complex sensor, observer, and video recording setup for the comprehensive assessment of behavioral changes in people with advanced stages of dementia in a naturalistic setting. Despite the complexity of the data acquisition scheme and its embedding in the routine procedures of a nursing home, we yielded a high rate of complete data sets. The systematic debriefing identified user- and hardware-related factors that were responsible for difficulties in sensor bracelet mounting, charging, or reading out the sensor data. These factors are important for the planning of future studies in a similar environment. Our findings underscore the notion that nursing home dementia care units provide a relevant test bed for assessment of behavior detection techniques or interventions. Currently, we are training machine-learning classifiers on the sensor data using the observer-based annotations, and the annotated video data as reference to determine whether the sensor information can be used to identify behavioral changes. In future, highly predictive sensor features will be used to identify behavior abnormalities. Such information will provide an objective assessment of challenging behaviors and may become part of expert systems to help family or professional caregivers to identify challenging behaviors and select appropriate interventions.

Systematic review: We reviewed the literature using medical databases, such as PubMed, abstracts from conferences, and cited work in review articles. Several studies described short- and long-term observations of challenging behaviors in people with dementia in nursing care units. Few studies used multimodal assessment including video recordings. These studies typically involved short periods of time and restricted behaviors such as social interaction during planned group activities. Multimodal long-term assessment, including behavior annotations, video recordings, and sensor-based assessments of naturalistically occurring behaviors, is still missing.

Interpretation: Our findings suggest that multimodal behavior assessment integrated into routine care procedures of dementia care units is challenging but feasible. A particular challenge was the reliable local handling of sensor devices by staff members and the remote control of the technology platform to ensure data safety and privacy. This was particularly relevant for the video data that could not be fully anonymized. Our data point to several limitations and shortcomings; some could be amended in future studies, but some are also linked to limited resources and the need to avoid interference with routine care procedures. Our findings underscore the high potential of sensor data to objectively measure challenging behaviors and thus provide objective patient-powered real-world end points for disease monitoring and evaluation of intervention effects.

Future directions: We plan to use the behavior annotations and the annotation of video recordings to train data-driven algorithms to infer behavior categories from sensor data. The final goal is to integrate automated sensor-based behavior detection into a digital expert system to help professional and family caregivers to select appropriate nonpharmacologic interventions for challenging behaviors of people with dementia.