Toward the Development of SMART Communication Technology: Automating the Analysis of Communicative Trouble and Repair in Dementia.

BACKGROUND AND OBJECTIVES: Communication difficulties have been reported as one of the most stress-inducing aspects of caring for people with dementia. Notably, with disease progression comes an increase in the frequency of communication difficulty and a reduction in the effectiveness of attempts to remedy breakdowns in communication. The aim of the current research was to evaluate the utility of an automated discourse analysis tool (i.e., Discursis) in distinguishing between different types of trouble and repair signaling behaviors, demonstrated within conversations between people with dementia and their professional care staff. RESEARCH DESIGN AND METHODS: Twenty conversations between people with dementia and their professional care staff were human-coded for instances of interactive/noninteractive trouble and typical/facilitative repair behaviors. Associations were then examined between these behaviors and recurrence metrics generated by Discursis.
RESULTS: Significant associations were identified between Discursis metrics, trouble-indicating, and repair behaviors. DISCUSSION AND IMPLICATIONS: These results suggest that discourse analysis software is capable of discriminating between different types of trouble and repair signaling behavior, on the basis of term recurrence calculated across speaker turns. The subsequent recurrence metrics generated by Discursis offer a means of automating the analysis of episodes of conversational trouble and repair. This achievement represents the first step toward the future development of an intelligent assistant that can analyze conversations in real time and offers support to people with dementia and their carers during periods of communicative trouble.

Translational Significance

Communication difficulties in people with dementia are common. Automatic discourse analysis tools such as Discursis can accurately differentiate between different types of trouble and repair during conversations with people with dementia. Further development of discourse analysis tools is directed toward developing in-home assistive devices to improve communication in people with dementia.

Unprecedented growth within our aging society over the next three decades will herald an increase in age-related disease and disorders, such as dementia (Ferri et al., 2006). An exponential increase in the prevalence of dementia represents an emerging health care crisis with significant implications for health care systems, the global economy, and our society. Innovative advances in technology that compensate for cognitive impairments may constitute a necessary paradigm shift in the management of dementia, to provide an effective means of supporting this rapidly expanding population and their caregivers into the future (Mihailidis, Boger, Craig, & Hoey, 2008).

Dementia is a progressive, neurocognitive disorder involving the deterioration of memory, attention, language, executive functioning, social cognition, learning, and/or perceptual-motor skills (American Psychiatric Association, 2013). Caregivers typically learn to accommodate for the loss of these functions by monitoring and structuring the environment for people with dementia (Burgio, Allen-Burge, Stevens, Davis, & Marson, 2000). The high demands and stress accompanying this type of support frequently result in family caregiver burnout and associated ill health (Dillehay & Sandys, 1990; Williamson & Schulz, 1993). Recent advances in computer technology may provide an innovative means of supporting “aging-in-place” for people with dementia by prolonging independence and reducing levels of family caregiver burden.

Cognitive Prosthetics

Cognitive prosthetics, or assistive technologies for cognition, represent one such advance in computer technology. Assistive technologies for cognition have been described as having the potential to promote independence in daily tasks, improve well-being, and ease caregiver burden (Fleming & Sum, 2014). Examples of prosthetic technologies which have been used to support people with dementia, include audio/visual prompting technologies to assist with hand washing (Mihailidis et al., 2008; Mihailidis, Fernie, & Cleghorn, 2000), table setting, meal preparation, and self-care tasks (Lancioni et al., 2010, 2012); machine-based prompting to assist with self-care tasks (i.e., drinking water, teeth brushing, and upper body dressing; Bewernitz, Mann, Dasler, & Belchior, 2009); devices that provide medication reminders, night and day calendars (Gilliard & Hagen, 2004), fall detectors (Engstrom, Ljunggren, Lindqvist, & Carlsson, 2005); and personal tracking equipment (McShane et al., 1998).

Comparatively few technologies have been designed to support the communication difficulties experienced by people with dementia. During the course of dementia progression, an increase in the frequency of communication difficulty is coupled with a decrease in the success of attempts to repair communicative trouble (Orange, Lubinski, & Higginbotham, 1996; Orange, Van Gennep, Miller, & Johnson, 1998; Small, Gutman, Makela, & Hillhouse, 2003). This inability to communicate successfully ultimately leads to difficulty in maintaining relationships (Astell et al., 2008), depression, or social isolation, for both people with dementia and their caregivers (Clarke, 1991; Hendryx-Bedalov, 2000; Orange, 1991; Orange et al., 1996, 1998).

Communication Difficulties in Dementia

The capacity of individuals with dementia to communicate with others is slowly and irreversibly challenged with disease progression (Jones, 2015). These communication difficulties have been described as cognitive-linguistic in nature, as a result of impaired language, episodic memory, and semantic memory systems (Watson, Aizawa, Savundranayagam, & Orange, 2013). Distinct differences in cognitive-communication profiles exist across dementia subtypes (e.g., Alzheimer’s dementia, vascular dementia, frontotemporal dementia). Alzheimer’s dementia, however, has been identified as the most commonly diagnosed dementia subtype within the older adult population (Brayne et al., 1995), and as such constitutes the focus of this review.

During the early stages of Alzheimer’s dementia, spoken language is typically fluent with evidence of word finding difficulties and an increase in the use of semantically empty words (e.g., “thing” and “it”; Bayles & Tomoeda, 1983, 2014). Conversations are described as repetitious and less cohesive than those of healthy adults (Bayles, Tomoeda, & Boone, 1985; Tomoeda, Bayles, Trosset, Azuma, & McGeagh, 1996), possibly as a consequence of rapidly forgetting what was just said (Bayles, Tomoeda, & Trosset, 1992). In midstage Alzheimer’s dementia, spoken language remains fluent; however, there is an exacerbation in word finding difficulties (Bayles & Tomoeda, 1983; Weiner, Neubecker, Bret, & Hynan, 2008), frequent repetition of ideas (Bayles, Tomoeda, Kaszniak, Stern, & Eagans, 1985), reduced sensitivity to context (e.g., unable to get a joke; Tomoeda et al., 1996), and a diminished ability to self-monitor and self-correct errors in language production (Bayles & Tomoeda, 2014). In the late stages of Alzheimer’s dementia, the ability to communicate meaningfully may be lost. A range of symptoms may be experienced, including mutism, palilalia (i.e., involuntary repetition of words, phrases, or sentences; Appell, Kertesz, & Fisman, 1982), echolalia (i.e., repetition of another person’s spoken words), or jargon (i.e., incomprehensible spoken language; Bayles & Tomoeda, 2014; Obler & Albert, 1985). Communication partners may have difficulty understanding the person with dementia, without prior knowledge of context (Bayles & Tomoeda, 2014).

To date, technologies developed to support the communication needs of people with dementia have been designed to facilitate socialization and general interaction. For example, interactions between robotic animals and people with dementia have been shown to improve mood and social exchange (Bemelmans, Gelderblom, Jonker, & de Witte, 2012; Broekens, Heerink, & Rosendal, 2009; Libin & Libin, 2004; Moyle et al., 2013). Telepresence robots have also been used as a means of engaging people with dementia and their families via a Skype interface, with positive effects on quality of life and mood reported (Moyle et al., 2014). The use of a multimedia device designed to utilize reminiscence and preserved long-term memory systems in people with dementia has improved conversations with professional care staff (e.g., increased initiation and conversational control for people with dementia and increased satisfaction from carers in relation to the quality of their interaction with people with dementia) (Alm, Dye, Gowans, & Campbell, 2007; Astell et al., 2008).

Despite these attempts to promote socialization and interaction in dementia, no technologies have been developed which specifically address the explicit types of communicative difficulties that people with dementia and their caregivers commonly experience. One of the most difficult aspects of everyday communication for many people with dementia and their caregivers is the ability to have meaningful conversations.

Turn-taking and nonverbal aspects of conversation are typically maintained in dementia (Astell et al., 2008; Azuma & Bayles, 1997); however, conversational partners report difficulty in understanding people with dementia (Orange et al., 1996) due to the use of nonspecific referents, unexpected topic shifts, and interference of words and ideas from preceding conversations (Fuld, Katzman, Davies, & Terry, 1982; Garcia & Joanette, 1994; Mentis, Briggs-Whittaker, & Gramigna, 1995; Nicholas, Obler, Albert, & Helm-Estabrooks, 1985; Orange et al., 1996; Ripich & Terrell, 1988). These linguistic disturbances have been defined as the primary sources of conversational trouble in dementia (Orange et al., 1996), contributing to communication breakdown.

Conversational Trouble and Repair in Dementia

“Repair in talk” is an area of conversation analysis which has been particularly useful in examining the interactions of individuals with communication disorders (Wilkinson, 2008) and may serve to provide unique insights into the development of technology, which can support communication trouble in dementia. Within the trouble source and repair paradigm (Schegloff, Jefferson, & Sacks, 1977), repair initiators (i.e., trouble-indicating behaviors) highlight that a problem with communication has occurred (e.g., when the listener requests a repetition or takes a guess at what the speaker meant) and repair patterns and strategies are then employed to resolve that misunderstanding. Patterns of repair can be either self-initiated (i.e., repaired by the current speaker) or other-initiated (i.e., repaired by the listener), and repair strategies/types (e.g., repetition or paraphrasing) implemented as a means of altering the trouble source (Schegloff, 1992). The human coding of identified episodes of trouble and repair through the conversation analysis technique has revealed unique profiles of “repair talk” in the conversations of people with dementia and their communication partners.

Orange and colleagues (1996) revealed significantly higher percentages of utterances containing trouble source and repair sequences in the conversations of individuals with middle-stage (36%) and early-stage (24%) Alzheimer’s dementia when compared with normal older adult (19%) speakers. This finding indicates that as conversational difficulties worsen with disease progression, so too does the need to repair misunderstandings. Conversational breakdown in Alzheimer’s dementia has been attributed to the use of ineffective repair strategies by family caregivers (Orange et al., 1996; Small et al., 2003). Indeed, family caregivers perceive the effectiveness of communication strategies to be higher in early stage than in middle to late stage Alzheimer’s dementia, indicating that communication efficiency diminishes with disease progression (Savundranayagam & Orange, 2014). Conversation partners may therefore benefit from training or third party assistance to develop strategies that successfully target communicative trouble in Alzheimer’s dementia (Savundranayagam & Orange, 2014).

Watson and colleagues (1999) investigated the frequency and nature of trouble and repair in conversations between 10 individuals with Alzheimer’s dementia and 10 unfamiliar communication partners. This research revealed that individuals with Alzheimer’s dementia used a higher proportion of noninteractive trouble-indicating behaviors, whereby trouble is signaled independently of the communication partner (e.g., lack of continuation [Professional care staff: “What’s your favorite flower?”; Person with dementia: “I used to love reading”]), and communication partners used a higher proportion of interactive trouble-indicating behaviors, whereby the trouble signal required a response from the person with Alzheimer’s dementia to repair a misunderstanding (e.g., specific requests [Person with dementia: “I come from Melbourne”; Professional care staff: “Where do you come from?”]). In relation to repair, unfamiliar communication partners demonstrated the use of a wider range of repair types when compared with individuals with Alzheimer’s dementia, who tended to favor the use of Revision/reformulation and Repetition repair types. All repair types used by the unfamiliar communication partners were generally effective (i.e., success rate of 73% or more), with the exception of Inappropriate (e.g., Professional care staff: “How many kids did you say you had?”; Person with dementia: “Yes”; 0%) and Clarifying question (e.g., Person with dementia: “Can you go back to the part with the bookshop?”; Professional care staff: “Which bookshop? The one near the river, or the one opposite the police station?”; 33%) repair types. Importantly, the most frequently used and effective repair strategy used by the communication partners was Revision/reformulation (e.g., Professional care staff: “I am going home”; Person with dementia: “Pardon”; Professional care staff: “I am going to go home”; 96%).

Inherent heterogeneity exists in relation to the conversational behaviors of individuals with Alzheimer’s dementia and their caregivers (familiar and unfamiliar), as a consequence of variable cognitive impairments, preserved abilities, and the capacity of communication partners to compensate for communicative trouble (Muller & Guendouzi, 2005). In reference to Alzheimer’s dementia, “order” in conversation is achieved via perceived mutual understanding, which is typically accomplished via several behavior patterns including topic shifts; educated guesses; watching and waiting; and repeating, rephrasing, or inviting confirmation (Guendouzi & Muller, 2006). Of note, the use of repetitive questions (involving term recurrence) in Alzheimer’s dementia has been highlighted as a marker of conversational disorder or potential trouble source (Muller & Guendouzi, 2005). The ability to quantify repetition within and across conversational utterances may provide an objective measure of trouble, as well as repair.

The abovementioned studies provide valuable information pertaining to the way in which trouble is signaled in conversations with people with dementia, and the repair strategies that are most effective in remediating communication difficulty. These results have the capacity to inform attempts to automate the process of trouble and repair detection, via machine learning techniques. In particular, conversational trouble and repair sequences involving the repetition of content within- and between-speaker utterances are of particular interest to the current research. These behaviors can be objectively measured by machine learning algorithms developed to determine the degree of similarity between two utterances. Such algorithms are not intended to replace more rigorous discourse coding techniques, but rather, will offer tools to assist with automating discourse analysis.

Machine Learning and the Automatic Analysis of Trouble and Repair

The development of software that can analyze episodes of trouble and successful/unsuccessful repair in the conversations of people with dementia, on the basis of human-coding techniques, is a first step toward creating an intelligent assistant to support communicative trouble in dementia. The automatic analysis of trouble-indicating behaviors (Chinaei et al., 2017; Rudzicz, Chan Currie, Danks, Mehta, & Zhao, 2014) and communication strategies, which enhance conversational engagement in dementia (Atay et al., 2015; Baker et al., 2015), has already been achieved to some extent, via the application of intelligent dialogue analysis software. Given these findings, our research group aims to further develop customized intelligent communication technology that can assist in the identification and resolution of communicative trouble that occurs between people with dementia, and their caregivers, during everyday conversation.

Discursis Signatures of Trouble and Repair

Discursis software provides an automated text-analytic tool that facilitates the quantification and visualization of communication behavior during conversation, between two or more speakers (Angus, Smith, & Wiles, 2012a, 2012b). Discursis has been used in previous research to examine conversation dynamics between doctors and patients (Angus, Watson, Smith, Gallois, & Wiles, 2012c), open disclosure conversations surrounding adverse hospital events (Watson, Angus, Gore, & Farmer, 2015), and during television interviews and phone calls (Angus et al., 2012c). In relation to dementia, it has also been used to visually identify topics that facilitate conversational engagement (Baker et al., 2015), as well as to ascertain the effectiveness of various communication strategies (e.g., active listening) used by professional care staff in facilitating conversational engagement (Atay et al., 2015).

Via the input of conversational transcripts, the term-based version of Discursis has the ability to identify and tag the recurrence of individual words within conversational turns and generates an interactive visual representation of this input, which demonstrates similarity between turns, across the time course of the conversation. The recurrence plots generated by Discursis provide an overview of the structure and content of a conversation, the relative length of turns, and the degree of similarity between turns within and between speakers (Baker et al., 2015; see Figure 1 for an example recurrence plot). In addition to its application as a visualization tool, Discursis also has the capacity to generate quantitative measures relating to multiple aspects of conversation behavior, which are depicted as metrics (Angus et al., 2012b). The software processes a conversational transcript by detecting and quantifying the recurrence of terms or concepts within the conversation. Relevant to each turn within the conversation, Discursis counts the initial representation of a concept and thereafter how often this term or concept is referred back to either by the first speaker to generate this term (i.e., self-recurrence) or by the conversational partner (i.e., other-recurrence).

Figure 1.

Zoom in view of Discursis recurrence plot of conversation between person with dementia (PWD) and care staff (CS), demonstrating interactive TIB (complete repetition) and typical repair (inappropriate) aligned with Discursis metrics. TIB = trouble-indicating behavior; OBS = other backward short metric; OBM = other backward medium; OBL = other backward long metric; SBS = self-backward short metric; SBM = short backward medium metric.

In addition, Discursis also has the capacity to count how the content of each turn is represented in forthcoming turns. Once Discursis has processed a conversation, metrics of interest can be exported to undertake statistical analysis. Twelve metrics can be generated relating to semantic recurrence occurring along a combination of three possible dimensions: time scale (short, medium, long), direction (forward, backward), and speaker type (self, other). Short-term metrics represent conversation behavior between two consecutive turns, either in relation to one’s own turn (self-backward-short or self-forward-short) or the other speaker’s immediately preceding or consecutive turn (other-backward-short or other-forward-short). Consider the transcript below demonstrating an interaction (two conversational turns) between a person with dementia and a professional carer.

Other-backward-short recurrence metrics are calculated on the basis of repeated terms between two different (other) speakers (i.e., Person 1 [person with dementia] and Person 2 [professional care staff], across two consecutive conversational turns (i.e., turn 46 and 45, in a backward direction relative to the current turn [i.e., turn 46]). In contrast, self-backward-short metrics are calculated on the basis of repeated terms between the same (self) speaker (in this instance Person 2 [Professional care staff]), across two consecutive conversational turns (i.e., 46 and 44, in a backward direction relevant to the current turn [i.e., turn 46]). Consider this transcript:

Medium-term metrics relate to conversation behavior within 10 turns in either direction, and long-term metrics refer to all turns across an entire conversation. As per the transcripts mentioned earlier, medium-term metrics are calculated in the same way, however, on the basis of 10 consecutive conversational turns relative to speaker and direction (i.e., backward to forward). These metrics may have the potential to provide quantitative signatures relevant to different types of human-coded trouble and repair signals in conversation, on the basis of recurrent conversational content.

Term-based recurrence metrics generated by Discursis may be useful in objectively identifying conversational trouble-indicating behaviors and repair signaling behaviors in dementia. Working frameworks of conversational trouble and repair in dementia (Watson et al., 1999), based on human observation, present a range of signaling behaviors that involve utterance repetition (e.g., nonspecific requests for repetition; repetition with reduction; repetition with elaboration; addition/specification, etc.). The ability for a computational system to analyze these behaviors will provide the basis for the development of a SMART communication assistant to support conversational trouble in dementia.

Research Design and Methods

The aim of the present study was to determine whether Discursis metrics can distinguish between different types of human-coded trouble-indicating behaviors (i.e., interactive vs noninteractive) and repair types (i.e., typical vs facilitative), identified within conversations between people with dementia and their professional care staff.

The data presented in this research were collected as part of a larger project investigating memory and communication support strategies for use by professional care staff of people with dementia (Broughton et al., 2011; Liddle et al., 2012). Ethical approval was granted by the University of Queensland’s Behavioral and Social Sciences Ethical Review Committee (project number 2008000841) and by the service provider responsible for the participating residential care facilities.

Participants

Participants in this study included 20 people with dementia and 14 care staff from three not-for-profit residential care facilities in Queensland, Australia. Written informed consent was obtained from a legally authorized person prior to participation, and people with dementia also gave assent to participate on the day of data collection.

The age range of participants with dementia was 72–94 years (mean = 87.4, SD = 4.8). The participants with dementia consisted of 13 men and 7 women with a diagnosis of dementia taken from their medical records. Specific diagnoses were available for five participants with dementia (four vascular dementia and one Alzheimer’s disease). For the remaining 15 participants with dementia, a diagnosis of Dementia Not Otherwise Specified was given. Three of the participants with dementia spoke English as a second language: one at a basic level; one at an intermediate level; and one at an advanced level. English proficiency levels were determined by family member or self-report, via three multiple choice options (i.e., basic, intermediate, or advanced) on a demographics questionnaire. Each participant with dementia’s level of cognitive impairment was classified as either mild, moderate, or severe, according to scores achieved on the Mini-Mental State Examination (Folstein, Folstein, & McHugh, 1975). Scores obtained on the Mini-Mental State Examination ranged from 3 to 27, indicative of severe cognitive impairment in 10 subjects, moderate impairment in 9 subjects, and mild cognitive impairment in 1 subject (Mungas, 1991). No participants with dementia were excluded on the basis of scores obtained on the Mini-Mental State Examination or other screening tests.

The professional care staff group consisted of 11 women and 3 men who fulfilled the roles of personal care or nursing assistant (N = 12), registered nursing staff (N = 1), or diversional therapist (N = 1). Of the seven professional care staff who disclosed their age, six were between 45 and 59 years of age, and one professional care staff member was less than 25 years of age. Ten professional care staff participated in only one conversation, and two professional care staff participated in two separate conversations with two different participants with dementia. An additional two professional care staff participated in three separate conversations each. A total of 20 conversational dyads were subsequently formed and assigned pragmatically, based on the mutual availability of individual professional care staff and participants with dementia at the time of recording. In each dyad, conversation partners were familiar to one another and had each had prior contact within the care setting.

Procedure

Each participant with dementia and their professional care staff partner were instructed by a research assistant to have a conversation on any topic for a duration of 10 min. Conversations took place in comfortable, familiar settings, either in the participant with dementia’s room or communal lounge area. The conversations were recorded via an Olympus DS-30 digital voice recorder placed in close proximity to the participants. The recordings obtained varied in length from 5:04 to 13:01 min (mean = 9:04 min, SD = 2:19 min), depending on the capacity and willingness of the participant with dementia to communicate.

The audio recordings were transcribed into written text using a modified Jeffersonian transcription method (Atkinson & Heritage, 1984). The notation relevant to interpretation of the text and preparation of transcripts were identical to the procedure described in a previous publication pertaining to this data set (Baker et al., 2015).

Data Analysis

Trouble and repair variables

Each of the 20 conversation transcripts was manually coded by a speech pathologist in relation to the presence of trouble-indicating behaviors and repair types, adapted from the framework established by Watson and colleagues (1999; see Tables 1 and 2 for description of framework).

Table 1.

Trouble-Indicating Behavior (TIB) Codes Used and Examples Adapted From Watson and colleagues (1999)

Code name	Definition	Example	Original code from Watson et al. (1999)
Interactive trouble-indicating behaviors: signal trouble understanding the conversational partner and require a response from them
Nonspecific local request	General comments with no specific referent that indicate nonunderstanding of the previous turn(s), but do not indicate the specific point of difficulty.	What?	TIB1: Neutral or nonspecific requests for repetition—Local (minimal queries)
		Huh?
		Pardon?
		What did you say?
Nonspecific global request	Nonspecific comments that indicate difficulty understanding the previous section of talk.	Wait a minute. Go back to the part about…	TIB12: Request for repetition—Global
		You lost me about 30 seconds ago.
Specific request	A question that indicates difficulty understanding a specific part of the previous turn(s) by using a question word and specific referent.	Speaker 1: I come from Melbourne	TIB5: Request for specific information
		Speaker 2: Where do you come from?
Repetition with reduction	A partial repetition of the original utterance (i.e., the trouble source turn) with rising intonation, that is followed by a response or a pause of medium length or greater.	Speaker 1: I went to a party last night.	TIB2: Requests for confirmation— Repetition with reduction
		Speaker 2: Last night?
		Speaker 1: Yeah
Complete repetition	A complete repetition of the original utterance (i.e., the trouble source turn) with rising intonation, that is followed by a response or a pause of medium length or greater.	Speaker 1: I don’t think so.	TIB3: Request for confirmation— Complete repetition
		Speaker 2: You don’t think so? (Long Pause)
Repetition with elaboration	A complete repetition of the original utterance (i.e., the trouble source turn) and the inclusion of additional semantic content.	Speaker 1: I went to a party last night.	TIB4: Request for confirmation— Repetition with elaboration
		Speaker 2: You went to a party last night at the restaurant?
Hypothesis formation	The speaker paraphrases or elaborates on the conversational partner’s previous utterances by stating what he/she believes the conversational partner meant, and uses rising intonation to seek confirmation that this hypothesis is correct. There are three subcategories:	Speaker 1: I used to—used to—teaching	TIB9: Hypothesis formation
	A: The speaker uses the same words as the original utterance but they are reordered	Speaker 2: You used to be a teacher?
	B: The content remains the same but the words are changed (paraphrasing)
	C: The content is elaborated upon
Noninteractive trouble-indicating behaviors: signal trouble understanding independent of the other speaker
Lack of uptake	The listener does not respond to a question at all (indicated by a pause of medium length or greater) or provides a minimal response that does not adequately address the question.	Speaker 1: What are you going to do today?	TIB8: Lack of uptake/lack of continuation
		Speaker 2: Mmm.
Lack of continuation	The listener’s response to a question is inappropriately off-topic, so the conversation is not continued as expected.	Speaker 1: What’s your favorite flower?	TIB8: Lack of uptake/lack of continuation
		Speaker 2: I used to love reading.
Tangential response	The listener’s response is on-topic but does not address the question asked.	Speaker 1: What was your brother’s name?	Nil
		Speaker 2: Oh, I wonder where he’s gone.
Metalinguistic comment	A comment that explicitly expresses that the speaker is having trouble with the “talk,” including understanding the message of the other speaker or difficulty finding a word.	I’m sorry, I don’t understand.	TIB10: Metalinguistic comment
		I don’t know what you mean.
		What’s the word that means…
Metacognitive comment	A comment that expresses that the speaker is unable to remember content to respond to a question or provide information.	I don’t remember.	Nil
	This does not include instances in which it is appropriate to respond in this way, such as if the speaker is being asked for a personal preference or he/she shouldn’t necessarily know the answer.	My memory’s bad.
		I can’t think of it.
		I don’t know.

Table 2.

Repair Type Codes Used and Examples Adapted From Watson and colleagues (1999)

Code name	Definition	Example	Original code from Watson et al. (1999)
Typical repairs: involve full or partial repetitions for clarification
Repetition	A repeat of all or part of the trouble source utterance, with no extra information added.	Speaker 1: I like it.	Repetition (Repair 1)
		Speaker 2: What?
		Speaker 1: I like it.
Revision/reformulation	The semantic content of the trouble source utterance was held constant, but the utterance form had been changed (i.e., using different word for same meaning, changing syntactic structure). Can also be seen as hesitations, pauses (filled and unfilled) indicating self-repair in formulating current turn speak.	Speaker 1: Did you watch any TV today?	Revision/reformulation (Repair 2)
		Speaker 2: Huh?
		Speaker 1: Have you been watching any TV today?
Addition/specification	The trouble source utterance was repaired by including additional information. It may have included providing specific information or elaborating on concepts in the original utterance.	Speaker 1: We’ll go later.	Addition/specification (Repair 3) and cues/explanation (Repair 4)
		Speaker 2: What?
		Speaker 1: We’ll go to the shops later.
Inappropriate	The speaker did not adequately address the needs of the conversational partner as expressed by an interactive TIB.	Speaker 1: Who said that?	Inappropriate/withdrawal (Repair 5)
		Speaker 2: No.
Confirm/reject	The TIB required the speaker to answer “yes” or “no” to clarify some trouble in conversation (i.e. in response to a request for specific information (hypothesis formation). May have included minimal reinforcers, or repetitions of hypothesis stated in the previous turn.	Speaker 1: Do you mean he went home?	Confirm/reject (Repair 6)
		Speaker 2: Yes.
Facilitative repairs: encourage re-engagement in the conversation
Accommodation	In response to a tangential response or lack of continuation TIB, the speaker accepts the change in topic and responds appropriately to the other speaker without reverting to the original topic or question.	Speaker 1: Did he get it?	Nil
		Speaker 2: I’m getting hungry.
		Speaker 1: Ok, what would you like to eat?
Prompting	In response to the conversational partner not providing a response to a question, the speaker encourages or prompts the conversational partner to provide a response.	No?	Nil
		Do you remember?
		Person with dementia’s name?
New topic introduction	Following a noninteractive TIB, the speaker introduces a new topic or asks an unrelated question to continue the flow of the conversation.	Speaker 1: What’s your favorite color?	Nil
		Speaker 2: (Long Pause)
		Speaker 1: What’s on TV?
Posing a solution	Following a noninteractive TIB, the speaker provides a suggestion for a possible response to aid the conversational partner in answering the question.	Speaker 1: What did you do yesterday?	Nil
		Speaker 2: I don’t remember.
		Speaker 1: Did you go for a walk?
Topic continuation	Following a noninteractive TIB, the speaker continues the conversational flow by providing further comments or questions on the topic.	Speaker 1: What’s your favorite color?	Nil
		Speaker 2: (Long Pause)
		Speaker 1: My favorite color is red.

Note: TIB = Trouble-indicating behavior.

Interrater and intrarater reliability analyses were completed on 30% (N = 6) of the transcribed conversational data sets. Reliability samples comprised randomly selected conversations. Cohen’s kappa statistic was used to measure agreement both within and between raters, regarding types of trouble-indicating behaviors and repairs observed. The following criteria were used to interpret kappa in this study: kappa < 0 (poor agreement); 0.0–0.20 (slight agreement); 0.21–0.40 (fair agreement); 0.41–0.60 (moderate agreement); 0.61–0.80 (substantial agreement); and 0.81–1.00 (near-perfect agreement; Landis & Koch, 1977).

Interrater reliability

Two speech pathologists previously trained in the nature of trouble and repair in conversation were asked to rate 30% of the conversational samples. Statistically significant (p < .001) levels of agreement (kappa > 0.81) were obtained for both variables.

Intrarater reliability

One of the speech pathologists above also rerated 30% (N = 6) of the conversational data sets. Again, Cohen’s kappa statistic was used to measure the level of intrarater agreement, in relation to types of trouble-indicating behaviors and repairs observed. Near-perfect levels of agreement (kappa > 0.81) were also identified across both variables.

Transcripts were also processed in Discursis to obtain metrics relative to the recurrence of semantic content between speakers during each conversation. Discursis processing parameters included the use of term-based recurrence for a maximum of 200 terms, and a stop word list of semantically empty words (e.g., “no,” “yeah”) and ambiguous terms (e.g., right).

Results

The final data set contained 4,129 conversational turns when the 20 conversations between participants with dementia and professional care staff were combined. The distribution of human-coded trouble and repair behaviors within this data set collapsed across speakers (i.e., participants with dementia and professional care staff) can be seen in Table 3. To meet the aims of the study, trouble-indicating behaviors were classified and grouped as interactive or noninteractive, and repair types as typical or facilitative. Human coding was achieved using a binary classification system, with the presence of the relevant behavior (trouble-indicating behavior or repair) coded as 1 and an absence of the behavior coded as 0. Repairs constituted 8.06% (333/4,129) of the data set (64% [213/333] of all repair types were typical and 36% [120/333] were facilitative). Trouble-indicating behaviors represented 8.43% (348/4,129) of all conversational turns within the data set (52.6% [183/348] of trouble-indicating behaviors were interactive and 47.4% [165/348] were noninteractive trouble-indicating behaviors).

Table 3.

Distribution of Trouble-Indicating Behaviors and Repair Types Across Groups, and Significant Chi-Square Tests of Association Demonstrating Relationships Between Different Trouble-Indicating Behaviors, Repair Types, and Discursis Metrics

Human coding of conversational turns		Discursis coding metrics and human codes
		Short	Medium	Long
Of 4,129 turns, 681 turns met the criteria to be coded as below		Direction of relationship	Direction of relationship	Direction of relationship
Interactive trouble total counts	183	Chi-square	Chi-square	Chi-square
Hypothesis formation	55	p < .001	p < .001	p < .001
Nonspecific local request	48	Presence of other/backward metric-positive relationship	Presence of other/backward metric-positive relationship	Presence of other/backward metric-positive relationship
Specific request	29	(55)	(66)	(71)
Complete repetition	25	Absence of other/backward metric-no relationship	Absence of other/backward metric-negative relationship	Absence of other/backward metric-no relationship
Repetition with reduction	20	(128)	(117)	(112)
Repetition with elaboration	6
Nonspecific global request	0
Noninteractive trouble total counts	165	Chi-square	Chi-square	Chi-square
Lack of uptake	97	p < .001	p < .001	p < .001
Tangential response	46	Presence of other/backward metric-negative relationship	Presence of other/backward metric-negative relationship	Presence of other/backward metric-negative relationship
Lack of continuation	20	(10)	(13)	(25)
Metalinguistic comment	2	Absence of other/backward metric-no relationship	Absence of other/backward metric-positive relationship	Absence of other/backward metric-no relationship
		(155)	(152)	(140)
Typical repair total counts	213	Chi-square	Chi-square	Chi-square
Confirm/reject	80	p < .001	p < .001	p > .004
Addition	56	Presence of self/backward metric- positive relationship	Presence of self/backward metric- positive trend	No significant relationship (as per Bonferroni adjustment)
Repetition	41	(69)	(79)
Revision	29	Absence of self/backward metric-no relationship	Absence of self/backward metric-no relationship
Inappropriate	7	(144)	(134)
Facilitative repair total counts	120	Chi-square	Chi-square	Chi-square
Accommodation	45	p < .001	p < .001	p > .004
Topic continuation	25	Presence of self/backward metric- negative relationship	Presence of self/backward metric- negative relationship	No significant relationship (as per Bonferroni adjustment)
New topic introduction	17	(12)	(21)
Posing a solution	17	Absence of self/backward metric-no relationship	Absence of self/backward metric-no relationship
Prompting	16	(108)	(99)

Notes: Positive and negative relationships indicate higher or lower than expected counts (respectively) in relation to present/absent Discursis metrics and human-coded trouble-indicating or repair type behaviors, as determined by standardized residual value of ≥ ±1.96. No relationship denotes actual counts within expected range (i.e., standard residual < ±1.96). Numbers in parentheses denote number of present/absent Discursis metrics and human-coded behavior matches.

aPositive relationship trend with standardized residual value of +1.90.

A series of chi-square tests of independence was subsequently conducted to determine whether human-coded episodes of trouble and repair were associated with Discursis metrics. Statistical comparisons were made on the basis of a binary coding system with the presence of interactive/noninteractive trouble-indicating behaviors, typical/facilitative repair types, or Discursis metric (value > 0) being coded with a 1 and an absence of these variables being coded with a 0. Multiple comparisons necessitated the use of a Bonferroni adjusted alpha level of .004 per test (.05/12).

The purpose of this analysis was to identify whether Discursis metrics could reliably distinguish between different types of trouble-indicating and repair behaviors, on the basis of term recurrence, across conversational turns. Human-coded behaviors known to involve term repetition (i.e., interactive trouble-indicating behavior and typical repairs) were hypothesized to be positively associated with the presence of Discursis metrics (i.e., number of counts of metrics matched with these behaviors would be higher than expected). In addition, human-coded behaviors that did not typically involve term repetition (i.e., noninteractive trouble-indicating behavior and facilitative repairs) would be negatively associated with the presence of Discursis metrics (i.e., number of counts of metrics matched with these behaviors would be lower than expected).

Significant relationships were observed for trouble-indicating behavior type and other-backward/ short (χ2 (1) = 32.89, p < .001), –medium (χ2 (1) = 39.29, p < .001), and /long (χ2 (1) = 24.29, p < .001) Discursis metrics. An examination of standardized residual values was also undertaken to determine which variables were contributing to the overall significant result. Higher than expected counts (i.e., positive relationship) in relation to the associations between interactive trouble-indicating behaviors and other-backward/ short/ medium and /long metrics, and noninteractive trouble-indicating behaviors and the absence of the other-backward/medium metrics were revealed (see Table 3). In contrast, lower than expected counts (i.e., negative relationship) were observed in relation to noninteractive trouble-indicating behaviors and other-backward/ short/ medium and /long metrics, and interactive trouble-indicating behaviors and the absence of the other-backward/medium metrics (see Table 3).

In relation to repairs, significant associations (i.e., positive relationships) were observed between repair type and self-backward/short (χ2 (1) = 20.91, p < .001) and /medium (χ2 (1) = 14.02, p < .001) Discursis metrics. Standardized residual values revealed higher than expected counts relating to associations between typical repairs and self-backward/short metrics (see Table 3). In contrast, lower than expected counts were observed in relation to facilitative repairs and self-backward/ short and /medium metrics (see Table 3).

Discussion and Implications

The aim of the present study was to determine whether an automated discourse analysis tool (i.e., Discursis) had the capacity to differentiate between human-coded trouble-indicating behaviors and repair types, within everyday conversations between people with dementia and their professional care staff. Significant associations were identified between trouble-indicating behaviours and other-backward/ short /medium and /long metrics, relating to semantic recurrence. In addition, significant associations were also observed between repair types and self-backward / short and / medium metrics. These significant associations were largely attributed to (a) higher than expected counts of interactive trouble-indicating behaviors aligned with other- backward/ short/ medium and /long metrics, and typical repairs aligned with self-backward/short metrics and (b) lower than expected counts of noninteractive trouble-indicating behaviors aligned with other- backward /short/medium and /long metrics, and facilitative repairs aligned with self-backward /short and /medium metrics. These findings suggest that Discursis is capable of measuring trouble-indicating and repair behaviors that entail the recurrence or revision of content from within preceding conversational turns. In contrast, Discursis was less reliable (lower than chance level) at quantifying noninteractive trouble signaling behaviors and facilitative type repairs, where topic or theme recurrence between conversational turns is lacking. These findings provide the precursor to the development of smart technologies that can support communicative trouble in dementia, via the real-time monitoring and analysis of conversational content.

Integral to the development of any technology that aims to support conversational trouble in dementia is the notion of common ground. Efficient conversations are navigated by mutual knowledge shared between speakers (i.e., common ground; McKinley, Brown-Schmidt, & Benjamin, 2017). Over the course of a conversation, this mutual knowledge grows incrementally on the basis of shared information between conversational partners (Clark, 1996). Conversational partners achieve mutual interpretation of ongoing talk by conjointly breaking the conversation into smaller units, demonstrating understanding of the talk, or requesting the repair of misunderstandings (Schegloff et al., 1977). Establishing and maintaining common ground in a conversation involves the use of feedback signals (Hyden, Plejert, Samuelsson, & Orulv, 2013), which include indicators of misunderstanding (trouble indicators), or requests for support (repair) to achieve mutual understanding. Understanding is achieved via the continual monitoring and modification of contributions from all conversational participants (Samuelsson & Hyden, 2017), and this negotiation of contributions typically occurs over a number of conversational turns (Clark, 1996; Clark & Schaefer, 1989a, 1989b). Just as humans practice these actions within everyday conversations, a device or technology that is designed to support communicative trouble must be able to monitor and generate feedback signals (either directly or passively), in the same way.

Interactive trouble-indicating behaviors may be defined as verbal signals that demonstrate to the other speaker that a misunderstanding has taken place on the basis of a previous conversational turn, and requires a response from that speaker to remedy that misunderstanding (Watson et al., 1999). Examples of these trouble-indicating behaviors include specific requests, complete repetitions, and hypothesis formation (i.e., taking a guess at what was meant; see Table 1). On the basis of our research findings, the presence of interactive trouble-indicating behaviors were associated with the presence of other-backward/short/ medium and /long Discursis metrics. These results were predicted given the nature of interactive trouble (i.e., complete repetition; see Figure 1) and the high likelihood for semantic recurrence with dynamic interactions between people with dementia and their care staff, in an effort to verbally signpost perceived misunderstandings.

By definition, noninteractive trouble-indicating behaviors are signaled independently of the other conversational partner and do not require a response from that partner in an effort to remediate the perceived misunderstanding. Rather, trouble is signaled by the use of inappropriate responses (e.g., providing no response to a question), which can disrupt the flow of conversation. Relevant to the current research, these types of behaviors were hypothesized to be associated with a lack of conversational recurrence as measured by Discursis. Our results indicated that the presence of noninteractive trouble-indicating behaviors were associated with an absence of other-backward /short/ medium and /long Discursis metrics, supported by the hypothesis of an absence of semantic recurrence (see example of Lack of uptake in Figure 2). An additional finding was the significant association between the presence of noninteractive trouble-indicating behaviors and absence of other-backward/medium metrics, which may reflect the nature of noninteractive trouble signaling and repair. Specifically, unsuccessful repair sequences may be abandoned (Simmons-Mackie & Kagan, 1999) in an effort to reestablish conversational flow, such that the introduction of new topics may involve a lack of thematic/semantic recurrence across extended conversational turns. This result lends support to the future utility of Discursis in distinguishing between interactive (high predicted degree of semantic recurrence) and noninteractive (low predicted degree of semantic recurrence) trouble-indicating behaviors within dementia conversations.

Figure 2.

Zoom in view of Discursis recurrence plot of conversation between person with dementia (PWD) and care staff (CS), demonstrating noninteractive TIB (lack of uptake), and facilitative repair (Posing a solution) and an absence of Discursis metrics. TIB = trouble-indicating behavior.

Similar to trouble-indicating behaviors, knowledge regarding repair types used by people with dementia and their carers in response to conversational trouble is also critical to the development of a SMART communication assistant in being able to offer effective repair strategies in real time. A repair sequence is initiated when a conversational contribution is not accepted (e.g., repetition is requested; Samuelsson & Hyden, 2017). The contribution is repaired, so that communication partners can achieve shared meaning, or common ground (Schegloff et al., 1977). The most commonly observed and preferred repair sequence, typically involves speakers self-corrections of their individual errors or trouble sources (Schegloff et al., 1977; Zahn, 1984). Indeed, positive associations between self-backward Discursis metrics and repair signaling behaviors were identified in the current research.

In a similar manner to interactive versus noninteractive trouble-indicating behaviors, a clear dichotomy was observed in relation to the reliability of Discursis to differentiate between typical versus facilitative repair types in conversation. More specifically, the presence of Discursis metrics (i.e., self-backward /short and /medium) were significantly associated with the presence of typical repairs and an absence of facilitative type repairs. Again, this result was expected given the nature of typical repairs to generally involve the revision of prior conversational turns (e.g., repetition) and hence semantic recurrence, and the converse nature of facilitative type repairs to demonstrate a lack of semantic recurrence (e.g., Posing a solution) across turns (see Figures 1 and 2). With respect to machine learning, this probability differential in relation to the way in which Discursis discriminates between typical and facilitative repair types may provide a future means via which these behaviors can be detected or generated in real time. These findings are important in informing the development of conversational technology, with respect to the types of trouble-indicating behaviors common to people with dementia, and the repair strategies that are routinely used to remediate these trouble sources. Defining the parameters regarding the conversational feedback signals habitually utilized within this population will be critical to the development of algorithms that provide a means of automatically detecting these behaviors.

In typical interactions, repair sequences (i.e., contribution → trouble-indicating behavior [self/other] → Repair [self/other] → Acceptance → New contribution) are generally conducted within three conversational turns (Samuelsson & Hyden, 2017). In line with previous dementia research, however, the alignment of trouble-indicating behaviors and repair types with Discursis metrics over extended conversational turns (i.e., medium and long turn lengths) within the present study indicates that trouble sources may require multifarious attempts to achieve resolution (Samuelsson & Hyden, 2017). Sixty-five percent of repair sequences within conversations between people with dementia and their family caregivers involve greater than three turns, with 6.5 conversational turns representing the average length of repair sequences (Samuelsson & Hyden, 2017). This finding supports the facility for a SMART assistive communication device for people with dementia to be able to monitor conversational content across extended turn lengths, beyond what typical conversations require.

In general, the results of this research indicate that Discursis, a text-analytic tool, can be used to quantify interactive trouble-indicating behaviors and typical repair types, used within the conversations of people with dementia and their care staff, on the basis of recurrence metrics. In contrast, Discursis was not sensitive to the nonrecurrent nature of noninteractive trouble-indicating behaviors and facilitative repair types, which do not typically share the same level of semantic reiteration across conversational turns. These results point to a computational distinction between interactive trouble-indicating and typical repair feedback signals, and noninteractive trouble-indicating and facilitative repair feedback signals. This finding is useful to inform the forthcoming development of machine learning algorithms that may be used to detect conversational trouble and repair signals within real-time conversations.

Threats to mutual understanding are common place within conversations involving individuals with communication impairments, including dementia (Barnes & Ferguson, 2014; Samuelsson & Hyden, 2017). Effective trouble resolution approaches typically involve (a) an evaluation of the speaker’s linguistic difficulty; (b) assisting the speaker to successfully overcome this difficulty; and (c) the facility to save “face” while undertaking this evaluation of difficulty and facilitating its resolution (Bremer, Broeder, Roberts, Simonot, & Margaret, 1987). Equipping caregivers (both family and professional) with a guided means of accurately identifying conversational trouble and offering effective options for repair would enhance communicative interactions for people with dementia and their conversation partners. Indeed, memory and communication (MESSAGE) strategy training programs have been previously developed and applied by our research group, for the purposes of educating care staff of people with dementia (Conway & Chenery, 2016; Smith et al., 2011) with positive results. Subsequent to communication training, carers reported significant increases in knowledge and preparedness to provide care to people with dementia. In line with these findings, the development of a SMART communication assistant that can automatically detect conversational trouble, and offer strategies for repair when required, would be a welcome adjunct to dementia care environments where communication difficulties increase with advancing disease, and effective repair is not always an effortless or instinctual practice for caregivers.

A limitation of the present study, given the relatively small size of the data set, was an inability to undertake statistical analysis relating to specific trouble-indicating behaviors and repair subtypes (as opposed to general group types such as interactive trouble-indicating behavior or typical type repair) and their potential association with Discursis recurrence metrics. Information of this nature would be informative for the purposes of machine learning and the potential development of recurrence thresholds or bands relevant to these specific behaviors. The findings of the current research suggest that an automatic/computational text-analytic tool is capable of distinguishing between certain types of trouble-indicating and repair behaviors represented within conversations between people with dementia and their carers. Next steps involve the evolution of this technology to (a) accommodate the quantification of trouble-indicating and repair behaviors that do not share semantic recurrence across conversational turns; (b) develop speech to text analytics in real time; and (c) analyze nonverbal trouble-indicating signals such as intonation patterns, eye gaze, and so forth.

Any effective SMART communication assistant must be able to monitor and modify feedback signals, akin to that accomplished in human-to-human interaction. Individuals with dementia exhibit specific interactional difficulties that become customary to family caregivers and people with dementia themselves, over time (Hyden, 2011). These difficulties usually increase in frequency and severity as the disease progresses, increasing the division of interactional labor toward the caregiver. Our vision of a SMART communication assistant is a form of technology that can ease the demands of this interactional labor for caregivers via the ability to automatically monitor feedback signals that indicate known trouble and to “scaffold” (Pea, 2004) conversational contributions in a way that engages people with dementia with prior turns (e.g., uses yes/no questions; Hyden, 2011b), or rather, fosters successful repair via the implementation of effective repair signals.

Funding

This work was conducted with support from an Australian Research Council (ARC) Discovery Project (DP150102604) and the ARC Centre of Excellence for the Dynamics of Language (Project ID: CE140100041).

Conflict of Interest

None reported.

45	Person 1 [Person with dementia]:	My mother.
46	Person 2 [Professional care staff]:	Oh, your mother was born in Wales?

44	Person 2 [Professional care staff]:	Your family is from Wales?
45	Person 1 [Person with dementia]:	My mother.
46	Person 2 [Professional care staff]:	Oh, your mother was born in Wales.