| Literature DB >> 36009145 |
Sharice Clough1, Caitlin Hilverman1,2, Sarah Brown-Schmidt3, Melissa C Duff1.
Abstract
Speakers design communication for their audience, providing more information in both speech and gesture when their listener is naïve to the topic. We test whether the hippocampal declarative memory system contributes to multimodal audience design. The hippocampus, while traditionally linked to episodic and relational memory, has also been linked to the ability to imagine the mental states of others and use language flexibly. We examined the speech and gesture use of four patients with hippocampal amnesia when describing how to complete everyday tasks (e.g., how to tie a shoe) to an imagined child listener and an adult listener. Although patients with amnesia did not increase their total number of words and instructional steps for the child listener, they did produce representational gestures at significantly higher rates for the imagined child compared to the adult listener. They also gestured at similar frequencies to neurotypical peers, suggesting that hand gesture can be a meaningful communicative resource, even in the case of severe declarative memory impairment. We discuss the contributions of multiple memory systems to multimodal audience design and the potential of gesture to act as a window into the social cognitive processes of individuals with neurologic disorders.Entities:
Keywords: adaptation; audience design; common ground; gesture; hippocampus; language; memory; multimodal; perspective taking; social cognition
Year: 2022 PMID: 36009145 PMCID: PMC9405987 DOI: 10.3390/brainsci12081082
Source DB: PubMed Journal: Brain Sci ISSN: 2076-3425
Figure 1Magnetic resonance scans of hippocampal patients. Images are coronal slices through four points along the hippocampus from T1-weighed scans. Volume changes can be noted in the hippocampal region for Patients 1846 and 2363 and significant bilateral MTL damage including the hippocampus can be noted in Patient 1951. R = right, L = left, A = anterior, P = posterior, and NC = neurotypical brain.
Demographic, Neuroanatomical, and Neuropsychological Characteristics of Participants with Hippocampal Amnesia.
| Neuropsychological Scores | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Demographic Characteristics | Anatomical | Intelligence | Memory | Language | |||||||
| Participant | Sex | Birth Year | Hand. | Ed. | Etiology | Damage | HC Volume | WAIS-III FSIQ | WMS-III GMI | BNT | TT |
| 1846 | F | 1963 | R | 14 | Anoxia | Bilateral |
| 84 |
| 43 | 41 |
| 2363 | M | 1956 | R | 18 | Anoxia | Bilateral |
| 98 |
| 58 | 44 |
| 2563 | M | 1955 | L | 16 | Anoxia | Bilateral | N/A | 102 |
| 52 | 44 |
| 1951 | M | 1952 | R | 16 | HSE | Bilateral |
| 106 |
| 49 | 44 |
| Group Mean |
| 97.5 |
| 50.5 | 43.3 | ||||||
Note: Hand. = handedness. Ed. = years of completed education. HSE = Herpes Simplex Encephalitis. HC = hippocampus. +MTL = damage extending into the greater medial temporal lobes. N/A = no available data. Volumetric data are z-scores as measured through high-resolution volumetric MRI and compared to a matched neurotypical group [45,46]. WAIS-III FSIQ = Wechsler Adult Intelligence Scale–III Full Scale Intelligence Quotient (mean: 100, standard deviation: 15). WMS-III GMI = Wechsler Memory Scale–III General Memory Index (mean: 100, standard deviation: 10). BNT = Boston Naming Test (max score: 60). TT = Token Test (max score: 44). Bolded scores are impaired as defined as two or more standard deviations below normative data.
Sample Demonstrations of Heating up Leftovers in the Microwave for Adult and Child Listeners by a Patient with Amnesia and a Neurotypical Participant.
| Amnesia | Neurotypical | ||
|---|---|---|---|
| Adult | Child | Adult | Child |
| Um, depending on what leftovers dictates the time. Okay, so we’ll just say something simple—mac and cheese. Set it for about forty-five seconds for a dish of mac and cheese. Open the door and put it, uh, put the mac and cheese in a bowl, a glass bowl. It has to be a glass bowl or a Pyrex bowl, one of the two, and, uh, nonmetal. Then you put that into the microwave. Close the door. Time is already set. So, you push start, and then it rings when it’s finished. | First you take a, uh, Pyrex plate, not metal. And you, uh, put your leftovers on the plate that you take out of the refrigerator. And you put it on the amount of time you want to heat it which in this case is probably about a minute and a half or two. Take it out of the microwave. Stir it up to make sure it’s equally heated, and then check to see if it’s the right temperature. If it’s not, then you reheat it, for another, say, fifteen seconds. Once it’s hot enough, then you can eat it. | Um, I would take the leftovers, uh, out of the refrigerator. I would, uh, put them in a microwave-safe vessel. Uh, I would place that vessel… Uh, I would carry that vessel to the microwave. I would open the microwave door. I would place the vessel into the microwave. I would close the door. I would use the keypad to select an appropriate amount of time. And I would push the start button. | So, I want you to go to the refrigerator and take out the leftovers that you want to eat. And those are in a plastic dish, and we can’t put plastic in the microwave. So, I want you to take your leftovers, however much you want to eat, and put them in this glass bowl. Now I want you to take the leftovers that you’ve put in the bowl, and open the microwave door, and put them inside. Close the door. And then I think we need to cook these for one minute. So, push one-zero-zero. And then I want you to push start, and they will be cooking. And when they’re done, you can open the door and take them out. |
Word count as a function of Group (Amnesia vs. Neurotypical) and Listener (Adult vs. Child); Negative binomial family mixed effect model, fit by maximum likelihood (Laplace Approximation).
| Fixed Effects | Estimate | Std. Error | z Value | Pr (>|z|) | |
|---|---|---|---|---|---|
| (Intercept) | 4.591 | 0.239 | 19.252 |
| |
| Group | 0.307 | 0.269 | 1.139 | 0.255 | |
| Listener | 0.176 | 0.116 | 1.514 | 0.130 | |
| Group:Listener | 0.196 | 0.133 | 1.468 | 0.142 | |
| Random effects | Variance | SD | |||
| Participant | (Intercept) | 0.199 | 0.447 | ||
| Task | (Intercept) | 0.015 | 0.121 |
Figure 2(A) Mean total words produced per demonstration for adult and child listeners by patients with amnesia and neurotypical participants. (B) Mean total steps produced per demonstration for adult and child listeners by patients with amnesia and neurotypical participants. Bars represent standard error of the mean. Points indicate mean performance of individual participants, with neurotypical participant points corresponding to the matched patient with amnesia.
Total steps as a function of Group (Amnesia vs. Neurotypical) and Listener (Adult vs. Child); Poisson family mixed effect model, fit by maximum likelihood (Laplace Approximation).
| Fixed Effects | Estimate | Std. Error | z Value | Pr (>|z|) | |
|---|---|---|---|---|---|
| (Intercept) | 1.845 | 0.164 | 11.248 |
| |
| Group | 0.264 | 0.171 | 1.548 | 0.122 | |
| Listener | 0.115 | 0.144 | 0.799 | 0.425 | |
| Group:Listener | 0.227 | 0.162 | 1.404 | 0.160 | |
| Random effects | Variance | SD | |||
| Participant | (Intercept) | 0.065 | 0.256 | ||
| Task | (Intercept) | 0.021 | 0.145 |
Figure 3(A) Proportion of speech categories produced for adult and child listeners by neurotypical and amnesia groups. (B) Mean number of essential steps produced per demonstration for adult and child listeners by patients with amnesia and neurotypical participants. (C) Mean number of optional steps produced per demonstration for adult and child listeners by patients with amnesia and neurotypical participants. (D) Mean number of semantic statements produced per demonstration for adult and child listeners by patients with amnesia and neurotypical participants. Bars represent standard error of the mean. Points indicate mean performance of individual participants, with neurotypical participant points corresponding to the matched patient with amnesia.
Essential steps as a function of Group (Amnesia vs. Neurotypical) and Listener (Adult vs. Child); Poisson family mixed effect model, fit by maximum likelihood (Laplace Approximation).
| Fixed Effects | Estimate | Std. Error | z Value | Pr (>|z|) | |
|---|---|---|---|---|---|
| (Intercept) | 1.370 | 0.138 | 9.899 |
| |
| Group | 0.070 | 0.104 | 0.671 | 0.502 | |
| Listener | 0.050 | 0.182 | 0.273 | 0.785 | |
| Group:Listener | 0.045 | 0.209 | 0.217 | 0.828 | |
| Random effects | Variance | SD | |||
| Task | (Intercept) | 0.043 | 0.208 |
Optional steps as a function of Group (Amnesia vs. Neurotypical) and Listener (Adult vs. Child); Poisson family mixed effect model, fit by maximum likelihood (Laplace Approximation).
| Fixed Effects | Estimate | Std. Error | z Value | Pr (>|z|) | |
|---|---|---|---|---|---|
| (Intercept) | 0.788 | 0.315 | 2.505 |
| |
| Group | 0.486 | 0.293 | 1.660 | 0.097 | |
| Listener | 0.230 | 0.238 | 0.965 | 0.335 | |
| Group:Listener | 0.362 | 0.260 | 1.393 | 0.164 | |
| Random effects | Variance | SD | |||
| Participant | (Intercept) | 0.198 | 0.445 | ||
| Task | (Intercept) | 0.138 | 0.371 |
Semantic statements as a function of Group (Amnesia vs. Neurotypical) and Listener (Adult vs. Child); Poisson family mixed effect model, fit by maximum likelihood (Laplace Approximation).
| Fixed Effects | Estimate | Std. Error | z Value | Pr (>|z|) | |
|---|---|---|---|---|---|
| (Intercept) | 0.352 | 0.372 | 0.946 | 0.344 | |
| Group | 0.343 | 0.362 | 0.948 | 0.343 | |
| Listener | 0.174 | 0.337 | 0.515 | 0.606 | |
| Group:Listener | 0.332 | 0.368 | 0.902 | 0.367 | |
| Random effects | Variance | SD | |||
| Participant | (Intercept) | 0.300 | 0.547 | ||
| Listener | 0.097 | 0.312 | |||
| Task | (Intercept) | 0.156 | 0.395 |
Gesture Rate as a function of Group (Amnesia vs. Neurotypical) and Listener (Adult vs. Child); Linear mixed effect model, fit by REML.
| Fixed Effects | Estimate | Std. Error | df | t Value | Pr (>|t|) | |
|---|---|---|---|---|---|---|
| (Intercept) | 9.090 | 2.064 | 13.968 | 4.404 |
| |
| Group | 2.335 | 2.363 | 13.142 | 0.988 | 0.341 | |
| Listener | 4.844 | 2.142 | 13.320 | 2.262 |
| |
| Group:Listener | −6.701 | 2.493 | 13.160 | −2.688 |
| |
| Random effects | Variance | SD | ||||
| Participant | (Intercept) | 14.818 | 3.849 | |||
| Listener | 11.993 | 3.463 | ||||
| Task | (Intercept) | 0.618 | 0.786 |
Figure 4Mean gesture rate produced per demonstration for adult and child listeners by patients with amnesia and neurotypical participants. Bars represent standard error of the mean. Points indicate mean performance of individual participants. Symbols correspond to each person with amnesia and their demographically matched neurotypical peers.
Gesture Rate as a function of Group (Amnesia vs. Neurotypical) and Listener (Adult vs. Child) with neurotypical group set as reference; linear mixed effect model, fit by REML.
| Fixed Effects | Estimate | Std. Error | df | t Value | Pr (>|t|) | |
|---|---|---|---|---|---|---|
| (Intercept) | 11.425 | 1.279 | 14.056 | 8.933 |
| |
| Group | −2.335 | 2.363 | 13.142 | −0.988 | 0.341 | |
| Listener | −1.857 | 1.276 | 12.722 | −1.456 | 0.170 | |
| Group:Listener | 6.701 | 2.493 | 13.160 | 2.688 |
| |
| Random effects | Variance | SD | ||||
| Participant | (Intercept) | 14.818 | 3.849 | |||
| Listener | 11.993 | 3.463 | ||||
| Task | (Intercept) | 0.618 | 0.786 |
Figure 5(A) Mean proportion of representative gestures produced per demonstration for adult and child listeners by patients with amnesia and neurotypical participants. (B) Mean proportion of two-handed gestures produced per demonstration for adult and child listeners by patients with amnesia and neurotypical participants. Bars represent standard error of the mean. Points indicate mean performance of individual participants. Symbols correspond to each person with amnesia and their demographically matched neurotypical peers.
Representative gestures as a function of Group (Amnesia vs. Neurotypical) and Listener (Adult vs. Child); binomial family mixed effect model, fit by maximum likelihood (Laplace Approximation).
| Fixed Effects | Estimate | Std. Error | z Value | Pr (>|z|) | |
|---|---|---|---|---|---|
| (Intercept) | 0.966 | 0.635 | 1.522 | 0.128 | |
| Group | −0.086 | 0.546 | −0.158 | 0.874 | |
| Listener | 0.437 | 0.417 | 1.048 | 0.295 | |
| Group:Listener | −0.302 | 0.458 | −0.660 | 0.509 | |
| Random effects | Variance | SD | |||
| Participant | (Intercept) | 0.716 | 0.846 | ||
| Listener | 0.192 | 0.438 | |||
| Task | (Intercept) | 0.702 | 0.838 |
Two-handed gestures as a function of Group (Amnesia vs. Neurotypical) and Listener (Adult vs. Child); binomial family mixed effect model, fit by maximum likelihood (Laplace Approximation).
| Fixed Effects | Estimate | Std. Error | z Value | Pr (>|z|) | |
|---|---|---|---|---|---|
| (Intercept) | −1.313 | 0.462 | −2.844 |
| |
| Group | 1.623 | 0.436 | 3.725 |
| |
| Listener | 0.417 | 0.286 | 1.456 | 0.145 | |
| Group:Listener | −0.908 | 0.313 | −2.901 |
| |
| Random effects | Variance | SD | |||
| Participant | (Intercept) | 0.457 | 0.676 | ||
| Task | (Intercept) | 0.278 | 0.527 |
Mean and standard deviation values for outcome variables stratified by task and group.
| Amnesia | Neurotypical | |||||||
|---|---|---|---|---|---|---|---|---|
| Shoe | Coffee | Microwave | Lamp | Shoe | Coffee | Microwave | Lamp | |
| Word Count | 133 (150) | 95 (37) | 119 (93) | 96 (62) | 143 (87) | 176 (93) | 141 (63) | 127 (67) |
| Total Steps | 6.6 (1.8) | 7.7 (2.4) | 5.5 (1.7) | 6.0 (1.1) | 8.6 (3.1) | 11.2 (5.5) | 7.9 (2.9) | 7.5 (4.1) |
| Gesture Rate | 12.3 (5.0) | 8.7 (4.8) | 6.2 (6.2) | 9.0 (5.6) | 11.9 (5.2) | 12.1 (5.9) | 11.7 (6.2) | 10.1 (4.5) |
| Representative Gestures | 0.81 (0.35) | 0.73 (0.32) | 0.66 (0.30) | 0.56 (0.38) | 0.89 (0.16) | 0.62 (0.27) | 0.53 (0.25) | 0.58 (0.21) |
| Two-Handed Gestures | 0.49 (0.26) | 0.15 (0.27) | 0.08 (0.13) | 0.04 (0.06) | 0.70 (0.16) | 0.54 (0.23) | 0.55 (0.33) | 0.48 (0.29) |
Word Count = Total number of words produced; Total Steps = Total number of steps produced; Gesture Rate = Gestures per 100 words; Representative Gestures = Proportion of total gestures that were representative (iconic + deictic); Two-Handed Gestures = Proportion of total gestures produced with both hands. Values in parentheses are standard deviations.