How pattern information analyses of semantic brain activity elicited in language comprehension could contribute to the early identification of Alzheimer's Disease.

Alzheimer's disease (AD) is associated with a loss of semantic knowledge reflecting brain pathophysiology that begins years before dementia. Identifying early signs of pathophysiology induced dysfunction in the neural systems that access and process words' meaning could therefore help forecast dementia. This article reviews pioneering studies demonstrating that abnormal functional Magnetic Resonance Imaging (fMRI) response patterns elicited in semantic tasks reflect both AD-pathophysiology and the hereditary risk of AD, and also can help forecast cognitive decline. However, to bring current semantic task-based fMRI research up to date with new AD research guidelines the relationship with different types of AD-pathophysiology needs to be more thoroughly examined. We shall argue that new analytic techniques and experimental paradigms will be critical for this. Previous work has relied on specialized tests of specific components of semantic knowledge/processing (e.g. famous name recognition) to reveal coarse AD-related changes in activation across broad brain regions. Recent computational advances now enable more detailed tests of the semantic information that is represented within brain regions during more natural language comprehension. These new methods stand to more directly index how pathophysiology alters neural information processing, whilst using language comprehension as the basis for a more comprehensive examination of semantic brain function. We here connect the semantic pattern information analysis literature up with AD research to raise awareness to potential cross-disciplinary research opportunities.

Introduction

Alzheimer's disease (AD) is the most common type of dementia, and a major worldwide health concern. At the time of writing, in the USA alone 5.5 million individuals are affected, and this is forecast to cost 200 billion USD over 2017 (http://www.alz.org/facts/overview.asp). Both figures are only expected to rise as average lifespans increase. Medical health priorities are (1) early identification – there is currently no precise way of forecasting who will succumb to AD-dementia even though neuropathologic warning signs first arise years before cognitive decline; (2) intervention – in order to delay or decelerate irreversible neuropathology and the onset of clinical symptoms.

Goals and intended readership of this article

This article focusses principally on the early identification of AD pathophysiology or AD-dementia. More specifically on research using fMRI to test for early appearing abnormalities in the way that semantic memory is represented and processed in the brain. The first purpose of the current article is to review related literature to evaluate the contribution made by semantic task-related fMRI measures to predicting AD pathophysiology or dementia. This is undertaken in the context of new research framework guidelines (Jack Jr. et al., 2018). The second purpose is to identify how new computational methods that have recently provided ways to decipher semantic information present in young healthy adults' brain activity might be reoriented to attack clinically relevant questions in AD research. The article has been written by authors with different expertise in respective disciplines and is intended to raise awareness to cross-disciplinary opportunities for both audiences. The discussion may be relevant to other neurodevelopmental and neurodegenerative problems, but these are beyond the scope of this article.

Existing clinical, genetic, and pathophysiological factors for early detection of clinical AD

The progression of AD is marked by a cascade of neuropathologic processes that develop over decades and often lead to AD-dementia (Sperling et al., 2011). Memory-based cognitive tests have historically played a major role in clinically categorizing a developmental stage of “amnestic mild cognitive impairment” (MCI, Albert et al., 2011; Petersen, 2004) prior to AD-dementia (McKhann et al., 1984; McKhann et al., 2011). However, a recent National Institute on Aging – Alzheimer's Association (USA) research framework now emphasizes biological factors rather than cognitive symptoms in defining stages of AD progression for intervention and observational research studies (Jack Jr. et al., 2018). The framework is grounded on the presence/absence of three neuropathologic factors: beta-amyloid plaques in the brain, neurofibrillary tau deposits, and neurodegeneration. Critically, amyloidosis and tau are first to appear (Sperling et al., 2011) and distinguish AD from other neurodegenerative diseases that also lead to dementia (Jack Jr. et al., 2018). Though of the two, cerebral amyloidosis is probably the most specific trait of AD, because tau appears in other dementias. Whilst under the new framework amyloidosis and tau positivity are essential for diagnosis of AD-dementia, they do not guarantee dementia. Indeed, some people with high levels of AD-pathophysiology live full lives without cognitive deficits (Sperling et al., 2011). More specifically amyloidosis positivity predicts conversion to AD-dementia at a sensitivity of 95% with a specificity below 60% (Ma et al., 2014) whilst tau positivity predicts conversion at a sensitivity of 75% and specificity of 72% (Ritchie et al., 2017).

Hereditary risk factors such as the genetic presence of allele ε4 of apolipoprotein E4 (APOE ε4) and a family history of AD-dementia are not key components of the new NIA-AA research framework. This is because they indicate risk of pathophysiology rather than presence of pathophysiology, (Jack Jr. et al., 2018). However, they have been extensively researched (Michaelson, 2014) and form the basis of many of the (pre-framework) studies reviewed in this article. APOEε4 together with MCI predict conversion to AD-dementia with a sensitivity of 53% and specificity of 67% (Elias-Sonnenschein et al., 2011).

Because a firm causal relationship between AD-pathophysiological/genetic biomarkers and AD-dementia has yet to be established there is substantial motivation to discover new predictors of AD-dementia. Emphasis is on biomarkers that are detectable early enough to enable a timely deployment of intervention. In addition, there is motivation to develop biomarkers that are less invasive/expensive than the current methods used to measure amyloidosis or tau (Jack Jr. et al., 2018). These require Positron Emission Tomography scans and the injection of a radiotracer or alternatively extracting cerebrospinal fluid.

Challenges in identifying early warning signs of AD in fMRI data

fMRI may uniquely be able to reveal early signs of pathophysiology induced neural dysfunction and/or functional alterations to brain circuitry made to compensate for pathophysiological damage (Wierenga and Bondi, 2007). Alternatively, fMRI could provide a non-invasive route to estimating the presence of AD-pathophysiology indirectly through measures of abnormal brain activity. Despite this potential, progress using fMRI has been slow. A US National Institute on Aging and Alzheimer's Association commissioned team identified fMRI as a “less well validated biomarker of neuronal injury” (Albert et al., 2011). Different to measures of pathophysiology, fMRI firstly requires the identification of cognitive task(s) for participants to undertake during scanning that induce AD-related warning signals. Secondly, fMRI requires the development of quantitative methods to identify what these warning signals look like.

Identifying neural correlates of AD-dementia (as opposed to preclinical AD) is relatively straightforward to accomplish by contrasting fMRI scans of clinical patients with healthy seniors. Even prior to the first semantic task-based fMRI study on AD-dementia patients (Saykin et al., 1999) it was established that functional differences are complex and widespread across the cortex. These include: the absence of normal activation, which is often interpreted as dysfunction; increased peak activation or expanded regions of activation which may reflect compensatory processes (Elman et al., 2014; Reuter-Lorenz and Park, 2014); and, anomalous activation in the form of spatially shifted peak foci and/or the recruitment of remote regions, which could reflect the engagement of alternate brain networks to cope with pathology (Sperling et al., 2011).

Based on the conspicuous differences between healthy seniors and AD patients in fMRI scans, a rational starting point in building an early-stage AD biomarker would seem to be to test for small scale variants of the abnormal brain activity patterns observed in AD-dementia patients. Unfortunately, the complex patterns of abnormal activity observed in AD-dementia patients also appear to develop in complex ways (as further discussed throughout this article) and there is currently no good way of mapping abnormal response patterns backwards in time to estimate what they originally looked like. Longitudinal studies measuring disease development over many years can address this problem. However, they are disadvantaged by being difficult to administrate and ultimately do not address the urgency of the situation.

Because of the difficulties associated with longitudinal studies, preclinical studies have tended to test for anomalies in task-related fMRI activation that coincide with pathophysiological and/or genetic risk factors (by contrasting individuals with high/low amyloidosis or between APOE ε4 carriers and non-carriers). Episodic memory tasks have proved popular in this respect because episodic memory loss is a hallmark of AD-dementia. Episodic memory-related fMRI measures reflect amyloidosis (Elman et al., 2014; Huijbers et al., 2015; Mormino et al., 2012). They also reflect the number and/or composition of APOEε4 alleles at various life stages (Trivedi et al., 2008) including in young adults (Dennis et al., 2010; Filippini et al., 2009; Mondadori et al., 2007), the middle aged (Johnson et al., 2006; Trivedi et al., 2006; Xu et al., 2009), and the elderly (Bondi et al., 2005; Bookheimer et al., 2000; Han et al., 2007). However, whereas meta-analyses of episodic memory studies of clinical-AD patients have identified systematic patterns (Schwindt and Black, 2009), preclinical studies have been less consistent. Activation differences between preclinical APOE ε4 carriers and non-carriers have varied in both direction and location (Trachtenberg et al., 2012). Episodic memory tasks also have the disadvantage of being tiring and frustrating for elderly adults to perform (Sugarman et al., 2012). This may affect their reliability in revealing cognitive deficits in this age group.

Other less studied tasks include tests of auditory verbal working memory (Wishart et al., 2006), visual working memory (Filbey et al., 2010; Filbey et al., 2006) and attention (Gordon et al., 2015). Alternatively, resting state functional connectivity has been put to wide use (see Badhwar et al., 2017 for a meta-analysis). Resting state is advantaged because it requires minimal participant effort, but has been limited by factors such as inter-participant variability (Damoiseaux, 2012). The focus of the current article is however on “semantic-tasks” which employ standardized stimuli, but are relatively effortless for elderly participants to undertake. Whilst not being as popular as episodic memory and resting state studies in the AD literature, semantic task-based fMRI has received sustained attention. The current focus on semantic tasks stems from an interest in pinpointing what new semantic model-based computational methods might have to offer for AD research.

Semantic task-based fMRI experiments

Semantic memory in broad terms refers to the brain's long-term store of learned general knowledge (see Tulving, 1972, and for more recent neurobiological reviews: Binder and Desai, 2011; Binder et al., 2009). The primary focus of the current article is narrowed to the branch of semantics associated with accessing and processing the meaning of words. This is in part because reading or listening to words is a versatile way to stimulate semantic brain systems in a controlled way. Also, because the new computational methods we discuss later in this article are particularly suited to interrogating neural representations of linguistic meaning. However, this is not to say that semantic brain activity cannot be stimulated by non-linguistic stimuli including, but not limited to, pictures or video (Carlson et al., 2014; Huth et al., 2012; Mitchell et al., 2008). Or alternatively that linguistic stimuli cue only semantic memories. For instance autobiographical memories and fictitious experiences can be triggered through verbal stimuli (Hassabis et al., 2007).

Studies of semantics have been an attractive basis for the early detection of AD for several reasons. Physiologically, semantic tasks activate a distributed cortical network (Binder and Desai, 2011; Binder et al., 2009) that encompasses brain regions that are vulnerable to early AD pathophysiology (illustrated later in Fig. 5). Behaviorally, similar to other dementias, AD disrupts semantic memory/processing (e.g. Chertkow et al., 2008; Corbett et al., 2012; Hodges et al., 1990, Hodges et al., 1992; Nebes, 1989; Salmon et al., 1999; Chan et al., 1993; Verma and Howard 2012). This makes semantic decline a relatively sensitive measure of dementia (whether AD-related or otherwise), because semantic memory, unlike episodic memory, remains relatively intact in healthy agers (Craik, 1992; Glisky, 2007; Nilsson, 2003; St-Laurent et al., 2011; Thornton and Light, 2006); Experimentally, semantic tasks are relatively effortless for elderly individuals to perform (Sugarman et al., 2012) because words' meaning is processed automatically in response to stimuli.

Fig. 5

Left. Difference in Amyloid and Tau accumulation and neurodegeneration in 30 amyloid PET-positive patients with mild probable AD comparative 12 amyloid PET-negative healthy controls (Iaccarino et al., 2018). Right. The semantic network as identified and interpreted by Binder and Desai (2011).

A typical semantic task-based fMRI study could involve repeatedly stimulating participants with written or spoken words and scanning their brain activity as they perform a judgement task on the word(s) at hand (e.g. “is it living?”). An important analytic consideration then concerns how to separate fMRI activation associated with meaning from other neural processes associated with processing surface features of words (visual appearance of written words or their sounds), lexical access, and other non-semantic task-related activity. Traditionally this has been accomplished by differencing fMRI scans of words drawn from different semantic categories, or differencing fMRI scans elicited by meaningful words with non-meaningful (nonsense) words. If orthographic/phonetic/linguistic properties of stimuli are appropriately controlled across categories, this contrast leaves behind semantic activation. If stimuli are not well controlled, it becomes ambiguous whether the resultant activation is associated with semantics or something else. Criteria that are essential to control to confidently reveal semantic brain activation are detailed in Binder et al. (2009). For the purposes of a study applying a semantic task purely as a vehicle to elicit fMRI activation distinguishing AD, it may not be essential to pin abnormal activation down to semantics. However, in the current review we shall place emphasis on this. A driving reason for this is evidence (discussed later) that healthy brain systems may adopt the role of failing diseased systems. In order to characterize this process, it is necessary to identify what information is represented in healthy activation to identify where it has been re-mapped to in the face of disease.

AD is associated with developmental transitions from hyper to hypo activation observed in different brain regions when using different semantic tasks

The results of semantic task-based fMRI studies conducted on both preclinical and clinical AD-dementia patients are collated in Table 1. These make the general point that AD pathophysiology, hereditary risk of AD and probable AD-dementia are related to complex changes in semantic task-based activation in widely distributed cortical regions. Here we shall selectively home in on studies shedding light on how semantic task-based activation changes prior to AD-dementia because these are most relevant to biomarker design.

Table 1

Semantic task-based fMRI studies of AD. Studies were identified via a pubmed search using the keywords “semantic”, “fMRI”, “Alzheimer's” conducted in Sept 2017, and a related Google search. Criteria for inclusion were that the studies were related to AD and used words/digits to elicit semantic activation (though some of the articles used pictures as stimuli as well as words).

fMRI task paradigm	AD-related participant group(s) (cognitively healthy seniors, unless stated)	Contol participants (demographically matched cognitively healthy seniors)	Activation difference associated with AD-related group				Notes
			Frontal	Temporal	Parietal	Occipital
Word categorization (Concrete/Abstract)
Lind et al. (2006a)	APOEe4	Non-APOEe4	Low		Low		fMRI analysis used a non-linguistic baseline (fixation cross)
Reanalysis of above (Lind et al., 2006b)	APOEe4 with episodic memory decline	APOEe4 without episodic memory decline			Low
Famous Names
Woodard et al. (2009)	APOEe4 and family history of AD, MCI	Neither with MCI or at genetic risk	High	High	High
Seidenberg et al. (2009)	Family history, Family history and APOEe4	No genetic risk (incl. Family history)	High	High	High
Rao et al. (2015)	APOEe4, 5 year longitudinal study	No genetic risk (incl. Family history)		High to Low	High to Low	High to Low	“High to low” indicates change over 5 year period
Reanalysis of above (Woodard et al., 2010)	Cognitive decline after 18 months	No cognitive decline after 18 months		Low score on cortical principal component			Famous name fMRI complements other risk factors in predicting decline
Reanalysis of above (Hantke et al., 2013)	Cognitive decline after 18 months	No cognitive decline after 18 months		Low score on cortical principal component			Famous name fMRI improves on episodic task fMRI in predicting decline
Smith et al. (2011)	APOEe4 X Physical activity (self-report)	No genetic risk (incl. family) X Physical	High	High			Physical activity increases activation especially for APOEe4 carrriers
Smith et al. (2013)	MCI X Exercise intervention	Control X Exercise intervention		No significant differences between MCI and Control			After exercise intervention activation is decreased in both MCI and controls
Pyramids and Palm Trees
Adamczuk et al. (2016)	Amyloidosis, APOEe4, BDNF	Negative: Amyloidosis, APOEe4, BDNF		High (associated only with amyloidosis)			No differences associated with APOEe4 and BDNF
Vandenbulcke et al. (2007)	MCI	Cognitively intact, fewer APOEe4 carriers		Low
Nelissen et al. (2007)	Clinically prob. AD-dementia, amyloidosis	Negative dementia/amyloidosis		Low (L) / High (R)			Only left MTG negatively correlates with amyloidosis
Reanalysis of above (Nelissen et al., 2011)				High right temporal anterior to posterior connectivity			Functional connectivity analysis
McGeown et al. (2009)	Clinically probable AD-dementia	Cognitively healthy	High & Low				fMRI analysis used a non-word baseline
Category-exemplar / category-function congruence
Saykin et al. (1999)	Clinically probable AD-dementia	Cognitively healthy	High & Low	Low	High & Low	Low	Rest-state used as fMRI analysis baseline.
							AD patients possibly incapable of category-exemplar task
Living/non-living word categorization
Grady et al. (2003)	Clinically probable AD-dementia	Cognitively healthy		Atypical fronto-temporo-parietal network			Functional connectivity analysis of pooled semantic/episodic task data
Pleasantness judgement of words
Grossman et al. (2003a)	Clinically probable AD-dementia	Cognitively healthy	High & Low	High & Low	Low	Low	Stimuli were animal / implement nouns
Grossman et al. (2003b)	Clinically probable AD-dementia	Cognitively healthy	High & Low	High & Low	Low		Stimuli were motion / cognition verbs
Digit recognition/addition/recall
Starr et al. (2005)	Clinically probable AD-dementia	Cognitively healthy	High & Low			High	fMRI analysis contrasted semantic and episodic/working memory task
Statement-property congruence
Olichney et al. (2010)	Clinically probable AD-dementia	Cognitively healthy	High & Low	High & Low	High & Low	Low	fMRI analysis contrasted novel and already seen stimuli
Similarity in shape/color between word pairs
Peelle et al. (2014)	Clinically probable AD-dementia	Cognitively healthy				Low	Stimuli were “natural kinds” or manufactured objects

The relationship between associative-semantic fMRI activation and amyloidosis has been studied in both cognitively healthy individuals (Adamczuk et al., 2016) and early stage AD-dementia patients (Nelissen et al., 2007) using the Pyramids and Palm Trees Test (Howard, 1992). Participants underwent fMRI as they judged which two of three object names/pictures were most similar in meaning (see Fig. 1 for more details). Although temporal, parietal and frontal brain regions were activated in judgement making, only hyperactivation in posterior left mid temporal gyrus of cognitively healthy individuals correlated positively with amyloid load (Adamczuk et al., 2016 and Fig. 1). In contrast, AD-dementia patients were hypoactivated at a nearby site in left superior temporal sulcus, with the degree of hypoactivation correlating negatively with amyloid uptake (Nelissen et al., 2007). In addition to this, a contralateral site in the right superior temporal sulcus of AD-dementia patients was hyperactivated. Because hyperactivation correlated positively with good performance on a behavioral test of anomia (Boston Naming Task, Kaplan et al., 1983) it was hypothesized to be compensatory. Taken together, the two studies suggest that amyloidosis in left posterior lateral temporal lobes is associated with a transition from hyper to hypoactivation as cognitive symptoms develop. As hypoactivation has also been detected in MCI patients using the same task (Vandenbulcke et al., 2007) this transition may occur prior to the onset of cognitive symptoms.

Fig. 1

Pyramids and Palm Trees example stimuli and results from Adamczuk et al. (2016). (Top) “Stimuli and tasks in fMRI experiment. Associative-semantic task with words (blue) and with pictures (purple). Visuoperceptual task with words (cyan) or pictures (yellow). Resting baseline with fixation point (red). Subjects were asked to press a left- or right-hand key depending on which of the 2 lower stimuli matched the upper stimulus more closely in meaning (blue, purple) or in size on the screen (cyan, yellow). A given concept triplet was presented in either the word or the picture format, and this was counterbalanced across subjects. Arrow in the top of the figure shows a timeline of 1 fMRI run, with each condition indicated in its respective color. The order of conditions was randomized for each run and subject. Translation: deur = door, hek = fence, raam = window.” (Bottom) “Area in the left posterior MTG of significant correlation between amyloidosis (SUVRcomp) and fMRI response during associative-semantic minus visuoperceptual condition (Contrast 1) (cluster peak −57, −45, 9, ext = 64 voxels, cluster-level Pcorrected = 0.006). The color scale indicates the T-values. MNI coordinates are indicated in the left upper corner and orientation of the brain in the right upper corner.” Figures reproduced with permission. We note here that whilst the visuoperceptual condition controls for the visual appearance of word/picture stimuli, it is likely to have placed lower demands on working memory. This is because unlike the associative-semantic condition it did not require the meaning of three words to be stored in working memory and compared). Consequently, the contrast map (bottom) may partially reflect this.

Hereditary risk of AD in cognitively intact individuals has been studied using two semantic fMRI tasks. The first contrasted cognitively healthy APOE ε4 carriers and non-carriers as they categorized words as concrete or abstract (Lind et al., 2006a; Lind et al., 2006b). APOE ε4 carriers had reduced activation in the left parietal cortex and left/right anterior cingulate. However, because the analysis was based on subtracting fMRI activation elicited in viewing a fixation cross away from activation elicited during the categorization task it is unclear to what extent the resultant activation is associated with decision making and/or orthographic processing and/or semantics.

The second task used to study hereditary risk is the Famous Name task (Douville et al., 2005; Woodard et al., 2007; Nielson et al., 2006). This scans participants as they identify whether or not peoples' names are famous. In analysis an fMRI contrast map is computed of brain regions that are more activated by famous names than unfamiliar names and vice versa. AD risk was associated with an abnormally broad spread of famous name activation by both Woodard et al. (2009) and Seidenberg et al. (2009). Woodard et al. (2009) detected famous name activation in the frontal, temporal and parietal lobes of a high-risk MCI group (Fig. 2). They then demonstrated that this activation tended to be regionally reduced in asymptomatic APOE ε4 carriers with a family history of dementia, and disappeared altogether in low-risk controls. In a similar vein, Seidenberg et al. (2009) revealed that famous name activation in cognitively intact APOE ε4 carriers with a family history of AD had a similar cortical distribution to Woodard et al. (2009). They too then found that regional activation tended to be reduced in APOE ε4 carriers without a family history of AD and was reduced further still in low risk controls (Fig. 3).

Fig. 2

a. From Woodard et al. (2009). “Regions (shown in blue) demonstrating significant differences between the Famous and Unfamiliar Name conditions, conducted separately for each of the three groups. Brain activation projected on the lateral and medial surfaces of the left and right hemispheres.” Figure reproduced with permission. The b annotation is newly inserted in the current article to facilitate comparison with Fig. 3, Fig. 4.

Fig. 3

From Seidenberg et al. (2009). “Results of voxel-wise analysis demonstrating significant differences between the famous and unfamiliar name conditions, conducted separately for each group: control (CON), family history (FH), and family history and APOEε4 (FH + ε4) groups. Yellow = regions showing greater activation to famous than unfamiliar names; blue = regions showing greater activation to unfamiliar than famous names. Brain activation projected on the lateral and medial surfaces of the left and right hemispheres.” Figure reproduced with permission.

Fig. 4

From Rao et al. (2015). “Voxelwise subtraction of the Famous and Non-Famous Name hemodynamic response functions for the Low Risk and APOE ε4 groups at baseline (0 months), 18 months, and 57 months.” Figure reproduced with permission.

A subsequent longitudinal study (Rao et al., 2015) scanned APOE ε4 carriers with a family history of AD, and controls (lacking either risk factor) three times over five years. At study onset all individuals were cognitively intact. However, by 57 months 8/24 APOE ε4 carriers and 1/21 controls had been diagnosed with MCI. Rao et al.'s (2015) key discovery was that the broader cortical spread of famous name activation observed in APOE ε4 carriers at study onset gradually faded out over time (Fig. 4). By five years, zero famous name activation was detected in the APOE ε4 group. Differently for the control group, famous name activation either remained constant over the five-year period or increased in the left and right fusiform/lingual gyri. The transition to hypoactivation was argued to stem from a phase of neural compensation reverting to burn out (Rao et al., 2015) consistent with the STAC-r theory (Reuter-Lorenz and Park, 2014).

Collating results across studies, both the Pyramids and Palm Trees and Famous Name tasks have linked AD-related factors to a developmental change between hyperactivation and hypoactivation. The clinical phase at which the conversion to hypoactivation occurs remains ambiguous given that Woodard et al.'s (2009) MCI group (Fig. 2) were hyperactivated, and Rao et al.'s (2015) 57 month group (8/24 of who had MCI) were not. However, it appears that hypoactivation can occur prior to the emergence of cognitive deficits, at least in some individuals. This is statistically supported by two separate studies that aimed to retrospectively forecast which of Rao et al.'s (2015) baseline Famous Name study participants would cognitively decline by 18 months (Woodard et al., 2010 and Hantke et al., 2013). In these studies cognitive decline was measured using neuropsychological tests of verbal learning/memory, as well as a dementia rating scale capturing multiple cognitive domains (rather than explicit tests of semantic memory). Woodard et al. (2010) and Hantke et al. (2013) found that both low cortical and low hippocampal fMRI activation were predictive of subsequent cognitive decline. Also, lending weight to the case for using semantic task-based fMRI as a biomarker, Famous Name fMRI-based measures contributed predictive information that was not available from APOE ε4 allele status, hippocampal atrophy and other demographic variables (Woodard et al., 2010). They also predicted cognitive decline more accurately than other fMRI-based measures derived from an episodic memory task (Hantke et al., 2013).

A second noteworthy observation is the dramatic difference in the cortical distribution of brain regions that tests of associative-semantic object knowledge and famous names link to amyloidosis/MCI (Fig. 1) and hereditary risk/MCI respectively (Fig. 2, Fig. 3, Fig. 4). In interpreting these differences, it is necessary to bear in mind that hereditary risk-based and amyloidosis-based contrasts are not equivalent (because APOE ε4 carriers could have been amyloid negative and vice versa) and also the MCI groups tested in the different studies could have differed in their pathophysiology profile. Nevertheless, this raises the possibility that tests of famous names and associative-semantic knowledge might offer different windows on neuropathology because they tap into different neural processes/networks. This prompts further consideration of how AD pathophysiology could affect the cortical organization of the semantic network and why different semantic tasks could be useful in estimating the state of AD pathophysiology at different disease stages.

How the progressive spread of AD-pathophysiology could be reflected by semantic network reconfiguration and charted using different semantic tasks

Post mortem studies (Braak and Braak, 1991; Braak et al., 2006, Braak et al., 2011) and more recently in vivo PET imaging (Cho et al., 2016; Sepulcre et al., 2016; Grothe et al., 2017; Iaccarino et al., 2018;) have charted how amyloidosis and Tau spread throughout the brain as AD progresses. Amyloid accumulation is initially diffuse, observed in middle and inferior temporal, middle frontal, orbitofrontal, inferior frontal, and superior frontal cortices. Subsequently, amyloidosis spreads throughout the entire neocortex starting with the precuneus. Conversely tau accumulates in a more stepwise fashion. Tau arises in entorhinal/pararahippocampal cortex, spreads though the medial temporal lobe to posterior parietal cortex and in later stages reaches the anterior cingulate and prefrontal and motor cortices.

The progression of AD-pathophysiology is at least partially mirrored in fMRI by select changes in the functional connectivity of resting state subnetworks (Jones et al., 2011). These changes affect different subnetworks (posterior/ventral/anterior) at different stages of the disease process (Damoiseaux et al., 2012). In a similar vein it is reasonable to hypothesize that particular subsystems of the semantic network that are vulnerable to AD-pathophysiology will also be disrupted once diseased. Consistent with this, behavioral experiments have detected deficiencies in the storage and/or retrieval (e.g. Chertkow and Bub, 1990; Hodges et al., 1992; Giffard et al., 2002; Laisney et al., 2011) of many knowledge domains including knowledge of people (e.g. Greene and Hodges, 1996; Thompson et al., 2002; Ahmed et al., 2008; Joubert et al., 2010; Clague et al., 2011; Barbeau et al., 2012), objects (e.g. Hart, 1988; Hodges et al., 1992; Ahmed et al., 2008; Joubert et al., 2010), buildings (e.g. Ahmed et al., 2008; Sheardova et al., 2014), living/non-living things (e.g. Silveri et al., 1991; Grossman et al., 1998) and events (e.g. Leyhe et al., 2010; Barbeau et al., 2012). The nature and severity of these deficits varies with disease stage (e.g. Giffard et al., 2002; Joubert et al., 2010; Corbett et al., 2012) which presumably reflects the spread of pathophysiology across select neural substrates. It naturally follows that tests designed specifically to activate vulnerable regions could help to estimate the current state of pathophysiology.

To illustrate this point, Fig. 5 shows the accumulation of AD-pathophysiology in probable AD (Iaccarino et al., 2018) side by side with a rendition of the semantic network (Binder and Desai, 2011). For the particular (group-level) state of AD pathophysiology illustrated, we cautiously surmise that a task that would usually activate the precuneus (green) would be a suitable probe for amyloidosis, which is high in the precuneus. Likewise, that a task activating tau-afflicted posterior ventral temporal cortex somewhere around the “motion” zone (yellow) would be a more suitable probe for tau. A considerable body of research exists to inform which tests could be relevant for testing different regions/disease stages. This research has documented differences in the brain regions activated by different semantic categories/features such as animals or tools (Martin et al., 1996), shapes or colors (Martin, 2016), body parts (Hauk et al., 2004), actions (Desai et al., 2010), sounds (Kiefer et al., 2008), and emotional valences (Vigliocco et al., 2014).

To the authors' knowledge, how different stages of AD-pathophysiology affect the neural representation of different semantic categories has yet to be studied systematically. Nevertheless, proof of principle that different semantic categories elicit different kinds of altered brain activity has come from two studies undertaken on patients with probable AD-dementia (identified clinically). In one study participants judged the pleasantness of different nouns that were either animals or implements (Grossman et al., 2003a). In the other, participants judged the pleasantness of motion/cognition verbs (Grossman et al., 2003b). Both studies detected AD-related differences in activation for each category. The details of these differences were complex and thus beyond the scope of this review. However, to give an impression Grossman et al., 2003a found that in AD-dementia patients a cluster of animal activation in left ventral temporal cortex was displaced posteriorally, whereas for implements remote frontal regions were hyperactivated. Hypoactivation was also observed across broad expanses of frontal, temporal and parietal cortex (though neurodegeneration was not measured).

An additional source of inspiration coming from recent resting state studies is evidence that the emergence and spread of AD-pathophysiology may be tightly related to activity in highly interconnected brain “hubs”. The “cascading network failure model” (Jones et al., 2016) likens neurobiological decline to “cascading failures seen in power grids triggered by local overloads proliferating to downstream nodes eventually leading to widespread power outages”. More specifically they suggest that functional brain failure originates with a drop in the within-region connectivity of a hub in the posterior cingulate/precuneus. Other hubs in retrosplenial/inferior parietal cortex and dorsomedial prefrontal cortex appear to transiently take on the role of the failing posterior hub, before possibly all failing themselves (Damoiseaux et al., 2012). The transferal of roles across hubs was evidenced by increases in between hub connectivity with the failing posterior region. These increases in turn were found to accompany amyloid accumulation. Other studies have revealed that increases in functional connectivity may critically be associated with tau. Amyloid-positive individuals show increased connectivity when Tau levels are low but decreased connectivity when Tau is elevated (Schultz et al., 2017). This could be because densely interconnected hubs accrue more tau pathology, which appears to progressively weaken their connectivity (Cope et al., 2018).

Hub failure is relevant because supramodal “hubs” that integrate semantic information across sensory, motor, and emotional processing systems are a central feature of most contemporary theories of semantic processing (Binder et al., 2009; Binder and Desai, 2011; Pulvermüller 2013; Lambon Ralph et al., 2017). Such cross-modal integration is necessary for us to understand, for instance, that someone saying “I've lit the firework” is likely to prempt a fast moving, noisy, bright and colorful outcome that is both spectacular and dangerous. Posterior temporal, anterior temporal, inferior temporal, inferior frontal, inferior parietal cortex and the precuneus have all been considered in the role of semantic hubs (although different authors emphasize the importance of different regions, including the anterior temporal lobes, Patterson et al., 2007; Lambon Ralph et al., 2017). This hub distribution both overlaps with “cascading network failure” hubs, and more generally with regions that are vulnerable to AD-pathophysiology (Fig. 5). This prompts the hypothesis that cascading network failure may also be visible within the semantic network, with network hubs transiently taking on the role of failing regions. It follows that the migration of semantic function across hubs could be a helpful way to characterize disease progression. Semantic tasks that emphasize “hub-like” integration of information across multiple modalities/categories could play a key role here.

Whilst “cascading semantic network failure” remains as a hypothesis for the future, evidence suggestive of compensatory network re-organization comes from Pyramids and Palm Trees-based tests of associative-semantic object knowledge (see also Fig. 1). Nelissen et al. (2007) suggested the posterior right temporal sulcus may adapt to support compromised left hemispheric function in amyloidosis positive AD-dementia patients. In a reanalysis of the same data, Nelissen et al. (2011) found that increased functional connectivity between the hyperactive right temporal sulcus site and the right anterior temporal pole was correlated with offline performance scores in the Boston Naming Test. These changes were hypothesized to reflect functional reorganization to cope with left temporal amyloid-related damage. An additional study of probable AD-dementia patients (clinically diagnosed) provides evidence of compensatory network recruitment in a semantic/episodic memory task. Using functional connectivity measures, Grady et al. (2003) demonstrated that when control participants categorized words as living/non-living they recruited a left hemisphere network that included prefrontal and temporal cortices. Differently, patients recruited bilateral dorsolateral prefrontal cortex, and temporo-parietal cortex, and the degree of recruitment correlated with improved task performance.

Results of other studies on clinically diagnosed AD-dementia patients are broadly consistent with the above two hypotheses (and summarized in Table 1). This is in the respect that different tasks have revealed different AD-related differences in different cortical regions. Also, that these differences have often included both patterns of hypo and hyper activation, consistent with network reorganization. Tasks applied have included judging category-exemplar and category-function congruence (Saykin et al., 1999), digit recognition/addition (Starr et al., 2005), another implementation of the Pyramids and Palm Trees Test (McGeown et al., 2009), evaluating statement-property congruence (Olichney et al., 2010), and judging shape/color similarities between word pairs Peelle et al. (2014).

In summary, this section has raised two (inter-related) hypotheses. First that tests using different semantic categories to activate different subsystems of the semantic network can help estimate the state of AD-pathophysiology. Second that tasks placing demands on semantic hubs could be particularly revealing of network re-organization, because hubs appear to be hot spots for tau accumulation.

Future work is necessary to firstly characterize how semantic task-based fMRI activation patterns are altered by amyloidosis and tau (the latter has yet to be studied in semantic tasks), and secondly to evaluate whether preclinical activation differences complement amyloidosis and tau in forecasting cognitive decline. Based on the successes of the studies reviewed in this article (Table 1) we believe this work is clinically warranted. However, we contend that new pattern information-based analytic methods and experimental paradigms testing natural language comprehension may be better suited for moving forward.

Limitations of existing analytic metrics and semantic tasks for early identification of AD

Current metrics of regional hyper/hypoactivation have a complex developmental relationship with AD

Current analyses have been successful in identifying how semantic activation in broad brain regions globally differs between high/low AD risk/pathophysiology groups. However, because both abnormally high and low regional activation have been linked to AD, use of this type of metric as the basis of a biomarker is complicated. Whilst the discrepancy is thought to originate from a systematic transition in disease development from a neural compensation phase (high activation) to one of neural exhaustion (Rao et al., 2015), the unfortunate consequence is that at some point high/low risk groups are confusable. As a specific example the APOE ε4 group at 57 months in Rao et al.'s (2015) study (Fig. 4) are indiscriminable from low risk controls of Woodard et al. (2009) (Fig. 2). This is also not a problem specific to semantic tasks because similar hyper to hypo transitions have been observed in longitudinal tests of episodic memory that required memorization of unfamiliar name/face pairs (O'Brien et al., 2010), and see also (Sperling, 2007; Sperling et al., 2003).

Current tasks are specialized but consequently conduct a limited analysis of semantic brain function

Whilst it remains unclear whether the Famous Name and Pyramids and Palm Trees Tests differentially identify different AD risk factors/pathophysiology (APOEe4 was revealed only by Famous Names, and amyloidosis was revealed only by the Pyramids and Palm Trees Test but has not been tested using Famous Names), the studies reviewed in this article suggest that different tasks offer a complementary picture on neural (dys)function associated with AD. On the flipside, this implies that each individual task is limited in its scope to spot dysfunction. For instance, the Famous Name task can be considered as a neurological examination of only the small subspace of semantic activation associated with famous names (and not animals, objects, actions, places, events and so on). Specialized tasks may thus overlook symptoms that would only show up by stimulating different semantic categories/types of processing and miss the opportunity to test for deficits in one category/process relative to others.

Future needs to counteract limitations

Pattern-based metrics of semantic information content within brain regions

It is now established that neural stimulus responses are not only reflected by global activation across broad brain regions (as measured by the studies reviewed in this article), but also by finer grained multivoxel activity patterns within regions (Haxby et al., 2014; Haxby et al., 2001). These finer patterns are thought to reflect neural population codes (Mur et al., 2009). Given that early AD-pathophysiology arises in brain regions associated with semantic function, we hypothesize that new pattern-based metrics of within region information content will more directly index how neural information processing is compromised in afflicted regions. In so doing, they will complement existing measures of regional hypo/hyper activation. We illustrate how this could apply here, in Fig. 6. This Figure simulates three hypothetical ways that regional brain activation could transition to hyper and then hypoactivation by differently dilating, scaling and modulating the signal/noise of the same underlying regional activation pattern. A cross is used to represent a hypothetical neural population code. Although differences in the clarity of the cross in the three conditions are visually obvious, they would go undetected if only whole region activation was measured.

Fig. 6

How current metrics of whole region activation could overlook changes in information within brain regions.

Natural language comprehension tests for a more comprehensive examination of brain function

Because experimental tasks that are based on different semantic categories/processes differentially activate different brain regions that are differentially vulnerable to AD pathophysiology at different disease stages we have argued they are likely to have complementary value for early identification of AD (section 5.2). However scanning participants as they undertake a battery of specialized tasks would be both time consuming and expensive. A more expeditious way to both test over a broad array of different concepts as well as different semantic/linguistic subprocesses could be through scanning natural language comprehension (e.g. reading or listening to a narrative). This has traditionally been challenging because methods to systematically analyze the different neural systems underpinning natural language comprehension have been sparse. However, pattern-information-based fMRI methods have recently provided a foundation for analyzing the semantic and linguistic systems underpinning language comprehension (Huth et al., 2016; Wehbe et al., 2014). We hypothesize they may also provide a basis for testing for AD pathophysiology induced disruption to the brain's language network.

Pattern-based metrics of within region semantic information content

Pattern information analyses test for differences in multivoxel patterns of activity associated with different stimuli which can occur in absence of regional average activation changes (e.g. Mur et al., 2009). Pattern-based analyses of healthy adults have revealed considerable evidence that fine grained information associated with multiple semantic categories is encoded in neural activity in a similar set of brain regions to those linked to AD risk by the Famous Name and Pyramids and Palm Trees studies. Perhaps more importantly many of these regions are vulnerable to early AD-pathophysiology. For instance, Bruffaerts et al. (2013) detected detailed semantic information associated with multiple categories of animals in the left perirhinal cortex (which anatomically neighbors the entorhinal cortex where tau first arises). More generally semantic information associated with various categories of living and non-living things has been detected in multiple subdivisions of the temporal and parietal lobes, as well as some frontal and occipital regions (Mitchell et al., 2008; Just et al., 2010; Anderson et al., 2015; Carota et al., 2017; Connolly et al., 2012; Devereux et al., 2013; Fairhall and Caramazza, 2013; Fernandino et al., 2015; Xu et al., 2018; Zinszer et al., 2016). Because AD pathophysiology is thought to damage connectivity between neurons (in brain regions within the semantic network), and multivoxel fMRI activation patterns are thought to reflect neural population codes (Mur et al., 2009), we hypothesize that a rise in AD pathophysiology will be reflected by a selective drop in the semantic information content of fMRI activity in afflicted brain regions.

Pattern-based analyses typically identify semantic content in fMRI data by testing how closely multivoxel activity patterns correlate with a semantic model of stimulus word meaning. This is a departure from the univariate contrast-based analyses discussed in this article. This difference could be significant if it turns out that multivoxel measures of information content provide an objective way to index (dys)function in neural information processing. In particular, if only low, but not high neural information content is associated with dysfunction this could help bypass the ambiguity associated with current analyses, where both high and low activation are associated with AD risk at different stages of the disease (compensation/burnout).

Building a semantic model that captures word meaning well enough to interrogate brain activity is of course a tall order. However, fortunately semantic modeling has been a topic of extensive research in computational linguistics and cognitive science. This has led to a selection of practically effective solutions that have been put to thorough test in fMRI analyses. The main semantic modeling approaches, their strengths and weaknesses and application to fMRI data are summarized in Box 1. A useful review is Jones et al. (2015).

Feature norm models are based on human report/ratings of how words and their referents are experienced. Participants either generate target word associates which are reinterpreted by the investigators as semantic features (e.g., Cree and McRae, 2003; Vinson et al., 2003). Or, features are fixed in advance and participants rate the importance of those features to concepts (e.g. Binder et al., 2016; Lynott and Connell, 2013). In the latter case a participant might rate on a scale of 0 to 6 the degree to which “thunder” is associated to “vision”, “audition”, “action” and “emotion”. Then thunder could be represented as a vector of these four ratings. Feature-norm models have helped explain fMRI data associated with concrete concepts (e.g. Chang et al., 2011; Bruffaerts et al., 2013; Fernandino et al., 2015; Fernandino et al., 2016a, Fernandino et al., 2016b) and sentences describing events (Anderson et al., 2016a; Anderson et al., 2018). Whilst feature-norm models are well suited to testing modal and supramodal aspects of conceptual representation, they may be disadvantaged for more linguistically oriented abstract concepts.

Taxonomic category models represent concepts in terms of their category membership. Thus, “cat” could be represented as a binary vector listing membership to categories such as “mammal” and “animal”. Large-scale taxonomic hierarchies have been manually assembled and are freely available (e.g. WordNet, Fellbaum, 1998). Taxonomic category models have been applied in similarity-based analyses of concrete concepts (Devereux et al., 2013; Fairhall and Caramazza, 2013) and in the predictive modeling of object/action related content of films (Huth et al., 2012). However, they are less well suited to capturing modal components of experience or representing abstract concepts that do not belong to clear categories (e.g. “morality”).

Text-based distributional semantic models (e.g. Fig. 7) are computational models that approximate words' “linguistic meaning” in terms of the textual contexts they appear in. Thus, pyramids and camels are related because they co-occur together in similar passages of text, and camels and donkeys are related because they both occur in “riding” contexts. Therefore “camel” may be semantically represented as a vector listing the frequencies with which camel co-occurred with each of a set of other words (where “pyramid” and “riding” would score highly). Typically, text-based models are built from multi-million word text corpora. A number of different approaches had been developed to do this, and in practice these often represent word co-occurrences indirectly rather than through explicit counts (Landauer and Dumais, 1997; Lund and Burgess, 1996; Blei et al., 2003; Jones and Mewhort, 2007; Turney and Pantel, 2010; Mikolov et al., 2013; Pennington et al., 2014). Text-based semantic models have been put to widespread use in explaining fMRI data associated with concrete concepts (e.g. Mitchell et al., 2008, Pereira et al., 2013, Carlson et al., 2014; Anderson et al., 2013; Anderson et al., 2015; Anderson et al., 2016b), concrete nouns/verbs (Carota et al., 2017); abstract concepts (Anderson et al., 2017), sentences (Pereira et al., 2018) and narratives (Wehbe et al., 2014; Huth et al., 2016; de Heer et al., 2017). However, they are limited in their ability to capture modal components of experience that cannot be learned from text.

Fig. 7

A simple computational text-based semantic model of word meaning.

Image-based semantic models represent words' visual identity (what dogs looks like) in terms of image-based statistics computed across large databases of manually captioned images (Bruni et al., 2014; Sivic and Zisserman, 2003; Kiela and Bottou, 2014). At a basic level this can be understood by considering an image to be formed from an arrangement of shape/color fragments (aka “visual words”) in much the same way that textual words are arranged to form documents. This enables textual words in image-captions to be represented as a vector listing their co-occurrences with the different visual fragments forming corresponding images. Thus, buildings become associated with hard edges, and stone textures, whereas mammals become associated with round edges and fur. Image-based models have been applied to identify fMRI activation associated with the visual appearance of nouns' referents (Anderson et al., 2013; Anderson et al., 2015; Anderson et al., 2017). Image-based approaches are of course only relevant for modeling imageable concepts.

Connectionist models of semantic knowledge and concept acquisition are inspired by biological neural networks. Models typically comprise multiple layers of nodes (analogous to neurons) that are interconnected by weights (analogous to synapses). Traditionally models are “trained” (e.g. Rumelhart et al., 1986) to map between words presented at an input layer and their semantic properties presented at an output layer (e.g. Rogers et al., 2004; Rogers and McClelland, 2004). Semantic properties can be determined by the experimenter or behavioral norms. Training is accomplished with an optimization procedure that adapts connection weights. Thus, the training process models concept acquisition, and the trained weights model knowledge representation. Recurrent connections can be introduced to model system dynamics. Disruption to neurons and connections has been used to model the consequences of neurological disorders (Rogers et al., 2004; Chen et al., 2017). Traditional connectionist models have had limited application to conceptual brain activity (though see Devereux and Clarke, 2018, and also Chen et al., 2017, who applied connectionist models to interpret functional connectivity data in neurological disorders). However, connectionist methods now form the basis of many state-of-the-art text (Mikolov et al., 2013) and image-based models that have been applied to fMRI (including deep learning image models, Kiela and Bottou, 2014; Anderson et al., 2017; Devereux et al., 2018).

Multimodal models seek to capitalize on the complementary strengths of different modalities of information. The benefits of combining textual and feature-norm data were initially observed in behavioral experiments by Andrews et al. (2009). More recently image and text-based models been fused to help explain conceptual fMRI data associated with nouns (Anderson et al., 2013; Anderson et al., 2015; Anderson et al., 2017). Text and feature-norm have been combined to model fMRI elicited in sentence reading (Anderson et al. under review). Visual data and behavioral property norms have been combined in a connectionist approach that models that transition from visual to semantic processing in visual object naming (Devereux and Clarke, 2018).

Alt-text: Box 1

To simplify the current discussion, we shall be concerned with just one approach, text-based distributional semantic models. These computational models are highly developed, applicable to a diverse selection of linguistic stimuli and freely downloadable. As such they provide a good default starting point for most semantic-model-based neuroimaging analyses. Text-based models (Fig. 7) operationalize the intuition that words with similar meaning appear in similar textual contexts (e.g. cats and dogs but not whales occur in pet-related sentences) by measuring patterns of word co-occurrences in multi-million word bodies of text (Landauer and Dumais, 1997; Lund and Burgess, 1996; Mikolov et al., 2013; Pennington et al., 2014; Turney and Pantel, 2010). Thus, in Fig. 7 the semantic representation for “bee” is a vector of co-occurrence counts between “bee” and other words such as “flying”, “insect”, “honey” and so on. Because “wasp” also co-occurs with “flying” and “insect” it has a similar semantic vector to “bee”, reflecting their similarity in meaning. “Bee” and “bear” also share a relationship in their co-occurrence with “honey”. Here the semantic features of the text-based model correspond to words such as “flying” and “insect” against which co-occurrences are measured. A co-occurrence model may have many thousands of semantic features, though these would typically be reduced using a data reduction approach (such as Singular Value Decomposition).

Text-based models have enjoyed widespread success in explaining healthy adults' fMRI data associated with nouns (e.g. Anderson et al., 2016a, Anderson et al., 2016b; Carlson et al., 2014; Mitchell et al., 2008; Pereira et al., 2013), verbs (Carota et al., 2017); abstract nouns (Anderson et al., 2017), sentences (Pereira et al., 2018) and narratives (de Heer et al., 2017; Huth et al., 2016; Wehbe et al., 2014). However, they are not without weaknesses, and may not be ideal for targeting questions of modal and supramodal conceptual representation (see Box 1 for alternative approaches such as feature-norm models).

To index semantic content in fMRI activity using a semantic model requires relating the semantic model to multivoxel activation patterns. This is typically achieved using either representational similarity analysis (RSA, e.g. Kriegeskorte et al., 2008), or multiple regression (e.g. Mitchell et al., 2008). Whilst RSA and multiple regression differ methodologically in how they correlate models with brain activity, from the current standpoint we consider them as different means to a similar end. Specifically, this is to index the information content in brain activity associated with the model (where high information content is revealed by a high model vs brain correlation). In the context of fMRI-based studies of linguistic meaning, RSA has been applied to analyze scans of isolated words, whereas multiple regression has been applied to analyze not only isolated words but also continuous streams of written/spoken language. In accordance with this, we briefly introduce RSA below in the context of analyzing isolated words. In the next section when we consider the potential advantages of testing language comprehension, we outline the regression-based approach. Similarities and differences between RSA and multiple regression are considered in detail elsewhere, e.g. Diedrichsen and Kriegeskorte (2017).

RSA (Fig. 8) involves inter-correlating fMRI activity patterns for all stimulus word pairs, and likewise inter-correlating semantic vectors for all word pairs. This yields two vectors of correlation coefficients, one for fMRI the other for the model. The match between model and brain is computed by correlating these two vectors. This yields a single correlation coefficient that serves as an approximate measure of the semantic information content in the brain region. Thus, if a brain region represents animal identity, neural activity patterns associated with different types of fish, birds and insects would be expected to cluster according to these categories and correlate with a semantic model that captures this taxonomic structure (e.g. see Bruffaerts et al., 2013). We propose as a working hypothesis that as AD pathophysiology accrues, meaningful clustering of neural activity patterns will disappear. This will be indicated by relatively weak model to brain correlations in afflicted brain regions (e.g. in Fig. 7, “bee” and “wasp” cluster tightly for the model only, but not fMRI).

Fig. 8

Representational similarity analysis (RSA), indexing the semantic information content in a brain region using a semantic model (e.g. Fig. 7).

Natural language comprehension tests for a more comprehensive examination of brain function

Pattern information analysis was previously discussed in the context of isolated word meaning, which is probably best considered as testing semantic memory. However, because word-level studies tend to be based upon relatively slow and repeated stimulation with single words (e.g. one word every 5 to 10 s resulting in fMRI data for 60 words or so, Mitchell et al., 2008) this not only constitutes a limited examination of semantic memory, but also does not address how semantic memory is manipulated relying on more than just memory lookup. Tasks requiring cross-modal knowledge manipulation and integration of multiple concepts could be important for spotting dysfunction (Corbett et al., 2012; Lambon Ralph et al., 2017), because they place demands on interconnected brain regions that are susceptible to AD-pathophysiology (Cope et al., 2018).

To understand natural language the brain must rapidly access semantic memories associated with a diverse array of lexical concepts (people, objects, events, places, and so on) and integrate them together to form composite meanings based on syntactic constraints and context. This engages an array of interacting, but at least partially separable neural systems supporting orthographic/phonetic, lexical semantic, syntactic, pragmatic and discourse level processing (e.g. Fedorenko et al., 2012; Fedorenko and Thompson-Schill, 2014; Lopopolo et al., 2017; Wehbe et al., 2014). We therefore hypothesize that tests of natural language comprehension (e.g. reading/listening to a story) stand to provide a more comprehensive examination of the brain's language network for pathophysiology induced re-organization or damage than existing specialized tests. Additionally, they are patient friendly and comprehension is simple to behaviorally monitor using questions of story content.

The complexities of natural language comprehension make fMRI data particularly challenging to analyze. However, model-based pattern information fMRI analyses have recently begun to introduce ways to interrogate neural processes underpinning sentence and narrative comprehension (Anderson et al., 2016a, Anderson et al., 2016b; Anderson et al., 2018; de Heer et al., 2017; Huth et al., 2016; Lopopolo et al., 2017; Pereira et al., 2018; Wang et al., 2017; Wehbe et al., 2014). These have either concentrated on just modeling semantic representation (Anderson et al., 2018; Anderson et al., 2016a, Anderson et al., 2016b; Huth et al., 2016; Pereira et al., 2018) using lexical semantic models similar to those of Fig. 7). Alternatively, they have modeled multiple processes operating in parallel (de Heer et al., 2017; Lopopolo et al., 2017; Wehbe et al., 2014). For instance, visual word-forms have been modeled in terms of pixelwise measures of words appearances (Devereux et al., 2013) and/or words' lengths (Wehbe et al., 2014). Speech/phonetics has been modeled in terms of articulatory features and/or spectral characteristics of the audio waveform (e.g. de Heer et al., 2017). Parts of speech (noun, verb, adjective), and grammatical relations between words (e.g. the subject or object of a verb) have been automatically estimated from stimulus text (e.g. Nivre et al., 2007) and applied in fMRI analyses (e.g. Lopopolo et al., 2017; Wehbe et al., 2014).

Language comprehension fMRI data has typically been related to models using multiple regression. Language models are first time-aligned to fMRI data, and then multiple regression is applied to fit a predictive mapping between the now synchronous model and fMRI data. Multiple regression can either be implemented forwards, to fit a many-to-one mapping between many model features and a single voxel's activity (a separate regression is trained for each voxel, see Fig. 9, left). Or backwards, where a many-to-one mapping is fit between many voxels and a single semantic model feature. Here a separate regression is trained for each feature (see Fig. 9, right). The linguistic content of fMRI data is typically evaluated through cross-validation, whereby the regression mapping between model and fMRI is fit on a large chunk of the dataset (e.g. 90% of the narrative) whilst the remaining data is held out for testing. For a forward mapping, held-out model data is mapped to predict held-out fMRI data and the correlation between predicted and actual fMRI computed to give a metric of prediction accuracy (Fig. 9, left). For a backward mapping, held out fMRI data is mapped to predict the language model and the correlation taken between predicted and actual model data (Fig. 9, right). These approaches have been used to map out how the semantic and linguistic processing profile varies across the cortex (Wehbe et al., 2014). This can be quantified in terms of how accurately different model features can be predicted from regional brain activity (Fig. 9, right).

Fig. 9

Predicting fMRI activation elicited whilst listening to natural speech using acoustic, grammatical and semantic features (left). Predicting acoustic, grammatical and semantic features from fMRI activation (right).

As working hypotheses, we posit that early stage AD pathophysiology (in particular amyloidosis and tau) selectively damages linguistic/semantic information processing in afflicted brain regions, and that this will be reflected by relatively weaker model-brain prediction accuracies in those regions. We also hypothesize that model-based approaches may provide a route to testing whether the brain's language processing network spatially re-organizes to compensate for encroaching AD-pathophysiology. In either case, quantifying whether regional processing weaknesses in the brain's language network are spatially related to the spread of AD-pathophysiology could provide grounds for new AD-specific biomarkers that help forecast when AD-pathophysiology will result in dementia.

Concluding remarks

This article has collated evidence that semantic tasks induce neural response patterns that are relevant to the early identification of AD and complement other risk measures. We have argued that current approaches could be extended analytically through estimating the information content contained in multivoxel activity patterns rather than just measuring regional activation. As such they could provide a new means to objectively evaluate regional brain function. We have also argued that language comprehension tasks could support more comprehensive neurological examinations for semantic dysfunction than the more specialized tests on specific categories in current usage. A caveat is that the analytic approaches we have advocated, which are all based on matching brain activity to a predictive model, are fundamentally limited by the quality of the model. Nevertheless, we believe that current techniques are sufficiently advanced to make a starting point and will only improve. We contend that pattern-based indices quantifying how neural information processing is regionally compromised will provide a foundation for new sensitive and specific AD-biomarkers.