Data-driven approaches for identifying links between brain structure and function in health and disease.

Brain imaging technology provides a powerful tool to visualize the living human brain, provide insights into disease mechanisms, and potentially provide a tool to assist clinical decision-making. The brain has a very specific structural substrate providing a foundation for functional information; however, most studies ignore the very interesting and complex relationships between brain structure and brain function. While a variety of approaches have been used to study how brain structure informs function, the study of such relationships in living humans in most cases is limited to noninvasive approaches at the macroscopic scale. The use of data-driven approaches to link structure and function provides a tool which is especially important at the macroscopic scale at which we can study the human brain. This paper reviews data-driven approaches, with a focus on independent component analysis approaches, which leverage higher order statistics to link together macroscopic structural and functional MRI data. Such approaches provide the benefit of allowing us to identify links which do not necessarily correspond spatially (eg, structural changes in one region related to functional changes in other regions). They also provide a "network level" perspective on the data, by enabling us to identify sets of brain regions that covary together. This also opens up the ability to evaluate both within and between network relationships. A variety of examples are presented, including several showing the potential of such approaches to inform us about mental illness, particularly about schizophrenia.

Introduction

Magnetic resonance imaging (MRI) data is a powerful tool which can provide us with detailed information about both brain structure and brain function. The use of high-resolution T1-weighted scans provides comprehensive anatomic information about brain tissue which can be used to estimate the volume of specific regions,[1] cortical thickness,[2] and networks of regions that covary together across individuals.[3] Diffusion MRI (dMRI) gives information about white matter structure, and can be used to assess structural connectivity by using approaches to estimate various diffusion parameters including anisotropy[4] as well as probabilistic tractography using tensor-based models[4,5] and higher-order connectivity through multi-shell imaging and modeling with orientation distribution functions.[6] Finally, functional MRI (fMRI) provides information about brain function via blood oxygenation level-dependent (BOLD) imaging over time[7] and has been used to perform exquisite mapping of intrinsic connectivity networks.[8] All three of these techniques compile important information about brain function and structure and have been widely used in the study of mental illness.

The structure-function synergy

While the most obvious perspective is that structural information leads to subsequent functional changes, the plasticity of brain structure should also be kept in mind, as the functional activity may well reshape structure.[9] It is also well known that structural sulcal/gyral boundaries definable at the macroscopic level do not necessarily correspond well to architectonic studies at the microscopic level.[10] fMRI studies are improved if individualized functional boundaries are incorporated into the spatial normalization strategies.[11] This complex synergy between structure and function increases the complexity of studying mental illnesses such as schizophrenia which are known to impact both structure and function. provides a simple illustration of these relationships; structure can be (and often is) claimed to be a fixed substrate onto which function operates. However, the reality is more complex as we know functional changes can also reshape brain structure, with such changes visible within days even at the macro level.[9] This then highlights the complexity of studying psychopathology, as illness can impact structure or function direction, or the directional relationships between these two modalities.

One of the challenges of complex mental illnesses such as schizophrenia is their highly intricate and diffuse presentation. That is, schizophrenia impacts much of the brain, and there are numerous examples of structural and functional deficits associated with the illness.[12-14] The use of data-driven methods can be particularly useful in this regard, as one can analyze the whole brain with an unbiased lens allowing for weak contributions from many different brain regions. Likewise, the relationship between function and structure can be elucidated with flexible analysis approaches such as independent component analysis (ICA).[15] ICA is a type of blind source separation[16] which allows for the estimation of signals from complex data sets with minimal assumptions on their specific form. In particular, with ICA the goal is to extract maximally independent sources from the data, typically written where is the data, is an estimated mixing matrix, and are the estimated maximally independent sources. In practice, the widely used ICA algorithms such as infomax[17] and fastICA[18] often include two complementary constraints, independence (ie, the maps are maximally independent of one another) and sparsity (ie, the maps have a small number of region with high values), which when combined provide for relatively well-behaved and interpretable results (eg, a component can be interpreted as a brain network[19]) as demonstrated by the extensive number of studies using this approach.[20-25]

Data-driven approaches like ICA can be used to extract brain networks from sMRI, dMRI, and fMRI data and also link together brain structure and brain function.[26] Approaches that can capture all of the possible relationships illustrated in Figure 1 are needed as this is where data-driven approaches provide a distinct advantage. In addition, hybrid approaches that combine the flexibility of data-driven approaches with the ability to focus on specific networks and/or regions are also quite powerful in this context.[27,28] There are a large number of studies to date which have used data-driven approaches such as ICA to study neuropsychiatry and neurological brain disorders.[12,29,30] While the neuropsychiatry field is still very heterogeneous in findings, there is some evidence that the data-driven approaches are producing replicable and canonical results which reliably show differences in schizophrenia[31] and show promise for the use of imaging to make diagnostic or treatment predictions.[32]

In this paper, we briefly introduce data-driven approaches to study brain structure and function with an emphasis on ICA-based approaches. Then we provide examples of several specific variations that have been used to analyze sMRI, dMRI, and fMRI data and also to link together these structural and functional modalities to study the healthy and diseased brain.

Data-driven approaches for analyzing brain structural and functional data

A data-driven approach includes minimizing the upfront assumptions and allowing the data to be the primary driver of the results. This includes approaches such as clustering,[33] principle component analysis (PCA),[34] ICA,[35] and more. In the context of brain imaging data, ICA has proven to be highly versatile and is now a widely used approach.[36] One of the likely reasons for this is that the resulting maps tend to be relatively well-behaved and interpretable due to the incorporation of both independence (which emphasizes non-systematically overlapping networks) and sparsity (which encourages more focal networks) in the estimation of the components.[37]

Source-based morphometry

The use of ICA for sMRI analysis, called source-based morphometry (SBM),[3] can be considered a multivariate extension of voxel-based morphometry.[38] Essentially individual subject data are segmented, and gray-matter maps are organized into a matrix and entered into an ICA analysis that produces maximally spatially independent component maps. These maps include regions that show similar gray matter covariation across subjects. The degree to which they are “expressed” in the data is captured via their loading parameters ( SBM has been studied with multi-site data and a number of the components have been found to be canonical, some consistently impacted by schizophrenia (such as the medial frontal/insular/temporal lobe component 1 in Figure 2).[31]

SBM can also be applied to dMRI data. For example, SBM can be used to analyze fractional anisotropy (FA) maps, which are essentially providing information about the directionality of the diffusion within each voxel. shows results from an SBM of FA study of patients with schizophrenia and healthy controls. A number of the SBM components showed group differences, with all of these showing reduced FA in schizophrenia. This included primarily frontal white matter (thalamic radiation, inferior fronto-occipital fasciculus, and the cingulum bundle) and temporal white matter (left inferior longitudinal fasciculus and the left inferior fronto-occipital fasciculus) in addition to forceps major, forceps minor, corticospinal tracts and the superior longitudinal fasciculus. A key advantage to this approach is that one does not have to make strong assumptions associated with the use of tract atlases. The results for ICA of SBM however do not inform us about the directionality of the diffusion data (an approach for this will be discussed later[39]); rather they identify regions that show similar between-subject covariation in FA values.

Group ICA of fMRI data

In the era of the connectome, which is essentially a structural map of brain connectivity,[40] resting fMRI has entered, and enabled us to estimate a “functional” connectome. Both of these, as measured by MRI, provide a “macro” view of the connectome, a domain in which multiple spatial and temporal domains overlap with one another in a complex manner, nowhere near a microscopic map of neuronal connectivity, and difficult to predict. Using group ICA, one can extract networks that are functionally coherent, that is, identity networks that are showing correlated activity. The output from group ICA[24,41] allows one to evaluate each network separately, essentially to evaluate the within-network connectivity, as well as to study the between-network connectivity, also called functional network connectivity (FNC; Importantly, one can also leverage single-subject spatial maps and timecourses that are estimated from the group ICA through a process called back-reconstruction.[42] Group ICA thus allows estimation of changes in functional connectivity at the voxel level within specific networks, changes in the relationship between pairs of brain networks, as well as other information such as spectral response, and in the case of a task, modulation by task stimuli. One can then evaluate changes in the within and between network connectivity which are related to symptoms, cognitive scores, individual subject prediction of diagnosis or treatment response.

Time-varying connectivity

Recent years have seen a rapidly growing interest in the analysis of brain connectivity which can capture transient changes in connectivity within a given experiment.[44,45] Such chronnectomic information provides a more natural way to analyze fMRI data, especially resting fMRI data which is unconstrained and likely contains many different unmeasured “tasks.” While the field continues to develop,[46] recent work in this area has found high replicability,[47] increased sensitivity to individual subject classification,[48] and evidence of multiple sources of variation including sleep/arousal,[49] emotion,[50] and even mind wandering.[51] In one study, a sliding-window approach was used, followed by clustering, to estimate patterns of connectivity from which individual subjects move into and out of over time. When evaluating the relationship of these “states” with schizophrenia, we found the patients tended to spend more time in the more weakly connected states and vice versa for the healthy controls[43] ( This expands our perspective on schizophrenia beyond the commonly observed weaker connectivity patterns. Additional relationships between cortical and subcortical structures (eg, putamen, thalamus) also showed strong state and group dependencies.[43] Other work has identified the presence of “sticky states” whose presence is correlated with, eg, negative symptoms in the patients.[52] Extensive ongoing work from many groups continues in this area and undoubtedly, such work will provide further insights into brain connectivity and hopefully schizophrenia. In a later section, we will discuss links between time-varying connectivity and brain structure.

Structural networks subserving functional networks

Interestingly, if one compares the results from SBM of sMRI data and group ICA of fMRI data, it is clear that some of the SBM components are quite similar to the resting fMRI networks that are widely studied. The sMRI maps resemble the fMRI maps, but with less specificity (that is typically multiple fMRI networks comprise a single sMRI network), as one might expect. shows an example of results from approximately 600 individuals comparing sMRI and fMRI networks.[53]

The group ICA model can also be used to analyze probabilistic dMRI data in a data-driven manner[39] which, in contrast to SBM of FA, preserves the directional information contained in the dMRI data. In that case, one computes a region-by-region (or voxel-by-voxel) matrix for each individual which contains the diffusion probabilities for traversing the space and enters these matrices into group ICA. This is called connectivity matrix ICA (cmICA). The results from a subset of corpus callosum components output from a cmICA analysis of almost 600 individuals are shown in This model provides a powerful way to decompose such data which is not restricted to specific regions of interest or specific atlases of tracts, allowing the data to speak for itself. Importantly, it also enables us to directly relate seed-based connectivity to group ICA results, and provides a computationally much faster way to compute these measures.[39] Using the single-subject output from the back-reconstruction step allows calculation of group differences in the identified components.[39] One can also combine both functional and structural information within a single analysis using such a model.

Deep learning

More recent models include the development of deep learning approaches which can be applied to neuroscience data.[54] Deep learning refers to an approach in which multiple models are essentially combined together to create a more complex (deeper) model. Deep learning is typically implemented as a stacked neural network, and as such is closely related to ICA (ICA can be modeled as a single-layer neural network). This allows for the capture of more complex and nonlinear relationships within the data. Interestingly, the restricted Boltzmann machine (RBM), a building block of deep learning models, provides results that are highly similar to those of ICA.[55] shows an example of the application of a deep belief network (essentially stacks of RBMs) to sMRI data from a range of patients including schizophrenia, schizoaffective disorder, and bipolar disorder, as well as healthy controls. Interestingly, as the depth of the model goes from one to two to three, the separation between the two groups increases. In addition, despite the more complex model, there continues to be overlap between some subjects, highlighting the complexity of categorizing these individuals who have disorders which share overlapping symptoms and for which we do not yet have an underlying cause. Such approaches can help us with potentially refining diagnostic criteria by characterizing the degree to which individuals are well separated using the biological data.[56]

Data fusion

Importantly, one can also study the relationship between brain function and brain structure more directly, allowing the data from multiple modalities to interact with one another, called data fusion.[15] An approach called joint ICA provides a way to do this, by jointly extracting from multiple modalities maximally independent components that share a common covariation among subjects. The joint components share a common loading parameter which quantifies the degree to which each subject expresses the joint component (; top). An example of joint ICA analysis of brain connectivity during an auditory oddball task and gray matter concentration is shown in Figure 9 (bottom). In this case, the analysis included patients with schizophrenia and healthy controls and the loading parameter associated with one of the components showed a significant group difference. Interestingly, parietal regions showing reduced gray matter in controls were implicated as associated with increases in task activation. Importantly, the (apparent) gray matter reductions were due to decreases in the white matter,[57] further emphasizing the importance of incorporating sMRI, dMRI, and fMRI together.

There are a few approaches capable of combining more than two modalities including linked ICA,[58] independent vector analysis,[59] PCA,[30] parallel ICA, and an extension of joint ICA, called multiset canonical correlation analysis/joint ICA (mCCA+jICA).[60,61] The mCCA+jICA model approach leverages the strength of mCCA, which identifies maximally correlated multimodal latent variables from the data, with joint ICA, which enables leveraging of the higher order statistical information in the data. These approaches provide a fully data-driven way to identify linked multimodal variables. An extension of this approach allows for the incorporation of prior knowledge to constrain the problem further by adding a reference function to the mCCA approach. This approach was used to identify multimodal patterns which are linked to a composite cognition score based on cognitive scores assessed with the MATRICS Consensus Cognitive Battery (MCCB).[62] An extensive network of gray matter, white matter, and functional regions were identified and showed significant differences in schizophrenia patients versus healthy controls ( The analysis and results replicated in a second cohort, underscoring the robustness of the resulting patterns. Interestingly, in addition to implicating specific structural and functional regions in the brain, one can also observe more similarity between individuals in the structural features and more variability in the functional features, reflective of the more state-like aspect of the fMRI measurement.

Structural substrates for time-varying connectivity

Another question of interest is to what degree, and in what ways, is the highly dynamic connectivity which has been observed in the past few years[44,63] dependent on a structural substrate. This is especially important in the context of psychopathology, as there are known structural differences that have been observed.[64-66] Further analyzing the same data set as summarized in Figure 5, we implemented a deep-learning model designed to capture nonlinear links between brain structure and dynamic state. For example, one can use a model that estimates both alignment between different sources of information (eg, the caption and figure in an image) and the sequential relationship between dynamic data (eg, translation of two text phrases).[67] Using this model, we identified two covarying structural networks in temporal lobe, both showing significant differences in alignment between patients with schizophrenia and healthy controls. Interesting, the structural links parallel the amount of time each group spent in a given state. That is, state 3 was occupied more often by the healthy controls and also showed a stronger structure/function alignment with insula/temporal lobe, whereas state 2 was occupied more of the time by schizophrenia patients and showed a stronger structure/function alignment with medial temporal lobe. Such results highlight the highly synergistic relationship between brain structure and brain function and the need for more work in this area.

Conclusion

In conclusion, this paper discussed the synergistic relationship between brain structure and function, provided a selective review of data-driven approaches for capturing multivariate relationship both within and between brain structural and functional measure, and presented multiple analytic examples from prior studies. There are still only a small number of studies that directly evaluate the relationship between brain structure and brain function; without this information we will undoubtedly be ignoring important information that can inform us about the healthy brain and the ways in which it is impacted by psychopathology.

Figure 11

Figure 11.

Nonlinear alignment of brain structure regions to dynamic connectivity patterns (left) reveal significant changes in the alignment of structure in the temporal lobe region in schizophrenia versus controls.[67]