BACON: A tool for reverse inference in brain activation and alteration.

Over the past decades, powerful MRI-based methods have been developed, which yield both voxel-based maps of the brain activity and anatomical variation related to different conditions. With regard to functional or structural MRI data, forward inferences try to determine which areas are involved given a mental function or a brain disorder. A major drawback of forward inference is its lack of specificity, as it suggests the involvement of brain areas that are not specific for the process/condition under investigation. Therefore, a different approach is needed to determine to what extent a given pattern of cerebral activation or alteration is specifically associated with a mental function or brain pathology. In this study, we present a new tool called BACON (Bayes fACtor mOdeliNg) for performing reverse inference both with functional and structural neuroimaging data. BACON implements the Bayes' factor and uses the activation likelihood estimation derived-maps to obtain posterior probability distributions on the evidence of specificity with regard to a particular mental function or brain pathology.

Bayes fACtor mOdeliNg

functional magnetic resonance imaging

gray matter

voxel‐based morphometry

white matter

INTRODUCTION

The notion of “cerebral areas” is central in the brain imaging research on normative and clinical population. The complex activity emerging from the neuronal territories is the forming part of the physiological basis of cognitive and sensorimotor functions. On the other hand, neuronal regions may present different degrees of anatomical variation, depending on the pathology by which they are affected.

In the last decades, powerful in vivo MRI‐based methods have been developed; they yield maps of the brain structure and function underlying different conditions and processes, respectively. For example, functional (fMRI) maps, generated in resting‐state or task‐related manner, provide a link between activity of certain parts of the brain and specific mental functions (Poldrack, Mumford, & Nichols, 2011), whereas voxel‐based morphometry (VBM), measuring differences in local concentrations of brain tissue, allows to establish relationships between abnormalities in the brain structure and the occurrence of pathological conditions (Ashburner & Friston, 2000).

With regard to functional or structural MRI data, forward inferences try to determine which areas are involved given a mental function or a brain disorder (Henson, 2006). In other words, forward inferences aim to answer questions like “Which are the brain areas involved in such mental process?” or “Which are the altered brain areas in a given pathology?”. In this case, answers are sought by means of frequentist statistical procedures, such as the maximum‐likelihood method. However, the major drawback of forward inference is its lack of specificity, in that it tends to suggest involvement of areas that are not specific for the variable under investigation. For instance, a brain area can be found activated during different processes, resulting in a large overlap of functions as shown by Anderson, Kinnison, and Pessoa (2013) and Cauda et al. (2012). Similarly, a neuronal subpopulation can be damaged by a wide range of brain disorders (Cauda et al., 2019; Crossley et al., 2014; Goodkind et al., 2015; Liloia et al., 2018) and this significantly reduces the utility of forward inference and of the associated maps.

Thus, a different approach is needed to determine the specific cognitive function or pathology associated with a pattern of activation or alteration in the brain and, in particular, to answer questions such as: “To what extent the activation of (a pattern of) brain areas is specific to pain perception?” or “How much the alteration of (a pattern of) brain areas is specific to a certain pathology?”. This is precisely the aim of reverse inference‐based analyses, which are theoretically founded on the Bayes' theorem.

Reverse inference has been adopted in a number of neuroimaging studies, starting from the seminal work by Poldrack (2006), who applied reverse inference to meta‐analytical data obtained from the functional sector of the BrainMap database (Fox et al., 2005; Fox & Lancaster, 2002; Laird, Lancaster, & Fox, 2005; Vanasse et al., 2018). In particular, Poldrack examined how specific the activation of the Broca's area was for the language function. Moreover, a recent study by Cauda et al. (2020) points out that reverse inference can determine specific morphometric pattern of alterations due to a given brain disorder.

In the years following Poldrack's work (2006), the use of Bayes' theorem in neuroimaging has been subjected to a lively debate, mainly centered on the choice of priors, which is a critical aspect in the application of the theorem (Gelman, 2017; Hutzler, 2014; Lieberman, 2015; Lieberman & Eisenberger, 2015; Machery, 2014; Poldrack, 2013; Shackman, 2015; Wager et al., 2016; Yarkoni, 2015a, 2015b). Difficulties in making correctly a reverse inference have been extensively discussed after the publication of Lieberman and Eisenberger (2015) (Gelman, 2017; Lieberman, 2015; Poldrack, 2013; Shackman, 2015; Wager et al., 2016; Yarkoni, 2015a, 2015b), who claimed that activity of the anterior dorsal cingulate is selective for pain. Results by these authors are based on analyses performed with the help of Neurosynth, a freely accessible platform and database for the automated synthesis of fMRI data (http://www.neurosynth.org; Yarkoni, Poldrack, Nichols, Van Essen, & Wager, 2011).In assessing the feasibility of reverse inference, it should be noted that it requires not just a sample of MRI data related to task or pathology (“everything that IS A”), but also data concerning the negation of the sample (“everything that IS NOT A”). This can be achieved through an extensive search on databases with peer‐reviewed contents (e.g., PubMed, PsycInfo, Web of Science, BrainMap, etc.).

To date, the only available method for computing an estimate of specificity (i.e., association test) is Neurosynth. This platform, albeit valuable, has some limitations concerning its structure and the possibility to manipulate the query about the data in a too limited way. Furthermore, Neurosynth stores fMRI results but lacks structural data, which are essential in order to investigate patterns of brain alteration (Cauda et al., 2018). Further limitations concern methodological and statistical issues. The association test can be performed directly via the Neurosynth interface, but proper reverse inference requires programming of the “core tools” of Neurosynth. More importantly, Neurosynth is based on the frequentist approach and inherits its limitations. Frequentist methods, besides requiring a large set of data to perform consistent inference, do not yield probabilities of the validity of hypotheses, but solely a rejection criterion for the null hypothesis, which can lead to misinterpretations. Finally, the kernel used for the association test is a spherical kernel with fixed radius, so that the resulting map indicates for each voxel the number of peaks within r millimeters (mm).

In order to provide the neuroimaging field with a sound reverse inference method, in the present study we propose a new computational tool called BACON (Bayes fACtor mOdeliNg) which can perform reverse inferences both with functional and structural neuroimaging data. BACON is based on the Bayesian statistic, especially on the Bayes' factor (BF), and uses activation likelihood estimation (ALE) derived‐maps generated by the GingerALE software package (Eickhoff et al., 2009; Eickhoff, Bzdok, Laird, Kurth, & Fox, 2012; Turkeltaub et al., 2012).

It is worth noting that the proposed Bayesian approach presents several advantages with respect to the frequentist methods: (a) it provides not just a dichotomous reject/do‐not‐reject decision with respect to null‐hypotheses, but it computes the evidence in favor of each of the hypotheses under consideration; (b) it can be used for the evaluation of multiple hypotheses; (c) data supporting the hypotheses of interest can continuously be updated (i.e., Bayesian updating); (d) differently from Neurosynth, the ALE uses a Gaussian kernel with the variance modeled by the number of subjects in the experiment, which allows to model the variability of the peaks distribution; finally, (e) the method provides the final probability of activation (or alteration) taking into account the independent activation (or alteration) of the foci.

Here we show the rationale and the statistics behind the use of BF in the analysis of meta‐analytic data and present a different and improved approach to the calculation of the BF on neuroimaging data. This approach is implemented in a new tool, named BACON (Bayes fACtor mOdeliNg), designed to calculate the BF starting from meta‐analytic neuroimaging data. BACON was developed as a plug‐in for the software Mango (ric.uthscsa.edu/mango; Multi‐image Analysis GUI), that is, a viewer for medical images endowed with several analysis tools (Lancaster et al., 2010; Lancaster et al., 2011; Lancaster et al., 2012). As a proof of concept, we test BACON using both functional and structural data sets extracted from BrainMap.

The BrainMap database stores peak coordinates (x, y, z) from published neuroimaging studies, along with the corresponding meta‐data that summarize the experimental design (Fox et al., 2005). The BrainMap environment has been designed to facilitate quantitative synthesis of neuroimaging results reported in the literature (Vanasse et al., 2018), especially through the ALE method. Compared to Neurosynth, BrainMap presents several advantages: (a) it contains functional meta‐data reporting both activation and deactivation which can be searched separately; (b) it contains structural meta‐data reporting both gray (GM) and white (WM) matter results, as well as increase and decrease morphometric variations; (c) a dedicated team supervises the correctness of the included data; (d) it has a set of software tools, such as Scribe, Sleuth, GingerALE (Eickhoff et al., 2012; Fox et al., 2005; Fox & Lancaster, 2002), Mango, Papaya (Lancaster et al., 2010; Lancaster et al., 2011; Lancaster et al., 2012), and ALE‐derived methods, such as the meta‐analytic connectivity modeling (MACM) (Laird et al., 2013; Robinson, Laird, Glahn, Lovallo, & Fox, 2010) and the morphometric co‐alteration networking (MCN) analysis (Cauda et al., 2018; Cauda, Nani, Manuello, et al., 2018; Manuello et al., 2018; Tatu et al., 2018). All these resources make BrainMap a simple but powerful instrument of research.

BACON is therefore an important complement to BrainMap, as it provides a new method for performing reverse inference‐based analyses. This is particularly insightful for the study of brain disorders, since BACON allows to distinguish between areas that are altered by the majority of brain disorders and areas that are specifically affected by certain diseases. What is more, when considering neuropathology under the aspect of its temporal progression, the BF index can identify which cerebral areas are likely to be early affected by the disease (Cauda et al., 2020).

MATERIALS AND METHODS

Reverse inference and BF definition

The basis of reverse inference is the Bayes' theorem. To place this in the field of neuroimaging, let us consider two hypotheses: H 0 affirming the occurrence of a specific state (e.g., a pathological condition such as schizophrenia, or a mental function such as the processing of painful stimuli), and H 1 affirming the absence of that state (i.e., the negation of H 0). Let D be a measurement of activation (or structural alteration) in a brain area or voxel. What is the probability that, given that measurement (D), the specific state is actually occurring (i.e., H 0 is true)?

According to Bayes' theorem, we have:and, correspondingly,

The quotient of these probabilities represents the Bayes' theorem in terms of relative belief:Let us then consider the priors P(H 0), P(H 1). If no knowledge of these priors is available, it is natural to consider them as identical (Cauda et al., 2020; Jaynes, 2003): Therefore, the BF01 can be expressed as:BF01 gives the degree of evidence for the two hypotheses: if BF 01 is greater than 1, the evidence favors H 0; if BF 01 is less than 1, the evidence favors H 1.

Considering Formulas (3) and (4), BF 01 can be also written as: Since the sum of the two posterior probabilities must be 1 formula 6 can be written as: (See Figure 1 for a graphical interpretation of the formulas here described).

FIGURE 1

Graphical representation of the role of the maps, previously obtained in the pipeline, for the Bayes' Factor calculation as implemented in the plug‐in. In the formula, the numerator represents the probability calculated from the unthresholded ALE map showing the effect of interest. In the denominator, the probability calculated from this same map is summed with the probability calculated for its negation (i.e., the unthresholded ALE map computed on the experiments not showing the effect of interest). The final map represents values of probabilities, obtained by means of the BF 01 computation, and can be thresholded depending on the desired level of probability. Maps are showed for visualization purpose only, and are not based on the data described in the Section 2

Inverting Formula (8) we obtain:The advantage of this expression is that we can directly obtain a posterior probability by computing the BF. This allows to choose the probability level according to which we want to verify our hypothesis.

Using ALE‐derived map for the calculation of the BF

For the calculation of the BF, we used the maps obtained by means of the ALE computation (Eickhoff et al., 2012). In this method each focus of every experiment is modeled by the ALE as a Gaussian probability distribution:where d indicates the Euclidean distance between the voxels and the considered focus, and σ indicates the spatial uncertainty. Subsequently, we determined for every experiment a modeled alteration (MA) map as the union of the Gaussian probability distribution of each focus of the experiment. The union of these MA maps provided the final ALE map. The significance of the ALE activations (or alterations) has been then tested against an empirical null distribution, thus yielding a p‐value (see also Eickhoff et al., 2012) for a detailed description of the algorithm). As the ALE creates smooth estimates of the local abundance of peak activations and is equivalent to a density estimation, for our purposes we normalized this density so as to obtain a probability function.

Plug‐in description

BACON is a plug‐in for Mango viewer (Lancaster et al., 2010; Lancaster et al., 2011; Lancaster et al., 2012) that allows to compute reverse inference, based on the calculation of the BF. It is coded in Python and can be installed and executed using the Script Manager of Mango. This plug‐in is intended to be used in conjunction with the other software packages and tools developed by the BrainMap Project.

BACON requires as input a couple of ALE‐derived maps. The most straightforward method to obtain this is to search data in the BrainMap database by means of the software Sleuth (http://www.brainmap.org/sleuth/) and then perform the ALE with GingerALE (www.brainmap.org/ale; see Figure 2). An extensive ad‐hoc search using a different database is also possible. Crucially, if interested for example in GM abnormalities in schizophrenia, the first query has to contain the criterion “IS schizophrenia”, while the second query has to contain “IS NOT schizophrenia” (see the queries used in the plug‐in testing section and in the Supporting Information). Therefore, the first map will be the result of the ALE computed on the data representing the domain of interest. The second map will be the negation of the first one (i.e., everything but the domain of interest). It is important to not invert the order of the maps during the upload, as this would affect the calculation and produce invalid results. Of note, the two ALE maps must be unthresholded and in nifty format. The output map contains in each voxel the value of the posterior probability obtained as previously described, and can be saved as floating‐point 32‐bit image (a detailed description of the plug‐in usage is provided in the BACON Manual).

FIGURE 2

Examples of pipeline for the calculation of specificity using the BACON plug‐in

Plug‐in testing

To assess the functionality of our plug‐in we performed two different analyses of reverse inference: a first one on fMRI data about pain tasks, and a second one on VBM data about the comparison between subjects with schizophrenia and healthy subjects. For the former, in order to have the two required data sets, we made the following two queries in the functional BrainMap database sector (April 2018):

[Experiments Context IS Normal Mapping] AND [Experiments Activation IS Activation Only] AND [Subjects Diagnosis IS Normals] AND [Experiments Imaging Modality IS fMRI] AND [Experiments Paradigm Class IS Pain Monitor/Discrimination];

[Experiments Context IS Normal Mapping] AND [Experiments Activation IS Activation Only] AND [Subjects Diagnosis IS Normals] AND [Experiments Imaging Modality IS fMRI] AND [Experiments Paradigm Class IS NOT Pain Monitor/Discrimination].

The VBM sector of BrainMap was queried for the analysis of schizophrenia (SCZ). The search algorithms were constructed as follows:

[Experiments Context IS Disease] AND [Experiment Contrast IS Gray Matter] AND [Experiments Observed Changes IS Controls > Patients] AND [Experiments Observed Changes IS Controls < Patients] AND [Subjects Diagnosis IS Schizophrenia];

[Experiments Context IS Disease] AND [Experiment Contrast IS Gray Matter] AND [Experiments Observed Changes IS Controls >Patients] AND [Experiments Observed Changes IS Controls < Patients] AND [Subjects Diagnosis IS NOT Schizophrenia].

For the overview of the selection strategy, see also the Supporting Information Methods and Figure S1. Detailed information about the meta‐data of the selected fMRI experiments are viewable in the Tables S1 and S3. For more information about the meta‐data for the VBM set, see the Table S2.

Starting from the retrieved functional data, we calculated the ALE for the conditions “pain” and “no‐pain,” using the random effects algorithm as implemented in GingerALE (v.2.3.6, http://brainmap.org; Eickhoff et al., 2012; Eickhoff et al., 2009; Turkeltaub et al., 2012). Subsequently, we determined the posterior probability using BACON (see also the Supporting Information Material and Figure 3). The same procedure was repeated for the data on schizophrenia. In order to verify with an external tool if the voxels with high values in the map obtained for the domain “pain” were actually more specific for that condition, we employed the behavioral analysis tool of Mango (http://ric.uthscsa.edu/mango/plugin_behavioralanalysis.html; Lancaster et al., 2011; Lancaster et al., 2012). Of note, this method relies on the spatial extent of the map it is applied to. For this reason, in the absence of a specific cut‐off value to be used, it is possible to repeat the analysis for more than one threshold. In the present case, each voxel with a value in the posterior probability map obtained through BACON is significant. Therefore, the output map for the pain domain has been thresholded at 0.7 and 0.8, to test more extreme (and specific) voxels. Finally, we repeated the behavioral analysis on the association map for the term “pain,” available on Neurosynth, so as to compare the results obtained with the two methods.

FIGURE 3

Posterior maps of the specificity of the PAIN threshold at p(PAIN| activation) = 0.7 (70%) and p(PAIN| activation) = 0.8 (80%)

With regard to structural data on schizophrenia, we tested the related probability through the disease analysis plug‐in available in Mango (http://ric.uthscsa.edu/mango/plugin_diseaseanalysis.html). For the reasons explained above, the same two thresholds were used in this case as well.

RESULTS

Pain domain

The resulting posterior probability map for the pain domain shows the involvement of bilateral insula (BA 13), left cingulate gyrus (BA 24), right middle and inferior frontal gyrus (BA 10), bilateral thalamus (right medial dorsal nucleus and bilateral ventral lateral nucleus), right postcentral gyrus (BA 5), right putamen (lentiform nucleus), and left superior temporal gyrus (BA 22) (Figure 6; see Tables S4 and S5 for the different thresholds).

The posterior probability map was submitted to the Mango behavioral analysis plug‐in. Results are different for the different thresholds. Using the probability map threshold at p= .7, we show that only 7 out of 51 behavioral subdomains exceed the level of statistical significance (i.e., z ≥3, corresponding to a threshold of p < .05 with Bonferroni correction for multiple comparisons). Notably, the subdomain Somesthesis (pain) showed the highest z‐value. Using the probability map threshold at p= .8, only the Somesthesis (pain) and Somesthesis (other) subdomains report significant results. Finally, we submitted the map generated by Neurosynth (i.e., association map) to the behavioral plug‐in. Overall, results show a lower specificity compared to the probability map obtained with BACON (see also Figure 4).

FIGURE 4

Comparison of the results of the behavioral analysis on the maps related to the cognitive domain “pain.” Top panel refers to the map produced by BACON and thresholded at p = .7. Middle panel refers to the map produced by BACON and thresholded at p = .8. Bottom panel refers to the map available on Neurosynth. Colors refer to the domain, as they are organized in the behavioral analysis tool. All the showed sub‐domains are statistically significant (z ≥3)

Schizophrenia condition

The resulting posterior probability map for SCZ shows the involvement of the right middle temporal gyrus (BA 21), bilateral precentral gyrus (BA 6), left middle frontal gyrus (BA 10), left superior temporal gyrus (BA 22 and BA 42), and bilateral precuneus (BA 7) (Figure 5; see Tables S6 and S7 for the different thresholds).

FIGURE 5

Posterior probability maps of the specificity to Schizophrenia, thresholded at p(SCZ| alteration) = 0.7 (70%) and 0.8 (80%)

The posterior probability map obtained with VBM data of SCZ was submitted to the disease analysis plug‐in of Mango. For both the thresholds used (i.e., 0.7 and 0.8), only SCZ exceeds the level of statistical significance (i.e., z ≥3 corresponding to a threshold of p < .05 with Bonferroni correction for multiple comparisons; Figure 6).

FIGURE 6

Comparison of the results of the behavioral analysis on the maps related to schizophrenia. Top panel refers to the map produced by BACON and thresholded at p = .7. Bottom panel refers to the map produced by BACON and thresholded at p = .8

DISCUSSION AND CONCLUSION

In the present study, we present a novel application of the Bayes' theorem to the analysis of neuroimaging data and, more specifically, to the computation of their specificity with respect to a given process or pathological condition. The calculation of the BF, which here has been specifically carried out for ALE maps, made possible to show results in terms of probability; this, in turn, has led to the notion of different degrees of specificity. This approach was implemented in BACON, a user‐friendly but powerful plug‐in, which can be used not only in the pipeline of BrainMap meta‐data, but also with other databases storing peer‐reviewed neuroimaging results. The behavioral analysis confirmed the reliability of the tool. In fact, the output map thresholded at 0.7 (i.e., a probability of the 70%) was associated with only 7 out of 51 possible cognitive domains, the strongest being pain. Increasing the threshold to 0.8 reduced the association to just two domains. The disease analysis carried out on the SCZ map showed confirmatory evidence as well. This result was not obvious, given the high degree of overlap between the pattern of structural alteration associated with different psychiatric and neurodegenerative disorders (Cauda et al., 2019; Cauda et al., 2020).

Of note, the probability maps obtained through BACON are less prone to misinterpretation than those constructed following the canonical frequentist approach. In fact, in the Bayesian framework the results directly represent the probability of the phenomenon. This allows to answer in a straightforward way the following question: how probable is that the brain areas in the map are specifically involved in a given process? Moreover, the BF compares two hypotheses (H 0vs. H 1) that are fully conditioned by the observed data. For this reason, it is possible to investigate what it is specific as well as what it is nonspecific. On the contrary, in the more often used approach based on the p‐value, if the null hypothesis (H 0) is rejected, nothing can be said about the alternative hypothesis. Finally, the results obtained with the BF are not affected by the problem of multiple comparisons, as it is the case with the frequentist approach.

It should be noted that thus far the analyses based on the BF reported in the neuroimaging literature were carried out considering data as binary (i.e., a voxel is active or nonactive during a given task; Poldrack, 2006). On the contrary, the ALE method used here considers the probability of activation (or alteration) of a voxel across a set of experiments; this enables a more precise modeling of the distribution of the activation peaks. What is more, the use of the ALE algorithm allows to correctly account for the spatial uncertainty between multiple results, as well as to control the possible bias introduced by under‐powered experiments due to small sample size.

In conclusion, we propose an alternative method to investigate meta‐analytic data, whose final result is a posterior probability map that allows to make reverse inferences in a simple and straightforward way. The use of BACON could significantly contribute to the improvement of our understanding of the brain, both in its healthy functioning and in its modifications during the development of pathological conditions.

CONFLICT OF INTEREST

The authors declare no potential conflict of interest.

Appendix S1: Supporting Information

Click here for additional data file.

Appendix S2: Supporting Information

Click here for additional data file.

Appendix S3: Supporting Information

Click here for additional data file.