Fusing concurrent EEG-fMRI with dynamic causal modeling: application to effective connectivity during face perception.

Despite the wealth of research on face perception, the interactions between core regions in the face-sensitive network of the visual cortex are not well understood. In particular, the link between neural activity in face-sensitive brain regions measured by fMRI and EEG markers of face-selective processing in the N170 component is not well established. In this study, we used dynamic causal modeling (DCM) as a data fusion approach to integrate concurrently acquired EEG and fMRI data during the perception of upright compared with inverted faces. Data features derived from single-trial EEG variability were used as contextual modulators on fMRI-derived estimates of effective connectivity between key regions of the face perception network. The overall construction of our model space was highly constrained by the effects of task and ERP parameters on our fMRI data. Bayesian model selection suggested that the occipital face area (OFA) acted as a central gatekeeper directing visual information to the superior temporal sulcus (STS), the fusiform face area (FFA), and to a medial region of the fusiform gyrus (mFG). The connection from the OFA to the STS was strengthened on trials in which N170 amplitudes to upright faces were large. In contrast, the connection from the OFA to the mFG, an area known to be involved in object processing, was enhanced for inverted faces particularly on trials in which N170 amplitudes were small. Our results suggest that trial-by-trial variation in neural activity at around 170 ms, reflected in the N170 component, reflects the relative engagement of the OFA to STS/FFA network over the OFA to mFG object processing network for face perception. Importantly, the DCMs predicted the observed data significantly better by including the modulators derived from the N170, highlighting the value of incorporating EEG-derived information to explain interactions between regions as a multi-modal data fusion method for combined EEG-fMRI.