A comparison between scalp- and source-reconstructed EEG networks.

EEG can be used to characterise functional networks using a variety of connectivity (FC) metrics. Unlike EEG source reconstruction, scalp analysis does not allow to make inferences about interacting regions, yet this latter approach has not been abandoned. Although the two approaches use different assumptions, conclusions drawn regarding the topology of the underlying networks should, ideally, not depend on the approach. The aim of the present work was to find an answer to the following questions: does scalp analysis provide a correct estimate of the network topology? how big are the distortions when using various pipelines in different experimental conditions? EEG recordings were analysed with amplitude- and phase-based metrics, founding a strong correlation for the global connectivity between scalp- and source-level. In contrast, network topology was only weakly correlated. The strongest correlations were obtained for MST leaf fraction, but only for FC metrics that limit the effects of volume conduction/signal leakage. These findings suggest that these effects alter the estimated EEG network organization, limiting the interpretation of results of scalp analysis. Finally, this study also suggests that the use of metrics that address the problem of zero lag correlations may give more reliable estimates of the underlying network topology.

Introduction

Modern network science is a fundamental tool for the understanding of both normal and abnormal brain organization[1-3]. In this context, alongside other imaging techniques such as magnetoencephalography (MEG) and functional MRI (fMRI), electroencephalography (EEG) has been widely used to study brain networks[1,4]. It is well accepted that scalp-level EEG analysis does not allow inferences in terms of underlying neuroanatomy[5], thus suggesting the use of source reconstruction techniques[6], yet the former approach has not been abandoned. The main problems with scalp-level analysis of functional networks are related to two considerations: (i) the already mentioned problem that location of EEG channels do not relate trivially to the location of the underlying sources; and (ii) spurious estimates of functional connectivity can occur between the channels due to the effects of field spread[7] and volume conduction[6], where more than one channel can pick up the activity of an underlying source, although these effects can still be present in source-space[8], where it is often referred to as signal leakage. Although scalp- and source-level analyses use different assumptions, conclusions drawn regarding the (global) topology of the underlying networks should, ideally, not depend on the approach that is used.

The aim of the present paper was to compare network measures extracted from scalp and source EEG signals, as defined by the minimum spanning tree (MST), and using a variety of functional connectivity (FC) metrics. The MST represents a network approach that provides an unbiased reconstruction of the core of a network[9,10]. MST parameters are sensitive to alterations in network topology[9] and several studies have already implemented this approach in order to study network alteration at the scalp-[11-19] and source-level[20-23]. Under certain conditions, the MST forms the critical backbone of the original network[24-26]. Moreover, it addresses several methodological limitations (i.e. biased estimates of network topology due to differences in connection strength or link density)[27], yet still captures changes in topology of the original network[9].

In order to assess the effect of field spread and volume conduction/leakage we included FC metrics that are sensitive to these effects, namely the phase locking value (PLV)[28] and the amplitude envelope correlation (AEC)[29], and metrics that are relatively insensitive to these effects, namely the phase lag index (PLI)[30] and AEC after leakage correction[29,31]. Subsequently, several MST parameters were used to characterize the global topology of the reconstructed functional networks.

We used a freely available EEG dataset[32,33] that includes one minute eyes-closed and eyes-open resting-state recordings from 109 subjects. Source-level time-series of neuronal activity were reconstructed for 68 ROIs[34] by means of the weighted Minimum Norm Estimate (wMNE)[35-38] and segmented into five non-overlapping epochs. MSTs were reconstructed for four FC metrics, both at the scalp- and source-level, and parameters that characterise global topology were subsequently compared between the two domains. Finally, with the aim to examine the effect of using one of the two approaches (analysis at the scalp- or source-level) for different experimental conditions, and differently from previous works[39,40] which adopted a simulation framework, we also show the results for both eyes-closed and eyes-open resting-state data.

Material and Methods

Dataset

A dataset created by the developers of the BCI2000 instrumentation system (www.bci2000.org), consisting of 64-channels EEG recordings from 109 subjects, was used in this study[32,33]. All the EEG recordings analysed during the current study are available in the PhysioNet repository, https://www.physionet.org/pn4/eegmmidb/. The dataset contains fourteen different experimental runs for each subject, comprising of two one-minute baseline runs (eyes-open and eyes-closed resting-state conditions). For the aim of the present work, we considered both eyes-closed and eyes-open resting-state runs. The data are provided in EDF + format and contain 64 raw EEG signals as per the international 10–10 system, sampled at 160 Hz.

EEG pre-processing

EEGLAB (version 13_6_5b)[41] was used to re-reference to common average reference and filter (with fir1 filter type) the EEG signals (band-pass filter between 1 and 70 Hz and 60 Hz notch filtering). Successively, ADJUST (version 1.1.1)[42], a fully automatic algorithm based on Independent Component Analysis (ICA), was used to detect and remove artifacts from the filtered signals.

Source reconstruction

Source-reconstructed time-series were obtained by using Brainstorm software (version 3.4)[43]. First, a head model was created using a symmetric boundary element method in Open-MEEG (version 2.3.0.1)[44,45] based on the anatomy derived from the ICBM152 brain[46]. Time-series of neuronal activity were reconstructed using whitened and depth-weighted linear L2 minimum norm estimate (wMNE)[35-38], with an identity matrix as noise covariance. Sources were constrained to the cortex and source orientation was perpendicular to the cortical surface[47]. To limit the effect of differences in network size[27] between scalp- (64 channels) and source-analysis, source-reconstructed time-series were projected onto 68 regions of interest (ROIs) as defined by the Desikan-Killiany atlas[34], where time-series for voxels within a ROI were averaged (after flipping the sign of sources with opposite directions).

Connectivity metrics

Functional connectivity metrics that are either sensitive or insensitive to the effects of field spread and volume conduction/signal leakage, based on either amplitude or phase information, were used.

In particular, we used AEC, a measure of amplitude coupling that uses linear correlations of the envelopes of the band-pass filtered signals[29,31] and AECcorrected, a version that uses a symmetric orthogonalisation procedure to remove zero-lag correlations (implemented in the time domain[31]). Furthermore, we used the PLV[28], a measure that quantifies the consistency of phase differences (including zero-lag), and the PLI[30], a measure that quantifies the asymmetry of the distribution of phase differences between time series and that ignores zero-lag phase differences. The connectivity metrics were calculated for all epochs of each subject, after having band-pass filtered the scalp- or source-reconstructed time-series in the alpha band (8–13 Hz) and segmenting the one-minute recordings in five non-overlapping epochs of 12 seconds[48].

Functional Network Topology

The EEG channels (scalp-level analysis) and the atlas-based ROIs (source-level analysis) were considered as network nodes, and the functional connections as weighted edges within the network. Then the MST, a sub-network that connects all nodes whilst minimizing the link weights without forming loops, was reconstructed. Since we are interested in the strongest connections, the connection weights were inverted (1 – FC measure) before constructing the MST. The topology of the MST was characterised using the following parameters[49]: the leaf fraction (number of nodes with degree of 1 divided by the total number of nodes), the diameter (largest distance between any two nodes), the tree hierarchy (balance between hub overload and network integration) and the kappa (broadness of degree distribution). The use of MST was introduced as a simple and unbiased method to represent the essential features of brain networks. In particular, the MST, which captures the backbone of the network, allows to compare this feature across different conditions (i.e., behavioural states and neurological diseases). If the original network can be interpreted as a kind of transport network, and if edge weights in the original graph possess strong fluctuations, also called the strong disorder limit, all transport in the original graph flows over the MST[25] forming the critical backbone of the original graph[50]. Moreover, it has been shown that the MST characterization still keeps important information about the topology of the whole network, as derived using more conventional graph analysis[9]. The MST leaf fraction captures if the network has a central organization, the diameter reflects the efficiency of the global organization, the tree hierarchy captures the balance between network efficiency and overload of central nodes (hubs), and kappa reflects the resilience of the network against attacks. A more detailed description and interpretation of MST parameters may be found in[9,10]. All analyses were performed using Matlab R2016b (The MathWorks, Inc., Natick, Massachusetts, US) and the MIT Strategic Engineering Matlab Tools for Network Analysis[51].

Statistical analysis

Correlations between scalp- and source-derived measures were assessed using Spearman’s rank correlation coefficient. In order to test differences between correlations from the different connectivity approaches, the ones that are insensitive to zero-lag correlations in contrast to the ones that are sensitive, we used a percentile bootstrap approach for non-overlapping correlations[52], using 500 repetitions, and using the code available at https://github.com/GRousselet/blog/tree/master/comp2dcorr and described at https://garstats.wordpress.com/2017/03/01/comp2dcorr/.

Results

The associations between measures extracted from scalp- and source-reconstructed networks are represented as scatterplot in Fig. 1. The Spearman correlations between scalp- and source-level global connectivity (obtained by averaging all the values in the connectivity matrix excluding the diagonal) were high for all connectivity metrics, whereas correlations were low to moderate for the MST parameters. For the MSTs, the highest correlations were observed for leaf fraction for MSTs based on AECcorrected (rho = 0.346) and PLI (rho = 0.262).

Figure 1

Scatterplots of scalp- and source-based measures of FC and network topology. The strength of the correlation is reported as rho value. Note that estimates of network topology correlated only weakly to moderately (maximum rho = 0.346) between scalp- and source-level, even though global FC correlated strongly.

For all the MST parameters, MSTs based on AECcorrected and PLI showed higher correlations in comparison with AEC and PLV. In particular, PLV-based MST parameters showed the lowest correlations, with correlations strength approaching zero for all the evaluated network measures (maximum rho = 0.061, for MST kappa). Statistical differences, expressed using confidence intervals, between Spearman correlations derived from amplitude and phase based coupling approaches are summarized in Tables 1 and 2. For amplitude based FC metrics, the largest difference was observed for MST leaf fraction, whilst for the other MST parameters the differences were small. For phase-based FC metrics, the most marked differences were observed for MST leaf fraction and MST Kappa. For both the amplitude- and phase-based approaches, the MST Hierarchy showed only minimal differences. Figure 2 shows differences and bootstrap distributions for MST leaf fraction, for MSTs based on AECcorrected versus AEC (left panel) and for MSTs based on PLI versus PLV (right panel).

Table 1

Comparison between scalp- and source-level correlations for amplitude based FC metrics. Mean difference and confidence intervals are reported.

	AECcorrected	AEC	Statistics
	rho	rho	difference	CI
MST leaf fraction	0.346	0.151	0.195	[0.08 0.32]
MST diameter	0.223	0.146	0.078	[−0.03 0.18]
MST kappa	0.254	0.173	0.081	[−0.03 0.19]
MST hierarchy	0.141	0.099	0.043	[−0.07 0.15]

Table 2

Comparison between scalp- and source-level correlations for phase based FC metrics. Mean difference and confidence intervals are reported.

	PLI	PLV	Statistics
	rho	rho	difference	CI
MST leaf fraction	0.262	0.019	0.243	[0.13 0.37]
MST diameter	0.136	0.019	0.117	[0.00 0.23]
MST kappa	0.215	0.061	0.153	[0.03 0.27]
MST hierarchy	0.078	−0.005	0.082	[−0.04 0.19]

Figure 2

Differences and bootstrap distributions for MST leaf fraction, for AECcorrected versus AEC (left panel) and for PLI versus PLV (right panel). The difference between coefficients is marked by a thick vertical black line. The 95% percentile bootstrap confidence interval is illustrated by the two thin vertical black lines.

As shown in Table 3, Spearman correlations tended to be higher (especially for PLI) when the same analysis was performed at subject-level. For this analysis, FC metrics from the five epochs were averaged for each single subject, and the correlation between scalp- and source-level estimates was computed.

Table 3

A Comparison between epoch-level and subject-level analysis.

	rho values
	Epoch level	Subject level
AEC global	0.890	0.904
PLI global	0.878	0.918
AEC MST leaf fraction	0.346	0.325
PLI MST leaf fraction	0.262	0.367
AEC MST diameter	0.223	0.278
PLI MST diameter	0.136	0.253
AEC MST kappa	0.173	0.104
PLI MST kappa	0.215	0.181
AEC MST hierarchy	0.141	0.123
PLI MST hierarchy	0.078	0.151

In the latter case, PLI-based correlations tended to be higher and in general outperformed AEC-based correlations.

Comparison of the two experimental conditions, namely eyes-closed and eyes-open resting-state, shows (Fig. 3) that, for some connectivity metrics, adopting one approach over the other (scalp- versus source-level analysis) may show a different magnitude, and even an opposite direction, for the condition effect on network topology (here assessed by MST leaf fraction). It is interesting to note that, at least for those metrics that address the problem of field spread and volume conduction/signal leakage (AECcorrected and PLI) the condition-related shift in network topology were in the same direction for the scalp- and sensor-level analysis, whereas for the other metrics (AEC and PLV) the condition-related shifts in network topology were in opposite directions, depending on whether the networks were reconstructed at the scalp- or source-level[53].

Figure 3

Scatterplot of leaf fraction for eyes-closed and eyes-open resting-state, based on scalp (top row) and source analysis (bottom row) and for each connectivity measure (different columns). The blue and orange arrows indicate the direction of the shift in network topology when comparing the eye-closed and eyes-open condition. Note that for the AEC and PLV the leaf fraction is higher in the eyes-open condition than in the eyes-closed condition for the scalp-level MSTs, whereas the opposite condition effect is found for the source-level MSTs.

Discussion

Although network reconstructions at the scalp- and source-level rely on different assumptions, conclusions drawn regarding the (global) topology of the underlying networks should, ideally, not depend on the approach that is used. We found that global functional connectivity correlated strongly between the two domains, independent of the metric that was used. In contrast, global MST network descriptors extracted from scalp- and source-level EEG signals correlated (at best) moderately. In particular, in the case of connectivity metrics that do not limit spurious connections that are due to field spread and volume conduction/signal leakage (PLV and uncorrected version of AEC), the correlations were particularly weak. Although topological parameters for MSTs based on AECcorrected and PLI showed only moderate correlations between scalp- and source-level, they were still higher than for MSTs based on PLV and uncorrected AEC for all of the estimated network measures. These differences (between scalp-/sensor-level correlations) were most evident for phase-based synchronization metrics, where PLI allowed to obtain higher correlations than PLV, for two of the MST descriptors (leaf fraction and kappa). These findings, which show minimal consistency between network analysis at scalp- and source-level, still advise against the use of FC metrics that do not correct for spurious correlations as they tend to amplify the differences between the two domains (scalp- and source-level). Conversely, the use of metrics that limit spurious connectivity (AECcorrected and PLI) tends to reduce these differences. Higher correlations were observed when the analysis was performed at subject-level. Finally, the magnitude, and even direction, of condition-effects may change depending on the connectivity metric used (Fig. 3), thus potentially leading to completely different conclusions.

In this work, we used weighted minimum norm estimated to reconstruct the source activity. It should be noted that different source reconstruction methods[54], may provide different results in terms of global network topology and the correlation between scalp- and sensor-level estimates. In order to examine the potential effect of other inverse methods, we have replicated the analysis using the sLoreta approach[55] which, as reported in the Supplementary Information, provided results, in terms of correlations between scalp- and source-level metrics, that were equivalent to those obtained with the wMNE. Since the accuracy of source localization may increase with the number of channels[39,56], one would expect the correlations between networks reconstructed at the sensor- and source-level to be higher for MEG than for EEG. The reduced sensitivity of MEG volume conduction may further aid in this respect. It should also be noted that, differently from MEG, the choice of the reference electrode may influence EEG sensor-space connectivity estimates[57]. In this latter work[57], the authors also report a detailed analysis about the effects of different approaches (such as anatomical templates, inverse methods and software implementations) on source-reconstructed EEG signals and their connectivity properties. Even though our work makes use of a set of different metrics to estimate functional connectivity and network topology, the reported results are consistent with those reported by Mahjoory et al., who also suggest that the variability introduced by different methodological choices reflects uncertainty in the results, which should not be overlooked when interpreting scalp and source-level findings. It is also interesting to note that our findings seem to be in line with another recent work[58] that investigated EEG coherence based on local field potentials measured with microelectrode arrays, as well as based on scalp-level EEG, and concluded that scalp-level coherence was not reliably related to coherence between brain areas measured intracranially.

The present work suffers from some limitations. First of all, there was no ground truth in our study, and we interpret our results under the assumption that network estimates at the source-level are a better approximation of the unknown true network organization than scalp-level estimates. This assumption is in line with a previous study[39] that have highlighted that scalp-level network analyses may result in erroneous inferences about the underlying network topology. In particular, the model study by Antiqueira and colleagues suggests that scalp-based network structures, especially when under-sampled at surface sites, might not agree with the underlying three-dimensional network. Another limitation refers to the inherently different mapping approaches between scalp- (channels) and source-level (ROIs) analysis, that strongly hinder the comparison between the two domains. Even though the number of EEG channels differed only slightly from the number of reconstructed ROIs (64 versus 68), it has been shown that differences in network size can affect estimates of network topology[27]. The correlations between the scalp- and source-level estimates of network topology presented here may therefore be lower than those that would have been obtained in case the networks had been of equal size. In order to estimate the possible effect of small differences in network size between sensor- and source-level we reported (see Supplementary Information) the results obtained using a subset of nodes for the source analysis. In particular, we omitted four nodes from original atlas (right and left parahippocampal and lingual) thus obtaining a network with 64 nodes. The results show that the impact of network size is negligible, since both the direction and the strength of associations (as measures by Spearman correlations, for MST leaf fraction based on AECcorrected) are comparable (difference = −0.01 [−0.03 0.01]). Another consideration is that consistency for local measures (e.g. nodal centrality) may be lower than reported for the global network measures. In order to investigate whether the observed correlations between scalp- and source-space based metrics were depended on the specific thresholding approach for the network reconstruction (i.e., the MST), we also replicated the analysis using a novel technique, namely efficiency cost optimization (ECO). This approach was recently introduced with the aim to filtering information in complex brain networks[59]. Our results show (see Supplementary Information) that also with ECO the functional connectivity metrics that are less sensitive to the effects of field spread and volume conduction/leakage (AECcorrected and PLI) gave higher correlations between network metrics as obtained at scalp- and source-level, compared to AEC and PLV.

Although these results demonstrate that the specific thresholding approach does only have a small effect on the results for unweighted networks, it is unclear whether our results generalise to weighted and/or directed network analyses. Similarly, our results were obtained for specific connectivity metrics. We used metrics that are widely used, and that capture different mechanism of coupling between brain regions[60,61], based on phase coupling or amplitude correlations. However, other metrics of functional connectivity may capture different properties of statistical interdependence, and may have a different sensitivity to, for example, common signal sources and non-stationarity. Different metrics may therefore have different biases when estimating interactions between time-series, which may also affect the correlation between source- and sensor-level results. Hence, although our findings indicate that results from sensor-level unweighted network analysis should be interpreted with caution, future studies should investigate whether this is also the case for weighted and/or directed networks that are derived from a larger range of functional connectivity metrics.

Moreover, we investigated the effect of frequency content (limited at alpha band in the main text), extending the analysis to other bands, namely theta (4–8 Hz) and beta (13–30 Hz) bands (see Supplementary Information). These results show that the reported trend still holds also for other frequency bands. Finally, the ICA-based artifact-rejection that we used may have distorted the phases[62]. However, since our study is focused on the comparison between analyses at the scalp- and source-level and is not intended to unveil the true underlying network topology, this limitation should only slightly impact on the reported results. Future studies should elucidate the effects of ICA artifact rejection on subsequent connectivity and network analyses.

Conclusion

In conclusion, the present work confirms that, although functional connectivity can be estimated reliably, extreme caution should be used when interpreting results derived from scalp-level EEG network analysis, even when unbiased approaches such as MST analysis are used. However, assuming that the source-based network representation is a better approximation of the unknown true network organization, our findings also indicate that connectivity metrics that limit the emergence of spurious correlations (such as the corrected AEC and PLI) may allow for more reliable estimates of the underlying global network organization.

Supplementary information