Identifying commonality and specificity across psychosis sub-groups via classification based on features from dynamic connectivity analysis.

It is difficult to distinguish schizophrenia (SZ), schizoaffective disorder (SAD), and bipolar disorder with psychosis (BPP) as their clinical diagnoses rely on symptoms that overlap. In this paper, we investigate if there is biological evidence to support the symptom-based clinical categories by looking across the three disorders using dynamic connectivity measures, and provide meaningful characteristics on which brain functional connectivity measures are commonly or uniquely impaired. Large-sample functional magnetic resonance image (fMRI) datasets from 623 subjects including 238 healthy controls (HCs), 113 SZ patients, 132 SAD patients, and 140 BPP patients were analyzed. First, we computed whole-brain dynamic functional connectivity (DFC) using a sliding-window technique, and then extracted the individual connectivity states by applying our previously proposed decomposition-based DFC analysis method. Next, with the features from the dominant connectivity state, we assessed the clinical categories by performing both four-group (SZ, SAD, BPP and healthy control groups) and pair-wise classification using a support vector machine within cross-validation. Furthermore, we comprehensively summarized the shared and unique connectivity alterations among the disorders. In terms of the classification performance, our method achieved 69% in the four-group classification and >80% in the between-group classifications for the mean overall accuracy; and yielded 66% in the four-group classification and >80% in the between-group classifications for the mean balanced accuracy. Through summarizing the features that were automatically selected in the classifications, we found that among the three symptom-related disorders, their disorder-common impairments primarily included the decreased connectivity strength between thalamus and cerebellum and the increased strength between postcentral gyrus and thalamus. The disorder-unique changes included more various brain regions, mainly in the temporal and frontal gyrus. Our work demonstrates that dynamic functional connectivity provides biological evidence that both common and unique impairments exist in psychosis sub-groups.

Introduction

Since schizophrenia (SZ), schizoaffective disorder (SAD), and bipolar disorder with psychosis (BPP) have overlapping clinical symptoms, it can be difficult to differentiate them for clinical diagnostic purposes (Laursen et al., 2009, Cosgrove and Suppes, 2013, Malaspina et al., 2013). SZ and BPP can show similar cross-sectional symptoms including delusions, hallucinations, and mood disturbance (Pearlson 2015). BPP has a high misdiagnosis rate and is often misdiagnosed as SZ (Mukherjee et al., 1983). A study (Meyer and Meyer 2009) reported that almost 45% of psychiatrists misdiagnosed bipolar patients and mentioning hallucinations decreased the likelihood of diagnosing bipolar disorder. Another study (Shen et al., 2018) showed that 20% – 30% of bipolar patients were mistakenly diagnosed as SZ. Given the ambiguities between the two disorders, Kasanin (Kasanin 1933) introduced the concept of SAD, defined by a combination of symptoms of schizophrenia and mood disorder, acknowledging their symptomatic overlap. The diagnosis of SAD is made when there are symptoms of major depression or mania, along with psychotic symptoms. SAD is likely to experience severe mood symptoms accounting for more than half of the total duration, while SZ may present brief mood symptoms. Once the psychotic symptoms predominate the illness period of SAD, the diagnosis leans towards SZ. As such, there is a great deal of confusion in differentiating SAD from BPP or SZ. Therefore, a fundamental question centers on which brain impairments commonly exist in these disorders and which are specifically changed among them.

Measured by the neuroimaging measures, the three disorders have been shown shared abnormality. Ivleva et al. (Ivleva et al., 2013) revealed that SZ and SAD populations showed overlapping gray matter reductions in many brain regions, while BPP patients showed limited gray matter reductions localized to the frontotemporal cortex. In another study (Amann et al., 2016), the three disorders were also studied by comparing with healthy group separately based on the voxel-based morphometry measures. Their finding supported that both SAD and SZ had widespread volume reduction in overlapping areas, whereas the changes of SAD resemble SZ more than bipolar disorder. A review paper (Birur et al., 2017) surveyed the recently published neuroimaging work with respect to schizophrenia and bipolar disorder, suggesting that white matter impairments of the two disorders show more consistency, and gray matter reduction is greater in schizophrenia than bipolar disorder. However, to the best of our knowledge, only a few studies involved SAD. Therefore, more work is needed to help further understand their commonality and specificity under the current DSM category.

Brain functional connectivity (Stephan et al., 2017, Du et al., 2018b) using functional magnetic resonance imaging (fMRI) data may be an alternative feasible measure to explore if there is biological evidence to support the symptom-based clinical categories and what are their common and specific brain functional impairments. Functional connectivity features were usually computed using the entire time series of fMRI data, called static functional connectivity analysis. Previous work used such features like spatial functional networks (Arribas et al., 2010, Khadka et al., 2013) and functional network connectivity (FNC) (Jafri et al., 2008) revealed from independent component analysis (ICA) to separate SZ and BPP patients from healthy controls. More recently, Xia et al. (Xia et al., 2019) investigated the shared and distinct functional network features such as clustering coefficient across schizophrenia, bipolar disorder, and major depressive disorder, revealing their trend toward randomized configurations but with different degrees. However, there has been much less work including SZ, BPP and SAD, as SAD patients are often categorized into an SZ group in previous studies due to concerns about the reliability of the SAD's Diagnostic and Statistical Manual of Mental Disorders (DSM) standard (Maj et al., 2000). Our previous work (Du et al., 2015) investigated five groups including healthy control (HC), SZ, BPP, schizoaffective disorder with manic episodes, and schizoaffective disorder with depressive episodes exclusively, using brain spatial networks estimated by a group information guided ICA (GIG-ICA) approach (Du and Fan 2013), resulting in a 68.75% classification accuracy based on a relatively small independent sample-size (N = 16) and revealing the disorder relationship to some extent. Thus far, exploring the brain functional commonality and specificity across BPP, SAD and SZ is still needed for understanding their mechanisms as well as for potential refining of their categories in future (Colibazzi 2014).

Recently, dynamic connectivity analysis (Sadaghiani et al., 2015, Preti et al., 2017) has shown increased sensitivity in identifying mental illness biomarkers compared to a static connectivity approach (Rashid et al., 2016, Du et al., 2017a, Du et al., 2017c). In a sliding time-window technique (Sakoglu et al., 2010, Liao et al., 2018, Li et al., 2019, Liao et al., 2019), dynamic functional connectivity can be obtained by estimating connectivity using windowed time series (Hutchison et al., 2013, Calhoun et al., 2014). Estimating connectivity states from dynamic connectivity patterns plays an important role in the biomarker extraction. By applying a K-means clustering method, previous studies have revealed abnormality in SZ compared to healthy population (Damaraju et al., 2014, Du et al., 2016) and differences between SZ and BPP (Rashid et al., 2014a, Rashid et al., 2014b) in dynamic connectivity states. Our previously proposed ICA method (Du et al., 2018a), which decomposes the time-varying connectivity patterns into different connectivity states while preserving subject variability and comparability, found that individuals with clinical high risk for psychosis show an intermediate pattern between HCs and SZ patients. Rashid et al. (Rashid et al., 2016) applied the regression coefficients of time-varying connectivity on the connectivity states to distinguish healthy, SZ (including SAD patients), and BPP groups, and obtained greater three-way classification accuracy compared to the traditional static connectivity method. Our other work (Du et al., 2017a, Du et al., 2017c) that performed statistical analysis on whole-brain dynamic connectivity measures provides interesting insights on BPP, SAD and SZ disorders, suggesting both hypoconnectivity (with decreasing trends) and hyperconnectivity (with increasing trends) from HC to BPP to SAD to SZ were present. The study also supports that dynamic analysis revealed more subtle group differences than static connectivity method. We expected that the advanced dynamic functional connectivity analysis would speed up the understanding of psychosis sub-groups.

In the present study, we investigate the psychosis sub-groups (SZ, BPP and SAD) by using dynamic functional connectivity measures under a classification framework, aiming to disclose the commonly and uniquely altered connectivity features across the current diagnoses. We perform both the four-group (HC, SZ, BPP, and SAD) and pair-group (e.g. HC vs. SZ) classifications comprehensively. An unbiased cross-validation procedure with abundant runs is applied to evaluate the effectiveness. We explore how well the psychosis sub-groups defined by clinical symptoms can be separated by only using brain measures and what are the brain-related commonality and specificity across these diagnoses.

Materials and methods

Materials

In this study, we analyzed resting-state fMRI data of 623 subjects including 238 HCs, 113 SZ patients, 132 SAD patients and 140 BPP patients from the multi-site Bipolar and Schizophrenia Network on Intermediate Phenotypes (BSNIP-1) study (Tamminga et al., 2013, Meda et al., 2014, Meda et al., 2015). Age and sex were matched among different groups (p = 0.19 for age examined by analysis of variance; p = 0.31 for sex examined by Chi Square test). The scanning period was about five minutes for all subjects. The detailed scanning information of each site is shown in the supplementary Table S1. All subjects provided informed consent, and were in a stable mental state and took stable medications at the time of the study. During the scanning, all participants were asked to rest with their eyes closed and stay awake. Patients were classified diagnostically using DSM-IV-TR criteria ascertained using the SCID (First et al., 2002). Medication use primarily included antipsychotic drugs (BPP 72.14%, SAD 87.12%, SZ 88.50%), antidepressant drugs (BPP 41.43%, SAD 56.82%, SZ 38.94%), and mood stabilizers (BPP 69.29%, SAD 56.06%, SZ 23.01%). More detailed information about the used data can be found in our previous study (Du et al., 2017a, Du et al., 2017c).

Methods

Data preprocessing

As described in the previous work (Du et al., 2017a, Du et al., 2017c), we preprocessed the fMRI data with the Data Processing Assistant for Resting-State fMRI (DPARSF) toolbox (Yan and Zang 2010) based on statistical parametric mapping software (SPM). The first six volumes were discarded, and then the remaining images were slice-time corrected and realigned to the first volume for head-motion correction. For each subject that we included, the translations of head motion were less than 3 mm, and the rotation of head motion did not exceed in all axis through the whole scanning process. As summarized in Table S2, there are no significant group differences in the head motion (p = 0.17 for the translation and p = 0.23 for the rotation, tested by analysis of variance). Subsequently, we spatially normalized the images to the Montreal Neurological Institute (MNI) EPI template (Friston et al., 1995), resliced to 3 mm × 3 mm × 3 mm voxels, and smoothed them with a Gaussian kernel with a full-width at half-maximum (FWHM) of 8 mm. Detrending and filtering (0.01 Hz~0.08 Hz) (Auer 2008) were performed afterwards. Finally, nuisance covariates including six head motion parameters, white matter signal, cerebrospinal fluid signal and global mean signal (Lydon-Staley et al., 2019) were regressed out.

Estimation of dynamic functional connectivity

For each individual, whole-brain dynamic functional connectivity was constructed using a sliding time-window method based on 116 regions of interest (ROIs) from the automated anatomical labeling (AAL) template (Tzourio-Mazoyer et al., 2002), as shown in the top of Fig. 1 (i.e., the step 1). First, a representative time series was computed by averaging all time series within voxels for each ROI. Then, a tapered window, created by convolving a rectangle (width = 20 TRs) with a Gaussian kernel (σ = 3 TRs), was moved in step of 1 TR to segment each representative time series into short time series. Next, regarding each window, a connectivity matrix (size: 116 × 116) reflecting connectivity strengths between all ROIs was obtained using a graphical LASSO model (Friedman et al., 2008), consistent with previous studies (Allen et al., 2014, Damaraju et al., 2014). Thus, time-varying connectivity matrices along different windows were obtained. To show the connectivity dynamics, for each subject we computed the standard deviation of each element in functional connectivity matrix across different windows. After that, we averaged the standard deviation measures across all subjects in each group for a summary (Fig. S1(A)). We also computed the mean of dynamic functional connectivity matrices across different windows for each subject, and then averaged them for each group to show (Fig. S1(B)). It is observed that functional connectivity showed an evident time-varying pattern for all groups and the standard deviation tended to be lower for the connections with higher strengths, indicating that dynamic connectivity analysis may provide more information than the traditional static method. Due to the symmetry of each connectivity matrix, its connectivity strengths were converted to a vector containing only upper triangular 6670 elements. Consequently, the time-varying connectivity patterns can be represented by a (window number × 6670) matrix, called a “dynamic matrix” hereinafter.

Fig. 1

Framework for dynamic connectivity computation and connectivity state extraction. Step 1: Computation of individual-subject dynamic connectivity matrix based on ROIs from AAL template. Step 2: Estimation of group-level connectivity states (GS) by performing Infomax ICA algorithm with ICASSO technique on the window-direction concatenated dynamic matrices of all subjects. Among the group-level connectivity states, we identified a dominant state that showed the highest contribution in the dynamics. Step 3: Computation of the related subject-level connectivity state (SS) using a multiple-objective optimization function based on the identified dominant group-level connectivity state and the individual-subject’s dynamic matrix.

Extraction of functional connectivity states

As outlined in the bottom of Fig. 1 (Steps 2 and 3), we extended our previously proposed decomposition method (Du et al., 2017a, Du et al., 2017c, Du et al., 2018a) to estimate the connectivity states from the time-varying connectivity patterns. In our method, the group-level connectivity states (GSs) were computed and then were used to guide the computation of each individual subject’s connectivity states so as to make the connectivity states comparable across different subjects.

In order to estimate the group-level connectivity states (i.e. the step 2 in Fig. 1), Infomax algorithm (Bell and Sejnowski 1995) was applied to the window-direction concatenated Fisher-transformed dynamic matrices of all subjects. The ICASSO technique (Himberg et al., 2004, Ma et al., 2011) was performed for yielding robust independent components (ICs), representing the group-level connectivity states denoted by . The number of states was set to five in this study, consistent with previous work (Damaraju et al., 2014, Miller et al., 2016, Du et al., 2017a, Du et al., 2017c, Du et al., 2018a). For subsequent analysis, each GS related IC (e.g. , size: 1 × 6670) was Z-scored to have zero mean and unit variance. Based on the obtained , according to the ICA model we computed the mixing matrix that is the weights of group-level connectivity states on the dynamic connectivity. Since the mixing matrix reflects the fluctuations of those connectivity states along different time windows, from the five group-level states we identified the dominant GS that showed the highest contribution in the sum of absolute weights across all windows and subjects. The dominant state has been demonstrated to be able to provide the most important information in dynamic connectivity (Du et al., 2017a, Du et al., 2017c, Du et al., 2018a).

In the next step (i.e., the step 3 in Fig. 1), we computed the corresponding subject-specific connectivity state (SS) based on the identified dominant GS and individual-subject Fisher-transformed dynamic matrix. Using a multiple-objective optimization function shown in (1) (Du et al., 2017a, Du et al., 2017c), we simultaneously optimize the independence of the subject-level state as well as the correspondence between the subject-level state to be estimated and the dominant group-level state that was already obtained.

Here, and denote the dominant GS and the related SS, respectively. is the negentropy of the estimated , representing its independence. reflects the similarity between the dominant group-level state and the individual state. The algorithm automatically generates Z-scored (Du and Fan 2013), by searching a optimal that is an unmixing vector operating on the dynamic matrix of the subject. This means , where is the whitened . In (1), is a Gaussian variable with zero mean and unit variance. is a nonquadratic function. In this work, for classification among different groups, we used the connectivity strengths in the dominant state for feature selection since the dominant state mostly contributes to the time-varying connectivity patterns.

Classification across healthy controls, bipolar disorder with psychosis, schizoaffective disorder, and schizophrenia

We investigated if dynamic connectivity measures can capture group differences across psychosis sub-groups and how well the four groups can be classified using these measures. An unbiased 10-fold cross-validation procedure with 100 runs was used to evaluate the classification performance, as shown in Fig. 2. In each of 100 runs, the samples (623 subjects) were divided into ten folds equally, each fold of which was used as the testing data and the remaining nine folds were used as the training data. Since there were more available features than samples, we used a support vector machine with recursive feature elimination (SVM-RFE) technique (Du et al., 2015) combined with an inner 10-fold cross-validation to perform the automatic feature selection within the training data of each run. In each run, the optimal features were determined only based on the training data so that the testing data were separated from the feature selection and model training.

Fig. 2

Flowchart for classifying the four groups including HC, SZ, SAD and BPP using dynamic connectivity measures. We used the 10-fold cross validation with 100 runs to maximize the reliability. Within each run, feature selection was first performed only based on the training data (i.e. each nine folds) using SVM-RFE and inner cross-validation procedure, then the trained model was tested and evaluated using the testing data (i.e. the remaining one fold).

The following describes the details of feature selection. The training data were divided into ten folds, nine of which were taken as the inner training data to perform SVM-RFE while the remaining one was used to test the trained model. Each iteration process removed the least significant 10% features according to the sorting sequence, and the maximum number of iterations was set to 35. After each iteration time, an SVM model was trained using the updated features on the inner training data, and then tested on the remaining fold. The above feature sorting and removal process was repeated 10 times, resulting in a classification accuracy matrix (size: 10 × 35) within the inner 10-fold cross-validation process. Subsequently, we chose the optimal feature subsets corresponding to the maximum mean classification accuracy (averaged across 10 times). By calculating the frequency of each feature appearing in the 10 feature subsets, we determined the selected features as those with the occurring frequency greater than 0.5.

In each run, we trained a SVM classifier based on the selected features using the outer training data. SVM has been successful in distinguishing various brain disorders (Mwangi et al., 2012, Zarogianni et al., 2013, Du et al., 2018b). For the multi-class classification problem, the one-against-all strategy that is often better than one-against-one (Milgram and Cheriet, 2006) was used since the SVM was originally designed for two-class problem. A linear kernel was utilized in SVM, with the regularization parameter C determining the tradeoff between the empirical error and the complexity term as 2. In theory, non-linear kernels may work better than linear kernels in solving complex classification task, but in practice many researchers in the neuroimaging field prefer linear kernels (Song et al., 2011, Orru et al., 2012, Wang et al., 2019) due to that feature number is often more than sample size. Based on the well-trained SVM, we examined its classification ability on the held-out testing data. Finally, we assessed the classification performance from different angles. The evaluated measures included the individual class accuracy, individual class precision, overall accuracy, balanced accuracy and balanced precision (Cuadros-Rodriguez et al., 2016) based on the predicted and diagnosis labels. The individual class accuracy reported the ratio of correctly classified subjects of a particular class to the total number of subjects in the class. The individual class precision was defined as the number of correctly classified subjects of a particular class divided by the total number of subjects predicted as the class. The overall accuracy was computed as the ratio of correctly classified subjects of all classes to the total number of subjects of all classes. Additionally, we also computed the mean of individual class accuracies (or precision values) across different groups, called as the balanced accuracy (or precision). Since 100 runs of 10-fold cross-validation were implemented, 100 × 10 values were obtained and then shown using a boxplot for each measure.

Next, we identified which connectivity features played a key role in terms of the distinction among the healthy population and the three symptom-related mental illnesses. The important features selected within all 100 × 10 classification processing were summarized and then visualized using BrainNet Viewer toolbox (Xia et al., 2013). Furthermore, we separately displayed the mean connectivity strength (across subjects) of each connectivity feature for each group to reflect the group differences.

We also examined if there is association between dynamic connectivity and medication. For the daily antipsychotic dose chlorpromazine (CPZ) equivalents, we applied a multiple linear regression model to test the association between the strengths of the important connectivity features in the dominant state and CPZ equivalents in probands with available dose-level medication data.

Classification on any paired groups

In addition to the complex four-group classification, we were interested in whether any two groups can be well separated using the dynamic connectivity measures. Six two-group classifications (HC vs. BPP, HC vs. SAD, HC vs. SZ, BPP vs. SAD, BPP vs. SZ, and SAD vs. SZ) were conducted using the same 10-fold cross-validation procedure (Fig. 2) with 100 runs. Similarly, the individual class accuracy, individual class precision, overall accuracy, balanced accuracy, and balanced precision were calculated to assess the classification ability.

Through the pair-wise classification, we identified the common and unique brain connectivity abnormalities among the three disorders by inspecting features automatically selected. First, the important features used in all 100 × 10 classifications were identified for each HC vs. disorder pair (HC vs. BPP, HC vs. SAD, and HC vs. SZ), separately. For example, if the occurring frequency of one feature was 1 across all classification runs in HC vs. BPP, it was called an important feature in classifying HC and BPP. After that, when one important feature coexisted in the HC vs. BPP, HC vs. SAD, and HC vs. SZ classifications, the feature was taken as being able to reflect the common abnormality among the three disorders. When one important feature was present in only two pairs related classifications (e.g. HC vs. BPP and HC vs. SAD), it was seen as the common changes for two disorders (e.g. BPP and SAD). In contrast, one important feature was regarded as disorder-unique if it was only present for the classification of one HC vs. disorder pair but not for the other two pairs. For example, if one feature was selected as the important feature in the HC vs. BPP classification but not for HC vs. SAD and HC vs. SZ classifications, it was taken as unique in BPP. These features can be indicators about which connectivity values are common to the illness groups and which connectivity values may differentiate the illness groups, benefiting our understanding of these disorders’ brain functional mechanisms.

Results

The connectivity states extracted from dynamic connectivity patterns

Fig. 3 (A)-(E) shows the identified five group-level connectivity states. The first state with the highest contribution was dominant as it showed greater fluctuation weights along all windows (see Fig. 3(F)). The dominant state pattern resembles the mean of dynamic connectivity (shown in Fig. S1(B)), however it was obtained through decomposing the time-varying connectivity patterns. The dominant state has been shown to be able to effectively providing a cleaner result relative to the static connectivity estimated using the entire time series in previous studies (Du et al., 2017a, Du et al., 2017c, Du et al., 2018a). So, in this work, the connectivity strengths between whole-brain regions in the individual dominant state were then utilized for feature selection, model training, and classification across different groups.

Fig. 3

The group-level connectivity states estimated from dynamic connectivity matrices. (A)-(E): Five group-level connectivity states (GSs), that were identified by decomposing the window-direction concatenated time-varying connectivity of all subjects into the maximally independent components. In (A)-(E), the contribution of each connectivity state is shown in the title. The contribution value was computed as the summed absolute weights of one state / the summed absolute weights of all states. (F): The fluctuations of these connectivity states (i.e. the mixing matrix A) along different windows for all subjects. As shown in (F), it is clear that the first state (i.e., GS 1) showed the greatest weights along the dynamics, thus GS 1 with the contribution as 47% was identified as the dominant connectivity state.

Result of multiple-group classification

Classification results from the 100 runs of 10-fold cross-validation are shown using boxplots in Fig. 4(A). Across the 1000 classification results, the mean overall classification accuracy was 69.01%, the mean balanced accuracy was 66.35%, and the mean balanced precision was 68.76% in distinguishing the HC, BPP, SAD, and SZ groups. It can be observed that the accuracy was significantly greater than chance (chance = 25%). Regarding the individual class accuracy, the mean value was 81.25% for HC, 65.11% for BPP, 63.36% for SAD, and 55.67% for SZ. The mean value of individual class precision was 74.04% for HC, 62.09% for BPP, 72.07% for SAD, and 66.82% for SZ. In this work, HC group tended to be separated more easily relative to other groups.

Fig. 4

The four-group classification results. (A) Summarized measurements of the four-group classification. Different boxes represent various measures including the individual class accuracy for each group, the individual class precision for each group, the overall accuracy, the balanced accuracy, and the balanced precision. For each measure, one boxplot was used to show the related values from 100 × 10 testing datasets. In each boxplot, the central line denotes the median, the square denotes the mean, and the edges of the box are the 25th and 75th percentiles. The whiskers extend to 1 inter-quartile range, and each outlier is displayed with a “” sign. (B) Important functional connectivity in the dominant state, that were selected as features in all 1000 classifications for differentiating HC, BPP, SAD and SZ. In each subfigure, the strength of each link reflects the mean connectivity strength across all subjects in the same group. Positive and negative connections are shown using red and blue colors, respectively. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

By calculating the frequency of each feature appearing in the 1000 feature subsets, we found 22 important features (see Fig. 4(B), Table 1, and the supplementary Fig. S2) reflecting the primary connectivity in differentiating the four groups. The positive connections primarily involved those between the right middle frontal and right inferior frontal gyrus, between the right precentral and superior parietal lobules, between the left rolandic operculum and the left transverse temporal gyrus, between the left middle temporal gyrus and the right middle temporal pole. The negative connections were mainly between the right inferior frontal gyrus and the vermis, between the right superior frontal gyrus and the left supramarginal gyrus, and between the left superior frontal gyrus orbital part and the right amygdala. These significant connections primarily consisted of frontal, temporal, parietal, and cerebellar regions, relating to hearing, cognition, and motion functions. Results showed that there were no significant associations (p-values were from 0.04 to 0.99) between these connectivity features and CPZ equivalents after multiple comparison correction (p less than 0.05 with Bonferroni correction).

Table 1

Summary of the 22 important connectivity features with the occurring frequency as 1 across all four-group classification runs. For each feature, the associated ROI ID and name in the AAL template are included. The averaged connectivity value across subjects is also included for each group. The connectivity values were Z-scored.

Important connectivity features in the four-group classification		The mean value of feature in each group
ROI name (ROI ID)	ROI name (ROI ID)	HC	BPP	SAD	SZ
Frontal-Mid-R (ROI-8)	Frontal-Inf-Oper-L (ROI-11)	0.23	0.29	0.49	0.17
Frontal-Sup-Orb-L (ROI-5)	Amygdala-R (ROI-42)	−0.55	−0.53	−0.54	−0.32
Precentral-R (ROI-2)	Parietal-Sup-R (ROI-60)	0.58	0.81	0.68	0.51
Frontal-Sup-R (ROI-4)	SupraMarginal-L (ROI-63)	−0.65	−0.60	−0.44	−0.66
Cuneus-L (ROI-45)	SupraMarginal-R (ROI-64)	−0.65	−0.30	−0.49	−0.36
Cingulum-Mid-R (ROI-34)	Caudate-L (ROI-71)	−0.11	−0.02	0.01	0.08
SupraMarginal-L (ROI-63)	Caudate-R (ROI-72)	−0.29	−0.08	−0.16	−0.19
Frontal-Sup-Orb-R (ROI-6)	Putamen-R (ROI-74)	−0.13	−0.23	−0.23	−0.34
Rolandic-Oper-L (ROI-17)	Heschl-L (ROI-79)	2.86	2.81	2.61	2.85
Frontal-Inf-Oper-R (ROI-12)	Temporal-Pole-Sup-R (ROI-84)	0.79	0.91	0.67	0.62
Olfactory-R (ROI-22)	Temporal-Pole-Mid-L (ROI-87)	0.26	0.03	0.09	0.13
Cuneus-L (ROI-45)	Temporal-Pole-Mid-R (ROI-88)	−0.33	−0.31	−0.62	−0.51
Temporal-Mid-L (ROI-85)	Temporal-Pole-Mid-R (ROI-88)	1.04	0.75	0.95	0.91
Putamen-R (ROI-74)	Temporal-Inf-R (ROI-90)	−0.50	−0.44	−0.52	−0.40
Rectus-L (ROI-27)	Cerebellum-Crus1-L (ROI-91)	−0.50	−0.39	−0.20	−0.46
Pallidum-R (ROI-76)	Cerebellum-Crus2-R (ROI-94)	−0.24	−0.57	−0.40	−0.60
Cingulum-Post-L (ROI-35)	Cerebellum-4-5-R (ROI-98)	−0.28	−0.10	−0.13	−0.09
Frontal-Sup-L (ROI-3)	Cerebellum-10-R (ROI-108)	−0.20	−0.50	−0.26	−0.38
Occipital-Mid-R (ROI-52)	Vermis-1-2 (ROI-109)	−0.12	−0.16	−0.14	−0.30
Frontal-Inf-Tri-R (ROI-14)	Vermis-4-5 (ROI-111)	−0.91	−1.07	−0.77	−0.83
SupraMarginal-L (ROI-63)	Vermis-6 (ROI-112)	−0.50	−0.23	−0.38	−0.40
Cerebellum-4-5-L (ROI-97)	Vermis-9 (ROI-115)	0.85	0.82	0.60	0.89

Results of paired-group classification

As mentioned above, we also performed six two-group classifications to further investigate the disorder common and specific impairments. The classification results (see Fig. 5 and Table 2) show that the mean overall accuracy values were all more than 80% and higher than the results from the four-group classification. Moreover, the SZ vs. SAD classification had the lowest mean in the overall accuracy (80.02%), probably due to the similarity between SZ and SAD. BPP and HC showed the highest mean overall accuracy (89.45%) relative to other pairs, suggesting that they could be easiest to be distinguished using the dynamic measures.

Fig. 5

Summary of the results from classifying any two groups using 100 runs of 10-fold cross-validation. Six classifications (HC vs. BPP, HC vs. SAD, HC vs. SZ, BPP vs. SAD, BPP vs. SZ, and SAD vs. SZ) were examined, with the HC vs. disorder results shown in (A) and the disorder vs. disorder results shown in (B). For each classification, seven measures including individual class (IndiC) accuracy of the first group (C1), individual class accuracy of the second group (C2), individual class precision of the first group, individual class precision of the second group, overall accuracy, balanced accuracy, and balanced precision are shown using boxplots. Taking the HC vs. BPP for example, C1 is HC group, and C2 is BPP group.

Table 2

Pair-wise classification results. For each evaluated measure, the mean value from 1000 classification results is included.

	HC vs. BPP	HC vs. SAD	HC vs. SZ	BPP vs. SAD	BPP vs. SZ	SAD vs. SZ
Individual class accuracy (C1)	93.25%	91.70%	91.16%	86.09%	86.79%	81.58%
Individual class accuracy (C2)	82.94%	79.35%	74.38%	86.62%	76.02%	78.20%
Individual class precision (C1)	90.62%	88.81%	88.02%	87.02%	81.95%	82.06%
Individual class precision (C2)	88.35%	84.99%	81.22%	86.88%	83.84%	79.09%
Overall accuracy	89.45%	87.15%	85.57%	86.35%	81.82%	80.02%
Balanced accuracy	88.09%	85.54%	82.77%	86.35%	81.40%	79.89%
Balanced precision	89.48%	86.92%	84.62%	86.95%	82.90%	80.57%

To quantify the commonality and specificity among the three related disorders, we summarized the important features used in all 100 × 10 classifications for each pair (HC vs. BPP, HC vs. SAD, or HC vs. SZ). There were 62, 60, and 52 important features (see Table S3, Figs. S3, and S4 for details) in the HC vs. BPP, HC vs. SAD, and HC vs. SZ classifications, respectively. Among these important features, there was no exactly the same connectivity feature used in all three pairs of classification. However, some features were commonly taken as the important features for two pairs of classification (Table 3). In particular, HC vs. SAD and HC vs. SZ had the same important feature of a connection between the left postcentral gyrus and right thalamus regions; HC vs. SAD and HC vs. BPP had the common important connectivity features between the left postcentral gyrus and left thalamus regions, as well as between right thalamus and left cerebellum; HC vs. BPP and HC vs. SZ showed a shared important feature in the connection between right thalamus and right cerebellum. Combining the results with the observation from Fig. S4 that shows the mean strengths of those important connectivity features, the three disorders showed commonality in the decreased connectivity strength between thalamus and cerebellum as well as the increased strength between postcentral gyrus and thalamus, although strictly there were no common important features used for all HC vs. disorder classifications. Furthermore, our results also suggest that BPP and SZ showed similarity in the connectivity changes between cuneus and insula, between cuneus and putamen, and between cuneus and supramarginal gyrus.

Table 3

The disorder-common and disorder-unique connectivity features. The common features included those used in both HC vs. BPP and HC vs. SZ, in both HC vs. SAD and HC vs. SZ, as well as in both HC vs. BPP and HC vs. SAD classifications. For reflecting the disorder-unique features, we included those used in both one pair of HC vs. disorder and four-group classifications.

Features used in both HC vs. BPP and HC vs. SZ		Features used in both HC vs. SAD and HC vs. SZ		Features used in both HC vs. BPP and HC vs. SAD
Name of ROI (ROI ID)	Name of ROI (ROI ID)	Name of ROI (ROI ID)	Name of ROI (ROI ID)	Name of ROI (ROI ID)	Name of ROI (ROI ID)
Insula-R (ROI-30)	Cuneus-L (ROI-45)	Postcentral-L (ROI-57)	Thalamus-R (ROI-78)	Postcentral-L (ROI-57)	Thalamus-L (ROI-77)
Cuneus-L (ROI-45)	SupraMarginal-R (ROI-64)			Thalamus-R (ROI-78)	Cerebellum-10-L (ROI-107)
Cuneus-L (ROI-45)	Putamen-L (ROI-73)			Cerebellum-9-L (ROI-105)	Cerebellum-10-L (ROI-107)
Cingulum-Post-L (ROI-35)	Cerebellum-4-5-R (ROI-98)
Pallidum-R (ROI-76)	Cerebellum-8-R (ROI-104)
Thalamus-R (ROI-78)	Cerebellum-8-R (ROI-104)
Features in both HC vs. BPP and four-group classifications		Features in both HC vs. SAD and four-group classifications		Features in both HC vs. SZ and four-group classifications
SupraMarginal-L (ROI-63)	Caudate-R (ROI-72)	Frontal-Mid-R (ROI-8)	Frontal-Inf-Oper-L (ROI-11)	Frontal-Sup-Orb-L (ROI-5)	Amygdala-R (ROI-42)
Temporal-Mid-L (ROI-85)	Temporal-Pole-Mid-R (ROI-88)	Rolandic-Oper-L (ROI-17)	Heschl-L (ROI-79)	Frontal-Sup-Orb-R (ROI-6)	Putamen-R (ROI-74)
Pallidum-R (ROI-76)	Cerebellum-Crus2-R (ROI-94)	Cuneus-L (ROI-45)	Temporal-Pole-Mid-R (ROI-88)	Occipital-Mid-R (ROI-52)	Vermis-1-2 (ROI-109)
Frontal-Sup-L (ROI-3)	Cerebellum-10-R (ROI-108)	Rectus-L (ROI-27)	Cerebellum-Crus1-L (ROI-91)
SupraMarginal-L (ROI-63)	Vermis-6 (ROI-112)

In addition to the shared alterations among these disorders, we also summarized the disorder-unique features in Table S4 and demonstrate them in Fig. S5. The HC vs. BPP, HC vs. SAD, and HC vs. SZ had 53, 56, and 45 unique features respectively, which were only used for its own classification. It is observed that in general these disorders showed various changes relative to HC. To look further into their specificity, we compared Table S4 with Table 1, and found five connectivity features were used for both the four-group and HC vs. BPP classification, thus presenting more specificity in BPP. In a similar way, there were four and three connectivity features for SAD and SZ, separately. As shown in Table 3, BPP-specific alterations involved connectivities between left supramarginal gyrus and right caudate, between left middle temporal gyrus and right middle temporal pole, between right pallidum and right cerebellum crus, between left superior frontal gyrus and right cerebellum, between left supramarginal gyrus and vermis. For SAD, significant unique connections were between right middle frontal gyrus and left inferior frontal gyrus, between left rolandic and left Heschl’s gyrus, between left cuneus and right middle temporal pole, and between left rectus and left cerebellum. SZ-unique changes mainly were between left superior frontal gyri and right amygdala, between right superior frontal gyrus and right putamen, and between right middle occipital gyrus and vermis.

Discussion

A wave of recent research has begun to investigate symptom-related psychiatric disorders (Pearlson and Ford, 2014, Du et al., 2015, Chang et al., 2018, Sorella et al., 2019, Xia et al., 2019). Fueled by the growing promise of dynamic connectivity analysis (Preti et al., 2017, Thompson and Fransson, 2018), there is hope to learn more about underlying brain changes and their disorder specificity. In this paper, we focused on investigating possible biological evidence across the complex symptom-similar groups including SZ, BPP and SAD using dynamic connectivity measures. We performed the four-group and pair-group classifications in order to further identify disorder common and specific changes that allow us to address the fundamental question of how these disorders are linked and different compared to healthy controls. A relatively large-sample (totaling 623 subjects) and an unbiased 10-fold cross-validation framework with abundant runs were used to help maximize a reliable evaluation. Our previously proposed decomposition method (Du et al., 2017a, Du et al., 2017c, Du et al., 2018a) was extended to extract the group-level states from all subjects and then individual-level connectivity states from time-varying connectivity patterns, which enables capturing accurate subject-specific characteristic.

Regarding the complex four-group classification task, our method reached up to 69.01% for the mean cross-validation classification accuracy. This is generally consistent with our previous work that utilized the spatial networks as features, which achieved 68.75% accuracy for the five-group classification but using fewer samples (Du et al., 2015). Although our classification accuracy is greater than the chance (25%), the classification using biological measures did not perfectly separate these clinical diagnostic groups. There are several issues that could cause the inaccuracy in separating these groups: the complexity of the multi-group classification problem, the difficulty of high-dimensional biological measures in representing the clinical symptoms, and the inherent unreliability of the clinically determined diagnosis itself (Mukherjee et al., 1983, Shen et al., 2018). Some groups have therefore developed biologically-based subtypes across psychotic illnesses (Clementz et al., 2016) using features depending purely on empirical knowledge, and not on clinical symptoms. Such individual “Biotypes” showed more neurobiological homogeneity than diagnosis-based categories (Clementz et al., 2015, Meda et al., 2016, Ji et al., 2019). This biological reclassification of psychosis is an important step forward, but does not yet invalidate traditional clinically-derived categories. Effectively refining the current DSM categories with help from fMRI is feasible albeit difficult (Marquand et al., 2016) as limited prior information can be used. Our work can be beneficial to the aim by providing information indicating their similarity and differences that could be prior information for future refinement of categories.

Our work provided insights about how these symptom-based groups are different and linked at a biological level. Our results show that the HC group seemed to be more accurately separated from the other groups. Among the four groups, the SZ subjects had the lowest individual class accuracy, and the BPP group showed the lowest individual class precision, further indicating the underlying biological similarity in the symptom-based categories. To date, a few studies have used dynamic functional connectivity measures to classify mental illnesses. Rashid et al. (Rashid et al., 2016) applied a clustering method to group dynamic connectivity patterns into different connectivity states. They then utilized the regression coefficients as features obtained by projecting the dynamic patterns onto the states to classify among HC, SZ and BPP (159 subjects in total), resulting in an average classification accuracy of 84.28%, but their study mixed SAD patients into the SZ group probably due to the difficulty in distinguishing the two highly-related disorders. Rabany et al. (Rabany et al., 2019) performed classification across HC, SZ and autism spectrum disorder (ASD) based on state-related measures (such as fraction rate of dwell time) derived from dynamic connectivity patterns, providing 81.8% accuracy for SZ, 50% for ASD, and 41.2% for HC. A recent work (Supekar et al., 2019) utilized the dynamics of interaction between default mode network, salience network and executive network to classify schizophrenia patients from healthy controls, resulting in 78% accuracy. However, it is hard to draw a conclusion from these studies on what functional connectivity differences directly led to the distinction across different groups, as the features used for classification do not reflect such information.

Our dynamic connectivity analysis method can help to explain which functional connectivity values contribute to the differentiation among the current clinically-based diagnoses. We captured 22 important connectivity features that were selected in all four-group classification runs indicating a key role in their group differences. These significant connections primarily consisted of frontal (e.g. middle, superior, and inferior frontal, and precentral), temporal (e.g. transverse and middle temporal), parietal (e.g. supramarginal gyrus and superior parietal lobule), cerebellar (e.g. vermis) regions. These regions played important roles in spatial orientation, language perception and processing, and auditory and sensory input processing. The finding is consistent with our previous study (Du et al., 2017a, Du et al., 2017c) that found primary group differences in functional connectivity strengths associated with postcentral gyrus, frontal, and cerebellar cortices using statistical analysis on dynamic connectivity measures.

We also performed additional pair-wise classification examinations to evaluate between-group relationship. Higher classification accuracy was obtained compared to the four-group results, and the performance in differentiating between disorder and HC tended to be better than the results between different disorders. Specifically, the SZ and SAD groups were most difficult to be separated from one another, suggesting that they are closer to each other in brain function compared to other pairs. Our finding accords with the fact that SAD is assigned into the same diagnostic class as SZ in the current DSM-5 (Heckers et al., 2013, Malaspina et al., 2013).

In order to characterize the commonality and differences among the three disorders, we summarized and compared the important features used in the pair-wise classifications. Our findings suggest that compared to HC, the commonly-changed connections among BPP, SAD and SZ consisted of those between postcentral gyrus and thalamus, and between thalamus and cerebellum. Our finding provides clear evidence that thalamic connectivity changes consistently exist in these psychiatric disorders, which accords with previous studies. One paper (Chen et al., 2019) studying SZ using resting fMRI connectivity suggests that thalamic connectivity with sensorimotor areas is related to the severity of cognitive deficits and clinical symptom, and the thalamic functional connectivity in the cerebellum is positively correlated with processing speed. Another study (Ferri et al., 2018) confirms the changes of SZ in connectivity between thalamic and post-central regions, and between thalamic and cerebellar regions in combination with their relationship to clinical features. Moreover, Tu et al. (Tu et al., 2018) found that major psychiatric disorders (SZ, BP I, BP II, and major depressive disorder) shared a similar pattern of thalamocortical dysconnectivity. A recent study (Xia et al., 2019) has shown significant impairment of thalamus associated with BP, SZ and major depressive disorder.

Interestingly, we also found disorder-unique brain alterations for BPP, SAD, and SZ. Five connectivity relationships were uniquely changed in BPP, involving the supramarginal gyrus, caudate, middle temporal gyrus, pallidum, and cerebellum crus. For SAD, there were four unique connections, which were relevant to the middle and inferior frontal gyri, rolandic, Heschl’s gyrus, cuneus, middle temporal pole, and rectus. SZ-unique changes mainly consisted of superior frontal gyrus, amygdala, and putamen. The findings may benefit the understanding of the current diagnosis categories. There has been prior evidence (Zhou et al., 2015) that the prefrontal cortex dysconnectivity is importantly associated with SZ.

Our paper is the first attempt utilizing dynamic connectivity features to investigate biological evidence among HC, BPP, SAD and SZ via classification. More importantly, our results highlight these disorder-common and disorder-specific abnormalities in brain connectivity. This provides clues for understanding the current symptom-based categories and hopefully providing prior information for future development of biologically meaningful categories. It is worth noting that since different hypotheses were proposed in analyzing functional dynamics, there have been various approaches such as clustering (Allen et al., 2014, Du et al., 2016, Rashid et al., 2016) and decomposition (Yaesoubi et al., 2015, Du et al., 2017b) that can be used for extracting connectivity states. In our work, we extended our previously proposed decomposition method (Du et al., 2017a, Du et al., 2017c, Du et al., 2018a) by performing ICA on all subjects’ data so as to yield direct and unbiased connectivity-related features for classification. Previous studies have shown better performance using dynamic connectivity, compared to static connectivity. To test this point using our data, we employed the AAL-based static connectivity measures to classify the four groups, under the same 10-fold cross-validation pipeline and SVM classifier. As shown in Fig. S6, the mean classification accuracy was 60.01%, the mean balanced accuracy was 56.92%, and the mean balanced precision was 59.04%. Regarding the mean value of individual class accuracy, HC was 74.89%, BPP was 54.08%, SAD was 50.01%, and SZ was 48%. Regarding the mean value of individual class precision, HC was 68.68%, BPP was 49.14%, SAD was 58.45%, and SZ was 59.90%. It is clear that the accuracy of static functional connectivity was lower than the results of dynamic functional connectivity. We also summarized the important features used in the four-group classification in Table S6. These significant connections primarily consisted of frontal, temporal and cerebellar regions.

Several aspects of our work may need future refinement. First, functional connectivity based on fixed AAL regions can be limiting. While there is no gold standard for the selection of ROIs, functional connectivity obtained using different parcellation strategies can result in various classification performances (Kalmady et al., 2019). In this paper, we employed AAL regions to estimate dynamic connectivity so as to have a convenient comparison with our previous study (Du et al., 2017a, Du et al., 2017c) that analyzed the same datasets. Although the classical AAL atlas has been widely applied for the classification between patients with brain disorders and healthy controls (Chen et al., 2011, Yu et al., 2017), atlas-based template might not perfectly adapt to signals in the individual fMRI data. In the future, we or others can adopt data-driven methods, such as ICA-driven regions (Salman et al., 2019), parcellations through clustering approaches (Yeo et al., 2011, Shen et al., 2013), and functional connectivity boundary mapping (Gordon et al., 2016), which could further improve classification performance by allowing the regions to adapt to the individuals rather than using fixed ROIs. Second, in this paper the group-level connectivity states were obtained by performing a group ICA on the dynamic connectivity patterns of all subjects so as to decrease the computation load, which is consistent to previous studies (Rashid et al., 2016, Osuch et al., 2018). For a more unbiased manner, the group-level states should be estimated using only the training data and then used to guide the computation of the individual states for both the training and testing data. To access the possible influence, we performed group ICA on the randomly selected training data for ten times. Our results show that the connectivity states from different training datasets were highly similar to the states computed using all subjects, with the correlations (reflecting the similarity) between the dominant states all higher than 0.98 (See Fig. S7). Third, although our results show some biological support for these existing categories of mental illness, given that symptoms are known to be an imperfect way to identify medical disorders of any type from, further work should focus on refining categories by taking advantage of neuroimaging measures (Insel, 2014). The question of whether traditional categories of mental illness should be ultimately replaced by those based purely on biology is a related issue (Clementz et al., 2016). Fourth, in our method, for facilitating the classification and enabling the feature correspondence, we used a fixed number of components (i.e. the number of states) while performing decomposition on all data. Previous work (Supekar et al., 2019) identified more inconsistent connectivity states between disease and healthy groups. Using our decomposition method, the dominant state would be similar between different groups (as shown in Fig. S8). How to estimate unique but comparable states from dynamics with the lack of group labels deserves further study. In this paper, we set the state number to five, consistent to many previous studies (Damaraju et al., 2014, Rashid et al., 2014a, Rashid et al., 2014b, Yaesoubi et al., 2015, Miller et al., 2016, Du et al., 2018a, Fu et al., 2019). How to determine an optimal component number is difficult in the blind signal processing problem. But, our additional experiments support that the estimated connectivity states under different number of states (M = 3, 4, 5, 6 and 7) tended to be stable under different settings (Fig. S9). Fifth, due to the limited sample size, the four groups included different subject numbers, which could affect the results to some extent. To verify this, we randomly selected 100 subjects from each group, and reran all analyses. In general, the results were consistent with our original results. Regarding the four-group classification, the mean overall classification accuracy was 71.25% in distinguishing among HC, BPP, SAD and SZ groups. For individual class accuracy, the mean value was 72.9% for HC, 73.62% for BPP, 72.78% for SAD and 65.7% for SZ; for individual class precision, the mean value was 74.66%, 69.11%, 74.06% and 71.96%, respectively. We summarized the 28 important features in Table S7, six of which consistently presented in the original results. These features also involved the frontal, temporal, and cerebellar regions. However, further evaluations are still needed when more data are available. Finally, as we focused primarily on the issue of classification, we did not assess associations between the functional connectivity and clinical symptoms.

CRediT authorship contribution statement

Yuhui Du: Methodology, Investigation, Writing - review & editing, Funding acquisition. Hui Hao: Writing - review & editing, Programing. Shuhua Wang: Writing - review & editing, Software. Godfrey D Pearlson: Writing - review & editing. Vince D. Calhoun: Writing - review & editing.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.