A new multimodality fusion classification approach to explore the uniqueness of schizophrenia and autism spectrum disorder.

Schizophrenia (SZ) and autism spectrum disorder (ASD) sharing overlapping symptoms have a long history of diagnostic confusion. It is unclear what their differences at a brain level are. Here, we propose a multimodality fusion classification approach to investigate their divergence in brain function and structure. Using brain functional network connectivity (FNC) calculated from resting-state fMRI data and gray matter volume (GMV) estimated from sMRI data, we classify the two disorders using the main data (335 SZ and 380 ASD patients) via an unbiased 10-fold cross-validation pipeline, and also validate the classification generalization ability on an independent cohort (120 SZ and 349 ASD patients). The classification accuracy reached up to 83.08% for the testing data and 72.10% for the independent data, significantly better than the results from using the single-modality features. The discriminative FNCs that were automatically selected primarily involved the sub-cortical, default mode, and visual domains. Interestingly, all discriminative FNCs relating to the default mode network showed an intermediate strength in healthy controls (HCs) between SZ and ASD patients. Their GMV differences were mainly driven by the frontal gyrus, temporal gyrus, and insula. Regarding these regions, the mean GMV of HC fell intermediate between that of SZ and ASD, and ASD showed the highest GMV. The middle frontal gyrus was associated with both functional and structural differences. In summary, our work reveals the unique neuroimaging characteristics of SZ and ASD that can achieve high and generalizable classification accuracy, supporting their potential as disorder-specific neural substrates of the two entwined disorders.

INTRODUCTION

Schizophrenia (SZ) and autism spectrum disorder (ASD) share a long history of diagnostic confusion (Trevisan et al., 2020). The term “autism” was first introduced as a symptom of SZ rather than as an independent disorder. In the DSM‐II (the second edition of the Diagnostic and Statistical Manual of Mental Disorders), autism was referred to as a childhood type of SZ (called childhood‐onset SZ), and ASD was first proposed as a distinct clinical diagnosis in the release of the DSM‐III. Although SZ and ASD are currently recognized as separate disorders, they have overlapping clinical symptoms. Undeniably, both SZ and ASD share cognitive impairments (Sasson et al., 2011), including notable impairment in theory of mind (Frith, 2004; Pilowsky et al., 2000), deficits in processing emotion (Kohler et al., 2010; Wallace et al., 2011), and language and learning disabilities (Stone & Iguchi, 2011). The basic problem is that the two disorders are defined by clinical symptoms and not by biology. There is thus a great need to reveal the neural difference between the two related disorders because a deeper understanding of each disorder would lead to better treatments. Although recent studies have investigated the association between ASD and SZ (King & Lord, 2011; Lanillos et al., 2020; Veddum et al., 2019), it is still largely unknown what their differences are on both brain function and structure and whether multimodal neuroimaging measures can effectively differentiate them.

Advanced brain functional measures derived from resting‐state functional magnetic resonance imaging (fMRI) data and brain structural information estimated from structural magnetic resonance imaging (sMRI) data have been used to explore SZ and ASD, however, most of the early studies worked on them separately (Anderson et al., 2011; Brent et al., 2013; Du, Fryer, et al., 2018; Du, Fu, et al., 2018; Greimel et al., 2013). Recently, increasing studies found an association between the two disorders (Krieger et al., 2021; Lanillos et al., 2020), raising a compelling demand to investigate their shared and unique abnormalities in the brain.

Based on brain structural measures, some previous studies (Cauda et al., 2017; Cheung et al., 2010; Zheng et al., 2018) evaluated SZ and ASD using meta‐analysis, revealing their similar changes in many aspects such as lower limbic‐striato‐thalamic gray matter volume (GMV) relative to healthy controls (HCs). In our recent work, we performed statistical analyses to investigate inter‐group differences among HC, SZ, and ASD in whole‐brain GMV of 3148 subjects (1661 HCs, 517 SZ patients, and 970 ASD patients) and gray matter density of 3374 subjects (1789 HCs, 555 SZ patients, and 1030 ASD patients), consequently disclosing substantial commonality and specificity of SZ and ASD in brain gray matter (Du et al. 2021). Nevertheless, in the study whether their differences in gray matter impairments can effectively differentiate the two disorders was not discovered. Another recent study (Yassin et al., 2020) used 97 patients with SZ spectrum, 36 patients with ASD, and 106 HCs to explore the ability of measures including cortical thickness (150 regions), surface area (150 regions), and subcortical volume (36 regions) in distinguishing the three groups or any two groups. The best results were 69% accuracy for the three‐class classification and 85% accuracy for the SZ versus ASD classification using cortical thickness. Although they achieved good classification performance, the features extracted by principal component analysis (PCA) could not reflect the most important features relating to classification.

By using brain functional connectivity as features, some studies applied a classification strategy to explore differences between ASD and SZ. Chen et al. (2017) used a support vector machine (SVM) to distinguish SZ or ASD patients from HCs, resulting in a good performance on a small sample size of data (80% accuracy in the 22 ASD patients vs. 21 HCs and 83% accuracy in the 35 SZ patients vs. 31 HCs). Relative to HCs, they found alterations of ASD primarily in intra‐salience network connectivity, and changes of SZ primarily in functional connectivity between the default mode and salience networks and functional connectivity within the sensorimotor network. Andriamananjara et al. (2018) employed SVM to distinguish individuals with SZ or ASD from HCs, showing an improved classification accuracy using dynamic connectivity (82% for 31 ASD vs. 23 HCs and 76% for 70 SZ patients vs. 74 HCs) compared to using static connectivity (78% for ASD datasets and 74% for SZ datasets). They revealed the discriminative connectivity involving the posterior cingulate gyrus, superior, and middle temporal gyrus, however, the used sample size was still small. Inspired by a prior study (Yahata et al., 2016) that separated ASD from HC with an accuracy of 85% but also found that the ASD‐derived classifier can moderately distinguish SZ from HC, Yoshihara et al. (2020) conducted dual classifiers that discriminated SZ or ASD from HC, and then quantified the relationship between SZ and ASD by evaluating the classification certainty using the dual classifiers on all SZ, ASD, and HC subjects. Their study suggested that SZ and ASD tended to show an overlapping relationship. All the above‐mentioned studies focused on classifying patients from HCs, and Mastrovito et al. (2018) conducted the first study classifying SZ and ASD directly using brain measures. They performed SVM on effective connectivity of 72 SZ and 37 ASD patients and yielded a 75% SZ versus ASD classification accuracy for a small test dataset (5 SZ and 27 ASD patients). In their work, the features used in distinguishing SZ and ASD were drawn from the features that separated SZ or ASD from HCs, thus those features may not accurately represent the most important differences between the two disorders. In our recent work (Du et al., 2021), we also performed statistical analyses to investigate inter‐group differences among HC, SZ, and ASD in brain functional networks and connectivity of 2980 subjects (1665 HCs, 537 SZ patients, and 778 ASD patients) and found the disorder‐common and disorder‐specific impairments relative to HC. Based on the disorder‐unique connectivity measures and the ASD‐weaker connectivity measures within the disorder‐common changes, SZ and ASD were distinguished well (i.e., the mean classification accuracy reached 75% across 12 classifications that employed different datasets as the testing data). However, the used features also came from the results of comparing disorder (SZ or ASD) with HC, rather than the results of directly linking SZ and ASD. Being different from using a two‐class classification strategy, Rabany et al. (2019) performed a three‐class classification based on measures derived from dynamic functional network connectivity, resulting in an accuracy of 81.8% for SZ, 50% for ASD, and 41.2% for HC using 33 SZ patients, 33 ASD patients, and 34 HCs. Since the study used dynamic measures including the fraction and dwell time of connectivity states as the features for the classification, no brain region or connectivity was directly linked to the disorder differences.

In summary, there are only a few studies that performed a direct comparison or classification between SZ and ASD. Although some studies have successfully classified ASD and SZ using brain functional or structural features, the used features cannot reflect their most important differences with each other, and no multimodal characteristics were used. Because there is a large overlap and complex relationships between SZ and ASD, it is particularly important to investigate what brain differences underline the neural substrate to make their diagnoses valid. Our other study made an effort in this area by proposing a new deep learning fusion model to combine brain functional and structural features for distinguishing SZ and ASD (Du, Li, et al., 2020). However, the elucidation of explicit features (reflecting the essential differences between the two disorders) in a deep learning model is often difficult. Also, although an unbiased five‐fold cross‐validation was employed for the evaluation, no fully independent dataset was used to verify the generalization ability of classification in our previous work (Du, Li, et al., 2020). Taken together, the divergence between these two related disorders at a brain level, especially on different modalities, is still not well understood.

In our present study, we aim to explore the difference between ASD and SZ based on both brain functional and structural characteristics by performing a straightforward classification between the two disorders. Specifically, our first goal is to investigate if the two disorders for which there has been a growing debate on their overlap and heterogeneity can be effectively distinguished by only using brain functional and structural measures. Going further, we are also interested in whether combining different functional and structural measures would improve classification performance. To achieve this, we propose a new fusion classifier that can effectively leverage information from different modalities. Our second goal is to explore what brain measures are different between the two disorders and what relationships exist between multimodal features. In this study, we tested the reliability of the trained classifier and identified differences using 715 individuals as the main training/testing data with unbiased 10‐fold cross‐validation, and also examined the generalization ability on 469 individuals as the fully independent data (collected at different sites) for evaluation. Furthermore, regarding the important features that were used to differentiate SZ and ASD, we investigated how each disorder differed relative to HCs to identify whether the two disorders show overlapping or specific abnormalities. We believe that our work will further the understanding of the mechanisms differentiating SZ and ASD.

MATERIALS AND METHODS

Data and preprocessing

The main training and testing data for the SZ‐ASD classification were from four datasets, including SZ patients from the Bipolar‐Schizophrenia Network for Intermediate Phenotypes‐1 (BSNIP‐1), SZ patients from the Function Biomedical Informatics Research Network (FBIRN), SZ patients from the Centers of Biomedical Research Excellence (COBRE), and ASD patients from the Autism Brain Imaging Data Exchange I (ABIDE I). For further validation, we also used the independent data from two datasets, including SZ patients from the Maryland Psychiatric Research Center (MPRC) and ASD patients from the Autism Brain Imaging Data Exchange II (ABIDE II) to evaluate the trained classifiers and the automatically selected features.

Both fMRI and sMRI datasets were included. All data was processed in the same preprocessing pipeline, using the statistical parametric mapping toolbox. For fMRI data, we removed the first few time points and then performed the rigid body motion correction to correct the subject's head motion, followed by the slice‐timing correction to account for timing difference in slice acquisition. fMRI data were subsequently warped into the standard Montreal Neurological Institute space using an echo planar imaging (EPI) template and were then resampled to 3 × 3 × 3 mm3 isotropic voxels. The resampled fMRI images were further smoothed using a Gaussian kernel with a full width at half maximum (FWHM) = 6 mm. For sMRI data, the T1‐weighted images were first segmented into gray matter, white matter, and cerebrospinal fluid by using the standard unified segmentation model (Ashburner & Friston, 2005). The Diffeomorphic Anatomical Registration Through Exponentiated Lie Algebra (DARTEL) algorithm (Goto et al., 2013) was employed to create a group template for spatial normalization of the segmented images of each subject. Then, the flow fields generated by DARTEL were used to estimate individual‐subject images. After that, individual‐subject gray matter images were spatially normalized to the MNI space, modulated, resliced (1.0‐mm isotropic voxels), and smoothed (6‐mm FWHM Gaussian kernel). Finally, the obtained GMV was used as voxel‐based morphometry (VBM).

After selecting fMRI and sMRI data with good quality (such as slight head motion in fMRI), we preserved 335 SZ and 380 ASD patients for the main training/testing data and 120 SZ and 349 ASD patients for the independent data. We also investigated the brain changes of each disorder group relative to the healthy group to identify the most discriminative features classifying the SZ and ASD patients. For this goal, 851 HCs from the BSNIP‐1, FBIRN, COBRE, and ABIDE I datasets were used. Tables 1 and S1 include the demographic and motion information and the scanning information of the selected data, respectively.

TABLE 1

The demographic and motion information of ASD, SZ, and HC groups used for classification and statistical analysis.

		Subject number	Mean age of subjects	Gender of subjects (male/female number)	Motion transition: mean/SD	Motion rotation: mean/SD
The main training and testing data	ASD patients (ABIDE I)	380	17.80	331/49	0.20/0.13	0.21/0.14
	SZ patients (BSNIP‐1, COBRE, and FBIRN)	335 (159, 53, 123)	36.38 (34.65, 35.09, 39.15)	185/150 (42/117, 47/6, 96/27)	0.16/0.14	0.15/0.14
The independent data	ASD patients (ABIDE II)	349	15.89	298/51	0.19/0.15	0.21/0.16
	SZ patients (MPRC)	120	38.08	82/38	0.10/0.11	0.08/0.11
HC (ABIDE I, BSNIP‐1, COBRE and FBIRN)		851 (443, 195, 79, 134)	27.59 (18.03, 38.34, 37.89, 37.47)	621/230 (360/83, 106/89, 57/22, 98/36)	0.17/0.12	0.18/0.13

Note: For the main data, the p‐value of age difference between SZ and ASD is 2.84 × 10−96 tested by two‐sample t‐test, and the p‐value of their gender difference is 0 tested by Chi‐square test. For the independent data, the p‐value of age difference between SZ and ASD is 8.38 × 10−62 tested by two‐sample t‐test, and the p‐value of their gender difference is 3.96 × 10−05 tested by Chi‐square test. Among the three groups in main data (ASD, SZ, and HC), the p‐value of age difference is 2.78 × 10−78 tested by ANOVA, and the p‐value of gender difference is 0. The motion translation measure of each subject was computed by averaging translation parameters across time points as well as , , and axes. The motion rotation measure of each subject was computed by averaging rotation parameters across time points as well as pitch, yaw, and roll. The motion differences among HC, ASD, and SZ were computed using ANOVA. For the main data, the p‐value is 7.11 × 10−05 for motion transition and is 5.10 × 10−08 for motion rotation. For the independent data, the p‐value is 4.92 × 10−09 for motion transition and is 1.57 × 10−14 for motion rotation.

Methods

In this study, we first estimated brain functional network connectivity (FNC) and voxel‐based GMV for each subject to reflect brain function and structure. Next, the SZ and ASD groups were classified using the single‐modality method and our proposed multimodality fusion method, respectively. For both methods, SVM in combination with an automatic feature selection algorithm was applied to the main datasets under a 10‐fold cross‐validation framework to evaluate the classification performance. Additionally, the independent data were used to validate the generalizability of the constructed classifiers and selected features. Finally, we investigated the features that played a key role in distinguishing SZ and ASD to address if the two disorders showed unique abnormalities.

Computation of brain functional and structural measures

In the work, brain functional networks were first estimated using the NeuroMark method and software (Du, Fu, et al., 2020) (see the shared resources on www.yuhuidu.com for the NeuroMark software), and then FNC (Jafri et al., 2008) was computed based on the time courses of brain functional networks. NeuroMark utilizes group information guided ICA (GIG‐ICA) (Du & Fan, 2013; Du, et al., 2015; Du, Fryer, et al., 2018) with reliable network templates as references to the preprocessed fMRI data to estimate subject‐specific functional networks. Our method can easily compute comparable network measures for individuals while maximally optimizing the unique network property for each subject. The unbiased spatial network templates were estimated using two large independent groups of HCs, that is, 1005 HCs from the Human Connectome Project (HCP) and 823 HCs from the Genomics Superstruct Project (GSP), with the number of independent components (ICs) in ICA set to 100, followed by the selection of 53 reliable, reproducible, and meaningful networks as templates. The network templates can be downloaded at the shared resources on www.yuhuidu.com. Guided by the 53 network templates, 53 corresponding networks were obtained for each subject using the multiple‐objective optimization in GIG‐ICA. Each resulting functional network reflects brain regions having high intra‐network connectivity. Using NeuroMark, the temporal fluctuation of each subject‐specific network can be reflected by the component‐associated time course. We calculated FNC of each subject by computing the Pearson correlation among the postprocessed time courses of 53 networks. Each time course was processed by converting to a Z‐score, regressing six motion parameters, de‐trending, de‐spiking, and band‐filtering (0.01–0.15 Hz) before computing correlations (Allen et al., 2011). It should be pointed out that motion has systematic effects on fMRI measures (Van Dijk et al., 2012), so in our study we carefully dealt with this issue by performing head motion correction in the preprocessing steps, selecting subjects with small head motions for analyses, removing the artifacts by applying ICA to decompose data into ICs including motion‐related components, and regressing six motion parameters on the time series of brain functional networks before computing FNC measures. Since each element of FNC matrix (size: 53 × 53) represents the connectivity between a pair of functional networks, this resulted in 1378 connectivity features for each subject due to the symmetrical property of FNC matrix.

From sMRI data, we obtained the GMV measures represented by a 3D image (size: 121 × 145 × 121) for each subject. Next, we down‐sampled each 3D image to a smaller one to decrease computation load. In our work, a 3 × 3 × 3 window was used to transform the original matrix into a 40 × 45 × 40 matrix. We then concatenated the voxel‐level GMV values within the brain mask into a vector (size: 1 × 15,342) as the brain structural measures. As such, we obtained two types of neuroimaging measures including FNC and GMV, which were taken as possible features for the classification between SZ and ASD so as to be indicators disclosing brain differences between the SZ and ASD groups and were also used for the investigation of their brain changes relative to HC. A summary of the above‐mentioned neuroimaging measure computation is shown in Figure 1.

FIGURE 1

The pipeline for computing brain functional connectivity and structural features. (a) The computation of functional network connectivity (FNC). (b) The computation of gray matter volume (GMV).

Since neuroimaging measures could be affected by age (Giorgio et al., 2010), gender (Filippi et al., 2013), and MRI acquisition (Streitburger et al., 2014), we carefully regressed out their effects from each neuroimaging measure (e.g., GMV in one voxel) in advance by performing two‐stage linear regression analyses  that are as same as our previous work (Du, Fu, et al., 2020; Du et al., 2021) to minimize the effects of age, gender, and site on the classification and group difference analyses. First, for each dataset (e.g., FBIRN), we regressed out the age, gender, site information, the interaction between age and site, and the interaction between gender and site for all subjects using a multiple linear regression model. In particular, for one specific neuroimaging measure, we constructed a model . Here, denotes the predicted value of the dependent variable, corresponding to the neuroimaging‐derived values of all subjects in the dataset (e.g., FBIRN). , , , , and are the explanatory variables, corresponding to the age, gender, site information, the interaction between age and site, and the interaction between gender and site of all subjects. It should be noted that since there are multiple sites for each dataset, we included all the site information in our study. is a constant term, and are coefficients relating the explanatory variables. We assume that the error term has a mean value of 0. Using the model, the relationship between the dependent variable and explanatory variables, reflected by , was estimated. Then, for the neuroimaging measure of each subject, we removed the effects of the age, gender, site information, the interaction between age and site, and the interaction between gender and site, thus resulting in new neuroimaging values of all subjects in the dataset, denoted by . Since the above‐mentioned regression was performed on each dataset (e.g., FBIRN) separately, we further handled the data variations across different datasets. To address this, we estimated the dataset effect by only using the new neuroimaging values of HCs (in ) from all six datasets. Based on the concatenated new data of all HCs, denoted by , we constructed another linear regression model . Here, represents the new data of HCs of all six datasets, includes the dataset information, and is the error term. From the model, and were obtained. After that, for each of all subjects (including both HCs and patients), we further regressed out the dataset effect from the neuroimaging value that were already removed the age, gender, and site effects (i.e., the data from ). In our study, we used dummy variables for the category‐related explanatory variables (i.e., gender, site information, the interaction between age and site, the interaction between gender and site, and dataset information). The above processing was performed for each neuroimaging measure separately. As such, we obtained the final data relating all neuroimaging measures for all subjects, which were used for the classification and group difference analyses.

Classification between SZ and ASD using single‐modality neuroimaging features

We separately investigated the performance of the functional connectivity and gray matter measures in differentiating SZ and ASD. As mentioned above, we computed FNC and GMV measures for the classification process. SVM was used in our work since it has shown satisfactory performance in classifying many mental disorders (Mwangi et al., 2012, Chen et al., 2017;  Du, Fu, et al., 2018; de Filippis et al., 2019). In addition, features can be selected and represented more straightforwardly in SVM compared to complex techniques such as deep learning. We evaluated the classification performance using both the testing data in the main data and additional independent data for a comprehensive assessment.

Figure 2 shows the classification framework with the single‐modality measures (FNC or GMV) as input. As shown in Figure 2a, 335 SZ and 380 ASD patients from four different datasets were used as the main training/testing data under a 10‐fold cross‐validation framework. The 10‐fold random partition process was implemented for 100 runs to maximize reliability. Within each run, all 715 subjects were divided into 10 folds equally, in which one randomly selected fold was used as the testing data, and the remaining nine folds were used as the training data. It should be noted that we kept the sample partitions consistent between the two modalities to facilitate further comparison and fusion. To automatically select optimal features, we applied two‐tailed two‐sample t‐tests followed by SVM with recursive feature elimination (SVM‐RFE) technique in inner 10‐fold cross‐validation using the training data (Du et al., 2015; Zhou & Tuck, 2007). In SVM‐RFE, each iteration process removed the least significant 10% features. After each time of iteration, an SVM classifier was trained using the updated features on the nine inner‐training folds, and then the constructed SVM classifier was tested on one remaining inner‐testing fold. The above feature sorting and removal process was repeated 10 times, consequently resulting in 10 feature subsets each of which corresponded to the maximum classification accuracy (of the inner‐testing fold) of each time. Subsequently, the features with greater than 50% occurrence frequency in the 10 feature subsets were taken as the final selected features. We chose 50% as the threshold to guarantee a sufficient number of features remained as well as good classification performance. Next, using those features, an SVM classifier was built using the (main) training data. In our experiments, the radial basis function (RBF) kernel was used in SVM, and the (within the range of ) and gamma (within the range of ) parameters were optimized via a nested five‐fold cross‐validation procedure using grid search with a step size of 23. In our work, we choose a relatively large range for both and gamma and a small optimization grid with 15 points for both and gamma to optimize the parameters of the SVM classifier. That means we searched for optimal setting among many possible settings of for both and gamma, consequently maximizing the capability of achieving good classification performance. Finally, we examined the classification ability of the well‐constructed classifier on the (main) testing data. This resulted in 100 classification evaluations across different runs.

FIGURE 2

Classification flowchart using single‐modality features. The classification was performed on both the (a) main and (b) independent datasets. Figure 2c represents the detailed process of “selecting features” in Figure 2a. For the main training/testing data, a 10‐fold cross‐validation framework was used for evaluating the classification performance. The features extracted from the training data were regarded as the intergroup differences and then validated by using the independent data. The used single‐modality measures are functional network connectivity (FNC) or gray matter volume (GMV).

As the main data was randomly shuffled and partitioned into 10 folds, the classifiers should adapt to different data. For further validation, we evaluated the generalization ability of the constructed classifiers and selected features using the independent data including two additional datasets that were totally separated from the main data. Figure 2b shows this basic pipeline. Since 100 classifiers and feature sets were obtained from the training processing in the single‐modality related classification, we also obtained 100 classification results using the independent data.

To assess the classification performance for the testing (in the main data) and independent data, a series of metrics including the accuracy, sensitivity, specificity, precision, F‐measure, and G‐mean were computed (Cuadros‐Rodríguez et al., 2016). Accuracy is computed as the ratio of correctly classified subjects of all classes to the total number of subjects of all classes. The proportions of correctly classified subjects in the positive class (SZ here) and negative class (ASD here) are measured using sensitivity and specificity, respectively. Precision describes the proportion of actually being positive out of all the subjects that are predicted to be positive. F‐measure is a comprehensive reflection of the ability to distinguish positive and negative subjects and describes the robustness of the classification model. G‐mean describes an equilibrium between agreements and errors in classification. These measures are computed using the following equations.

Here, TP denotes the number of subjects that are judged to the positive class and that judgments are correct; TN denotes the number of subjects that are judged to the negative class and that judgments are correct; FP denotes the number of subjects that are judged to the positive class and that judgments are wrong; FN denotes the number of subjects that are judged to the negative class and that judgments are wrong. In our work, regarding each metric, 100 runs of 10‐fold cross‐validation resulted in 100 values for both the (main) testing and independent data, and we summarize and visualize the 100 values of each metric using one boxplot.

Classification between SZ and ASD using multimodality features by our fusion method

In this section, we propose a new fusion method to effectively take advantage of both functional and structural measures for the classification goal. As shown in Figure 3a, the basic idea is to combine the classifiers' outputs from different modalities by using a linear weighting method and then generate the final predicted label. The weights were determined based on how well each modality worked in the distinction using preexisting data. To generate the weights we normalized the accuracy of the FNC and GMV classifications. For the testing data (see Figure 3b), the accuracy was the maximum classification accuracy corresponding to the optimal parameters within the training data. For the independent data (see Figure 3c), the accuracy was the classification accuracy on the testing data. As we computed 100 classifiers for each modality, we also had 100 results for both testing and independent data using our fusion method. Note: the features used in the fusion method were as same as that in the single‐modality methods because we only combined the classifiers but not the features. Therefore, the two kinds of features can be easily presented as the disorder divergence using their original values, without the need for complex inversion.

FIGURE 3

Fusion classification framework by using both modalities (i.e., FNC and GMV). (a) Shows how individual subject from the testing (of the main data) and independent data is classified by combining the use of FNC and GMV features. Basically, the predicated group scores are sum‐weighted to obtain updated scores which are then used to determine the final label. (b) and (c) demonstrate how the weights corresponding to different modalities are computed for the testing and independent data, respectively.

Similar to the single‐modality method, we computed six metrics to assess the classification results from the fusion classifier and showed the details in boxplots. Furthermore, we performed a paired t‐test between the fusion method and the single‐modality (FNC or GMV) method for each metric to quantify whether there is a significant improvement in using the fusion method relative to the single‐modality method or not.

Investigation of brain changes in SZ and ASD relative to HC

Through the two‐class classification, we revealed the neuroimaging markers representing the primary brain differences of the two intertwined disorders, and going further we also investigated whether the two disorders have similar or divergent changing trends relative to the healthy group in terms of these biomarkers. To do this, we summarized the common features resulting from different classification runs and investigated them across three groups (851 HCs, 335 SZ, and 380 ASD patients) using statistical analyses, aiming to illuminate what brain changes differ in the two disorders and further if brain functional and structural differences show associations in common brain regions.

Regarding the FNC features, we took those which were present in all 100 FNC feature sets across 100 classification runs as the important FNCs. For each important FNC, we comprehensively investigated the group differences between SZ and ASD by performing two‐tailed two‐sample t‐tests. After that, we chose the FNCs that had the most significant differences in SZ versus ASD (with the smallest p‐values) to evaluate if the two disorders showed similar or disparate changing trends relative to HC in those FNCs (p‐value <.05 with Bonferroni correction).

Regarding the GMV features, considering that the number of voxels taken as features was relatively large, we summarized the features at the brain region level by using the automated anatomical labeling atlas 3 (AAL3) for a brain parcellation (Rolls et al., 2020) to simplify the subsequent analyses. We included the voxels representing more than 10% overlapping features (across 100 runs) to form important brain regions. The overlap degree of GMV features was set to a smaller threshold (10%) relative to that of FNC features because there were more optional features in GMV compared to FNC and the final selected features were relatively few for both modalities. For each important brain region, we investigated the group differences between SZ and ASD by performing two‐tailed two‐sample t‐tests. Then, we chose the important regions that had the most significant differences in SZ versus ASD (with the smallest p‐values) and a relatively large number of voxel features (including more than 100 voxels) to evaluate if the two disorders showed similar or disparate changing trends relative to HC in those brain regions. For each important brain region that we selected, we calculated the mean of voxel features within it for each subject and summarized the group differences using two‐tailed two‐sample t‐tests (p‐value <.05 with Bonferroni correction) on any pair of groups.

RESULTS

Performance of classification between SZ and ASD

Figure 4 shows the classification results obtained from the single‐modality method (using FNC or GMV) and our fusion method for both the (main) testing and the independent data. For each metric, the mean value across all 100 classification runs is displayed in Table 2. Regarding the (main) testing data, the mean values of all metrics were greater than 75% from using FNC, furthermore, our fusion method greatly outperformed the single‐modality methods. The results from the paired t‐tests (see Table S2) between the classification using our fusion method and that using the single‐modality method also support that the fusion method achieved significantly better results. Regarding the independent data, the classification accuracy was lower than that of the testing data, probably due to the varied data property (from different sites). As expected, our proposed fusion method still improved the classification performance (see Table 2 for the accuracy and Table S3 for the paired t‐test results), compared to single‐modality methods. The low precision and F‐measure of the independent data resulted from the uneven size of subjects between the two groups (120 SZ and 349 ASD patients). In sum, our method generally performed better than the single‐modality methods for both the (main) testing data and the independent data.

FIGURE 4

Evaluated metrics of the classification results using the FNC, GMV measures, and our fusion method. We show the results obtained using the testing data of the main datasets and the independent datasets in (a) and (b), respectively. For each metric, 100 values from 100 classification runs are shown in one boxplot.

TABLE 2

Evaluated metrics computed based on the classification results of the (main) testing data and independent data by using the FNC measures, GMV measures, and our fusion method.

	Accuracy (%)	Sensitivity (%)	Specificity (%)	Precision (%)	F‐measure (%)	G‐mean (%)
FNC using testing data	80.00	77.30	82.34	79.69	78.17	79.58
GMV using testing data	70.65	68.03	72.92	68.91	68.25	70.23
Our fusion method using testing data	83.08	80.58	85.26	83.00	81.52	82.73
FNC using independent data	69.58	62.43	72.04	43.47	51.23	67.04
GMV using independent data	63.07	62.00	63.44	36.95	46.27	62.67
Our fusion method using independent data	72.10	64.60	74.67	46.78	54.24	69.43

Note: Here, the mean value across all 100 classification runs is summarized for each of the six metrics.

Divergence between SZ and ASD in both FNC and GMV

Since the features were selected by maximizing the classification accuracy via inner cross‐validation within the training data and as expected those features yielded high classification accuracy for both testing and independent data in distinguishing SZ and ASD, those features should be able to reflect primary group differences in the brain between the two disorders.

In terms of the FNC measures, there were a total of 33 important FNCs (see details in Table S4) that were present in all 100 feature sets of 100 classification runs. Here, we only show the top 10 FNCs corresponding to the smallest p‐values of the two‐sample t‐test between SZ and ASD. Table 3 includes the detailed information of the top 10 FNCs, including the related brain networks, the mean FNC strength in each group, and the group difference of SZ versus ASD. Figure S1 shows the two‐sample t‐test results of the 10 FNCs using boxplots. The 10 FNCs were mainly related to sub‐cortical networks (including the connectivity between sub‐cortical networks and DMN, between sub‐cortical networks and cerebellar networks, between sub‐cortical networks and cognitive control networks, as well as between sub‐cortical networks and visual networks), and also included the connectivity within visual networks as well as between DMN and visual networks. To be more concise, we separated the 10 FNCs into SZ‐higher FNCs (see Figure 5a1) and SZ‐lower FNCs (see Figure 5a2), compared to ASD. In particular, there were four of ten FNCs presenting higher strengths in SZ than ASD, and they were located between the caudate and precuneus, thalamus and calcarine gyrus, caudate and posterior cingulate cortex, thalamus and right middle occipital gyrus. The remaining six FNCs had lower strengths in SZ than ASD, including the connectivity between subthalamus and cerebellum, caudate and middle frontal gyrus, middle occipital gyrus, and precuneus as well as cuneus and lingual gyrus. As shown in Figure 6, within the top 10 FNCs, four FNCs in HC were located intermediate between SZ and ASD. Remarkably, for all the four FNCs that were related to default mode network (DMN), including the connectivity between DMN and subcortical networks as well as between DMN and visual networks, SZ and ASD showed the opposite changing direction (relative to HC). More interestingly, regarding the remaining six FNCs, SZ, and ASD showed the same changing direction (relative to HC), but the changing strengths of SZ generally were greater than ASD, indicating a more severity in SZ.

TABLE 3

The top 10 important FNCs used in the classification.

Brain network 1	Brain network 2	Mean FNC strength of HC group	Mean FNC strength of ASD group	Mean FNC strength of SZ group	p‐value of two‐sample t‐test between SZ and ASD	T‐value of two‐sample t‐test between SZ and ASD
Caudate (SC‐IC 99)	Precuneus (DM‐IC 40)	−0.369	−0.381	−0.291	1.82 × 10−15	8.136
Subthalamus (SC‐IC 53)	Cerebellum (CB‐IC 7)	0.176	0.141	0.035	1.18 × 10−13	−7.568
Caudate (SC‐IC 99)	Middle frontal gyrus (CC‐IC 88)	0.173	0.168	0.095	3.11 × 10−10	−6.384
Thalamus (SC‐IC 45)	Calcarine gyrus (VI‐IC 16)	−0.307	−0.263	−0.176	1.05 × 10−09	6.185
Caudate (SC‐IC 99)	Posterior cingulate cortex (DM‐IC 94)	−0.190	−0.198	−0.130	3.14 × 10−09	6.000
Thalamus (SC‐IC 45)	Right middle occipital gyrus (VI‐IC 12)	−0.193	−0.165	−0.090	5.06 × 10−09	5.918
Middle occipital gyrus (VI‐IC 5)	Precuneus (DM‐IC 40)	−0.107	−0.069	−0.144	1.04 × 10−08	−5.792
Middle occipital gyrus (VI‐IC 5)	Precuneus (DM‐IC 51)	0.036	0.050	−0.025	2.02 × 10−08	−5.675
Subthalamus (SC‐IC 53)	Cerebellum (CB‐IC 18)	−0.145	−0.182	−0.264	2.42 × 10−08	−5.642
Cuneus (VI‐IC 15)	Lingual gyrus (VI‐IC 8)	0.811	0.805	0.731	2.69 × 10−08	−5.623

Note: For each FNC, we included the relevant brain networks, the mean FNC strength of each group, and the p‐value and T‐value in SZ versus ASD (tested by two‐sample t‐test). We sorted the important FNC features (in Table S4) according to the p‐values obtained in the SZ versus ASD comparison. Here, we only include the top 10 FNCs with the lowest p‐values.

FIGURE 5

The top 10 functional network connectivity (FNC) features and the top 15 brain regions with important gray matter volume (GMV) features used in the classification. For the top 10 FNCs, we show each FNC's mean strength of HC, SZ, and ASD groups in (a1) and (a2). (a1) Includes the connectivity in which the mean functional connectivity strength of SZ patients was higher than ASD patients. (a2) includes the connectivity in which the mean functional connectivity strength of SZ patients is lower than ASD patients. Regarding the top 15 brain regions, the mean GMV voxel values across subjects in HC, SZ, and ASD groups are displayed in (b). It should be noted that all 15 regions presented higher GMV values in ASD than SZ

FIGURE 6

Differences between disorder group (SZ or ASD) and HC group for the top 10 FNCs. The difference of each FNC was calculated by the mean FNC strength across subjects in disorder group minus the mean FNC strength across subjects in HC group

Regarding the GMV measures, since we summarized the voxel features by assigning them to brain regions, more meaningful information can be provided. Table S5 shows detailed information for all 77 brain regions that were identified by our method. Because some brain regions only had a few voxels overlapping, we only selected the regions with more than 100 voxels and the smallest p‐values in SZ versus ASD to show their group differences here. A total of 15 brain regions were retained, and their region locations, mean GMV of each group, and group differences (from two‐sample t‐tests between SZ and ASD) are listed in Table 4. We also show the two‐sample t‐test results of the 15 brain regions in Figure S2 using boxplots. These regions mainly included the frontal gyrus, temporal gyrus, and insula. Remarkably, all 15 regions presented higher GMV values in ASD than SZ. See Figure 5b for representative brain regions. More surprisingly, the mean GMV value of HC was located between ASD and SZ for all those regions, as shown in Figure 7.

TABLE 4

The top 15 important brain regions each of which consisted of more than 10% overlapping GMV features in the classification and also included abundant voxels and the lowest p‐values in the SZ versus ASD comparison.

Brain region	Mean GMV of HC group	Mean GMV of ASD group	Mean GMV of SZ group	Number of voxels in each region	p‐value of two‐sample t‐test between SZ and ASD	T‐value of two‐sample t‐test between SZ and ASD
Frontal_Sup_Medial_L	0.590	0.599	0.559	188	1.06 × 10−19	−9.355
Temporal_Mid_R	0.597	0.613	0.570	241	1.58 × 10−18	−9.030
Frontal_Sup_Medial_R	0.588	0.596	0.558	178	6.22 × 10−18	−8.862
Frontal_Sup_2_R	0.611	0.618	0.578	590	7.40 × 10−18	−8.840
Temporal_Pole_Sup_L	0.695	0.704	0.661	388	1.22 × 10−17	−8.778
Temporal_Pole_Mid_R	0.652	0.660	0.615	219	1.29 × 10−17	−8.772
Frontal_Sup_2_L	0.579	0.587	0.550	471	3.10 × 10−17	−8.661
Temporal_Pole_Sup_R	0.673	0.683	0.641	545	3.41 × 10−17	−8.649
Temporal_Sup_R	0.631	0.649	0.611	151	1.87 × 10−16	−8.432
Temporal_Inf_R	0.618	0.631	0.595	127	2.14 × 10−16	−8.415
Frontal_Mid_2_L	0.668	0.675	0.635	283	3.08 × 10−16	−8.368
Insula_R	0.619	0.624	0.581	145	5.30 × 10−16	−8.298
Temporal_Sup_L	0.714	0.723	0.686	164	2.50 × 10−15	−8.093
Temporal_Mid_L	0.682	0.694	0.657	223	6.44 × 10−15	−7.967
Frontal_Mid_2_R	0.578	0.582	0.548	303	8.41 × 10−14	−7.615

Note: Automated anatomical labeling atlas 3 (AAL3) was used to parcellate the brain. For each brain region, we list the relevant region name, the mean GMV of each group (HC, ASD, and SZ), the number of voxels in each region, and the p‐value and T‐value in the SZ versus ASD comparison (tested by two‐sample t‐test).

FIGURE 7

Differences between disorder group (SZ or ASD) and HC group for the top 15 brain regions. The difference of each brain region was calculated by the mean GMV across subjects in disorder group minus the mean GMV across subjects in HC group. The GMV in each region was represented by the averaged GMV across all voxels that were included in important features

DISCUSSION

SZ and ASD were historically considered to be the same disorder, emphasizing their complex relationship (Chisholm et al., 2015; Veddum et al., 2019). The underlying biological basis of neither disorder is currently known. In recent years, many studies have suggested an important association between the two disorders in the gene, clinical symptoms, and brain (Dominguez‐Iturza et al., 2019; Veddum et al., 2019; Yoshihara et al., 2020; Zheng et al., 2018), hence, there is a great interest in understanding their distinct aspects at a brain level using neuroimaging analysis methods (Cauda et al., 2017; Chen et al., 2017; Zheng et al., 2018). Yet most of the studies to date have applied a statistical analysis or meta‐analysis approach to investigate the two disorders. Although some studies applied a machine learning method (classification) to investigate the brain abnormality of two disorders (relative to HC), there is rare work that revealed what brain measures can directly and effectively distinguish the two disorders. Furthermore, to the best of our knowledge, all existing studies included a relatively small number of subjects, and few studies utilized both brain functional and structural measures.

In the article, we studied the two disorders from a classification perspective, aiming to explore whether they can be effectively differentiated using brain functional connectivity and gray matter measures, and going further to disclose what brain measures are significantly different between them. In particular, we proposed a new approach to jointly take advantage of multimodality features. An unbiased strategy was conducted to guarantee the reliability of classification and feature selection by using the 10‐fold cross‐validation within the main data, and the independent data collected from separate sites were used for validation of generalization capability.

The results showed that our proposed fusion method that effectively utilized FNC and GMV obtained a high classification accuracy and significantly outperformed the single‐modality methods. Our fusion method resulted in 83.08% and 72.10% mean accuracy for the testing data (in the main datasets) and the fully independent data, respectively, however, FNC yielded 80.00% mean accuracy for the testing data and 69.58% mean accuracy for the independent data and GMV obtained 70.65% mean accuracy for the testing data and 63.07% mean accuracy for the independent data. To sum up, through the optimal feature selection, both brain functional connectivity and gray matter features can efficiently distinguish SZ and ASD subjects, and our fusion method further improved the classification performance. Our work supports that combining different modalities would be a more powerful way for the diagnosis of brain disorders, and the selected important features worked well in distinguishing the two disorders.

Given the effectiveness of the selected features in differentiating SZ and ASD for both the out‐of‐sample testing data and independent data, the important features that were consistently presented across multiple classification runs may reflect primary differences between the two disorders. Based on the important features, we comprehensively summarized the differences between the two disorders, and also highlighted their specific changes relative to the healthy population.

FNCs showing between‐disorder differences were mainly related to sub‐cortical networks, including the connectivity between sub‐cortical networks and DMN, between sub‐cortical networks and cerebellar networks, between sub‐cortical networks and cognitive control networks, and between sub‐cortical networks and visual networks. In the meantime, there were also significant between‐disorder differences in FNCs within visual networks as well as FNCs between DMN and visual networks.

Among the top 10 FNCs reflecting differences between SZ and ASD, seven FNCs were associated with sub‐cortical networks (primarily including the thalamus and caudate regions), providing clear evidence of disorder divergency in the subcortical regions that are related to cognitive, affective, and social functions. Among the seven FNCs, five presented the same changing direction in SZ and ASD (relative to HC), and the remaining two showed an intermediate situation in HC (between SZ and ASD). Moreover, regarding the five FNCs showing a similar changing trend in SZ and ASD, SZ changed to a greater extent. In a series of higher cognitive and sensory processes, the thalamus coordinates the transmission of information across multiple functional circuits (Sugranyes et al., 2011). The cortical–subcortical dysconnectivity of SZ has been found in previous studies (Damaraju et al., 2014; Woodward et al., 2012). Some studies have shown the occurrences of ASD (Fu et al., 2019) and SZ (Sugranyes et al., 2011) are related to the abnormality of the thalamus. In addition, there is evidence that the caudate is impaired in ASD (Acevedo et al., 2018; Pereira et al., 2019) and SZ (Acevedo et al., 2018; Mueller et al., 2015). In summary, our results provide further evidence that SZ differs from ASD in terms of the interaction between the subcortical system and other systems.

Our work also highlights the important role of DMN in underlying the connectivity differences between SZ and ASD. In our finding, all four FNCs that were associated with DMN (i.e., two FNCs between DMN and sub‐cortical networks as well as two FNCs between DMN and visual networks) showed an intermediate strength in HC (relative to the two disorders). In particular, relative to HC, the interaction between DMN and subcortical regions was diminished in ASD but was enhanced in SZ; relative to HC, the interaction between DMN and visual regions was decreased in SZ but was increased in ASD. It is known that one of the main clinical symptoms of ASD and SZ is cognitive difficulty and DMN makes a significant contribution to cognitive activity (Andrews‐Hanna et al., 2010; Jenkins, 2019). In previous studies, Nomi and Uddin (Nomi & Uddin, 2015) demonstrated dysconnectivity between DMN and sub‐cortical regions in adolescents with ASD compared to HC, and Hyatt et al. suggested shared neural deficits might correspond to different dimensional social deficits between ASD and SZ in mentalizing‐associated DMN regions (Hyatt et al., 2020). Therefore, our finding manifests the disparity of DMN between SZ and ASD.

Although visual impairment has been observed in ASD (Simmons et al., 2009) and SZ (Butler et al., 2008) alone, our study discloses more about how the two disorders differ in visual function. As mentioned above, the interaction relation between visual networks and DMN showed different changes (with an increase in ASD but a decrease in SZ). We also found that SZ and ASD showed a similar increasing trend (relative to the healthy group) in the connectivity between visual networks and sub‐cortical networks, and SZ changes to a greater extent (than ASD). In addition, the interaction within visual regions was decreased for both SZ and ASD (relative to HC), with a greater change in SZ. A work from Yamamoto et al. showed increased functional connectivity between the left thalamus and the occipital cortices in SZ compared to HC (Yamamoto et al., 2018). And Wei et al. found the local functional connectivity reduced in the visual area in SZ compared to HC (Wei et al., 2018). In addition, the study of Mastrovito conducted a direct comparison between ASD and SZ on the functional connectivity of visual networks and found an increase within high visual networks as well as between DMNs and high visual networks (Mastrovito et al., 2018) in ASD compared to SZ, which is consistent with our results.

Regarding the GMV measures, ASD group shows increases compared to SZ group in all top 15 brain regions, including frontal gyrus (left and right medial superior frontal gyrus, left and right dorsolateral superior frontal gyrus, and left and right middle frontal gyrus), temporal gyrus (left and right middle temporal gyrus, left and right temporal pole: superior temporal gyrus, left and right superior temporal gyrus, right temporal pole: middle temporal gyrus and right inferior temporal gyrus) and right insula. Furthermore, regarding all the 15 brain regions, HC group fell intermediate between SZ and ASD. Our previous study also found GMV decreases of SZ group and GMV increases of ASD group relative to HC in the frontal and temporal gyrus but no voxels showed increased gray matter in SZ and decreased gray matter in ASD (relative to HC) (Du et al., 2021). Frontal gyrus, which regulates behavior (Jana et al., 2020) and relates to cognitive competence (Thye et al., 2018), has been shown to be associated with social cognition abnormalities in ASD and SZ. In previous studies, Liu et al. found that compared to HC, the GMV of ASD shows increases in the left superior and middle frontal gyrus (Liu et al., 2017). In addition, the increase in the volume of the temporal lobe in ASD (Ecker et al., 2013; Palmen et al., 2005) and the reduction in the volume of the temporal lobe in SZ (Mennigen et al., 2019; van Haren et al., 2007) also have been found compared to HC. Taken together, our work directly and clearly suggests that the frontal gyrus, temporal gyrus, and insula show differences in gray matter between SZ and ASD, going further their changes in the two disorders are toward different directions (relative to HC).

Interestingly, our results support that the middle frontal gyrus plays an important role in distinguishing SZ and ASD in using both brain functional and structural measures, for the first time highlighting that this region may represent different neural underpinnings of SZ and ASD. In our work, our findings supported that the functional connectivity between caudate and middle frontal gyrus decreased in both the SZ group and ASD group compared to the HC group and the severity of the SZ group was stronger than that of the ASD group. We also found that the GMV showed an increasing trend in ASD and a decreasing trend in SZ compared to HC for the middle frontal gyrus. There are many studies also have reported the abnormality of the middle frontal gyrus in the two disorders from either functional or structural measures. For functional measures, Kyriakopoulos et al. found that the patients with early‐onset SZ showed reduced functional activation compared to HC in the middle frontal gyrus (Kyriakopoulos et al., 2012) and Noonan et al. found the activation of ASD in the left middle frontal gyrus was weaker than that of HC group (Noonan et al., 2009). For structural measures, Cauda et al. (2011) found increased GMV in the middle frontal gyrus of pediatric ASD patients relative to HC and Ubukata et al. (2013) suggested gray matter reductions of SZ group in the middle frontal regions compared to HC.

Some aspects may deserve further exploration in future work. First, in this work, we did not assess associations between the neuroimaging measures showing group differences and the clinical symptoms of SZ or ASD, as the symptom scores of the two disorders are not very comparable in our data. In the future, more work can be done to investigate whether their differences in brain link to their different clinical manifestation. Second, as we aimed to find the most important neuroimaging measures that differ between ASD and SZ, we built two‐class (SZ vs. ASD) classifiers, not three‐class (SZ, ASD, and HC) classifiers. So, the ability of our identified measures in distinguishing disease and HC groups was not tested. Third, the fusion was conducted at the model level (not the feature level) in our work, since complicated feature fusion could hinder the intuitional explanation of biomarkers. Fourth, other neuroimaging measures such as cortical thickness and surface area could be jointly utilized in the future to improve the performance, as one recent study showed their potential in distinguishing SZ and ASD (Yassin et al., 2020). But we also think proposing a fusion method on both feature and model levels would further improve the distinguishing ability.

In summary, we investigated the discriminative biomarkers between SZ and ASD in brain functional connectivity and gray matter measures using a direct classification strategy. We identified the most important differences that can successfully distinguish the two groups of patients, and the results were validated using both an unbiased 10‐fold cross‐validation in main data and the fully independent data. More importantly, our proposed fusion method performed better than the method only using single‐modality features, supporting the potential of using the multimodal combination. We also found that the middle frontal gyrus was involved in both functional connectivity and gray matter differences. Taken together, our work is important because it localizes the brain differences between two related disorders that have shared a long and tangled history in their diagnoses.

CONFLICT OF INTEREST

The authors declare no potential conflict of interest.

Appendix S1 Supporting Information

Click here for additional data file.