A brain-based general measure of attention.

Attention is central to many aspects of cognition, but there is no singular neural measure of a person's overall attentional functioning across tasks. Here, using original data from 92 participants performing three different attention-demanding tasks during functional magnetic resonance imaging, we constructed a suite of whole-brain models that can predict a profile of multiple attentional components (sustained attention, divided attention and tracking, and working memory capacity) for novel individuals. Multiple brain regions across the salience, subcortical and frontoparietal networks drove accurate predictions, supporting a common (general) attention factor across tasks, distinguished from task-specific ones. Furthermore, connectome-to-connectome transformation modelling generated an individual's task-related connectomes from rest functional magnetic resonance imaging, substantially improving predictive power. Finally, combining the connectome transformation and general attention factor, we built a standardized measure that shows superior generalization across four independent datasets (total N = 495) of various attentional measures, suggesting broad utility for research and clinical applications.

Introduction

Attention has a ubiquitous role in perception and cognition[1]. We are endlessly exposed to all kinds of overflowing sensory information, and the ability to deploy attention over space and to sustain it over time is crucial in everyday life. We have, however, limited cognitive capacity, and therefore must selectively process information most relevant to our actions. Attentiveness explains behavioral performance fluctuations both within and across individuals[2], and attention deficits are common in mental illness and symptomatic of brain damage[3-5].

Despite this central importance of attention, clinicians and researchers lack a standardized way to measure a person’s overall attentional functioning. Although no mental process can be reduced to a single number, both research and clinical practice can benefit from having standardized and quantifiable measures to facilitate comparison across and within individuals[6,7]. For example, intelligence research and education practice benefits from the ability to measure g factor, as a general index of fluid intelligence[8]. A comparable index is lacking for attention, despite its pervasive role in modulating most perceptual and cognitive processes.

The fact that there are so many different tasks to examine attention functioning reflects that attention is not a unitary construct but rather multifaceted[1]. People’s attentional abilities may vary along the multiple dimensions of attention. These differences in attentional functions among individuals can be measured by extensive behavioral tasks; however, an overload of tasks not only requires a substantial amount of time, but also may introduce fatigue, which may affect task performance. Therefore, it is important to understand what is common or unique among the different attention tasks and try to predict them with minimal testing.

The literature lacks a systematic investigation of the general and specific factors of attentional processes and their underlying neural architectures across an array of tasks. One behavioral study examined a set of cognitive tasks known to employ executive functions, including attention and working memory. This study showed that the nine tested tasks are not completely independent but share common and separable components[9]. Another behavioral study also revealed a general factor that is shared by multiple attention task paradigms as well as specialized factors that are unique to specific tasks where the shared component explained substantial variance in performance across most of the tasks[10]. However, this common (general) attention factor cannot be derived from an individual task, and the behavioral nature of this study did not allow exploration of the underlying neural mechanisms across tasks. The frontal and parietal cortices are well-known to control attention[11,12], but most imaging studies have not systematically compared their engagement across multiple attention tasks. In addition, only recently have studies begun to predict individual attentional behaviors from brain scans[13-16].

Here we seek to develop a brain-based, standardized attention profile measure that can quantify a person’s performance across the different attentional demands. We adopted a connectome-based predictive modeling approach (CPM) that develops computational models to accurately predict an unseen, novel individual’s trait and behavior solely from their brain activity[17]. This is based on the whole-brain pattern of functional connectivity (synchronized fluctuation of time-series signals from distributed brain regions), which is unique to each individual and predictive of their behaviors[18-21]. CPM accurately predicts a variety of individual behaviors and traits, including intelligence[18,22], attention[14,23-25], memory[26-28], language[29], creativity[30], and personalities[31-33].

One of the earliest CPM studies introduced a model to predict an individual’s ability to sustain attention over time (saCPM)[14], demonstrating that the brain’s functional organization is predictive of behavioral performance in the gradual-onset continuous performance task (gradCPT)[34]. In addition, the saCPM generalized to predict individual differences in the stop-signal task[23] and the Attention Network Task[24,35], and symptom severity differences in patients with attention-deficit/hyperactivity disorder (ADHD)[14,36]. Sustained attention, however, is just one aspect of human attention[1], and we still need a general measure that can inform a single individual’s attentional ability across multiple tasks.

For a comprehensive assessment of attention, we collected original behavioral and fMRI data from more than 90 participants performing three representative and well-validated attention-demanding tasks during fMRI scanning. The three tasks include the gradCPT to measure sustained attention, multiple object tracking (MOT) to measure divided attention and tracking, and a visual short-term memory (VSTM) task to assess working memory capacity as a form of internal attention[1,37]. We developed a suite of attention models to predict individual behaviors in these tasks from functional connectivity. Moreover, we leveraged the network models to probe brain systems that support common and separable factors for attention functions measured during the tasks across individuals.

Furthermore, we demonstrated new ways to draw patterns of brain networks supporting multiple attentional processes from resting-state data alone. This is essential because attention tasks vary widely and are difficult to standardize across studies and settings. It is also impractical to ask participants, especially patients or children, to engage in many different attention tasks, especially inside a brain scanner. If a profile of attentional measures can be derived from resting-state data, it would have significant utility for future research and potential for clinical applications. To achieve this, we utilized a novel method called connectome-to-connectome (C2C) state transformation modeling[38]. The C2C framework accurately generated individual attention-related task connectomes from their rest connectomes and significantly improved behavioral predictions.

Finally, we propose a general attention model, integrating CPM and C2C modeling to measure a common attention factor across tasks from resting-state fMRI. The proposed model successfully generalizes to four external, independent datasets (total N = 495) with various attentional measures, across three attention task behaviors and ADHD Rating Scale (ADHD-RS), suggesting its applicability to the broad range of attentional measures. We compared the general attention measure to other best-performing single-task-based CPMs, including saCPM[14]. The external validations demonstrated that the proposed general measure best predicts a novel individual’s attentional abilities in diverse settings.

Results

CPMs of three attentional tasks

We first built a battery of predictive models of attentional functions (Supplementary Table 1). We constructed nine CPMs that differed in the fMRI data on which they were defined and the behavior they were trained to predict. In all three tasks, predicted behavioral scores significantly correlated with actual task scores when CPMs used task fMRI (gradCPT: Pearson’s r(90)=0.592, p<0.001; MOT: Pearson’s r(90)=0.469, p<0.001; VSTM: Pearson’s r(90)=0.365, p=0.014, p-values corrected for family-wise error rate [FWER] in multiple tests using 1,000 permutations, the top row in Figure 1 and Extended Data Figure 1), indicating that CPMs using task fMRI accurately predict individual task scores. The CPMs using movie fMRI also accurately predicted individual performance in all three tasks (gradCPT: Pearson’s r(90)=0.392, p=0.006; MOT: Pearson’s r(90)=0.345, p=0.022; VSTM: Pearson’s r(90)=0.350, p=0.019, FWE corrected, bottom row in Figure 1). Rest fMRI-based models predicted gradCPT and MOT performance (gradCPT: Pearson’s r(90)=0.394, p=0.005; MOT: Pearson’s r(90)=0.318, p=0.047, FWE corrected, middle row in Figure 1), but failed to predict VSTM performance (Pearson’s r(90)=0.158, p=0.636, FWE corrected, the middle row in Figure 1).

Figure 1.

Prediction accuracy of nine CPMs.

Rows represent the fMRI data used in model construction and prediction, and columns represent the target attention task. Models’ prediction accuracies were assessed by correlating model-predicted behavioral scores and observed scores. P values were obtained using 1,000 permutations (corrected for nine tests). GradCPT: gradual-onset continuous performance task, MOT: multiple object tracking, and VSTM: visual short-term memory.

Extended Data Fig. 1

Predictive anatomy of three task-based CPMs

A. The scale bar in gradCPT, MOT and VSTM represents the relative ratio of predictive functional connections to all possible number of functional connections between networks with a sign representing whether the connection is in a positive or negative network. The scale bar in overlap represents the actual number of predictive functional connections with a sign representing whether the connection is in a positive or negative network. GradCPT: gradual-onset continuous performance task, MOT: multiple object tracking, and VSTM: visual short-term memory. MF: medial-frontal network, FP: frontoparietal network, DM: default mode network, VI: visual I, VII: visual II, VAs: visual association, SA: salience network, Subc: subcortex, Cbl: cerebellum. B. The number of predictive connections of three task-based CPMs in positive and negative networks.

A predictive model that generalizes to different settings has greater utility than a model that only works for a specific task. We investigated whether the nine basic CPMs defined to predict performance on each task generalize to predict performance on other tasks from fMRI data measured during different attention tasks and task-neutral states (resting-state or movie-watching). The CPMs trained using task fMRI successfully generalized to different attention tasks, except between MOT and VSTM (ps<0.05 FWE corrected using 1,000 permutations, the top left 3 by 3 submatrix in Figure 2). Interestingly, both the MOT and VSTM models predicted individual performances in gradCPT numerically better than their own corresponding tasks. For example, the CPM trained using VSTM fMRI to predict VSTM performance predicted unseen subjects’ gradCPT performance from gradCPT fMRI (prediction q=0.216 and Pearson’s r(90)=0.515) better than VSTM performance from VSTM fMRI (prediction q=0.094 and Pearson’s r(90)=0.365).

Figure 2.

Cross-prediction results of nine original CPMs across all cognitive states and attention tasks.

A. Models’ prediction accuracies were assessed by prediction q. Negative q was set to zero in this figure. Rows represent combinations of fMRI data and behavior scores used in model construction, and columns represent combinations of fMRI data and behavior scores used in model validation. On-diagonal elements represent the nine within-task prediction results (corresponding to Figure 1) and off-diagonal elements represent the cross-task predictions. For example, when a CPM trained using VSTM fMRI to predict VSTM performance was applied to gradCPT fMRI to predict gradCPT performance, prediction performance was q=0.22 (and r=0.52 in B). Similarly, when a CPM trained using rest fMRI to predict VSTM performance was applied to movie fMRI to predict MOT performance, performance was q<0 (and r=0.22 in B). The models with task fMRI successfully generalized to different attention tasks, except CPMs between MOT and VSTM (the top left 3 by 3 submatrix), and the models with movie fMRI also generalized to different tasks to lesser degrees (the bottom right 3 by 3 submatrix). P values for significance were obtained using 1,000 permutations and corrected for multiple tests (***: p<0.001; **: p<0.01; *: p<0.05; ~: p<0.1). GradCPT: gradual-onset continuous performance task, MOT: multiple object tracking, and VSTM: visual short-term memory. B. The same result as A, but the models’ prediction accuracies were assessed by correlation r between model-predicted and observed behavioral scores across individuals.

CPMs of a common attention factor predict task behaviors

To investigate if we can build a model that predicts overall attention function, we performed variations of predictive modeling based on a behaviorally derived common attention factor, defined as the mean of the z-scored performance scores across the three tasks. We then trained nine predictive models to predict this common attention factor. All modeling procedures were the same as the original CPMs except the use of the common attention factor as the training behavior. The common factor CPMs with task fMRI successfully generalized to different attention task fMRI (Pearson’s r(90)s>0.37, ps<0.05 FWE corrected using 1,000 permutations, Figure 3A). The common factor CPMs better generalized to different tasks (numerically higher prediction performance) than the basic CPMs of each task (c.f., Figure 2). The predictive connectivity features of common factor models were distributed among multiple brain networks, mainly in the salience, subcortical, cerebellar, frontoparietal, visual II, and motor networks (Figure 3B).

Figure 3.

Cross-prediction results of five CPMs trained to predict a common attention factor using different fMRI data.

A. All models were trained to predict a shared variance (a common attention factor) in three task behaviors but tested to predict individual behaviors in each task from different fMRI data. Models’ prediction accuracies were assessed by prediction q and correlation r between observed and predicted common factor measures. P values of significance were obtained using 1,000 permutations and corrected for all 5×5 tests (***: p<0.001, **: p<0.01, *: p<0.05, and ~: p<0.1). The models with task fMRI successfully generalized to predict different task behaviors when the models were applied to task fMRI (the top left 3 by 3 submatrix). B. Predictive functional connections of the common attention factor. The scale bar represents the relative ratio of predictive functional connections to all possible number of connections between networks with a sign representing whether the connection is in a positive or negative network. GradCPT: gradual-onset continuous performance task, MOT: multiple object tracking, and VSTM: visual short-term memory. MF: medial-frontal network, FP: frontoparietal network, DM: default mode network, VI: visual I, VII: visual II, VAs: visual association, SA: salience network, Subc: subcortex, Cbl: cerebellum.

We then examined if the common-factor CPMs specifically predicted shared variance between tasks or also captured task-specific variance for each behavior. We first examined the degree to which common factor models captured shared variance among the three tasks by estimating the correlation between model-predicted scores and the observed common factor. Predicted common factor scores and observed common factor scores were significantly correlated across combinations of training and testing fMRI (Extended Data Figure 2A). Correlations between predicted and observed common factors were, in general, numerically higher than the correlations between the predicted common factor scores and observed scores of each task (Figure 3A), implying that the common factor CPMs capture variance shared by three tasks more than variance unique to each task. We additionally assessed if the common factor models also predict unique variances in different tasks. We ran a partial correlation between predicted and observed behavioral scores, controlling for the observed common factor. The models did not capture variances specific to each task in general (Extended Data Figure 2B). These observations demonstrate that CPMs of a common attention factor indeed predict an overall attentional ability underlying different attention tasks, but do not capture task-specific variance.

Extended Data Fig. 2

Cross-prediction results of five common attention factor CPMs

A. Cross-prediction results when models were applied to predict the common attention factor from different fMRI data. Models’ prediction accuracies were assessed by prediction q and correlation r between observed and predicted common factor measures. P values of significance were obtained using 1,000 permutations and corrected for all 5×5 tests (***: p<0.001, **: p<0.01, *: p<0.05, and ~: p<0.1). Rows represent different fMRI data used to predict a common attention factor used in model construction, and columns represent the same but in model validation. B. Cross-prediction results, taking into account shared variance (the common factor) between task behaviors. Models’ prediction accuracies were assessed by partial correlation between observed and predicted behavior scores while controlling for the shared variance. P values of significance were obtained using 1,000 permutations and corrected for all 5×9 tests (***p<0.001, **: p<0.01, *: p<0.05, and ~: p<0.1). Rows represent different fMRI data used to predict a common attention factor used in model construction, and columns represent combinations of fMRI data and behavior scores used in model validation. GradCPT: gradual-onset continuous performance task, MOT: multiple object tracking, and VSTM: visual short-term memory.

Factors driving high generalizability of attention CPMs

The CPMs’ high generalizability (Figure 2 and 3A) may seem trivial given the significant correlations in the behavioral scores between tasks (Extended Data Figure 3). To examine whether the generalizability of CPMs between tasks is fully driven by the correlations in behavior between them, we constructed variants of CPMs. In this analysis, CPMs were trained to predict unique variance in behavior specific to each task. The task-specific variance was defined by the residual after regressing the two non-target task behaviors out from target task behaviors. The task-specific variance had zero correlation with the two remaining non-target behaviors (Supplementary Table 2). All other steps except the target behavior remained the same as the original modeling. A cross-task prediction pattern of task-specific models was similar to the original models (Figure 4AB). The models with task fMRI successfully predicted behaviors in different tasks, except between MOT and VSTM (Pearson’s r(90)s>0.31, ps<0.05, FWE corrected for multiple tests using 1,000 permutations, the top left 3 by 3 submatrix in Figure 4AB). That is, even after we explicitly orthogonalized the behavioral measures, we still observed cross-task generalization. This result suggests that the generalizability of the attention CPMs to different tasks cannot be simply explained by the correlation between training and testing behaviors (Supplementary Table 2).

Extended Data Fig. 3

A similarity of individual behaviours between different tasks

The similarity was assessed by Pearson’s correlation of individual performances between attention tasks. Individual behaviors were significantly correlated between every pair of tasks. GradCPT: gradual-onset continuous performance task, MOT: multiple object tracking, and VSTM: visual short-term memory.

Figure 4.

Cross-prediction results of CPMs trained to predict task-specific variance.

A&B. Models’ prediction accuracies were assessed by prediction q (panel A) and correlation r (panel B). P value was obtained using 1,000 permutations and corrected for multiple tests (***: p<0.001, **: p<0.01, *: p<0.05, and ~: p<0.1). Rows represent combinations of fMRI data and behavior scores used in model construction, and columns represent combinations of fMRI data and behavior scores used in model validation. GradCPT: gradual-onset continuous performance task, MOT: multiple object tracking, and VSTM: visual short-term memory. C. Predictive anatomy of task-specific CPMs. MF: medial-frontal network, FP: frontoparietal network, DM: default mode network, VI: visual I, VII: visual II, VAs: visual association, SA: salience network, Subc: subcortex, Cbl: cerebellum.

To further test the effect of the common attention factor, we re-assessed the prediction accuracy of the task-specific CPMs while controlling for the shared variance between tasks using partial correlation. After controlling for the common factor, the majority of cross-predictions were not significant (Extended Data Figure 4). This suggests that brain networks underlying the common attention factor and networks underlying unique variances are partially overlapping. To confirm, we compared the predictive anatomy of task-specific CPMs (Figure 4C) with that of a common factor CPM (Figure 3B). Functional connections in the salience, subcortical, and cerebellar networks emerged as the predictive networks shared between the task-specific and the common factor CPMs (Supplementary Figure 1). Taken together, shared and unique variances in the three attention tasks could be, in part, explained by the same brain system that consists of connections between salience, subcortical, and cerebellar networks, facilitating generalizable model predictions across tasks.

Extended Data Fig. 4

Cross-prediction results of task-specific CPMs

A. Cross-prediction results, taking into account shared variance between task behaviors. Models’ prediction accuracies were assessed by partial correlation between observed and predicted behavior scores while controlling for the shared variance. P value was obtained using 1,000 permutations and corrected for multiple tests ***: p<0.001, **: p<0.01, *: p<0.05, and ~: p<0.1). Rows represent combinations of fMRI data and behavior scores used in model construction, and columns represent combinations of fMRI data and behavior scores used in model validation. GradCPT: gradual-onset continuous performance task, MOT: multiple object tracking, and VSTM: visual short-term memory. B. Cross-prediction results when models were applied to predict the common attention factor from different fMRI data. Models’ prediction accuracies were assessed by correlation between observed and predicted common factor. P value was obtained using 1,000 permutations and corrected for all 9×5 tests (***: p<0.001, **: p<0.01, *: p<0.05, and ~: p<0.1). Rows represent combinations of fMRI data and behavior scores used in model construction, and columns represent different fMRI data used to predict a common attention factor used in model validation.

Multiple brain networks contribute to model predictions

We observed that CPMs of a common attention factor accurately predict an individual’s attentional behaviors in different tasks (Figure 3A). Given the common involvement of connections of salience, subcortical, and cerebellar networks across three tasks (Figure 3A), we expected that the same networks would play a key role in predicting an individual’s overall attention. To explicitly evaluate each network’s contribution to prediction accuracy and compare across networks, we computationally lesioned all the nodes in each of the ten networks[18,39] iteratively and then trained and tested three task fMRI-based CPMs in the same way the original CPMs were constructed. Lesioning the salience network significantly decreased prediction accuracy (mean decrease=0.021, p=0.018 using 1,000 permutations, Figure 5A and Supplementary Figure 2A). Lesioning cerebellar or subcortical networks also decreased prediction accuracy, although not significantly, compared to lesioning the other seven networks, suggesting that salience, subcortical, and cerebellar networks may contain behaviorally relevant and unique information that is absent in other networks. To confirm that the lower performance was not driven by the smaller number of model features after lesioning, we estimated the correlation between prediction performance and the number of post-lesioning connections (Supplementary Table 3). We did not find a significant positive correlation, supporting the finding that the salience network indeed plays a major role in predicting attentional behaviors across tasks.

Figure 5.

Network contribution on CPMs’ prediction performance.

Three CPMs were trained and tested using task fMRI (connectivity edges survived after lesioning each network [A], connectivity edges within each network [B], and connectivity edges connecting each network to the other nine networks [C]) to predict behaviors in three tasks, gradCPT, MOT, and VSTM, respectively. The prediction performances were averaged to summarize contribution of each network. A. Cross-prediction after lesioning a network. B. Cross prediction using connectivity within each network. C. Cross-prediction using connectivity of each network to the other nine networks. Significance was obtained from 1,000 permutations (***: p<0.001, **: p<0.01, and * : p<0.05). In a box-whisker plot, a box covers the first to third quartile (q1 and q3, respectively) of the data, and a center line represents the median. A red dot represents the mean. Whisker covers approximately 99.3% of data , extended to the most extreme point that is not an outlier. A data point is considered an outlier if it is greater than or less than . GradCPT: gradual-onset continuous performance task, MOT: multiple object tracking, and VSTM: visual short-term memory. MF: medial-frontal network, FP: frontoparietal network, DM: default mode network, VI: visual I, VII: visual II, VAs: visual association, SA: salience network, Subc: subcortex, Cbl: cerebellum.

We performed a complementary analysis to examine the predictive power of each network directly. We built predictive models only using within-network connectivity of each network, iteratively for each network. This analysis revealed that the frontoparietal, subcortical, salience, and motor networks are the most predictive networks (p<0.001 using 1,000 permutations, Figure 5B and Supplementary Figure 2B). Although prediction performance was lower than that of the original models (r=~0.43), the prediction accuracy by each network is notable given the significantly fewer number of predictive edges in these models compared to original CPMs (Supplementary Table 3 and Figure 3B).

Note that this analysis examined the predictive power of only edges located within a network of interest; in contrast, the previous lesioning analysis examined the importance of edges within a target network and edges connecting a target network to the other nine networks together. To further differentiate the roles of within- and between-network connectivity in prediction, we assessed the predictive power of between-network connectivity for each network. We constructed a new set of CPMs using functional edges that connect one network with the other nine networks, performing this analysis iteratively for each of the ten networks. We found that the connectivity of the salience network was again the most predictive of individual attention on average, even numerically better than the original models (Figure 5C and Supplementary Figure 2C). The models’ performances mirrored results from the lesioning analysis; the salience, cerebellar, and subcortical networks accurately predicted individual behaviors. Again, to confirm that the lower performance was not driven by the smaller number of between-network connections, we estimated the correlation between performance and the number of edges and did not find any significant positive correlation between them.

Brain networks predicting general attention

Given the importance of the salience, subcortical, cerebellar, frontoparietal, and visual II networks in predicting individual attentional behaviors, we asked if connectivity between these networks can predict individual behaviors in attention tasks with an accuracy comparable to the original whole-brain models. The CPMs using the connectivity between the five networks fully generalize across different task-related behaviors (Extended Data Figure 5). Of all models made by selecting any three out of five networks, the model using the salience, frontoparietal, and subcortical networks best predicted individual behaviors on average, and its prediction performance was comparable to the performance of the original whole-brain model (Extended Data Figure 5). This result suggests that connectivity between these three networks may be associated with a common component of attention. As a control analysis, we built a model using the other five networks (the medial-frontal, default mode, motor, visual I, and visual association networks) and found that the model’s prediction was significantly less accurate than the previous five-network model or even the three-network (the salience, frontoparietal, and subcortical networks) model (Extended Data Figure 5). This result is in line with the previous observation that the CPMs of a common attention factor accurately predict attention task performances (Figure 3) and corroborates these networks’ general importance in attentional functions.

Extended Data Fig. 5

Cross-prediction using connectivity between the frontoparietal (FP, 2), visual II (VII, 6), salience (SA, 8), subcortical (Subc, 9), cerebellar (Cbl, 10) networks

Prediction of a model using connectivity between the medial-frontal (1), default mode (3), motor (4), visual I (5), visual association (7) networks was also obtained as a control. A. Rows represent combinations of networks (indicated by numbers) used in each model. Models’ prediction accuracies were assessed by correlating model-predicted and observed behavioral scores. B. Prediction performance of each network obtained by averaging all models that used the network in A. C. The same result as A, but model accuracies were assessed by q2. D. Prediction performance of each network obtained by averaging all models that used the network in C. GradCPT: gradual-onset continuous performance task, MOT: multiple object tracking, and VSTM: visual short-term memory.

State transformations improve predictions from rest fMRI

The results above demonstrated that task-based connectomes led to better prediction of behaviors compared to rest connectomes. Nevertheless, resting scans still have the undisputable advantages of 1) enhancing data retention by reducing the demand on participants, especially in clinical populations, and 2) facilitating data coordination across studies and sites for a large-scale multi-site study. Therefore, we next asked if we could improve the predictive power of resting fMRI by applying a recently introduced approach, connectome-to-connectome (C2C) state transformation modeling[38].

The C2C model-generated task connectomes accurately resembled their corresponding empirical task connectomes (Extended Data Figure 6). More remarkably, individual attentional behaviors were better predicted by the generated task connectomes than by the empirical rest connectomes alone (ps<0.05 from 1,000 permutations, Figure 6). The results from the movie connectomes are also shown in Extended Data Figure 6 and Supplementary Figure 3.

Extended Data Fig. 6

Similarity between C2C model-generated task connectomes and empirical task connectomes

Error bar represents standard deviation from 1,000 iterations. A and C represent a spatial similarity between two connectomes assessed by Pearson’s correlation. Darker bars represent the similarity between empirical task and generated task connectomes, and lighter bars represent the similarity between empirical task and empirical rest connectomes. The higher similarity of the generated connectome indicates that the C2C model accurately generates the target task connectome from the rest connectome. B and D represent root mean square (RMS) difference between two connectomes. The smaller difference of the generated connectome indicates that the C2C model accurately generates the target task connectome from the rest connectome. In a box-whisker plot, a box covers the first to third quartile (q1 and q3, respectively) of the data, and a center line represents the median. A red dot represents the mean. Whisker covers approximately 99.3% of data , extended to the most extreme point that is not an outlier. A data point is considered an outlier if it is greater than or less than . GradCPT: gradual-onset continuous performance task, MOT: multiple object tracking, and VSTM: visual short-term memory. *: p<0.001 from 1,000 permutations

Figure 6.

Prediction of individual behaviors by applying the original CPMs trained using task fMRI to rest fMRI with a rest-to-task connectome transformation using C2C modeling.

A. Prediction performance was assessed by prediction q, and negative values were set to zero (i.e., q=0 for task-to-rest prediction without C2C modeling in all three tasks). Darker bars represent the behavior prediction accuracy with C2C-generated task connectomes. Lighter bars represent the behavior prediction accuracy with empirical rest connectomes. A darker bar in ‘Task-to-Rest’ represents the behavior prediction accuracy of a model trained using empirical task connectome when the model is applied to the C2C-generated task connectome. A lighter bar in ‘Task-to-Rest’ represents the behavior prediction of a model trained using empirical task connectome when the model is applied to the empirical rest connectome. The task connectomes generated by C2C models from rest data significantly better predicted individual behaviors than empirical rest connectome in all three attention tasks. A darker bar in ‘Rest-to-Rest’ represents the prediction of a model trained using empirical rest connectome when the model is applied to the C2C-generated task connectome. *: p<0.01 from 1,000 iterations. B. The same result, but prediction performance was assessed by correlation r. In a box-whisker plot, a box covers the first to third quartile (q1 and q3, respectively) of the data, and a center line represents the median. Whisker covers approximately 99.3% of data , extended to the most extreme point that is not an outlier. A data point is considered an outlier if it is greater than or less than . GradCPT: gradual-onset continuous performance task, MOT: multiple object tracking, and VSTM: visual short-term memory. *: p<0.05 from 1,000 iterations.

A general attention model

Lastly, unifying the models and findings above, we propose a general attention model to provide a single standardized measure of a person’s overall attention functioning based on resting-state data alone. This general model consists of a hybrid attention connectome (Extended Data Figure 7), the common attention factor, C2C transformation modeling, and CPM.

Extended Data Fig. 7

The general attention connectome lookup table

Out of a total 30,135 edges, 10,885 (36.1%) edges were pulled from gradCPT, 12,542 (41.6%) edges were from MOT, and 6,708 (22.3%) were from VSTM. The Ratio map was obtained based on All map. In each within- or between-network element in Ratio, the number of edges in the element for each task was counted and normalized by the total number of edges of each task. A task with the highest normalized value was assigned.

The general attention model accurately predicted individual performance in the three attention tasks based on the rest connectome. The general attention model was significantly better than the gradCPT, MOT, and VSTM task models applied to rest data (ps<0.001 for both prediction q and Pearson’s r(90) in all three comparisons, Figure 7A). The general model successfully predicted individual behavioral performance in both prediction q and correlation r assessment. In contrast, when the three single task-based models were applied to rest data, they could not successfully predict individuals’ actual scores (prediction qs=0, Figure 2A and 7; see Supplementary Figure 4 for movie data). Importantly, the general model captures the variance in behaviors across all three tasks better than any of the single task CPMs, which made weaker predictions for non-native tasks, even for predicting individual difference (correlation r in Figure 7A). The stronger predictive power and the higher generalizability of the general model suggest its potential broad applicability.

Figure 7.

The general attention model in internal validation.

A. Behavior prediction by the general attention model applied to a rest connectome. Each task name in x-axis represents a single task-based CPM. The general attention model and three CPMs predict individual behaviors from the rest connectome. Behavior prediction performances were averaged for three tasks prediction (predicting gradCPT, MOT, and VSTM scores). The general attention model significantly better predicted task behaviors than all task CPMs. *:p<0.001 from 1,000 permutations. In a box-whisker plot, a box covers the first to third quartile (q1 and q3, respectively) of the data, and a center line represents the median. Whisker covers approximately 99.3% of data , extended to the most extreme point that is not an outlier. A data point is considered an outlier if it is greater than or less than . GradCPT: gradual-onset continuous performance task, MOT: multiple object tracking, and VSTM: visual short-term memory. B. Predictive anatomy of the general attention model. The scale bar represents the relative ratio of predictive functional connections to all possible number of functional connections between networks with a sign representing whether the connection is in a positive or negative network. MF: medial-frontal network, FP: frontoparietal network, DM: default mode network, VI: visual I, VII: visual II, VAs: visual association, SA: salience network, Subc: subcortex, Cbl: cerebellum.

The general model generalizes to four external datasets

Above, the proposed general attention model was cross-validated within the attention dataset we collected (n = 92). However, a small sample size can produce large variance in the accuracies of predictive models, and therefore cross-validation results may not be reliable[19,40]. To more rigorously validate the general attention model’s generalizability and practical applicability, we ‘stress-tested’ the general model on four diverse and independent datasets. We compared the general model’s prediction performance with performances of two original CPMs (model 1 and model 4 in Supplementary Table 1) that yielded the best performance on average among all single task-based models (Figure 2 and 7) and sustained attention CPM (saCPM), which is arguably one of the state-of-the-art personalized fMRI-based attention prediction models[14]. The saCPM’s prediction performance and generalizability have been extensively demonstrated in multiple studies[6,14,15,24,41,42]. The saCPM is conceptually the same as model 1 in the current study, but constructed on different datasets. Both models were constructed using fMRI data and behavioral performances in gradCPT, but experimental task designs, including scan durations, were different.

The four datasets consist of rest connectomes and different attention-related measures: 1) gradCPT performance (d’) from 25 adults[14], 2) Attention Network Test (ANT) performance (RT variability) from 41 adults[24,25], 3) Short Penn continuous performance task (SCPT) performance (response time for true positive trials) from 316 adults[22], provided by Human Connectome Project (HCP)[43], and 4) ADHD-RS-IV scores[44] from 113 children and adolescents with and without ADHD diagnoses[14] provided by the ADHD-200 consortium[45]. Since the saCPM was defined in the first external dataset of this study, we examined its external performance only in the other external datasets (ANT, SCPT, and ADHD-RS).

The general model successfully generalized to predict different attentional measures in the four external datasets. It not only captured individual differences in attention function (external data 1, Pearson’s r(23)=0.472, p=0.012; external data 2, r(39)=0.340, p=0.012; external data 3, r(314)=0.106, p=0.029; external data 4 r(111)=0.230, p=0.002, 1,000 permutations, Figure 8) but also accurately predicted standardized scores from individuals (external data 1, prediction q=0.200, p=0.012; external data 2, q=0.129, p=0.012; external data 3, q=0.010, p=0.029; external data 4, q=0.051, p=0.002, 1,000 permutations, Figure 8). In contrast, the three CPM models trained using gradCPT or rest fMRI failed to predict actual individual attention abilities from rest fMRI in all four datasets (except the rest-based CPM in the 1st external dataset), although they could predict individual differences (correlation r in Figure 8). To further probe the model prediction, we visualized predictions of individual scores against their observed scores with a fitted line in each scatter plot (Extended Data Figure 8). In fitting a line, we did not constrain the intercepts. As attentional measures were z-scored in each dataset, a fitted line that closely passes the origin [0,0] with a positive slope, staying within the white quadrants, implies an accurate prediction of actual scores. In all four datasets, the general model’s fitted line passed the origin more closely than the other three CPMs. This indicates that the general model most correctly predicts that a person whose score is better than the population average (observed z>0) has a higher attention function (predicted z>0) and whose ability is below the average (observed z<0) has a lower attention function (predicted z<0).

Figure 8.

The general attention model generalizes to predict different attentional measures in four independent datasets.

Prediction performance was assessed by prediction q and r. Negative q values were set to zero. In q assessment, the general model (yellow) successfully generalized in four different datasets, while task or rest-based CPMs and saCPM predicting gradCPT did not in any dataset. The general model accurately predicted individuals’ actual attentional abilities observed in gradCPT, ANT, and SCPT and assessed by ADHD-RS in q. In contrast, the CPMs trained using gradCPT or rest fMRI, or saCPM did not generalize to predict individual abilities that were assessed by different measures in the external datasets in q evaluation. Model prediction was considered successful if performance assessed by r and q is statistically significant using 1,000 permutations (***: p=<0.001, **: p<0.01, and *: p<0.05).

Extended Data Fig. 8

Scatter plots of predicted and observed attention scores in four external datasets

Three models, the general attention model and two single task models (model 1 and 4 in Table 1) were trained within the internal dataset and then applied to rest connectomes in the four datasets. If a fitted line closely passes the origin (0,0) with a positive slope (staying within white quadrants), the model could be considered successfully predicting actual attentional abilities. There was no constraint on intercepts in fitting a line. The general model best generalized to predict various attentional measures in four independent external datasets.

We further compared prediction error of the general model and three CPMs by estimating MSE between predicted and observed scores in each dataset. The general model significantly reduced MSE compared to null predictions (ps<0.05 from 1,000 permutations) in all four datasets (the top left in Extended Data Figure 9). In addition, the general attention model produced the lowest prediction error in all external datasets. This result further supports the higher generalizability and practical applicability of the proposed general attention model over any single task-based CPM.

Extended Data Fig. 9

Prediction error, assessed by mean square error (MSE), of the general attention model in four independent datasets

The general model significantly reduced prediction error (assessed by MSE) compared to null models in four datasets. In all datasets, the general attention model produced the lowest prediction error among all models tested. ***: p<0.001, **: p<0.01, *: p<0.05, and ~: p<0.1 from 1,000 permutations

Discussion

We developed a suite of whole-brain connectome-based predictive models (CPM) that can predict both a common (task-general) and task-specific aspects of attention from an individual’s single-task or resting-state fMRI data. The network models accurately predicted sustained attention, divided attention and tracking, and working memory capacity. By leveraging these models, we uncovered the underlying brain networks supporting a general component across these attentional functions. We repeatedly observed that patterns of multiple brain networks, including the salience, subcortical, cerebellar, and frontoparietal networks, drive accurate prediction of individuals’ attentional abilities across tasks, suggesting that these networks support a common (general) attention factor. To further enhance the measurement of attention, we applied a novel analysis framework, connectome-to-connectome (C2C) modeling[38], and demonstrated that we can generate the patterns of individuals’ attention task connectomes from their rest connectome alone. More remarkably, the generated task connectomes substantially improved the prediction of individual attentional behaviors in either a task-specific or general manner. Finally, by combining connectomes from the multiple attention tasks in our study, a common attention factor, the C2C framework, and CPM behavior prediction, we were able to derive a general attention model that captured standardized individual behaviors better than any task-specific models applied to rest data, and showed superior generalizability across tasks, both within our study and in four external datasets with diverse attentional measures, making it a powerful measure with broad utility.

Our general attention model was based on three tasks, which cover many fundamental dimensions of attention: sustained attention, divided attention and tracking, and working memory. Amongst the specific tasks, the gradCPT task supports the strongest predictions and generalizability to other tasks here and in other studies[14,23,25]. Therefore, when only one attention task can be conducted in the scanner, we recommend the use of the gradCPT. When only rest data are available, the general attention model offers the most generalizable and valid measure of attention, distinct from other variables such as intelligence or age.

The gradCPT, MOT, and VSTM tasks tested here are only a small sample of the wide variety of attention tasks developed over decades of research. Therefore, the general attention measure might not span this entire range, and the long-term goal would be to build a truly “universal” model. However, having tested a wider variety of tasks, Huang et al. demonstrated that there appears to be only one general factor that is shared across them, and we believe that this is what underlies the general attention measure proposed here[10]. As further external validation, unprecedented for an fMRI study, we validated the general attention model across multiple measures of attention (performances in gradCPT, MOT, VSTM, ANT, and SCPT and ADHD-RS), lab or clinical data (healthy and ADHD diagnosis), different age groups (adults and children/adolescents), variable data acquisition sites (two centers in New Haven, one in St. Louis, and one in Beijing), and data processing procedures (AFNI, Bioimage Suite with SPM, and FSL). The fact that our general attention model still generalizes successfully despite these challenges demonstrates its broad applicability and potential.

Importantly, the general attention model captures an overall attentional variance that cannot be explained by other general phenotypes, such as g factor and age. In two external datasets, the general model’s prediction accuracy actually increased when controlled for age and intelligence, indicating that the proposed general attention measure is specific to an individual’s overall attention function and cannot be explained by other general aspects of cognition or maturity. Future work can further validate the general attention measure across different tasks and datasets such as the Philadelphia Neurodevelopmental Cohort[46] and Adolescent Brain Cognitive Development Study[47] to demonstrate the universality of the general attention measure.

CPMs successfully generalized to predict performance across three different attention tasks: sustained attention, tracking, and visual working memory (Figure 2). For example, a model trained to predict individual behaviors in gradCPT accurately predicted performance in both MOT and VSTM. This generalizability of predictive models across different attention tasks suggests that there are shared neural components supporting a general attention factor across the tasks. Previous studies revealed a general attention factor[10] and the neural system underlying attentional performance in diverse tasks[48], and more generally, cognitive control[49-51]. Going beyond these studies, our CPM approach looks at connectivity patterns that can further predict quantifiable performance across multiple tasks and an overall attentional ability in unseen, novel individuals.

The brain networks of the common attention factor mainly recruited the salience, subcortical, cerebellar, and frontoparietal networks (Figure 3B). CPMs tuned to the general attention factor exhibited prediction accuracy and generalizability that are higher than or comparable to the native task models in predicting each task behavior, suggesting a pervasive role of these networks in different attentional components.

To further understand the relative contribution of each canonical brain network for the general attention factor, we computationally lesioned each network in isolation and examined their impact on whole-brain CPM performance. We observed that the salience network, followed by the subcortical and frontoparietal networks, is the most important network in predicting individual attentional behaviors across all tasks in general, exhibiting high generalizability (Figure 5, Extended Data Figure 5, and Supplementary Figure 2). The results further show that these networks play a primary role in attention performance across tasks. The involvement of these networks in CPM is in line with previous findings that attention-related tasks induce or modulate a functional engagement of frontal and parietal areas[48,52-55] and subcortical areas[56-58].

Interestingly, we also found that connectivity between the cerebellum and other networks is an informative marker of individual attentional performance (Figure 5). Although the cerebellum is traditionally considered important for motor control, cerebellar involvement in higher cognition has also gained attention over the last three decades[59-63]. Advances in brain imaging techniques started to reveal cerebellar connections with higher association cortices and coactivation with cortical networks in various cognitive tasks[64]. Cerebellar involvement in attention has also been well documented, such as, attention-induced cerebellar activation[65], attentional modulation on the cerebellar activity[66], and attention deficits with cerebellar lesions[67].

To enhance the practicality of the current approach, we extended CPM modeling with a novel method called connectome-to-connectome (C2C) modeling. We showed that C2C modeling accurately predicted individuals’ attention-task connectomes from their rest connectomes. Notably, the predictive power of rest connectomes considerably increased when using C2C transformation. This result demonstrates the potential of using connectome state transformation to substantially improve the prediction performance of CPMs, achieving the best of both worlds: the high predictive power of task data and the relatively convenient acquisition of resting-state data. The C2C state transformation framework in the general attention model offers a novel solution to the current tradeoff between predictive power and difficulty in data collection, holding promising potential for practical applications. On one hand, task-based fMRI provides better prediction performance compared to resting-state data, perhaps due to unconstrained mind-wandering that may result in more variable mental states from scan to scan and from subject to subject at rest[25,29,68-71]. On the other hand, rest scans are much easier to collect consistently across studies and sites. For example, since clinical populations may have difficulty performing certain tasks[72], researchers or clinicians can obtain rest scans from patients because of its simplicity and minimal demands[73], a main reason why resting-state fMRI has gained popularity in clinical and other neuroimaging studies. C2C modeling combines the advantages of these two and improves the diagnostic value of rest scans, lessening the burden of conducting multiple scans or trying to standardize tasks across individuals.

Our approach of combining CPM and C2C modeling should be useful in studying other mental abilities such as memory and intelligence, and related neuropsychiatric disorders such as ADHD and dementia. This approach can derive estimates of multiple cognitive measures from a single rest scan, analogous to how physicians can assay multiple health measures from a single blood sample. With further development, the proposed general model may have potential clinical utility, for example, monitoring and evaluating the effect of treatments and interventions on brain function, or providing a cognitive measure for non-communicative patients to support a clinical decision.

Methods

This study was approved by the Yale University Institutional Review Board.

Subjects and experimental designs

One hundred and twenty-seven right-handed, neurologically healthy individuals with normal or corrected-to-normal vision participated in a two-session fMRI study for monetary reward (80 female, ages 18 to 35 years, mean=23.15 years, SD=4.43). Data from thirty-five participants were excluded from the analysis due to excessive head motion (>3 mm maximum head displacement and >0.15 mm mean framewise displacement [FD]) during fMRI scanning, a small number of fMRI volumes (<120 TRs) after censoring, task performances with lower or higher than 2.5 standard deviations from the group mean in both sessions, or low imaging data quality by visual quality check. The remaining 92 individuals with all behavioral and imaging data were included in the main analysis (60 female, ages 18 to 35 years, mean=22.79 years, SD=4.24). Two fMRI sessions were separated by approximately two weeks (mean=17.31 days, SD=20.21, median=12 days). Based on preliminary data, we performed a power analysis to predetermine a sample size of 84 participants at 0.8 power. The current sample size (92 participants analyzed, and 127 participants in total) is high relative to other single-site studies in the neuroimaging literature.

Each fMRI session started with an anatomical magnetization prepared rapid gradient echo (MPRAGE) followed by 10-min resting-state runs (two runs in session 1 and one run in session 2) and a 7:16-min watching-movie run (Inscapes)[74]. Afterwards, all participants performed three 10-min attention-related tasks while they were in the scanner. The three tasks were the gradual-onset continuous performance task (gradCPT), multiple object tracking (MOT), and visual short-term memory task (VSTM). The order of these tasks was counterbalanced across participants and sessions. An additional task, either the Attention Network Task (ANT)[35] or an n-back task, was collected after completing the three main tasks in session 2, but these tasks are not included in this study because of the smaller number of subjects who performed these tasks. Visual stimuli in task fMRI were presented using Psychtoolbox-3 in MATLAB R2016b. PsychoPy2 (version:1.85.3) was used to present Inscapes during movie-watching fMRI. All participants provided written informed consent and were paid for their participation.

Three attention tasks

Participants performed three attention-related tasks in the scanner. Task performance was assessed with sensitivity (d’), accuracy (%), and working memory capacity (Pashler’s K) for gradCPT, MOT, and VSTM, respectively. To calculate task performance scores, we averaged scores from the two sessions for each task. For those who had only one session that met our data inclusion criteria (29 subjects for gradCPT, 25 subjects for MOT, and 22 subjects for VSTM), we used the task score of the available session in the analysis.

Gradual-onset continuous performance task

The gradCPT is a task that measures sustained attention and inhibitory control[34,75]. In this 10-min task, participants saw grayscale photographs of scenes gradually transitioned from one to the next. The scenes consist of city scenes that appear in 90% of the total trials and mountain scenes that appears in only 10% of the total trials. Each scene transitioned every 800ms, and participants were asked to respond every time they saw a city scene by pressing a button with their right index finger and withhold responses to the mountain scenes. The task consisted of 740 trials. Sensitivity (d’) was calculated to assess task performance, as , since it reflects a subject performance more reliably than percentage accuracy given the imbalanced target to non-target ratio (9:1). We observed that sensitivity d’ and % accuracy are highly correlated (Pearson’s r(90)=0.90, p=6.63×10−34).

Multiple object tracking

MOT measures divided attention, tracking, working memory capacity, spatial attention, inhibition, and sustained selective attention[76]. In this 10-min task, participants tracked multiple target objects while all stimuli were moving. At the beginning of each trial, participants were presented with 12 randomly spread identical white discs on the screen. For each trial, three or five discs among the 12 flashed green and turned back to white, designating them as the target discs of that trial, while the remaining non-target discs remained white. All of the 12 discs then moved around the screen for 5,000ms, and then one of the 12 discs was probed. Participants were instructed to press a button with their right index finger if the probed disc was one of the original targets and press with their right middle finger if it was not. Participants had 2,000ms to respond. The task consisted of 56 trials, and performance was assessed by a percent accuracy.

Visual short-term memory task

VSTM measures visual working memory capacity that stores visual information[77]. In this 10-min task, participants saw discs of the same size but different colors on the screen for 100ms and were asked to remember the colors of individual discs after 600ms of fixation period. The number of discs for each trial varied from two to eight (two, three, four, six, or eight discs). The stimuli were replaced by a fixation mark for 900ms, and the discs reappeared with or without color changes. Participants were instructed to press a button with their right index finger if they detect any color changes between the two appearances of the discs and press the other button with their right middle finger if no change had occurred. Participants had 2,000ms to respond. The task consisted of 160 trials. For half of the total trials, original discs were replaced by different colors of discs, and for the other half of the trials, the original discs remained unchanged. Performance was assessed with a measure of working memory capacity, Pashler’s K[78], the average value of set size*(hit rate – false alarm rate) / (1- false alarm rate)[79].

Behavioral analysis

Given that all tasks required attentional ability, individuals who performed well in one task were expected to perform well on others. In order to confirm how behaviors in different attentional tasks are related, we computed Pearson’s correlation between individual performance scores on every pair of tasks, resulting in three between-task similarity metrics: between gradCPT and MOT, gradCPT and VSTM, and MOT and VSTM.

The between-task similarity estimate should be constrained by the reliability of the behavioral measures we adopted. Therefore, prior to computing a similarity of individual performance on the three tasks, we assessed the reliability of each behavioral measure by computing an intraclass correlation coefficient (ICC). For this reliability analysis only, we used subjects who had acceptable behavioral scores from both sessions for each task, resulting in a different number of available subjects for each task; 65 subjects for gradCPT, 69 subjects for MOT, and 71 subjects for VSTM. Within these subsamples, we estimated the ICC of individual performance between the two sessions for each task.

MR imaging parameters and preprocessing

MRI data were collected at the Yale Magnetic Resonance Research Center and the Brain Imaging Center at Yale with 3T Siemens Prisma system and 64-channel head coil. A high-resolution MPRAGE was collected at the beginning of each session with the following parameters: TR = 1800 ms, TE = 2.26 ms, flip angle = 8°, acquisition matrix = 256 × 256, in-plane resolution = 1.0 mm2, slice thickness = 1.0 mm, 208 sagittal slices. 10-min resting-state scans, two scans in session 1 and one in session 2, were collected after MPRAGE followed by an Inscapes movie-watching run (7:16 min). After these passive viewing scans, participants performed three 10-min main attention-related tasks, gradCPT, VSTM, and MOT, with a button box in their right hand. Each of the three tasks and resting-state scans included 600 whole-brain volumes acquired using an EPI sequence with the following parameters: TR = 1,000 ms, TE = 30 ms, flip angle = 62°, acquisition matrix = 84 × 84, in-plane resolution = 2.5 mm2, 52 axial-oblique slices parallel to the AC-PC line, slice thickness = 2.5 mm, multiband 4, acceleration factor = 1. This information was also provided in a previous study, which analyzed a subset of the current dataset; 49 subjects with two usable gradCPT runs at the time of the study[6].

Collected data were preprocessed with AFNI[80]. The preprocessing procedure included the following steps: Removing the first three volumes; censoring of volumes containing outliers in more than 10% of voxels; censoring of volumes for which the Euclidean norm of the head motion parameter derivatives are greater than 0.2 mm; despiking; slice-time correction; motion correction; regression of mean signal from the CSF, white matter, and whole brain and 24 motion parameters. FMRI data were aligned to the high-resolution anatomical image (MPRAGE) and normalized to MNI space. All the following analyses were performed in MATLAB R2016b.

Whole-brain functional connectome

Network nodes were defined using a 268-node whole-brain functional atlas, which covers the cortex, subcortex, and the cerebellum[81]. We excluded 23 nodes (due to imperfect acquisition of fMRI data on these areas from at least one subject), resulting in 245 nodes analyzed in this study. For each participant, an averaged time-series signal was calculated for each node, and Pearson’s correlation between all possible pairs of the 245 nodes were computed. The pairwise correlations were then Fisher z-transformed, resulting in a 245×245 symmetrical whole-brain functional connectivity matrix (29,890 unique edges). We calculated the connectivity matrix for each session separately and averaged them across two sessions for the final analysis. For those who had only one session that met our data inclusion criteria (29 subjects for gradCPT, 25 subjects for MOT, and 21 subjects for VSTM), we used the connectivity matrix from the available session in the analysis. Every individual had five connectivity matrices including three attention-related, one resting-state, and one movie-watching.

Brain-based prediction of individual behaviors across tasks

Connectome-based predictive modeling (CPM)

We constructed and validated CPMs using a 10-fold cross-validation (CV). In building CPMs, we held one fold (10%) of subjects out for model testing, with 82 or 83 participants in the training set. In training the CPM model, we first selected features (edges) that were significantly correlated with individual behaviors in a target task (Pearson’s, p<0.05). These features yielded both positive and negative edge masks depending on the signs of their correlation with behavior. For each subject in the training set, two networks’ strengths (one from the positive and the other from the negative network) were measured by averaging their respective connectivity strengths. Then, we fitted a general linear model between task performance (a dependent variable) and the two network strengths (independent variables). Once the two network masks and a general linear model were constructed, we applied the CPM to the held-out testing subject. The CPM estimated two network strengths for the test subject and predicted the subjects’ task performance from their network strength measures. Every fold was iteratively used as a test set in 10-fold CV. We repeated this 10-fold CV 1,000 times by randomly assigning subjects across 10 folds. Behavioral scores were z-scored for each task within a training set. Then, a mean and standard deviation computed within a training set were used to normalize behaviors of testing samples. Z-scoring was essential to provide a standardized behavioral measure, enabling the predictions across multiple tasks with incompatible scoring scales. We used raw scores only in the visualizations of a reliability of behavioral measures with scatter plots (Supplementary Figure 5A).

We assessed each model’s prediction performance by correlating model-predicted individual task scores and observed task scores. A significant positive correlation indicates that the model successfully predicts individual differences in behavioral performance. We also estimated prediction to further validate model prediction[82]. MSE of model-predicted scores were divided by MSE of guessing all z-scores=0, and then this normalized MSE was subtracted from 1 to yield q. The prediction q represents a model’s numerical accuracy in predicting an individual’s actual behavioral score compared to simply guessing their mean behavior. Hence, the prediction q complements the correlation-based model assessment and could inform a stronger practical utility of a predictive model. The 1,000 repetitions of 10-fold CV and model evaluation by r and q were applied to all following modeling analyses done within the n=92 dataset.

We previously demonstrated that CPMs are robust against the choice of feature selection threshold within the range of traditional statistical significance (e.g., p of 0.05 ~ 0.001)[22,25]. We tested a similar range of selection thresholds in the current study and confirmed that the results remained similar across the range (Supplementary Figure 6).

CPMs of a common attention factor

To examine how well a shared variance component of attention can explain behaviors on a variety of attentional tasks, we built CPMs using a common factor of the three tasks. In this analysis, we trained five predictive models to predict a common attention factor. All modeling procedure was the same as the original CPMs except the use of a common attention factor as a target behavior of interest. Before constructing a common attention factor, we z-scored behavioral scores for each task within a training set. Then, we normalized behaviors of testing samples using a mean and standard deviation computed within a training set. Z-scoring was essential to provide a standardized measure, enabling the generalization across different tasks with incompatible scoring scales. The common factor was defined by a mean of z-scored behaviors across the three tasks. The mean z-score was highly correlated with a shared factor extracted from factor analysis or the first principal component from principal component analysis (Pearson’s r(90)s>0.992, Supplementary Table 4, 5, and 6) and also in a training set of every iteration. To reveal a set of connectivity features that supports the common factor of attention across all three task fMRI conditions, we tracked an overlap of predictive connectivity between three task fMRI-based models of the common attention factor.

Significance testing with corrections for multiple tests

We evaluated the significance of model performance using permutations (one-tailed). We ran 1,000 permutations to construct 1,000 null models for the 81 model predictions. In each permutation, individual performances were randomly shuffled, and the null CPMs were trained and tested with connectivity matrices and the shuffled performances for 81 prediction cases. We assessed the performance of the null models by correlation r and prediction q.

Importantly, we used the permutations to correct for multiple tests (Supplementary Figure 7A). To do this, we first divided 81 predictions into three groups based on cognitive states of fMRI in training and testing datasets (1: nine original CPMs’ within-task predictions, on-diagonal elements represented with a blue line in Supplementary Figure 7A; 2: nine CPMs’ predictions when they were applied to different fMRI to predict same task from training data, represented with light green lines; 3: nine CPMs’ predictions when they were applied to different fMRI to predict different task, without any lines). We corrected family-wise error rate (FWER) for multiple comparisons in each case group separately using maximal statistic permutation test[83]. For each case group, the maximum null performance was selected in each permutation run. This yields 1,000 maximum performance null models for each group. We compared observed model performance with the 1,000 max null performance distribution of the corresponding group. The FWE-corrected significance of the observed model performance was calculated as p = (1 + the number of the null max performances better than the observed model performance) / 1,001. When we corrected for all 81 prediction cases simultaneously, the pattern of significant predictions remained similar (Supplementary Figure 7B).

Predictive anatomy of attention CPMs

We explored the predictive anatomy of CPMs to reveal the anatomical basis of the attention tasks and the general attention factor among three tasks. Each fold and repetition of 10-fold CV provided positive and negative network masks for each model. We extracted the most robust edges that appeared in 75% of 10-fold CV 1,000 iterations of each modeling. These robust predictive edges were then visualized with ten canonical networks (medial frontal, frontoparietal, default mode, motor, visual I, visual II, visual association, salience, subcortical, and cerebellum) defined in previous studies[18,39]. We tested different frequency thresholds (90 and 100%) in visualization and confirmed that the results remained similar (Supplementary Figure 8).

Controlling for behavioral correlations between tasks

In the original CPMs of three attention tasks, we examined the generalizability of CPMs across different tasks. Successful generalization, however, may be considered trivial, given the significant correlation of individual performances between tasks. To address this issue, behavioral correlations across tasks were taken into account. We regressed two non-target task behaviors from target task behaviors and considered residual target task-specific variance. The current models were trained to predict the residual variance specific to a target task. We assessed the significance of the model performance as described in Significance testing of CPMs’ behavioral prediction accuracy with correction for multiple tests with 1,000 repetitions of 10-fold CV, corrected using 1,000 permutations.

Although feature edges were correlated to the variance unique to a target task, it is possible that the feature edges were also associated with a shared variance. To address this possibility, we estimated partial correlations between model-predicted and observed behavioral scores while controlling for a common attention factor of the three tasks. We assessed the significance of the model performance as described in Significance testing of CPMs’ behavioral prediction accuracy with correction for multiple tests with 1,000 repetitions of 10-fold CV, corrected using 1,000 permutations.

Role of canonical brain networks in attentional behaviors

CPMs with computational lesion in brain networks

Next, we investigated which brain network is the most predictive of all three attention task scores. To assess the network-wise importance in behavioral prediction, we divided 245 brain nodes into ten canonical networks (the medial frontal, frontoparietal, default mode, motor, visual I, visual II, visual association, salience, subcortical, and cerebellum)[18,39] and computationally lesioned all nodes of a given network. We constructed and evaluated the nine CPMs of the reduced size of the connectivity matrix after lesioning one network. We repeated this procedure by lesioning each network iteratively. We restricted this analysis to task fMRI connectivity which yielded a successful prediction in the preceding analysis. We assessed the significance of the model performance as described in Significance testing of CPMs’ behavioral prediction accuracy with correction for multiple tests with 1,000 repetitions of 10-fold CV.

CPMs using within-network and between-network connectivity

In addition to the computational lesioning, we performed complementary analyses to examine the predictive power of brain networks. In this analysis, we restricted CPMs to use functional connections of only one brain network instead of the whole-brain connectivity. This analysis was further separated into two parts. First, we constructed CPMs based on connectivity within each brain network. Second, we constructed CPMs based on connectivity of one target network to the other nine networks. Hence, the first analysis was to examine predictiveness of within-network connectivity, and the second analysis was to examine predictiveness of between-network connectivity. We assessed the significance of the model performance as described in Significance testing of CPMs’ behavioral prediction accuracy with 1,000 repetitions of 10-fold CV.

Generating attention connectomes from resting-state fMRI

We utilized a novel method called connectome-to-connectome (C2C) state transformation modeling[38] to facilitate the estimation of attention-task connectomes and to improve behavioral predictions from resting-state data alone. In the previous analyses, we examined the generalizability of CPMs across multiple attention tasks. However, predictions of individual scores are typically impaired when the cognitive state of testing samples’ fMRI data is different from the training samples. The C2C framework generates task connectomes from the rest connectome or movie-watching connectome, and by employing the C2C approach, we can improve individual behavioral predictions from the rest or movie connectome[38].

The C2C model works in three steps in model application. First, the model extracts subsystems from the whole-brain resting-state connectome of individuals. The model, then, transforms the extracted subsystems to estimate task-specific subsystems. Finally, the model constructs whole-brain task-specific connectomes from estimated subsystems. The C2C modeling is based on two principal component analyses (PCA) and partial lease square regression (PLSR), each of which have been used in fMRI connectivity studies for various purposes such as noise removal, dimension reduction, and multivariate regression[25,84,85]. In model construction, the C2C model first defines and extracts state-specific subsystems and their scores separately for the resting-state and task-related state using two PCA. We applied one PCA on the rest connectomes of individuals in the training set. This corresponds to the first step of the C2C model described above. We applied another PCA separately on these same individuals’ task connectomes. This second PCA provides a reconstruction the whole-brain task connectome from the generated task subsystems, corresponding to the third step of the C2C model. Then, we employed PLSR to estimate the transformation of subsystems from the resting-state to the task state. The PCA-extracted subsystem scores of the resting and task states were put into the regression. This corresponds to the second step.

In this analysis, we constructed three C2C models to predict whole-brain connectomes of the three attention tasks from the rest connectomes. We assessed the success of task connectome generation in 10-fold CV. We held out one fold (9 or 10 subjects) for model validation and used nine folds to train C2C models. For model validation, we calculated the similarity between the model-generated connectomes and observed task connectomes and the similarity between observed rest connectomes and observed task connectomes, using spatial correlation. We also estimated the root mean square difference between the model-generated and observed task connectomes and compared it with a difference between the observed rest and task connectomes. Finally, we used the model-generated task connectomes to predict individual behaviors in the three attention tasks. We compared prediction accuracy of the C2C-generated task connectomes with the accuracy of the observed rest connectomes. Here, CPMs and C2C models were trained in the same training partition of 10-folds and tested in the held-out fold simultaneously. We ran the same C2C modeling and comparison procedure using movie data.

Building a general attention model

To maximize the practical utility of our suite of attention prediction models described above, we developed a general attention model that integrates the multiple task connectomes, CPM, and C2C model to (1) define a general attention connectome and generate it from a rest connectome and (2) predict overall attention performance in novel individuals.

The first step for model training was to generate for each participant a general attention connectome that combines the edges from the individual’s three attention task connectomes. There are several methods for doing so, described at the end of this section, and we chose the method that selects the edge with the highest absolute strength across the three tasks. To do this consistently across individuals, we first computed a group-average attention task connectome by averaging the edge strength from all the training participants for each edge in each task connectome. Then to select which task edge to use for the general attention connectome, we compared the absolute mean strength for each edge across the three average task connectomes. For example, if the average gradCPT connectome showed the maximum absolute strength for a particular edge, relative to the absolute edge strength in the other average task connectomes (MOT and VSTM), then we assigned the gradCPT edge strength to be the representative edge in the general attention connectome lookup table (Supplementary Figure 9), which specifies which of the task edges to use. Then for each participant, we used this population-level general attention connectome lookup table to generate the individual’s general attention connectome as a mosaic of the empirical edge values pulled from the individual’s three task connectomes.

The resulting individual general attention connectome can then be fed into the CPM and C2C pipeline like any other individual connectome. We trained one CPM to predict the common attention factor from the representative general connectome We also trained one C2C model that can estimate individual general attention connectome from novel rest connectomes. Once trained, the general attention model, combining general connectome construction, CPM, and C2C model, can predict a novel participant’s overall attention performance from a single rest connectome. This model was constructed and validated using 1,000 repetitions of 10-fold CV.

We explored different variants of ways to build the general attention model, and the primary model described above was chosen based on simplicity and performance, although performance did not vary significantly between models. To combine the task connectomes into the general attention connectome, one could average the edges, concatenate the three task connectomes, choose task edges that showed the largest variance across participants, or choose task edges that showed the maximum absolute strength for each participant (without consistency across them). These different methods showed only small numerical differences in performance with the averaging method showing the lowest performance. Finally, we tried predicting behavior using three inner linear regressions to predict the three task scores and then average them for the general measure; this method performed similarly to the primary model. We tested all the different combinations of these modeling choices and settled on the primary model described above because of its simplicity over the other models, again noting that prediction performance of the different model parameter choices was similar.

External validations in four independent datasets

Lastly, we substantially validated the proposed general attention model in four independent external validation datasets, three locally obtained (from the greater New Haven area) and two publicly available (total N = 495). Testing a model using one or more independent datasets is necessary given the large variance occurring in small-size samples and could boost a model’s reliability and practical utility[19,40]. The four datasets comprised rest connectomes and diverse attention-related measures. Since the three local datasets were obtained in previous works and the other two datasets were provided by different open data-sharing projects, we briefly described each dataset below.

The first set included gradCPT performance of 25 participants (13 females, mean age=22.7, ranges 18–32 years)[14]. Task performance was assessed by d’ sensitivity. The second set includes Attention Network Task (ANT) performance of 41 participants (28 females, mean age = 23.7 [±4.3] years, range 18–37 years)[24]. Task performance was assessed by correct-trial response time (RT) variability. Imaging data from these two datasets were acquired using similar parameters, including 1000 ms repetition-time (TR), to the current study but pre-processed differently. All participants in the two datasets gave written informed consent in accordance with the Yale University Human Subjects Committee and were paid for their participation. Information on subject recruitment, task design, and rest fMRI acquisition and processing is described in detail in previous work[14,24,25].

The third was the Human Connectome Project dataset[43], and information on fMRI acquisition and processing, and subject selection is described in detail elsewhere[22,86,87]. In the current study, we used Short Penn continuous performance task (SCPT) performance of 316 participants (154 females, mean age = 28.5 [±3.73], range 22–36 years) who completed all fMRI and behavioral sessions and were not related to one another (taking into account a family structure). Task performance was assessed by median RT for true-positive trials. The experimental protocol was approved by the Institutional Review Board at Washing ton University in St. Louis. All participants provided informed consent. The forth was a dataset provided by the ADHD-200 consortium[45]. Information on fMRI acquisition and processing, and subject selection is described in detail in the previous study[14] and at http://fcon_1000.projects.nitrc.org/indi/adhd200/. In the current study, we used ADHD-RS-IV score[44] of 113 children and adolescent (35 females, mean age 11.8 (±2.0) years, range 8–16 years) from the Peking University site. 75 participants were typically developing controls and 38 participants were with ADHD diagnoses. Each participant’s parent provided informed consent, and all children agreed to participate in the study. The data collection was approved by the Research Ethics Review Board of Institute of Mental Health, Peking University.

The attentional measures were z-scored in each dataset. We reversed the sign of z-scores of the ANT RT variability, SCPT RT, and ADHD-RS so that a higher score represents better attention performance. In the ADHD-200 dataset, we restricted our predictive network to 229 nodes by removing nodes missing in this dataset, resulting in 26,106 (=228×229/2) unique edges in each connectome. Otherwise, the gradCPT-based CPM, the rest-based CPM trained to predict gradCPT score, and the general attention model were identical to the models tested internally, but trained using the full internal samples (n=92). To have a clear view on what the general model adds, we directly compared the general model with the two best performance models (task fMRI- and rest fMRI-based CPMs predicting gradCPT performance) in the current study and sustained attention CPM (saCPM, https://github.com/monicadrosenberg/Rosenberg_PNAS2020)[14]. The saCPM was originally constructed using fMRI and performance in gradCPT[14]. That is, the saCPM is conceptually the same as the gradCPT-based CPM, model 1 in the current study, but built on the 1st external dataset. Since the saCPM was defined in the 1st external dataset, we examined its external performance only in the 2~4th external datasets.

We externally validated these four models in the four independent datasets where attentional function was measured by diverse attention-demanding tasks or ADHD-RS. Model performance was assessed by prediction q and correlation r. We assessed the significance of model performance using 1,000 permutations where null predictions were tested in each permutation. The variable data collection, analysis procedures, measures of attention, and age groups across multiple studies and sites enabled rigorous tests of a model’s generalizability. That is, if our general attention model successfully predicts attentional scores across the diverse datasets, then it emphasizes our model’s broad practical applicability.