Network-based assessment of collaborative research in neuroscience.

INTRODUCTION: The purpose of this study was to describe collaborative research in neuroscience within the context of the Center for Neurodegeneration and Translational Neuroscience (CNTN), a Center of Biomedical Research Excellence supported by the National Institute of General Medical Science. Drawing upon research on the science of team science, this study investigated the way that interactions around research emerged over the course of establishing a new research center. The objectives were to document changes in research activity and describe how human research support infrastructure functioned to support the production of science.
METHODS: Social network analyses were used to model coauthorship relationships based on publication histories from baseline (2014) through the current grant year (2017) for key personnel (n = 12), as well as survey data on collaborative engagement among CNTN members (n = 59).
RESULTS: Exponential random graph models indicated that over time, CNTN members were increasingly likely to form coauthorship relationships. Community detection algorithms and brokerage analyses suggested that the CNTN was functioning as intended to support scientific development. DISCUSSION: Assessment of team science efforts is critical to evaluating and developing appropriate support structures that facilitate successful team science efforts in translational neuroscience.

Introduction

Effective assessment of multidisciplinary collaborative research efforts requires the use of assessment strategies that can determine how collaborative research teams are functioning to meet goals, document changes in scholarly productivity, evaluate mentorship relationships, provide early notification of ineffective research supports and structures, identify sources of bottlenecks in information flow, and outline the extent to which resources are being used appropriately [1], [2]. In the context of the Center for Neurodegeneration and Translational Neuroscience (CNTN) funded through the National Institute of General Medical Sciences (NIGMS) Centers for Biomedical Research Excellence (COBRE) program, assessment acts to support the development of human capital and research infrastructure necessary for the success of neuroscience research and investigators. The CNTN is reflective of the emerging trend in collaborative, or team, science that has gained ground in biomedical research in part due to the growing evidence that impactful and innovative scientific advances are more likely to result from collaborative science efforts [3], [4], [5]. The science of team science, or documenting and evaluating the development and outcomes of collaborative research, has grown into its own robust field, catalyzed by evaluation and assessment policies and recommendations from extramural funding agencies and programs, such as the National Institutes of Health Clinical and Translational Science Awards (CTSA) [6], [7]. Although work in this area has used network analytic techniques, for example, documenting the types of networks formed via collaboration [8], [9] and productivity metrics of these networks [3], [10], [11], there remains much to learn from these techniques about how sustainable patterns of collaboration develop to support science.

Funders of biomedical research invest considerable resources into the preparation of emerging medical and academic researchers [12] and development of research infrastructure for neuroscience, which in this case, included human capital for research support. We refer to human capital for research support as teams of individuals who support the production of science. Critical individuals may include, but are not limited to, grant managers, clinical managers and staff, technicians, and students. Individuals such as grant managers and technicians rarely appear in assessments of team science [11] but are often critical to the production of research. Aims of the CNTN include supporting investigators working in human and animal models of neuroscience to produce initial data and assisting investigators in the development of advanced translational neuroscience skills, particularly in the areas of imaging and statistics. For many investigators, lack of research support, infrastructure, and the opportunity to develop advanced skills needed to conduct high-quality research are a detriment to producing scholarly products and grant proposals that are competitive for extramural funds [8], [12]. The existence of a robust science infrastructure is critical to facilitating these interactions. This study reports on assessment results of the growth and development in shared authorships among key CNTN members, as well as the functioning of CNTN research support networks designed to support the production of neuroscience research.

Program evaluation and assessment in collaborative neuroscience research

Within the biomedical sciences, program evaluation research has focused largely on either the impact of scientific research in basic and applied settings, or the collaborative nature of scientific research teams, or the career advancement of investigators [2], [13], [14], [15]. While no specific set of guiding principles exists solely for the purposes of evaluating scientific research, evaluation research to date has followed guidelines set by the American Evaluation Association broadly intended to cover all kinds of evaluation [16]. In recent years, assessment in government-funded research has grown to play an increasing role in evaluating research quality, reducing costs, and disseminating research credibility to the public [17]. Expenditures from the public purse must increasingly be justified by their measureable impact. Furthermore, a growing presence of translational science–specific evaluation literature [2], [18], [19] can be attributed to the requirement of a formal evaluation component for all National Institutes of Health CTSA [2].

The CTSA evaluation literature has produced a number of research articles supporting several evaluation designs appropriate for capturing and characterizing the nature of translational research programs [20], [21], [22], [23]. Multidisciplinary teams working on biomedical science form and develop in a dynamic manner over time, self-organizing around research topics, specialized skills, and knowledge domains [24]. Studies have demonstrated innovation in describing the complexity of translational teams through various approaches including mixed methods, case study, and network analysis designs [1].

Evaluation may play a critical role in describing interactions within innovative scientific teams. The major challenge for evaluators is appropriately documenting the nature of these interactions to identify patterns that can be used in the service of promoting effective collaborative science. A limiting factor is that little is known about the predictors of successful collaboration, mechanisms that support collaborative researchers' development, or barriers to collaborative success [25], [26]. While collaborative teams deliver greater levels of productivity over time and reap the benefits of increased visibility within the scientific community, there are few explanatory models to account for these outcomes [24], [27].

Mapping neuroscience research collaborations

Publication tracking is a commonly accepted form of quantifying research production and has been used to link publishing trajectories with career development [22], [26]. Quantity and quality of publications, often measured through journal impact factors and citation indices, are two normative indicators of impact in biomedical fields. Evidence also suggests a trend in high-impact coauthorship relationships in Alzheimer's disease research and related fields [5]. In Alzheimer's disease research, some of the most impactful work has emerged from long-standing collaborations. Collaborative research relationships foster opportunities to share ideas, generate intellectual stimulation, and cross-pollinate skill and knowledge development [28]. Scientific advancement may to some extent rest on scientists' abilities to functionally navigate the processes of forming research teams, effectively work to produce science, and efficiently distribute findings. From this perspective, a third metric of productivity and impact in biomedical research may be the extent to which scientists form and maintain publication and grant relationships.

Developing effective research teams that lead to these publication and grant relationships requires effort, negotiation, and time [8], [18], [29], [30]. Academic faculty and clinical researchers are typically expected to publish research results to advance in their careers. Collaborative research centers and institutions are designed to facilitate the process of building research teams and should engender scientific collaboration more effectively than that could be generated independently by investigators in neuroscience and other biomedical fields, particularly when investigators are working as specialists without a deep community of institutional peers. Evidence for increased collaborative publications may be one metric by which to measure the success of the center or institute in furthering science. By tracking changes in coauthorship relations over time, potential impact of center or institute structures may become apparent if increased collaboration or different collaboration patterns occur after the onset of the formal research center or institute. Social network analysis (SNA) is a viable technique for identifying scientific collaborative network structures. Introduced at the turn of the 21st century [31], [32], [33], SNA has rapidly gained research attention recent years as an emerging best practice for mapping collaboration and producing evidence for effective team science [9], [10], [11].

SNA has been applied to document productivity and viability of research teams' collaborative interactions over time, including prediction of interdisciplinary collaboration formation [3], [34], [35] and cooperative structures and interactions among network members [36], [37], [38]. Despite increased SNA investigations into research networks in medical and translational research [3], [10], [11], [34], [35], [36], [39], [40], [41], [42], [43], there are few SNA investigations into collaborative research specifically in neuroscience. Thus, there is relatively little information as to how scientists working in translational neuroscience may form collaborative partnerships that are indicative of successful team science. Publication counts are a conservative measure of productivity and career advancement [8], [26]. However, without adequate measures of the quality of collaborative interactions, there may be a failure to accurately understand how these productivity outcomes emerge from collaborative science or how productive collaborations can be encouraged and facilitated [44].

Research support networks

In this era of increased specialization, the formation of research support networks is critical to conducting high-quality research and developing competitive grant proposals for external funding [8], [21], [45]. Furthermore, center and institute funding may be maximized when collaborative teams emerge and share resources [29], [30]. Collaborative engagement of research support members, such as statisticians and technicians, with lead scientists and the extent to which members of a research community participate across multiple research projects or teams may be indicative of a healthy and sustainable research infrastructure [46]. Effective assessment includes the collection of multiple forms of evidence that can provide insight into the underlying factors that yield collaborative products [28].

The present study

The purpose of this study was to investigate indicators of research collaboration in translational A&D within an NIGMS-funded research center. The study maximizes the best practices for assessing collaborative research networks by using social network techniques for evaluating changes in scholarly productivity, membership relationships, research supports and structures, and workflow processes. The results contribute to the emerging application of team science in the context of neurodegenerative disease research and NIGMS centers [9], [34], [35]. Longitudinal, periodic reassessment provides information on the growth and reordering of collaborative networks and provides additional insights into the success of developing a scientific infrastructure. The data presented in this study contribute to understanding the development of emerging collaborative science efforts and smaller scale multidisciplinary research collaboratives in neuroscience. Two primary sets of research questions guided the analytic aspects of the study.

Research questions

Does being a member of the CNTN increase the likelihood of shared authorship? Does being a member of the CNTN increase the likelihood of sharing authorship with another CNTN member? Does change in network metrics and shared authorship over time indicate the CNTN is having a positive influence on shared authorship between members?

To address these questions, change in publication collaborations (2014 to 2017) were analyzed to determine if and how shared publications among CNTN members increased from baseline through CNTN implementation.

Does the a priori defined structure of the CNTN map onto the emergent community network structures of collaborative engagement among CNTN members? What brokerage processes between core CNTN areas drive the emergence of the observed CNTN community structure? Do these brokerage processes align with the predefined roles of CNTN cores areas?

To address these questions, collaborative engagement data were analyzed to examine the emergent community structures in the CNTN and determine how they map onto members' empirically defined roles in the CNTN. A brokerage analysis of the collaborative engagement data was also conducted to determine the process of CNTN workflow. Finally, a multinodal network including persons and their related projects was examined.

A Multilevel systematic approach to network analysis

Social network analysis was used to answer the study research questions. Social networks are often defined as relations among individuals, or nodes, where the ties between them are referred to as edges [47]. Networks can also contain relationships between individuals and other abstractions, including projects, objects, or psychological states. An n × n matrix is defined, such that Y is the value of the relation from node i to node j. Relations can be binary, ordinal, or continuous. Symmetric matrices contain undirected ties, whereas asymmetric matrices contain directed ties. A matrix can be visualized as a graph, or a sociogram that demonstrates relationships among nodes.

Exploratory or descriptive methods are used to summarize the network. Descriptive network measures can exist at the node and network level. Nodes within a network are often described using measures of how central a node is within a network. Measures of centrality can provide information about who brokers information between people or communities within a network and can be useful when making decisions about organization structure and group dynamics [47]. A common measure of centrality in a network is degree. Degree is the number of ties a node has. For an undirected matrix Y, degree for node i is Σj Yij.

The network level is often described using edge count, transitivity, and density metrics. Edge count is a sum of all observed relations in a matrix. Transitivity is the formation of closed triads in a graph, where a loop length 3 is a sequence of nodes (x, y, z,) such that (x, y), (y, z), and (z, x) are edges of the graph. A transitivity index for a given network can be calculated, where the number of observed transitive triads is divided by the number of potential transitive triads. Triads are rare in randomly generated networks and when observed indicate self-organization [48]. Density is of the proportion of observed ties out of all possible ties, calculated as ∑y/n(n−1). Density is a measure of how well connected a graph is and can indicate how well information flows, how much information is being shared, or how well supported individuals are, depending on the nature of ties [3], [35], [47].

Similarly, network-level structures can be identified, which demonstrate how ties form or how communities of people within a network self-organize during interaction. Exponential random graph modeling (ERGM) comprises a class of models used to inferentially test the formation of ties between actors in a network based on their attribute characteristics [48], [49]. ERGMs model the probability of observing network given the space of all possible networks Y, calculated by 2n(n−1), where n is the number of nodes. From this large distribution of graphs, the probability of observing the number of reciprocated ties and transitive structures in can be estimated. Furthermore, emergent community structures can be identified in networks by defining an interconnected topology combining order and randomness [50]. Networks can be decomposed into subcommunities, or sets of highly interconnected nodes. Modularity, then, is an empirically defined, compartmentalized internal structure that indicates the density of connections between nodes within modules and the sparseness of connections between nodes in different modules. High modularity is often interpreted as robustness to external perturbations to a network.

Network processes, such as how information or work flows through a network, can be examined using brokerage analyses [51]. In its most basic form, node v is a broker if for distinct nodes a and b, a → v → b where a and b are not related. If nodes in a network belong to distinct groups, then group membership may be used to distinguish between different types of brokerage roles. Let A → B → C describe the two-path relationship at the heart of a brokerage structure. A node from group B brokers the relationship from a node in group A to a node in group C. Gould and Fernandez [51] describe six types of brokerage relationships:

w: Coordinator role; the broker mediates contact between two individuals from his or her own group. Two-path structure: A → A → A

w: Itinerant broker role; the broker mediates contact between two individuals from a single group to which he or she does not belong. Two-path structure: A → B → A

b: Representative role; the broker mediates an incoming contact from an out-group member to an in-group member. Two-path structure: A → B → B

b: Gatekeeper role; the broker mediates an outgoing contact from an in-group member to an out-group member. Two-path structure: A → A → B

b: Liaison role; the broker mediates contact between two individuals from different groups, neither of which is the group to which he or she belongs. Two-path structure: A → B → C

t: Total (cumulative) brokerage role occupancy (any of the above two paths).

A brokerage score for a given node is the number of ordered pairs having the appropriate group membership brokerage relationship. Aggregate scores can be computed for defined groups within a network as well as at the network level. Expectations and variances of brokerage scores given the size and density of a network can also be computed [52].

Network graphs are often visualized using layout algorithms. These algorithms are specific to the nature of the observed networks. Collaborative human systems (crowds, protests, markets) where people collaborate, cooperate, or interfere are often characterized as small worlds [53], [54], [55]. The forced atlas 2 layout algorithm is a practical layout approach that can be used to visualize network data that represent small world phenomena. The forced atlas 2 algorithm is designed to simulate a physical system to spatialize a network. Nodes repulse each other while edges attract their nodes. These competing forces create a movement that converges to a balanced state, where the final configuration can help data interpretation [56]. Conducting network analysis requires collecting specialized forms of data that capture relationships between people. The following section describes the methods used to collect data that were used to address the research questions using the social network analytic approaches described previously.

Methods

The CNTN data were derived from self-reported publications and collaborations from members of the CNTN research collaborative. Because of the relatively small size of the collaborative, the demographic information of those who participated has been withheld to protect member identity. Data for the publication networks were derived from the members' curricula vitae (CV). Key personnel were identified who were likely to lead publishing efforts (N = 15), most of whom provided CV (n = 12). Publications and presentations/abstracts per calendar years spanning the life of the CNTN (2015 to 2017) listed on the CV were used. The 2014 year before the award of the CNTN support was included as a baseline measure. All the authorship information from the publications spanning years members contributed to the CNTN, including baseline, were entered into a data array organized by year and author. These data were manipulated to form adjacency matrices of all coauthors from all years (n = 672), producing an unweighted, nondirected adjacency matrix for each publication year. The attributes for authors were coded as 0 = non-CNTN author and 1 = CNTN author.

Member collaborative engagement data were gathered using an online self-report survey. The survey was designed according to recommendations for best practice [57]. A CNTN census membership list (N = 56) was compiled through a multistep procedure that included document review by CNTN evaluators and subsequent review by key personnel. Members were e-mailed a survey asking them to identify with whom they collaborated to carry out their CNTN duties. Collaborative engagement was defined as, “coordinated activity including conversational interactions, coordinated and supportive behaviors for joint activity and projects, and receiving and giving of feedback guidance or scaffolding. Computer-mediated interactions (i.e., e-mail) should be included.” In the survey, members first identified with whom they interacted and then reported the frequency of their engagement with those whom they had identified on a sliding scale of 100, ranging from “almost never” to “daily,” with various time intervals specified between. Members were then asked to indicate with whom they collaborated on specific projects: CNTN administrative functions, CNTN technical duties, CNTN-initiated research, and non-CNTN initiated research. Participants could include the names of members who may have been missing from the census list, yielding a final census of 59 individuals. Data were manipulated to form an adjacency matrix of all members, specified by the frequency of their collaborative engagement. A second adjacency matrix was also constructed, such that member relationships to each other and their specific projects could be modeled. Thirty-two CNTN members participated in the survey. Missing data were dealt with by inferring reciprocity (i.e., the matrix was transposed), creating an undirected, valued matrix, with more frequent collaborations represented by higher values. The attributes for member roles were coded according to their CNTN affiliation. These categories included one of three ongoing projects in the CNTN (projects 1–3), the administrative core (i.e., project leadership and assessment teams), the data management and statistics core (i.e., storing the clinical data), the clinical core (i.e., technicians and research-oriented personnel), and an unassigned category (no self-identification with a category).

Results

The publication networks and collaborative engagement of CNTN members were examined at the whole network level using network descriptive statistics, including edge count, density, and transitivity. Centrality was calculated for all nodes in the network using degree centrality. Statistics were calculated in R using the package statnet [58].

Publication network analysis and findings

An ERGM model was fit to CNTN member publication networks (see Table 1 for a summary of results). The ERGM was calculated in R using the package ergm [59]. The model includes an indicator that the author of a publication was a CNTN or a non-CNTN contributor. The indicator corresponds to both a factor effect, or that CNTN member influences tie formation (a shared publication), and a homophily effect, or a dyad covariate that two authors share CNTN membership. Model estimates are presented in Table 2. Significant effects are identified if the 95% confidence interval (i.e., ±2 SE) does not contain 0. In all years, CNTN affiliation significantly increased tie formation (which is expected given the publications came from members' CV). Controlling for the CNTN influence on tie formation, in years 2014 and 2015, there are no significant dyad covariate effects for CNTN affiliation. In years 2016 and 2017, there are significant dyad covariate effects for CNTN affiliation, where members of CNTN are more likely to form ties with other CNTN members. Network graphs for CNTN publication networks were visualized using a forced atlas 2 layout algorithm [57] using the open source software Gephi (see Fig. 1) [60]. Isolated nodes were removed from the graph to improve layout.

Table 1

Descriptive publication network characteristics by CNTN year

Metric	2014	2015	2016	2017
Edge count	2485.00	1655.00	1497.00	949.00
Transitivity	0.60	0.59	0.70	0.59
Density	0.01	0.01	0.01	0.00
#Connected nodes	290/43%	260/40%	193/29%	196/30%
#Products	55	59	54	107

Abbreviation: CNTN, Center for Neurodegeneration and Translational Neuroscience.

NOTE. Data from 2014 serves as a baseline year, before the CNTN was funded. Node count = 672 for all years. Primary axis scale = edge count. Secondary axis scale = products. Products include publications, presentations, and abstracts.

Table 2

Exponential random graph modeling results for all publication networks

Metric	Est	Std. error	P-value
	2014
Edges	−4.452	0.156	.000
CNTN tie effect	0.579	0.147	.000
CNTN homophily effect	−0.118	0.156	.451
	2015
Edges	−5.123	0.130	.000
CNTN tie effect	1.081	0.115	.000
CNTN homophily effect	0.077	0.130	.549
	2016
Edges	−5.466	0.131	.000
CNTN tie effect	1.389	0.113	.000
CNTN homophily effect	0.355	0.131	.007
	2017
Edges	−6.272	0.100	.000
CNTN tie effect	2.040	0.074	.000
CNTN homophily effect	0.490	0.100	.000

Abbreviation: CNTN, Center for Neurodegeneration and Translational Neuroscience.

NOTE. Node count = 372 for all years.

Fig. 1

CNTN member publication network layouts for all study years. Isolated nodes were removed from the analysis and visualizations. Blue nodes = CNTN authors; gray nodes = non-CNTN authors. Modularity (2014 = 0.609; 2015 = 0.748; 2016 = 0.573; 2017 = 0.738); average degree (2014 = 34.6; 2015 = 24.7; 2016 = 31.0; 2017 = 19.35); connected components (2014 = 6; 2015 = 5; 2016 = 2; 2017 = 4). Abbreviation: CNTN, Center for Neurodegeneration and Translational Neuroscience.

Visual inspection of the graph suggests increased collaboration of CNTN members during the publication process. The descriptive network statistics indicate that productivity within the publication networks increased over time while the density, edge count, and average degree decreased. Note, the density of these networks is similar to other investigations of research communities [3]. The number of connected components within the network decreased from six in 2014 to two in 2016 and four in 2017, suggesting that network is becoming more connected at the macro level.

Collaborative engagement network analysis and findings

To analyze the community structures within the network, the a priori affiliations of CNTN members (top down) were compared to the empirically defined community structures, or graph modularity (bottom up). The network was visualized using a forced atlas 2 layout algorithm [56] using the open-source software Gephi [60]. Isolated nodes were removed from the graph to improve layout. A modularity community detection algorithm [51] was used to analyze the bottom-up, emergent community structures within the network. The visualization was frozen and recolor coded based on the resulting a posteriori empirical community structures. Analysis of the percentage of nodes affiliated with structures between graphs provided evidence for the difference between the top-down, a priori defined structure of the CNTN and the bottom-up, self-organized community structure that emerged via collaboration engagement (see Fig. 2). Visual inspection of the graphs suggests that while a priori structures related to the administrative, data core structures, and research project 3 remained largely intact during community detection, five communities self-organized during member collaborative engagement. To examine the process by which this self-organization occurred, a brokerage analysis was conducted. Brokerage statistics were calculated in R using the package sna [53]. See Table 3 for a summary of results. Results indicated that members affiliated with the administrative core and project 3 brokered coordinated (A → A → A), representative (A → B → B), and gatekeeper processes (A → A → B). Members affiliated with the clinical core and those unassigned to any core brokered the itinerant (A → B → A) and liaison processes (A → B → C).

Fig. 2

CNTN collaborative engagement layout by affiliation and emergent community structure. Node size is proportional to node centrality. Node count = 59; edge count = 685; average degree = 11.61; modularity = 0.326; all statistics apply to both graphs presented. Abbreviation: CNTN, Center for Neurodegeneration and Translational Neuroscience.

Table 3

Brokerage analysis for collaborative engagement by CNTN affiliation

ID	Affiliation	Coordinator	Itinerant	Representative	Gatekeeper	Liaison
1	Administrative core	4.18	11.45	17.47	17.47	54.04
2	Clinical core	0.63	15.44	8.38	8.38	71.76
3	Data core	1.33	14.62	11.30	11.30	66.07
4	Project 1	0.95	15.06	9.87	9.87	68.85
5	Project 2	1.77	14.11	12.66	12.66	63.41
6	Project 3	4.18	11.45	17.47	17.47	54.04
7	Unassigned	0.06	16.20	3.54	3.54	81.25

Abbreviation: CNTN, Center for Neurodegeneration and Translational Neuroscience.

NOTE. Brokerage scores represent expected values conditional on network size and density. Two highest values per column are in bold. Repeated values represent identical levels of expected brokerage for an affiliation group. Total (any two path) = 104.6 for all affiliations.

The visual inspection of the sociograms in Fig. 2 indicated that the members of the CNTN with the greatest centrality were members of the administrative core. The comparison of the a priori structures with the empirical community structures suggested that the emergent groups that differed from the a priori categories centered around research projects with human populations. The emergent groups each include members from the specific projects, administrative, clinical, and data cores. The brokerage analyses indicate a process by which the relatively intact cores in the a posteriori communities were more likely to facilitate internal ties or either providing information that stayed within the receiving group or receiving information from a group that stayed internal. By contrast, the clinical core acted to facilitate more diverse connections, a workflow that facilitates the exchange of information between groups. As the function of the clinical core was to facilitate the production of the research itself, the empirically derived groups would suggest that teams have used this resource effectively.

A second visualization was then constructed, using the adjacency matrix that included relationships between CNTN members and their reported collaborations around project activities, treating the project activities as nodes. Centrality was calculated for the multinodal network and visualization was constructed using the same procedures as described previously (see Fig. 3). Visual inspection of the graph indicated that work on multiple projects was at the center of CNTN collaborative engagement. Betweenness scores for project activities were as follows: multiple project = 637.38; non-COBRE project = 346.91; CNTN Admin Project = 330.20; COBRE initiated project = 258.59; CNTN tech project = 191.92.

Fig. 3

CNTN multinodal network including persons and project activities. Node size is proportional to network centrality. Pink nodes = persons; green nodes = projects. Node count = 64; edge count = 1101; average degree = 17.20. Abbreviations: CNTN, Center for Neurodegeneration and Translational Neuroscience; Multiple, multiple projects; Non-Cobre, Non–CNTN-related work; Admin, administrative duties; Research, CNTN-related research; Tech, technology- and data-related activities.

Discussion

The CNTN was initiated in 2015 to support neuroscience research through the development of research infrastructure and investment in emerging scientists. Relatively unique to COBRE-supported centers, the CNTN developed a robust assessment strategy to encourage evidence-based decision making about how the program was functioning to support neuroscience and investigators, as well as regular outcome-based assessments of program influence on metrics used to measure the success of investigators. Contributing to the literature on the application of team science within neuroscience, productivity data were gathered annually to determine change in collaborative authorship patterns over time. Furthermore, in recognition that developing a functioning multidisciplinary and multi-institutional research collaborative is challenging [29], the assessment approach provides information about the processes underlying collaborative group formation in neuroscience through modularity and brokerage analyses of the entire research structure, ranging from research support personnel through the program director.

Similar to other investigations of collaborative research teams initiated after receipt of extramural funding [10], the CNTN shows evidence of increasingly cohesive collaborative relationships among members. The results are suggestive of adaptive change within CNTN publication collaborations. That is, the data suggest that there are growing number of within-CNTN collaborations and a pruning of ties that may not be required for current practice, leading to increased productivity. One interpretation of these findings is that the CNTN has been effective in creating opportunities for team science that has direct outcomes relevant to the field (through sharing of research results) as well as careers of emerging scientists (through productive coauthorship relationships).

Furthermore, our study suggests that even early in its implementation, the CNTN appears to be functioning to bring scholars together and support them in developing their scientific agendas through the provision of shared research human capital. The results signaled a natural progression of a newly formed collaborative structure. For example, the non-CNTN project work in which members engage may include collaborative work that predated the CNTN funding or work that allowed for the development of skills (such as imaging or statistical models) while the start-up required before gathering CNTN-specific data unfolded in the first years of the program. Work on multiple projects as the center of CNTN collaborative engagement signals that members are involved in significant cross-talk across predefined organizational structures, using specializations of team members and center resources. The study findings support a multitiered, complex structure that includes members coming together as a “team” broadly within the CNTN and in the multiple small worlds that emerge from the collaborative structure.

Continued investigation as the program continues will dive into the conditions that facilitate adaptive team science. Given the community formation and brokerage results from the collaborative engagement survey, it is reasonable to suggest that adequate support cores that coordinate administrative grant functions and provide imaging and statistical expertise function to distribute research demands to enable science to develop more efficiently. Furthermore, the role of the clinical core to coordinate resources among research projects, namely in the form of handling participant enrollment, data entry, and testing, is clearly identifiable by the liaison role the core plays. The clinical core serves as a bridge between members from multiple groups in the CNTN, connecting them with foundational support that enables the conduct of the science itself. As the teams mature, the development of shared norms, language, and expectations should become more integrated into their daily work patterns, potentially resulting in a more efficient time to publication [8] and enhanced proposal funding success.

With regard to the CNTN research project activities, modularity and brokerage analyses suggest that project 3 functions in a distinct way from the other research projects in that the brokerage patterns are similar to those of the administrative core. Notable is that project 3 is not housed at the lead research institution for the CNTN, which could suggest that projects apart from the lead institution may take on more managerial and coordination demands than projects with closer proximity to the research infrastructure. The potential impact of this differential role on collaboration or research productivity cannot be determined from these findings, but future research may be able to gain a more fine-grained understanding of these processes.

Limitations

Despite the cogent story that took shape around the multiple sources of data, the CNTN is still relatively young in its possible lifecycle and data are limited to date. Furthermore, the productivity metrics were constrained to authorship on publications and presentations/abstracts only. Collaborative partnerships on grant submissions are a key metric to demonstrate success of the CNTN and have been included in other network analyses of productivity. As the CNTN matures, these data are likely to be more robust for inclusion in the productivity analyses. Similarly, the productivity metrics did not account for quality of the publication (e.g., journal impact factor, number of citations). The CNTN is also a relatively small collaborative program. Some of the more complex findings from the initial results, such as the brokerage results, may be an anomaly to the particular nature of the CNTN and may not be informative to other programs engaged in team science. Furthermore, it is possible that types of brokerage, not captured by the existing metrics, may emerge from these partnerships. Finally, given the specific nature of the CNTN, it is not clear if these patterns would replicate in a similarly structured COBRE-funded program or other research center or institute. Without the presence of a control group of personnel without COBRE support, which is not plausible given the contextualized and specialized nature of the scientists, it is important to interpret the study results with caution when considering the effects of the formal research center structure on productivity. However, note that the intent of this study was not to yield generalizable findings but to provide an indication of the types of outcomes and collaborative relationships that might yield outcomes, when infrastructure supports the development of A&D research partnerships.

Summary

The totality of evidence from this study suggests that the CNTN has been effective in facilitating scientific collaborations in neuroscience. Over time, these collaborations, and those stemming from other centers like the CNTN, may yield high-impact scientific findings and advance the careers of emerging investigators in the field. Evaluation of the structure and function of the CNTN and similar collaboratives is critical for determining how to intentionally create communities that facilitate research engagement and to maximize the impact of these resources for all institutions and members of the collaborative.

Systematic review: The authors conducted a thorough search of the extant literature on assessment of collaboration in large-scale research center programs using traditional databases. The review revealed that network-based approaches to assessment of research collaboration are on the rise, but scarce in the fields of biomedical sciences and Alzheimer's disease.

Interpretation: Social network analyses methods found that research collaboration resulting in publications, abstracts, and presentations among research team members supported by a National Institute of General Medical Science Center of Biomedical Research Excellence award increased as the grant matured. A survey of engagement with research suggested that research support for the science was critical to creating a sustainable environment for collaboration to develop.

Future directions: Assessment of team science efforts is critical to evaluating and developing appropriate support structures that facilitate successful team science efforts in translational neuroscience.