CTS teams: a new model for translational team training and team science intervention.

INTRODUCTION: Clinical/translational science (CTS) is team-based, requiring effective collaboration and communication across many disciplines involving a variety of stakeholders. We implemented a pre-doctoral team-based training model with didactic and experiential curricular interventions to support the development of CTS research skills in a cross-disciplinary team environment. We assessed the potential impact of this new training model as a team science intervention that can catalyze new cross-disciplinary collaborations across the institution.
METHODS: Between 2016 and 2020, 32 pre-doctoral students and 26 co-mentors participated in the assessment of the CTS Team program over a two-year period of TL1 training grant support. Data collection and analyses followed a program logic model and used a variety of metrics for clinical and translational scientist career success.
RESULTS: CTS training in the context of CTS Teams supported improved self-efficacy for clinical research skills and resulted in a significant increase in the frequency of participation in cross-disciplinary collaborative activities by both trainees and mentors. Most CTS Team co-mentor pairs had not previously collaborated. Two-thirds of the co-mentors plan to continue collaborating, and most (85%) currently use or plan to use collaboration tools, for example, written collaboration plans, authorship agreements.
CONCLUSIONS: The CTS Team training model provides a unique clinical and translational science team training experience that embeds authentic cross-disciplinary research collaboration into PhD research projects. It establishes trainee cohorts that are diverse in terms of scientific disciplines and translational research phases, and creates a new cross-disciplinary community of practice across faculty members and research groups in multiple colleges.

Introduction

Clinical and Translational Science Award (CTSA) hubs are NIH-supported institutions that are charged with creating research and training environments for clinical and translational research, recognizing that such research is team-based and requires effective collaboration and communication across many scientific disciplines and involving a variety of stakeholders [1]. CTSA hubs are therefore encouraged to “foster collaborations by providing education in team science to promote the development and implementation of multidisciplinary teams” and to integrate “a team science approach into translational science research training” [1]. A nation-wide survey of CTSA hubs in 2012 revealed that most (86%) of education leaders reported that team science training is important and that about half (52%) of the institutions offered such training at that time, in the form of courses and seminars [2]. Such training was believed to be effective, but there was no consensus about training methods, content, or assessment approaches.

This emphasis on team science is driven by the recognition that the increasing complexity of health and societal problems has resulted in a shift in the way we do research toward more interprofessional and cross-disciplinary team approaches [3]. Partnering with social scientists from disciplines such as organizational psychology, sociology, philosophy, leadership studies, and communications, a “science of team science” community has emerged, focusing in part on enhancing research team effectiveness to solve complex problems [4,5].

This work is based on the premise that if future clinical and translational researchers will be engaged in team science during their research careers, they will benefit from deliberate training in such an environment. We therefore designed a CTSA TL1 training program that directly addresses the call to action by Begg et al. [6] to develop new models and methods for clinical and translational workforce development at a most informative stage of training: graduate education. We describe a “CTS Team” training model in which pairs of PhD and/or dual degree students embed authentic cross-disciplinary collaborative research experiences into their PhD dissertation research projects. Our starting goal was to develop and implement a new team training approach in clinical and translational science (CTS) combining didactic and experiential training in team science that included cross-disciplinary mentoring. Our second goal was to use measures proposed by Lee et al. [7] for assessing clinical and translational scientist career success, in order to identify metrics that will be useful in identifying changes in behaviors and attitudes during the relatively short timeframe of the training program (i.e., two years), which might be correlated with future measures of success, such as publications and career outcomes.

It became apparent that the potential impact of the CTS Team training model was far greater than the outcomes for the trainees. The CTS Team training model has proven to be an intervention in the collaborative network within the university, impacting the faculty members serving as mentors for the CTS Team members by catalyzing new cross-disciplinary collaborations across the institution. We therefore also examined the impact of CTS Teams on cross-disciplinary collaboration between the program faculty members, by investigating previous and planned future collaborations among CTS Team co-mentors and changes in cross-disciplinary collaborative activities by co-mentors.

Methods

Logic Model

The logic model that guided the development and implementation of the CTS Team training model is shown in Fig. 1. Inputs that are addressed in this report include identification of PhD and dual degree programs that have partnered with the TL1 program, the CTS PhD co-major, and a mixed methods assessment plan. The activity on which this report focuses primarily is experiential learning via co-mentored, authentic cross-disciplinary collaboration by CTS Teams, supported by a new Team Science course. The short-term outputs described in this report include the implementation of the new Team Science course, translational research projects (team-specific aims) that become embedded in TL1 trainee dissertation research projects, and changes in attitudes toward clinical research self-efficacy, team science, research orientation, grit, satisfaction, and motivation.

Fig. 1.

Logic model used for the design and implementation of the clinical and translational science team training model (CTS Teams) in the UF TL1 program. Elements addressed in in this report are highlighted in red bold font.

CTS Teams

Prior to 2016, University of Florida (UF) TL1 trainees were selected from diverse disciplines, required to identify at least one specific aim of their dissertation research that was clinical or translational, and provided with a didactic curriculum leading to the award of a PhD concentration (“co-major”) in “Clinical & Translational Science.” Individual TL1 trainees identified a research mentor and a clinical mentor (Table 1). The TL1 application and selection process transitioned in 2016 to support teams of TL1 trainees instead individual trainees (like most other NIH training grants). CTS Teams are composed of two PhD and/or dual degree (e.g., MD-PhD) students who are pursuing their PhD studies in different disciplines, in different colleges, and with different mentors. Students apply during their first or second year of PhD studies, before they have defended their doctoral dissertation research proposals. Prospective CTS Teams propose cross-disciplinary collaborative research projects that become embedded into their individual dissertation research as “team-specific aims.”

Table 1.

Individual TL1 trainees vs CTS team training

	Individual TL1 trainees								CTS teams*
Trainee-mentor group
Cohort (year TL1 training started)	2009	2010	2011	2012	2013	2014	2015	2016	2016	2017	2018	2019	2020
Number of trainees applied	17	7	5	10	14	13	11	6	8	15	10	6	16
Number of trainees started	6	4	3	4	3	2	7	5	2	8	8	6	8
Number of CTS teams started	–	–	–	–	–	–	–	–	1	4	4	3	4

Cohorts of Clinical and Translational Science (CTS) Teams that had completed the two-year TL1 training period by Fall 2020 are indicated by bold type (2016-2018); survey results for these three cohorts are described in the text and summarized in this table.

The CTS Team application includes individual components in which each team member briefly describes their individual PhD projects in terms of the research questions, hypotheses, specific aims, significance, and research strategy, so that reviewers can understand the context of the collaboration. Importantly, the team members together write a team research plan that outlines the team-specific aims, hypotheses that form the basis of their proposed collaborative research, the clinical and/or translational components of the research, goals and expected outcomes, specific aims, and how each team member will contribute to the collaboration. They describe their collaborative research strategy in terms of significance, specific aims, and experimental approach, as well as the synergy and interdependence needed to accomplish the team-specific aims. The team members’ primary research mentors agree to serve as co-mentors for the purpose of accomplishing the team-specific aims and write a single letter of support including a team mentoring plan. No additional financial incentives were given to the mentors to be a part of this program beyond the standard training grant support for their mentees (stipend, tuition & fees, travel allowance, and training-related expenses).

Team Science Course

The team science training program includes both didactic and experiential components. Didactic training includes the graduate course “Team Science,” in which lecture/discussion sessions are used to illustrate principles and strategies for cross-disciplinary collaboration and evidence from the team science literature for their effectiveness [8]. Topics include the following: Introduction to the Science of Team Science, Preparing for Team Science, Team Leadership, Building a Research Team, Strategic Team Science [9], Conflict Management, Writing Collaboration Plans & Authorship Agreements, Managing Research Teams, and Team Monitoring and Evaluation. Most importantly, students immediately apply team science concepts to their own dissertation research, with homework assignments such as writing a team vision & mission statement, collaboration plan, and authorship agreement, practicing a team monitoring exercise, and writing a team evaluation plan. CTS Team trainees prepare homework assignments as teams, focusing on their collaborative research experience. Other students in the course are encouraged to collaborate with other members of their research group and/or their mentors on all homework assignments. These experiential components of the team science course provide coaching and practice of team science skills during the first semester of the CTS Team training period and support the authentic cross-disciplinary collaboration by CTS Team members described in the previous section.

Procedures for Data Collection

All CTS Team trainees and co-mentors were invited to participate in separate evaluation surveys approved by the UF Institutional Review Board (IRB-02). Prior to participation in the survey, participants read and electronically signed an informed consent. Trainee surveys were distributed electronically to the trainees in the first month of the first year of the TL1 training period and during the last month of the second year of TL1 training. Data used in this report includes responses only from the first three cohorts of CTS Teams (2016, 2017, and 2018) who had completed the two-year TL1 training period by the Fall of 2020 (Table 1). All CTS Team co-mentors were invited to complete the mentor survey in the Fall of 2020.

Instruments

Survey items from multiple survey instruments were either administered as separate surveys for the first two cohorts of CTS Teams (2016 and 2017) or combined into a single survey for the third cohort (2018). In the combined survey, most measures were adjusted to a 7-point Likert scale. Survey descriptions are summarized in Table 2, including the Clinical Research Appraisal Inventory (CRAI) [10], Cross-Disciplinary Collaborative Activities Scale (CDCA) [11], Research Orientation Scale (ROS) [11], Grit Scale [12], Satisfaction With Life Scale [13], Overall Job Satisfaction [14], and Work Preference Inventory [15]. In addition, a single item was used to assess career satisfaction (Rubio, personal communication). Survey items are listed in Supplemental Table S1. Independent samples t-tests were used to compare results at the beginning and end of the training periods [16].

Table 2.

Metrics used for CTS team program evaluation

Measure	Characteristic assessed	# Items [Reference]	Results
Clinical Research Appraisal Inventory (CRAI)	Self-efficacy for clinical research tasks in six factor domains: data collection & analysis; reporting, interpreting, & presenting results; conceptualizing studies & collaborating; planning; funding; and protecting human subjects	12 [10]	*
Cross-Disciplinary Collaborative Activities Scale (CDCA)	Frequency of engagement in cross-disciplinary activities, such as reading journals and attending conferences outside of one’s primary scientific discipline	6 [11]	*
Research Orientation Scale (ROS)	Propensity to endorse a unidisciplinary vs. cross-disciplinary research perspective and attitude, with sub-indices measuring orientation toward uni-, multi-, and trans-/inter-disciplinary research	10 [11]	**
Grit Scale	Perseverance and passion for long-term goals	8 [12]	nc
Satisfaction With Life Scale	Intrinsic factors for career training success	5 [13]	nc
Overall Job Satisfaction	Overall job satisfaction	1 [14]	nc
Work Preference Inventory	Intrinsic motivation (e.g., self-determination, curiosity, enjoyment, interest) and extrinsic motivation (e.g., concerns about competition, evaluation, recognition)	10 [15]	nc
Career Satisfaction	Overall career satisfaction	1	nc

Significant pre-/post-training changes observed in Clinical and Translational Science (CTS) Team trainees; see Results section for details.

Changes observed, but not statistically significant; see Results section for details.

nc, No changes observed.

The CDCA instrument was also used to collect data from CTS Team co-mentors. A paired samples t-test was conducted to compare pretest and post-test scores in mentor CDCA before and after participating in the TL1 training program. Post-then-pre-design was used to collect valid data and protect from possible overestimation in self-reported responses [17]. Co-mentors were also asked questions about past collaborative relationships and outputs with co-mentors, their use of team science tools, and plans for future collaboration, as described below.

Results

CTS Team Training Approach

CTS teams

Prior to implementation of the CTS Team training model in 2016, the UF TL1 program provided up to two years of support and CTS training experiences to thirty-four individual pre-doctoral TL1 trainees (68% women, 32% men). They were selected from a total of 83 applicants (41% accept rate), with an average of 10.4 individual applications per year (Table 1). Selection criteria included research fit for CTS, past research performance, and research career goals. Accepted individual TL1 trainees included 31 PhD students (five with previous professional degrees) and three MD-PhD students (9% of trainees) in the PhD phase of their training, representing 13 PhD programs in seven different colleges (Fig. 2A). More than half of the trainees (19/34, 56%) were affiliated with the Biomedical Sciences program in the College of Medicine (MED), and five were in Public Health & Health Professions (PHHP) and Engineering (ENG), with one each from four other colleges. The average PhD time-to-degree for all individual TL1 trainees was 5.1 ± 1.0 years, which is comparable to other PhD students in their home graduate programs (data not shown), and indicates that additional CTS course requirements did not lengthen time-to-degree.

Fig. 2.

Clinical and Translational Science (CTS) Team members were more diverse by scientific discipline than individual TL1 trainees. A, Home colleges of individual TL1 trainees, 2009-2016. B, Home colleges of CTS Team members, 2016-2020. C, Social network analysis of CTS Team home colleges, drawn using NodeXL [18]. Nodes represent colleges, and lines (edges) represent CTS Teams that connect the colleges. Line thickness is proportional to number of teams with each home college pairing (edge weight). Abbreviations used for University of Florida colleges: AGR, Agriculture & Life Sciences; ENG, Engineering; HHP, Health & Human Performance; JC, Journalism & Communications; LAS, Liberal Arts & Sciences; MED, Medicine; NUR, Nursing; PHHP, Public Health & Health Professions; PHM, Pharmacy; VET, Veterinary Medicine.

Beginning in 2016, the UF TL1 program began up to two years of support and training in both CTS and team science to 32 pre-doctoral trainees in 16 CTS Teams (69% women, 31% men). They were selected from a total of 55 applicants (58% accept rate), with an average of 11 CTS Team member applications per year (Table 1). Selection criteria included research fit for CTS, team research aims, and collaborative research strategy. Accepted CTS Team members included 25 PhD students (one with previous professional degree) and seven MD-PhD students (22% of trainees) in the PhD phase of their training. Compared to individual TL1 trainees, the 32 CTS Team members represented a greater diversity of scientific disciplines, including 18 PhD programs in nine different colleges (Fig. 2B). The colleges of MED, PHHP, ENG, and Liberal Arts & Sciences (LAS) were represented by 5–8 trainees each (16–25%), with 1–2 trainees (3–6%) from each of four other colleges.

CTS Teams created a network of cross-disciplinary collaborations among the nine colleges, as illustrated by the social network analysis in Fig. 2C. Four teams paired TL1 trainees in MED and ENG, with two teams each in MED and LAS and in PHHP and LAS, and eight other unique pairings. CTS Team graduate program affiliations and team research topics are shown in Supplemental Table S2. Less than one-third of CTS Team members have completed the PhD degree to date, but preliminary results suggest that PhD time-to-degree was not extended by additional CTS course requirements and team-specific aims (data not shown).

Translational research phases

Using the translational stages described by the National Center for Advancing Translational Science (NCATS) [19], individual TL1 trainees and CTS Teams have performed and are performing research throughout the continuum of clinical and translational science, including preclinical research (T0), clinical research (T1-T2), clinical implementation (T3) and population or public health (T4). Using the rubric of Surkis et al. [20], individual TL1 trainee and CTS Team projects were classified for their translational phase, as summarized in Fig. 3. Although the differences were not statistically significant (p = 0.07, Fisher’s exact test), overall the CTS Team research projects were more diverse, with fewer preclinical (T0) projects than individual TL1 trainees had (44% vs 65%), about the same proportion of clinical (T1-T2) research projects (25% vs 21%), and more clinical implementation and public health (T3-T4) research projects (31% vs 15%).

Fig. 3.

Clinical and Translational Science (CTS) Team research projects were more diverse by translational research phase than projects by individual TL1 trainees. Percentages are shown for the translational research phases of research projects conducted by 34 individual TL1 trainees (blue) and 16 CTS Teams (orange). Counts are indicated by numbers above the bars.

Changes in Trainee Behaviors and Attitudes

All eighteen CTS Team members from the 2016, 2017, and 2018 cohorts responded to the surveys at the beginning and end of the two-year TL1 training period (100% response rate). An independent samples t-test was conducted to compare clinical translational research efficacy using the 12-item CRAI at the beginning and upon completion of the two-year CTS Team funding period. There was a significant difference in the pre-training scores (M = 4.60, SD = 0.86) and post-training scores (M = 5.36, SD = 0.96); t (63) = 3.44, p = 0.001, supporting the hypothesis that participation in a two-year translational team science training program contributes to the development of clinical research self-efficacy. Similarly, the CDCA measure revealed that the frequency of participation in cross-disciplinary collaborative activities significantly increased from the beginning of the program (M = 4.20, SD = 1.10) to the end of the two-year funding period (M = 4.65, SD = 1.056); t (63) = 1.71 at the statistically different level of p = .047.

As summarized in Table 2, no significant changes were observed in CTS Team members over the two-year training period using the following survey instruments (data not shown): ROS); Grit Scale; Satisfaction With Life Scale; Work Preference Inventory; and other items related to job and career satisfaction. The ROS revealed significant shifts from unidisciplinary to multidisciplinary or inter/transdisciplinary in individual TL1 trainee cohorts (data not shown), but the only significant difference observed with the pooled data for multiple cohorts was an increase in inter-disciplinary orientation from pre-training (M = 5.13, SD = 1.06) to post-training (M = 5.56, SD = 0.93); t (55) = 1.52 that approached statistical significance at p = 0.075.

Changes in Mentor Behaviors and Attitudes

CTS team co-mentor characteristics

Thirty-one different faculty members served as co-mentors for 16 CTS Teams from 2016 to 2020, as two faculty members have mentored two CTS Team members since 2016 and one team member had two mentors participating in the CTS Team. The co-mentors for CTS Teams included 16% (5/31) assistant professors, 45% (14/31) associate professors, and 39% (12/31) professors, with 45% women and 55% men.

Twenty-six of the 31 CTS Team co-mentors responded to the mentor survey (84% response rate). The survey respondents included 19% (5/26) assistant professors, 42% (11/26) associate professors, and 38% (10/26) professors, with 53% women and 47% men. Most CTS Team co-mentors (20/26, 77%) included clinical and/or translational in the way they described their research, and only six (23%) characterized their research as lab-based basic research or theoretical only.

The CDCA measure was used to compare cross-disciplinary collaborative activities of the CTS Team co-mentors before and after participating in the TL1 training program. There was a significant difference between pretest (M = 4.64, SD = 0.93) and post-test scores (M = 4.95, SD = 0.86); t (18) = 2.916, p = 0.009. These results support the hypothesis that faculty members also engage in more cross-disciplinary activities over the time of their participation as co-mentors for the cross-disciplinary TL1 Teams.

Co-mentor collaboration history and plans

All CTS Team co-mentors were asked about their collaboration relationships with each other prior to the formation of the CTS Team by their trainees. As shown in Fig. 4A, 71% of the 28 CTS Team co-mentor respondents had not previously collaborated. Those who had previously collaborated identified coauthored publications, funded grant applications, submitted but not funded grant applications, and a patent application as collaborative products.

Fig. 4.

More Clinical and Translational Science (CTS) Team co-mentors plan future collaborations than had previously collaborated. A, Previous collaborative relationships between CTS Team co-mentors (N = 28). B, Plans for future collaboration between CTS Team co-mentors (N = 19). Descriptive responses were categorized as “No” (orange) or “Yes” (blue).

CTS Team co-mentors for trainees who had completed at least one year of TL1 training were asked about their future plans to collaborate with each other. Seventy-four percent of the respondents planned to continue collaborating with their CTS Team co-mentor beyond the 2-year TL1 funding period for their TL1 trainees, including submission of a new NIH grant proposal (Fig. 4B). Most (85%) used or planned to use collaboration tools such as those used by TL1 trainees as a result of the Team Science course, for example, written collaboration plans, authorship agreements.

Paired responses to the questions about previous and planned future collaborations were available for 19 CTS Team co-mentors. Of the 13 respondents who reported no previous collaboration, eight (62%) had plans to continue collaborating. Of the remaining six CTS co-mentors who reported previous collaboration, all six (100%) planned to continue collaborating.

Discussion

The goals of our investigation were to evaluate the implementation of the CTS Team training model and to assess changes in behaviors and attitudes of both TL1 trainees and mentors. CTS Teams provide a unique clinical and translational science team training experience, supported by didactics via an applied Team Science course, and practical experience performing cross-disciplinary research that the trainees themselves design and implement with the guidance of co-mentors from different disciplines. Trainees take ownership of the CTS Team projects, because the collaborative research becomes embedded into their overall PhD training experience. The most distinguishing characteristic of the CTS Team approach is the application of team science principles to doctoral dissertation research via authentic cross-disciplinary collaboration. Echoing Norman and Lotrecchiano [21], we view the development of this experiential team science training program as a unique educational environment that was made possible with the foundational infrastructure of the Clinical Translational Science Award and TL1 training grant, although we note that implementation of such a team training environment does not absolutely require training grant support. We also show that the impact of this team training program extends beyond the traditional health science disciplines and beyond its immediate training objectives. Most importantly, CTS Teams provide essential opportunities to develop and practice clinical and translational research skills in a cross-disciplinary team science environment.

Overall, the CTS Team training model resulted in the establishment of trainee cohorts that were much more diverse in terms of scientific disciplines, as indicated by the trainees’ home colleges (Fig. 2), and the phases of translational research in which the trainees engaged in their doctoral dissertation research (Fig. 3). The team training model also attracted more interest in the TL1 program among MD-PhD students. Preliminary results from qualitative analysis of trainee feedback (data not shown) suggest high levels of satisfaction with the team training approach.

We sought to identify measures that reflect changes in behaviors and attitudes during the relatively short timeframe of the training program (i.e., two years) that may be correlated with future measures of success, such as publications and career outcomes. As suggested by Lee et al. [7], we wished to go beyond the usual measures of success in training such as academic performance and research progress as determined by the numbers of research abstracts presented at conferences and subsequent peer-reviewed publications, which sometimes take years to be produced. Personal factors such as the personality traits and intrinsic success factors are often overlooked in training evaluation plans, but according to social cognitive career theory, the major factors that influence career choices and decisions are self-efficacy and outcome expectations [22]. The confidence that trainees have in their ability to perform research-related tasks not only is related to decisions to pursue a research career but may also predict research productivity [23]. Based on a model for measuring physician-scientist career success [24], we recognized that domains of CTS training success may include extrinsic success factors (e.g., earned degrees, job placement, grant funding, publications) and intrinsic success factors (e.g., job, career and life satisfaction).

Our results revealed at least two metrics that detect significant changes in trainees during the TL1 training period. Firstly, CTS training in the context of CTS Teams effectively supported improved self-efficacy for clinical research skills, as measured by the short form of the CRAI. The value of CRAI for assessing CTS training is supported by numerous other reports of its use to assess training of medical students [25,26], MD-PhD students [27], nursing graduate students [28], CTSA KL2 Scholars [29], and early-stage investigators in the NIH PRograms to Increase Diversity among Individuals Engaged in Health-related Research (PRIDE) [30].

Secondly, the team science orientation of the UF TL1 program supported a significant increase in the frequency of participation in cross-disciplinary collaborative activities (CDCA) by both the TL1 trainees and their mentors. Although not statistically significant for pooled TL1 trainee cohorts, the ROS did reveal an increase in interdisciplinary orientation, suggesting that this metric may still prove to be useful. Preliminary data (unpublished) suggest that TL1 trainees start at a lower level of unidisciplinary orientation at the beginning of their training than T32 trainees. The ROS has been used as part of training assessment in Transdisciplinary Research in Energetics and Cancer (TREC) programs [31] and in social network analysis of centers for population health and health disparities [32].

All of the other metrics that we tested from those suggested by Lee et al. [7] were not able to detect changes over the two-year TL1 training period (Table 1). CTS Team training may simply not be significantly different enough from traditional research training to impact characteristics such as perseverance, satisfaction, and motivation. Although some of these measures might detect changes after a longer period of time and/or more extended training, they were discontinued as part of our program assessment. Additional metrics focusing on team behaviors are now being assessed with CTS Teams. Providing an evidence base for the use of various metrics is important to avoid survey fatigue in trainees [33]. It is important to streamline program assessments for CTS training by minimizing the number of surveys used, increasing survey participation, and allowing capacity for other evaluation methods, in order to provide valid assessments of training outcomes. This is especially challenging during graduate education, when career outcomes and research productivity are usually several years in the future.

The training outcomes for trainees were hypothesized in the program logic model we established for our TL1 program evaluation (Fig. 1). What we did not anticipate was the degree to which new collaborative relationships and the academic network would develop among CTS Team co-mentors (Fig. 2C). The training program influenced mentors to engage more frequently in cross-disciplinary collaborative activities. More importantly, the program led to the intention of continued collaboration after TL1 trainees graduate (Fig. 4) and has led to collaborative behaviors supporting coauthored research papers, a patent application, and NIH grant proposals. Preliminary results from qualitative analysis of additional survey questions (data not shown) suggest high satisfaction among mentors with the team training approach. Taken together, these results indicate very positive ripple effects of this pre-doctoral team science training programs on the collaboration network at our institution. Thus, the CTS Team training model is serving as a team science intervention.

The findings reported here contribute to the growing scholarship on CTS research training and workforce development in the CTSA program and meet the third and fourth recommendations by Begg et al. [6] for preparing trainees for team science: CTS Teams represent a new method of instruction to support the development of team science competencies, and assessment metrics have been identified that will be useful for other CTSA programs to gauge success of their trainees in cross-disciplinary team science, as well as some proposed metrics that appear not to be useful. We note that progress is being made by the CTSA community on the other two recommendations by Begg et al. [6], including the recent definition of translation team competencies by Lotrecchiano et al. [34], and ongoing development of strategies for the sharing of team science curricular elements by members of the International Network for the Science of Team Science (INSciTS) Team Science Education and Training Special Interest Group [4]. Further development of assessment tools for team science is an area of intense investigation by ourselves and by working groups of the CTSA Workforce Development Enterprise Committee and INSciTS, for example, the development of competency-based assessments based on a recently proposed model for assessing pre-doctoral and postdoctoral science training [35].

We employed validated instruments and predetermined timepoints to collect survey data. However, the generalizability of the results reported in this paper is limited by the small sample size that stems from the overall program design. To address this limitation, our analyses were guided by relevant statistical considerations [16]. Another limitation of the results presented here is the availability of data for between-group comparisons. We are in the process of collecting data from trainees funded by individual research grants and other training programs (e.g., T32), and will include between-group data in future reports.

This study provides a fertile ground for future research, some of which we have already begun. We recognize the need to develop evaluation protocols and capture the data treating team members and their mentors as the unit of analysis. Our work in competency assessment [35] and cultural consensus analysis [36] incorporates this perspective. We also recognize the need to incorporate mentor-level outcomes and relevant metrics as part of the program logic model. We will continue to use social network analysis measures and will integrate other team observation and behavioral intention instruments. We are also engaged in active collaboration with other TL1 sites in sharing and discussing evaluation protocols to create a community of practice and pave the way for harmonized multi-site evaluation designs.

In summary, CTS Team training creates a new cross-disciplinary community of practice with connections beyond those of disconnected co-mentoring and asynchronous manuscript co-authorship. The disruption to existing processes that the TL1 training program brought about spanned the boundaries of established programs and labs, and intervened with both the flow of the research training and the research process on mentors’ labs. This has created new cross-disciplinary bridges connecting research groups across different colleges and has strengthened existing research collaborative relationships. Although CTS training is the primary focus of TL1 training programs, they may have broader ripple effects expanding social and semantic networks of participating mentors and lead to tangible research outputs such as joint research protocols, publications, and grants. The CTS Team training approach answers the call to action by Begg et al. [6] to develop new models and methods for clinical and translational workforce development during graduate education. We note that this approach is well suited for other graduate education programs designed to train future researchers who will need to use interprofessional and cross-disciplinary team approaches to solve increasingly complex health and societal problems.