Measuring Team Effectiveness in the Health Care Setting: An Inventory of Survey Tools.

BACKGROUND: Guidance for measuring team effectiveness in dynamic clinical settings is necessary; however, there are no consensus strategies to help health care organizations achieve optimal teamwork. This systematic review aims to identify validated survey instruments of team effectiveness by clinical settings.
METHODS: PubMed, MEDLINE, and ISI Web of Knowledge were searched for team effectiveness surveys deployed from 1990 to 2016. Validity and reliability were evaluated using 4 psychometric properties: interrater agreement, internal consistency, content validity, and structural integrity. Two conceptual frameworks, the Donabedian model and the Command Team Effectiveness model, assess conceptual dimensions most measured in each health care setting.
RESULTS: The 22 articles focused on surgical, primary care, and other health care settings. Few instruments report the required psychometric properties or feature non-self-reported outcomes. The major conceptual dimensions measured in the survey instruments differed across settings. Team cohesion and overall perceived team effectiveness can be found in all the team effectiveness measurement tools regardless of the health care setting. We found that surgical settings have distinctive conditions for measuring team effectiveness relative to primary or ambulatory care. DISCUSSION: Further development of setting-specific team effectiveness measurement tools can help further enhance continuous quality improvements and clinical outcomes in the future.

Background

Today, team-based care has become a key component of many transformations in health care delivery and emerging models of value-based care.[1] The complexity of health care services, including a continuing trend toward value-based care and pay for performance, has elevated the importance of team-based care in the deliverance of health services.

Previous research indicates that higher team effectiveness is associated with better health outcomes.[2-4] The impact of high-functioning teams on quality of care, worker satisfaction, and cost of care can be substantial when it comes to surgical care,[5] intensive care,[6] ambulatory care,[7] and primary care managing patient populations with chronic conditions.[8] Despite this knowledge and growing awareness of the importance of teamwork among health care leaders, there are no consensus strategies to help health care organizations achieve optimal teamwork.[9] One of the first steps in achieving optimal team performance is the ability to measure, track, and influence team effectiveness. Therefore, systematic reviews of survey instruments measuring team effectiveness across health care settings provide an important first step.[9-11]

Among the previous systematic reviews on survey instruments, Valentine et al[11] inventoried and described a list of teamwork survey tools used in health care settings. Their findings indicate what dimensions of teamwork have been assessed, along with the psychometric validity of each survey. However, this study did not specify which health care setting each measurement tool and article has addressed. Other review studies on team effectiveness did not identify which survey tools should be applied in which type of setting.[9,10] We are left with the question of which conceptual dimensions are most relevant to various types of health care settings and what survey instruments are most often deployed in these settings. The complexity and dynamic nature of these settings create different conditions for teamwork; therefore, understanding these specific requirements is key in developing measurement tools. There is an apparent need for guidance on understanding the contextual nature of teamwork skills and performance for different settings, including surgery, intensive care, emergency medicine, and ambulatory care settings.

This study seeks to identify validated survey instruments by clinical settings. This objective renews and complements findings from the study by Valentine et al,[11] building on psychometric properties and concepts used in available survey instruments. Our evaluation of survey instruments is supported by 2 conceptual frameworks: one rooted in outcomes research from the field of health services research[12] and the other in team theory and organizational psychology.[13] By identifying which content domains were assessed and how the domains differ by team environment, the findings of this study can assist in the development of more specialized team member training and operational design interventions directed to the most appropriate team composition, team member tasks, and responsibilities by clinical settings. The findings will also benefit practitioners who wish to ascertain the usefulness and relevance of a particular tool for their care teams in various health care settings.

Methods

Conceptual framework

Teams are defined as “two or more people with different tasks who work together adaptively to achieve specified and shared goals.”[14](p4) Compared with teams in other industries, health care teams have more dynamic work conditions that change frequently, have to change team membership in a short-term period, have various specialized members, and have interprofessional and even multidisciplinary cultures.[9] These unique conditions vary across health care settings. Teams in operating rooms and emergency medicine are more likely to experience changes in team memberships that may be assembled ad hoc, whereas teams in primary care can have more diverse team compositions which include physicians, nurse practitioners, medical assistants, and receptionists, compared with surgical teams.[9,15] Such different work conditions and diverse team compositions require integrating multiple conceptual frameworks for evaluating team effectiveness.

We employed 2 conceptual models as frameworks for evaluating the contextual nature of team effectiveness across different settings—(1) the Donabedian[12] model on quality of care and (2) the Command Team Effectiveness[16] (CTEF) model. Both conceptual frameworks highlight outcome domains in the streams of care that fit our focus on team effectiveness. Donabedian contends that it is important to identify essential elements constituting quality of care based on structure, process, and outcome. Structure refers to the attributes of the material sources, human resource, and organizational structure. Process denotes the actual activities in giving and receiving care, including patients’ activities and practitioners’ activities. Outcome refers to the impact of care on the patients’ health status. Since then, many studies have confirmed that structure, process, and outcome should be evaluated together when considering quality of health care.[17-19] Therefore, we assessed the relationship among the 3 subdimensions and investigated how survey instruments were used to measure these concepts across health care settings. This comprehensive framework also aids in determining which conceptual dimensions have been understudied when measuring team effectiveness.

In addition to the Donabedian model, we applied the CTEF model to refine the framework of analysis to be more relevant to surgical teams, which have the unique attributes of action teams.[20,21] According to organizational psychological literature, “action teams” refers to the teams, such as emergency medical, surgical teams, air crews, and military command and control, that require specialized professionals to collaborate in the context of high-acuity, complex tasks, ad hoc team compositions, and time-pressured conditions.[22,23] As our focus is identifying validated survey instruments according to the attributes of teams, the CTEF model allows us to closely investigate key subcomponents of team effectiveness in surgical settings according to their unique team characteristics.[24]

In the CTEF model, conditions include dimensions of mission framework, task, organizational characteristics, leadership, and the characteristics of team members. These dimensions are addressed in the generic team effectiveness frameworks in health care as well,[20,25] but the CTEF model measures outcomes based on 2 categories—task outcomes and team outcomes. Task-related outcomes include time-error costs, task accomplishment quality, accuracy, timeliness, and error rate, whereas team-related outcomes measure team satisfaction, team norms, roles, communication patterns, motivation, attitudes, emotional tone, and turnover. By differentiating task-related and team-related outcomes, the CTEF model provides a more specific criterion for the evaluation of Donabedian structure, process, and outcomes relevant to action teams in surgical settings.

Data collection

We conducted a systematic review of the literature, searching for survey instruments measuring team effectiveness. We conceptualized 3 relevant dimensions—survey instruments, clinical setting, and team effectiveness—as search points. Compared with Valentine et al,[11] we expanded the scope of search dimensions and database to investigate team effectiveness survey tools across different health care settings. Next, we identified key terms for each dimension. For the survey instrument dimension, we selected “survey,” “evaluation,” “instrument,” “assessment,” and “questionnaire” as the search terms. For the dimension of the clinical setting, we included key terms such as “clinical,” “health care,” and “surgical” to limit our search to only the health care domains. Because our focus is team effectiveness itself, we included the key term “team effectiveness” for the last dimension. The focus of this study was to identify and evaluate survey instruments of team effectiveness related to health outcomes; the general studies on team(s) or teamwork(s) without outcome domains were excluded.

We used 3 databases for this systematic review of the literature: PubMed, MEDLINE via OVID, and ISI Web of Knowledge. We included articles using the selected key terms from January 1990 to October 2016. In every search step, we combined key terms in the 3 dimensions using AND operator (eg, “survey” AND “clinical” AND “team effectiveness”). We limited our search to titles and abstracts. This search strategy identified articles in the overlapping areas of the 3 dimensions. The first round of searching produced 646 articles of interest. After deleting duplicates (492), a total of 191 articles were included in our final review.

Next, we conducted an abstract review to select those articles that matched our predefined inclusion criteria: (1) articles should be peer-reviewed, (2) articles should be empirical studies on teams, (3) the studies should demonstrate the use of survey instruments, and (4) the survey instrument should include a team effectiveness measure. In all, 22 articles met the criteria for full-text review. Researchers extracted information from the selected articles focusing on team compositions, types of settings, types of surveys, dimensions of team effectiveness, and significant impact on non-self-reporting outcomes (if applicable). In the final step, researchers evaluated reliability and validity of survey tools for each article. Figure 1 summarizes the research approach and search steps in a flowchart using the PRISMA methodology.[26]

Figure 1.

PRISMA flowchart for systematic review.

Analysis strategy

Following Donabedian and the CTEF model, we first categorized what type of questions researchers asked health care team members based on structure, process, and outcome. Then, we identified the setting to describe organizational preconditions of team effectiveness. Two of the researchers independently reviewed items in each survey tool identified in the literature to categorize based on these criteria, following terminologies in Mathieu et al.[25] We then qualitatively assessed the subcategories based on survey items that describe consistent terminologies.

To evaluate the reliability and validity of each survey instrument, we used 4 psychometric properties: interrater agreement (IRA), internal consistency, content validity, and structural integrity. These 4 properties have been used by other researchers to assess how accurately a survey instrument captures what it is intended to measure.[11,27,28]

Interrater agreement and interrater reliability (IRR) measure whether different raters provide similar or identical reports when faced with same survey instruments. Particularly, when assessing survey instruments of teamwork including multiple groups of professionals, researchers should report both IRA and IRR to justify the aggregated scores at the group levels.[29] When a team has different groups of professionals, such as physicians, nurses, specialists, or administrative staff, IRA and IRR indicate whether a survey produces similar responses on the conceptual dimensions across the groups of participants. Interrater agreement is measured by the r index,[30] which ranges from 0 to 1, where .7 is used as the minimum threshold for an acceptable value.[30,31] Interrater reliability is measured by an intraclass correlation coefficient (ICC) with a range of −1 and 1, where values of ICC should be greater than 0 as a threshold for acceptable similarity.[32] Interrater agreement and interrater reliability are the key properties in a survey measuring team effectiveness because the survey aims to assess team members’ behaviors and achievement as a group. When IRA and IRR produce satisfactory values, it assures the reliability of the survey instrument to measure a team of individuals.

Internal consistency indicates the degree to which survey items are correlated to each other. When items are strongly correlated, it assures that the survey reflects similar concepts across items. Cronbach α is most commonly used to measure internal consistency, ranged from negative infinity to 1, with values greater than .7 defined as acceptable consistency across items.[33]

Content validity refers to whether a survey accurately measures the substantive meanings of the conceptual dimensions of interest. To test content validity of a survey, triangulation is highly recommended, which requires the use of different methodologies other than a survey, such as interview, qualitative field research, expert-reviewed survey, formal pretest, or pilot survey, to measure the same conceptual dimensions of interest. When developing survey instruments of team effectiveness in different settings, researchers also conduct systematic reviews to develop items and apply previously validated scales. This is the only measure of validity in this study that reflects whether a survey captures the true dimensions that are of real-world interest.

Structural integrity refers to the extent to which survey items are clustered with a high covariance. When a survey aims to measure single conceptual dimensions, all survey items should be constructed in one dimension as expected. Structural integrity reveals the number of conceptual dimensions by assessing covariance among survey items and provides the evidence of construct dimensionality. Exploratory and confirmatory factor analyses provide the percentage of variance that can be explained by the constructed factors. When the factor loading value is greater than 0.40 and eigenvalue is greater than 1.0, the structural integrity of the survey is acceptable.[34,35]

Ethical considerations

Human subjects were not involved in this systematic review. Ethics review and study registration do not apply. The data sets used during the study are available on reasonable request to the corresponding author.

Results

We analyzed 22 articles with survey instruments measuring team effectiveness. The articles address a variety of clinical settings, types of respondents, variables of interest, and team compositions as described in Table 1. Of the 22 articles, 19 articles administered survey questions to team members in formal/informal clinical teams or potential team members (eg, health care faculties who train clinical professions), whereas 3 articles used third-party experts[52,53] and patients[46] as respondents who assessed the clinical team effectiveness.

Table 1.

Description of clinical setting and team composition.

Article characteristics					Team composition
Author(s), year	Settings	Types of respondents	Role of team effectiveness	Key concepts	MD	Nurses	Other
Surgical setting
La Sala et al, 2015[36]	Intensive care	Team members	DV	Team functioning	x	x	Health care operators
Andrew et al, 2012[37]	Operating room	Team members	IV	Teamwork, team communication, team effectiveness, quality expectations, possession of relevant skills	x
Hamilton et al, 2009[38]	Trauma resuscitations	Third-party experts	DV	Team skills during trauma resuscitation			Postgraduate year—2 surgical trainees
Michinov et al, 2008[23]	Anesthesia care	Team members	D	Transactive memory, perceived team effectiveness, job satisfaction	x	x
Simon and Stewart, 2007[39]	Operating room	Team members	DV	Stress, deference, confidence, leadership, communication, teamworking, preferred leadership style and subjective experience of teamworking	x	x	Consultant, sister, health care assistant, midwife, physiotherapists, matron/ward manager, operating dept. practitioner, and administrator
Davenport et al, 2007[3]	General/vascular surgery	Team members	IV	Emotional exhaustion; safety, teamwork, working conditions, recognition of stress effects, perception of management, job satisfaction	x	x	Physician assistants, nurse practitioners, surgical residents or fellows, anesthesia residents or fellows, certified registered nurse anesthetists, staff nurses
Primary care setting
Landry and Erwin, 2015[40]	Primary care	Team members	D	Team participation, team processes, team processes			Health care executives
Song et al, 2015[41]	Primary care	Team members	D	Team effectiveness, shared understanding, perceived collective identity, perceived team effectiveness	x	x
Beaulieu et al, 2014[42]	Primary care	Team members	DV	Team functioning	x	x	Social workers
Helfrich et al, 2014[19]	Primary care	Team members	IV	Team structure, team process, team effectiveness	x	x	Teamlets; physician assistant, nurse care managers, administrative clerks
Becker and Roblin, 2008[43]	Primary care	Team members	IV	Practice climate	x	x	Support staff
Other setting
Keebler et al, 2014[44]	US Army Medical Center	Team members	D	TeamSTEPPS (leadership, communication, situation monitoring, mutual support)	x	x	1700 multidisciplinary health care professions and support staff from the US Army medical facilities
El Ansari et al, 2016[45]	Mental Health	Team members	D	Community mental health teams (CMHTs) effectiveness	x	x	Administrative and clerical staff, social workers/mental health setting
Tremblay et al, 2015[46]	Ambulatory Cancer Clinic	Patients	DV	Patient Perceptions of the prompt access to care, person-centered response, quality of patient-provider communication, and quality of care environment	x	x	Navigators in oncology/ambulatory cancer clinic
Strasser et al, 2014[47]	VA Medical Services	Team members	IV	Team functioning	x	x	Occupational therapy, physical therapy, social workers/case management, speech-language pathologists/medical services at VA hospitals
Dounis et al, 2013[48]	Diabetes training program	Team members	DV	Perceived interprofessional team effectiveness	x	x	Health care faculties from 6 different disciplines; medicine (MD), nursing (NP, RN), dentistry, therapy, pharmacy/type 2 diabetes training program
Baker et al, 2011[49]	4 general hospitals	Team members	D	Challenging the process, enabling others to act, embracing change, doing the job, working with other			Health care workers (no specific description)/general hospital
Strasser et al, 2010[47]	Rehabilitation services at VA hospitals	Team members	D	Physician support, shared leadership, supervisor team support, teamness, team effectiveness	x	x	Occupational therapy, physical therapy, speech-language pathology, and social work or case management/rehabilitation services at VA hospitals
Strating and Nieboer, 2009[50]	Home care	Team members	D	Team climate		x	Management staff/home care
Shortell et al, 2004[4]	Long-term care	Team members	DV	Perceived team effectiveness	x	x	Individuals on the breakthrough series (BTS) intervention teams/chronic illness program
Temkin-Greener et al, 2004[51]	Chronic Illness Program	Team members	DV	Interdisciplinary team process, team effectiveness	x	x	Social workers, therapists, dietitians, aides, drivers, coordinators/long-term care
Hyer et al, 2003[52]	Geriatric health care teams	Third-party experts	D	Team dynamics (team process and skills in addressing conflict)	x	x	Social workers, pharmacists/geriatric health care

Abbreviations: D, development of a survey; DV, dependent variable; IV, independent variable; MD, medical doctor or physician.

We investigated how each article used team effectiveness as a variable of interest. Of the 22 articles, 9 articles focused on the development of the instrument and performed validity and reliability testing. As Table 1 indicates, those articles assessed dimensions of team effectiveness and analyzed the relationship among subdimensions of teamwork in terms of team functions, conditions of teamwork, leadership, and team effectiveness. As those articles mainly evaluated reliability and validity of newly developed survey questionnaires, most of the articles reported psychometric properties. Of the 22 articles, 5 articles measured team effectiveness as antecedents of health care outcomes, including task performance, patient trust, patient-centered care improvement, patient discharge rates, and length of stay. Eight articles were interested in measuring team effectiveness as consequences of care management, such as team training program, a simulation-setting training program, team functioning, team attitudes, team communication, and team quality.

Six articles focused on surgical settings, including trauma resuscitations, intensive care, anesthesia care, operating room setting, and general/vascular surgery. Five articles mainly discussed team effectiveness in primary care settings. The other 11 articles had a variety of clinical settings, such as a diabetes training program, a mental health hospital, a geriatric health care setting, chronic illness programs, Veterans Affairs (VA) hospitals, home care, long-term care, and ambulatory care settings. As team dynamics on team composition, functioning, structure, and process can vary depending on the clinical setting, we qualitatively analyzed which types of conceptual dimensions were most addressed in each type of clinical setting.

Team composition is key to measuring team effectiveness in the health care setting. We investigated what types of health care professionals were involved in teamwork. As Table 1 indicates, of the 22 articles, 17 survey instruments included physicians and nurses. Other professions often included in a team are clinical specialists, social workers, administrative clerks, health care executives, consultants, midwives, occupational therapy, dietitians, anesthesia residents, and receptionists. Such diverse team compositions indicate that it is important to consider professions’ characteristics, norms, cultures, and functions in a team when measuring team effectiveness.

The psychometric properties—IRA/IRR, internal consistency, content validity, and structural integrity—indicate whether a survey is a valid and reliable measure of team effectiveness. As noted in Table 2, there were few studies that reported all 4 psychometric properties: of the 21 articles, only 4 articles (19%) reported all 4 psychometric properties. Interestingly, only 5 articles reported IRA/IRR score, and 1 of them had unacceptably low r, which indicates the need for more high-quality survey instruments with high validity and reliability. Compared with IRA/IRR, internal consistency was more frequently reported: 16 out of 21 articles reported Cronbach α values. When the studies aimed to develop a survey tool (or measure team effectiveness as a dependent variable), the internal consistency test was often conducted to show how the survey items were constructed. Content validity was reported in most of the studies: 19 studies report that their survey items were constructed through literature review, existing survey with validity tests, 3-phase qualitative study based on a formative evaluative approach, and expert interview. Most studies had adopted existing surveys from literature or modified survey items to fit into the health care domain. In terms of structural integrity, about half of the selected studies reported the factor loading value of their survey items, and 11 articles reported covariance among survey items with factor loading value and eigenvalue.

Table 2.

Psychometric properties of survey instruments on team effectiveness.

Author, year	Survey name	Survey description	Interrater agreement/reliability	Internal consistency (Cronbach α)	Content validity	Structural integrity	Non-self-reported outcomes
La Sala et al, 2015[36]	Index of Interprofessional Team Collaboration	14 items measuring team functioning, 6-point Likert scale	NR	.95	Literature review to select existing survey measures of Index of Interprofessional Team Collaboration	Two dimensions, reflection on the processes and interdependence roles, were summarized in a general factor analysis	NR
Andrew et al, 2012[37,a]	A newly developed survey based on the previously validated Comprehensive Assessment of Team Member Effectiveness (CATME)	87-item survey measuring 5 categories (teamwork, team communication, team effectiveness, quality expectations, and possession of relevant skills) to assess team skills with an LVH (Laparoscopic Ventral Hernia) simulator, 5-point Likert scale	Cross-group differences tested using ANOVA: significant differences found in communication (P < .01) and quality of work (P < .05), but other 5 items were consistent across groups	.811	Literature review to select survey instruments based on the research question. 6 items among 87 items were deemed appropriate for the LVH task	Significant interitem correlation among 6 items (r = .46-.86)	Survey measures were correlated with task performance on the LVH simulator
Hamilton et al, 2009[38]	A new prototype instrument using Mayo High Performance Teamwork Scale	7 items to identify whether 7 attributes relevant to team skills in the trauma context are present or not; researchers validated this prototype by assigning a normalized ranking score to each of the 10 scenarios based on the score provided by each experienced clinician; binary fashion	Interobserver agreement was poor; rwg = .64	NR; but researchers indicated that the original form of Mayo instrument performed poorly in their context	Literature review to select a relevant preexisting survey items; Mayo instruments were used for operationalizing principles of crises resource management, communication skills and containment of errors. The instrument was developed and tested in the context of trauma resuscitation. Researchers modified the instrument as 7 items to fit in the trauma context	NR; no factor analysis conducted. Researchers indicated that some items in Mayo instruments were seldom applicable to their context. The reason is unclear	NR
Michinov et al, 2008[23]	Survey including transactive memory system, perception of team effectiveness and work attitudes (Minnesota Job Satisfaction Questionnaire)	The survey included a set of dimensions of transactive memory (15 items, 5-point Likert scale), perceived team effectiveness (1 item, 10-point Likert scale), and job satisfaction (20 items, 5-point Likert scale)	NR	.74-.88	Literature review to select survey items: Lewis; Job satisfaction survey items derived from Minnesota Job Satisfaction Questionnaire)	NR	NR; only self-reported perceptions on team effectiveness, job satisfaction and team identification
Simon and Stewart, 2007[39]	An abbreviated Operating Room Management Questionnaire (ORMAQ)	30 questions measuring 8 themes; stress, deference, confidence, leadership, communication, teamworking, preferred leadership style and subjective experience of teamworking	NR	NR	Literature review and select instrument from Helmreich and Merritt’s ORMAQ, measuring subjects’ experience of stress and attitudes to leadership and teamworking	NR; researchers indicated that the instrument is highly valid and reliable by citing other works but did not report any psychometric properties	NR
Davenport et al, 2007[3]	Safety Attitudes Questionnaire	30 items grouped into 6 measures of OSCF and an additional factor of emotional exhaustion; safety, teamwork, working conditions, recognition of stress effects, perception of management, and job satisfaction	NR	.54-.83	Researchers adopted Safety Attitudes Question (SAQ) and measure emotional exhaustion using Maslach Burnout Inventory	NR	Risk-adjusted surgical morbidity and mortality; only communication with attending and resident doctors was correlated with risk-adjusted morbidity
Landry and Erwin, 2015[40]	Cross-sectional Survey for multidisciplinary teams	Respondents were asked to answer the questions of team participation and team processes; team processes had a set of 6 categories; binary fashion (advantage/disadvantage)	NR	NR	The survey items were developed based on the literature review on the leadership development and training as well as multidisciplinary team	NR	NR
Song et al, 2015[41,a]	Primary Care Team Dynamics Survey	31 items for measuring conditions for team effectiveness, shared understanding, 3 supportive processes perceived collective identity and perceived team effectiveness, 5-point Likert scale, full model	Full survey; rwg = .23-.75; ICC = .58-.81	.71-.91	Literature review to select 10 survey instruments based on conceptual model. A series of expert reviews and cognitive interview with attending physicians, nurses, and front desk staff were conducted to select 31 items	Two round SEM. Factor analysis generated 7 factors. FL > 0.40, GFI = 0.91	NR
Beaulieu et al, 2014[42,a]	Team Climate Inventory (TCI) Questionnaire (short version)	19-item measuring 4 team functioning scales: participative safety (5-point) support for innovation (5-point), vision (7-point), and task orientation (7-point scale)	Full survey; rwg = .72-.99	.88-.93	Use the previously validated survey instrument. Literature indicates that TCI questionnaire is among the few survey tools that has been validated and used in many countries and contexts	Exploratory factor analysis were performed: r = .61-.83	NR
Helfrich et al, 2014[19]	Cross-sectional survey for VA primary care in the 3 sets of measures—team structure, team process, and team effectiveness	Three sets of dimensions were covered in the survey items; structure questions about the formation of a 4-member teamlet, process questions about distribution of tasks among teamlet members and delegation of clinical activities, and effectiveness questions about assessing respondents’ collective efficacy	NR	NR	Literature review to select existing items (SOAP-C) and create new survey items; using Donabedian model and working with VA operational leaders, researchers constructed survey items based on structure, process, and effectiveness	NR	5 clinic-level respondent-reported improvements in team’ abilities to deliver patient-centered care; team process and effectiveness measures had stronger associations with improvement than structure measures
Becker and Roblin, 2008[43]	Practice Climate Survey	There were 2 survey instruments, one for practitioners (41 items, 24 related to practice climate) and the other for support staff (31 items, 17 related to practice climate). The instruments are similar, with additional items concerning patient interaction on the practitioner instrument. All items consist of Likert scales, with 4, 5, or 7 response levels per item	NR; but researchers calculated an average of the practitioner scores and an average of staff scores and then computed the average of these 2 averages to obtain the team’s score	.80-.90	NR	No specific test scores, but it indicated that the subcategory of practice climate survey were medical record availability, tame for tasks, delegation, patient focus, coordination, team ownership and autonomy	Patient trust in primary care physician (survey measure); significant and positive relationship exists between practice climate and patient trust
Keebler et al, 2014[44]	TeamSTEPPS (Team Strategies and Tools to Enhance Performance and Patient Safety) Teamwork Perceptions Questionnnaire (T-TPQ)	A self-reportSurvey including 35 items that examines multiple dimensions of perceptions of teamwork within various health care settings	NR	.97	Literature review of evidence based on teamwork, patient safety, and team training to select 5 dimensions: Leadership, Mutual Support, Situation Monitoring, Communication, and Team Structure	CFI = 0.94; RMSEA = 0.06	NR
El Ansari et al, 2016[45]	Community mental health teams (CMHTs) effectiveness survey	Through 3-phrase qualitative study and evaluating survey scales’ psychometric properties, 20 items demonstrated good measures of CMHTs, capturing 7 themes; 5-point Likert scale	ICC(2) = .57; rwg = .99	.91	Conducting 3-phase qualitative study based on a formative evaluative approach that employed an iterative process with various stakeholder; individual (N = 157) from the 11 Mental Health Trusts participated in 10 workshops to develop the survey items	Splitting sample at random into 2 halves: the first half was employed for exploratory factor analysis and the second half was employed for confirmatory factor analysis; all items factor loading >0.4; EV > 1, CFI > 0.90; Var Exp = 39.4%	NR
Tremblay et al, 2015[46]	Cancer Services Responsiveness tool (CSR); patient-reported survey	19-item questionnaire evaluating patients’ perceptions of the responsiveness of cancer services, focusing on prompt access to care, person-centered response, quality of patient-provider communication, and quality of care environment; 4-point Likert scale	NR	.64-.85	Researchers adopted the instrument from WHO’s generic responsiveness instrument. They translated into French and tested a validity of the adopted French version	NR	NR
Strasser et al, 2014	Team Functioning Survey, focusing on medicine, nursing, therapy, social work, and case management staff in teams	5 measures of team functioning are measured through staff surveys collected pre- and postintervention 1 year apart, t1 and t2; physician support, shared leadership, supervisor team support, teamness, and team effectiveness; 7-point Likert scale	NR	NR	Using Strasser et al (2010), researchers used valid survey instrument in t1 and t2, and then assess the association between changes in team functioning measures and patient outcomes	NR	Functional improvement, discharge destination, and length of stay were measured from VHA Functional Status Outcomes Database; Team functioning increased community discharge and decreased length of stay
Dounis et al, 2013[48]	Pre- and postresearch assessment surveys to evaluate the effectiveness of an interprofessional health care faculty training program	The research assessment survey consisted of 4 closed-ended items (multiple choice responses). 9 additional items were used to assess participant attitude and 5 items were used to measure perception about interprofessional teams	Test-retest was used on a small convenience sample to assess internal reliability; r = .83	.75	NR	NR	NR
Baker et al, 2011[49,a]	A developed survey using 2 previous surveys: (Leadership Practices Inventory-Self [LPI]) and Performance and Relationship Questionnaire (PRQ)	16 items measuring leadership and followership based on 5 categories: challenging the process, enabling others to act, embracing change, doing the job, and working with others: LPI (10-point Likert scale), PRQ (5-point Likert scale)	NR	.53-.80	Literature review to select items: Leadership (Leadership Practices Inventory-Self [LPI]), Followership (the Performance and Relationship Questionnaire [PRQ])	A confirmatory factor analysis were performed: 5-factor model fit better, and the root mean square error of approximation were indicative of acceptable model (GFI = 0.87)	NR
Strasser et al, 2010[47]	Team Functioning Survey, focusing on rehabilitation staff	60 items if team functioning with 7-point Likert scale; physician support, shared leadership, supervisor team support, teamness, and team effectiveness	NR	.86-.94	Literature review to modify a previous survey of Team Functioning Survey	IRT and factor analysis supported the proposed model: EV > 1, Var Exp = 68%	Measures of team functioning were related to patient outcomes of motor FIM gain and discharge disposition
Strating and Nieboer, 2009[50]	Team Climate Inventory (TCI), focusing on quality improvement teams	14-item Dutch version with sample at baseline (T0) and end measurement (T1)	NR	.73-.80	Researchers translated Team Climate Inventory (TCI) into Dutch. Comparison of the 2 translations revealed no salient differences. Perceived team effectiveness measures derived from Lemieux Charles’ measures	Confirmatory factor analysis proposed 4-factor structure—vision, participative safety, task orientation, and support for innovation; factor loading between 0.68 and 0.87. RMSEA = 0.03	NR
Shortell et al, 2004[4]	A survey of individuals on the breakthrough series (BTS) intervention team	The survey included a set of questions on team members’ assessments of organizational culture and of their organization’s commitment to quality improvement and perceived team effectiveness after the completion of the intervention	NR	.85-.95	Literature review to use the competing values framework for assessing organizational culture. Researchers also used Lemieux-Charles team effectiveness instrument to assess team effectiveness	NR	The actual number and depth of changes made to improve chronic illness care; perceived team effectiveness was consistently associated with both a greater number and depth of changes made to improve chronic illness care
Temkin-Greener et al, 2004[51]	Assessment of Interdisciplinary Team Process and Performance in Long-Term Care, focusing on Program of All-inclusive Care for the Elderly (PACE)	59 items measuring 5 scales of team process measures and predictors of team performance, 5-point Likert scale	NR	.76-.89	Literature review to select survey items. An expert of panel of 12 professionals and specialist in education and English as a second language prereviewed the survey items	Construct validity was demonstrated through the results of the regression analysis; as postulated in the theoretical model, leadership, communication, coordination and conflict management were positively related to team cohesion and perceived team effectiveness	NR
Hyer et al, 2003[52]	Trainee Test of Team Dynamics	5 question test to capture knowledge of team process and skills in addressing conflict after watching a 5-minute videotape of a simulated interdisciplinary health care team meeting; researchers also create “gold standard” by asking 31 experts to rate the video	NR	NR	Geriatric Interdisciplinary Team Training (GITT) faculty agreed to a specific set of learning objective dimensions at a national meeting prior to the full launch of GITT. Survey items were created based on the learning objective domains	Principal component factor analysis was conducted; 2 concepts—ability to distinguish team dynamics and recognition of effective team behaviors are only significant. One-way ANOVA test also supported the result	NR

Abbreviations: ANOVA, analysis of variance; CFA, confirmatory factor analysis; EV, eigenvalues; FL, factor loadings; ICC, intraclass correlation coefficient; IRA, interrater agreement; IRR, interrater reliability; NR, not reported; r = James, Demaree, and Wolf’s interrater agreement indices; Var Exp, variance explained.

An article that report all 4 psychometric properties.

The results indicate that survey instruments are widely used when measuring the dynamics of teamwork and individual team member behaviors; however, when team effectiveness is self-reported using Likert-type questionnaires, team members can report biased answers on their teamwork or unintentionally overestimate their team effectiveness.[54,55] To avoid any human error and bias in survey responses, it is important to measure non-self-reported outcomes, and compare these with the self-reported team effectiveness results from a survey. Only 7 articles (31%) reported non-self-reported outcomes related their survey measures, which reveal that when team members reported high team effectiveness, the objective outcomes (eg, task performance, quality of care, length of stay, and patient discharge) were also improved. Our findings indicate that the adequacy of survey instruments on team effectiveness linked to actual outcomes is still understudied.

Tables 3 to 5 show the conceptual dimensions of measuring team effectiveness in surveys across different clinical settings. First, we focused on the surgical setting to investigate what conceptual dimensions were most measured. Applying the CTEF model in addition to Donabedian model, we identified the specific subdimensions of structure, process, and outcomes with the consideration of surgical teams’ high-acuity, complex task, and time-sensitive conditions. We found that structure and outcome dimensions were most often measured in the surgical settings. Particularly, team skills, task specialization, and working conditions were the unique dimensions found in the surgical setting. In terms of outcomes, “task competency” and “would refer others to this team” were measured only in surgical settings. In the process category, “team coordination” and “value the teamwork” dimensions were most measured across the selected articles in the surgical setting. The prevalence of these dimensions confirms that the surgical teams require interprofessional coordination and that team members must value teamwork to achieve high-quality care. Following the literature on action teams, the findings indicate that survey tools in surgical settings more often focus on team skills, specialization, and coordination to assess team effectiveness. The findings also support that task performance is considered highly important and differentiated from team performance when it comes to the surgical setting.

Table 3.

The conceptual dimensions of team effectiveness in the surgical setting.

Conceptual framework	La Sala et al, 2015[36]	Andrew et al, 2012[37]	Hamilton et al, 2009[38]	Michinov et al, 2008[23]	Simon and Stewart, 2007[39]	Davenport et al, 2007[3]
Structure
Recognizing leadership			x		x
Team skills/specialization[a]		x		x		x
Commitment to patients						x
Emotional exhaustion/stress					x	x
Clear roles and responsibilities		x	x
Working conditions (quality of the work)[a]						x
Process
Communication among teammates		x	x
Team coordination	x	x	x	x		x
Value the teamwork	x	x		x	x
Outcome
Team cohesion (collective efficacy)	x				x
Overall perceived team effectiveness		x		x
Improved task competency[a]	x	x	x
(Postteamwork) job satisfaction				x		x
Would refer others to this team[a]		x

In the surgical settings, we used both Donabedian and CTEF framework to identify contextual nature of teamwork sills and performance.

Concepts showed only surgical settings.

Table 5.

The conceptual dimensions of team effectiveness in the other clinical setting.

Conceptual framework	Keebler et al, 2014[44]	El Ansari et al, 2016[45]	Tremblay et al, 2015[46]	Strasser et al, 2014	Dounis et al, 2013[48]	Baker et al, 2011[49]	Strasser et al, 2010[47]	Strating and Nieboer, 2009[50]	Temkin-Greener et al, 2004[51]	Shortell et al, 2004[4]	Hyer et al, 2003[52]
Type of setting	US Army Medical facilities	Mental Health	Ambulatory Cancer Clinic	Medical services at VA	Type 2 diabetes training program	General hospital	Rehabilitation services at VA hospitals	Home care	Long-term care	Chronic Illness Program	Geriatric health care teams
Structure
Recognizing leadership[a]	x			x		x	x		x	x
Commitment to patients[a]	x	x	x	x			x
Embracing challenge[b]						x				x
Team composition	x									x
Clear roles and responsibilities[a]			x			x		x		x
Process
Shared understanding	x					x		x	x
Participating in decision making										x
Communication among teammates[a]	x	x	x		x			x	x	x
Conflict resolution effort				x		x	x		x	x
Team collaboration		x			x
Team coordination[a]		x		x	x		x		x
Sharing vision and goals	x	x				x		x
Support for innovation	x	x		x			x	x		x
Value the teamwork	x			x	x	x	x	x
Task orientation				x			x	x
Monitoring situation[b]	x
Outcome
Team cohesion[a]		x		x	x		x	x	x
Team efficiency					x						x
Overall perceived team effectiveness[a]						x		x	x	x	x
Patient safety[b]			x		x
Improved patient well-being[b]		x	x	x	x		x

Common dimensions across settings.

Unique dimensions founded in the other clinical settings except for surgical and primary care settings.

Table 4 presents the conceptual dimensions measured by surveys in the primary care setting. Interestingly, primary care setting surveys are more focused on team coordination and value of the teamwork. In the process category, team collaboration, participation in decision making, and support for innovation were mostly found in surveys administered in primary care settings. This indicates that primary care settings recruit both internal and external professions to the host primary care organizations, so how they communicate with each other and how much they share goals/vision is key to enhanced team effectiveness. The right mix of team composition is also an important issue in the primary care setting. Compared with the surgical setting, the primary care setting seems more focused on team performance than task performance. The primary care setting surveys mostly measured outcome as overall perceived team effectiveness, collective team efficacy, and team members’ job satisfaction. Teamwork can be differently perceived and evaluated based on the type of clinical setting, as illustrated by the differential appearance and prevalence in conceptual dimensions surveyed across primary care and surgical settings.

Table 4.

The conceptual dimensions of team effectiveness in the primary care setting.

Conceptual framework	Landry and Erwin, 2015[40]	Song et al, 2015[41]	Beaulieu et al, 2014[42]	Helfrich et al, 2014[19]	Becker and Roblin, 2008[43]
Structure
Recognizing leadership					x
Commitment to patients		x		x	x
Emotional exhaustion/stress				x
Team composition		x		x
Clear roles and responsibilities		x	x
Process
Shared understanding		x	x
Communication among teammates	x	x		x
Conflict resolution effort	x	x
Team collaboration	x				x
Participation in decision making		x	x	x	x
Team coordination	x				x
Sharing vision and goals	x		x
Support for innovation			x	x
Value the teamwork	x
Task orientation			x
Outcome
Team cohesion (collective efficacy)		x	x	x
Overall perceived team effectiveness		x		x
(Postteamwork) job satisfaction		x

Table 5 shows the conceptual dimensions of other clinical settings. Due to the large variety and range of clinical settings, we listed the type of setting under each article. Overall, we found that there were common dimensions across all these settings. Recognizing leadership, commitment to patients, and clear roles/responsibilities were found in most articles regardless of setting. These dimensions are common structural conditions that health care teams share to achieve or improve team effectiveness. In the outcome category, team cohesion and overall perceived team effectiveness can be found regardless of health care setting. This means that team-based performance is usually measured as part of a survey instrument regardless of health care setting. Patient outcomes were only addressed in the survey tools in the other settings. Patient safety was addressed in 2 surveys, whereas improved patient well-being was measured in 5 surveys. In terms of process, communication among team members and team coordination were commonly found across settings; these dimensions were found in both surgical and primary care settings as well. The findings indicate that communication and coordination can be key to promoting team effectiveness in health care where different types of professionals and specialties are required to work together.

Discussion

Effective teamwork in health care contributes to a positive organizational culture and improves patient safety and outcomes. Developing accurate methods for measuring team effectiveness will be crucial to help drive quality improvement. In addition, these methods may differ depending on the clinical setting in which they are deployed. We found that survey tools have been used to measure team effectiveness as an outcome or as a tool for developing models of team effectiveness. Most survey tools were implemented in primary care or surgical settings; thus, more work is required to develop valid survey tools in other clinical settings, such as ambulatory care, cancer care, rehabilitation service, and long-term care.

Regardless of the clinical setting, studies measuring team effectiveness using surveys should also include measures of the surveys’ psychometric properties. The inclusion of those properties adds credibility to the measurement instruments and helps future researchers study team effectiveness and develop new and improved measurement instruments. Of 22 articles, we found that only 4 articles (18%) reported all 4 psychometric properties and only 7 articles (31%) reported non-self-reported outcomes related their survey measures. This finding reveals that the adequacy of the survey instruments still needs to be assessed.

Regarding conceptual dimensions in survey instruments, we found that the focus on outcome measures in the survey instruments is different across settings. The surveys administrated in the primary care setting are more likely to focus on team performance than those administrated in surgical settings. In particular, most survey instruments in surgical settings distinguish task-specific components from team-related components when the surveys were administrated, which supports our use of the CTEF model as a conceptual framework. The distinctiveness of action teams in the surgical setting requires coordinating the different professions in a short-term period and time-pressured situation. In this dynamic process, team members need to monitor progress toward goals and provide real-time feedback so that any errors or misunderstanding are recognized and modified.[20] Additional research on surgical team effectiveness may be particularly useful because of the fast-paced nature of the operating room setting. Decisions in operating rooms are often made rapidly, with limited information, and hold serious consequences for the patient. Surgical teams comprise a variety of health care professionals including surgeon(s), operating room nurse(s), and the anesthesia care teams. Therefore, determining shared characteristics among high-functioning surgical teams would help providers and administrators improve efficiency, effectiveness, and quality across a variety of delivery models and settings. There are likely factors beyond surgical team composition that influence team effectiveness, and these could be captured through qualitative means such as survey assessments. Research to identify those factors, how they can be accurately measured, and how they affect team members and patients, needs to be explored.

Interestingly, the findings indicate that existing survey instruments are less likely to address patient outcome as a key subdimension of outcomes. Only 5 survey tools in other health care settings recognize patient safety and improved patient well-being as their subdimensions of outcomes, whereas none of the survey instruments in the surgical or primary care settings explicitly measure patient outcomes as their key conceptual dimension. This is a notable finding because teamwork and team effectiveness are highlighted in the context of value-based payment.[56,57] To tie team effectiveness to value that actually improves care for patients, more attention to patient outcomes is needed when developing survey tools of team effectiveness.

Our study selection criteria were limited to the identification and evaluation of survey instruments of team effectiveness related to health outcomes and should not be construed as covering general studies on team(s) or teamwork(s) without outcome domains. Also, it is possible that the choice of conceptual framework can oversimplify or conflate distinct features of different health care settings. Our study mitigates this using multiple models to inventory the instruments under study, supplementing the Donabedian model with CTEF to account for the surgical setting.

Conclusions

The Medicare Access and CHIP Reauthorization Act of 2015 (MACRA) and initiatives in the commercial insurance market have fueled the shift to value-based payments and have driven required changes in care delivery models, including team-based care. New alternative payment models require a renewed emphasis on care coordination and team effectiveness. We report on team effectiveness measurement tools in a variety of health care settings in this article. Our findings indicate that more valid, context-sensitive survey tools need to be developed for health care settings. Our findings also reveal that patient outcomes should be addressed more thoroughly as key dimensions of outcomes when measuring team effectiveness.

In addition, we found that surgical settings have distinctive conditions for measuring team effectiveness relative to other primary care or ambulatory care. As evidenced by programs such as Enhanced Recovery After Surgery (ERAS), the Perioperative Surgical Home (PSH), and Medicare’s Comprehensive Care for Joint Replacement Model (CJR), the operating room has become a critical setting for team-based care delivery[58-60]; thus, more validated survey instruments focused on surgical action teams are needed. Further development of specific team effectiveness evaluation tools in various settings, such as chronic illness care, home care, long-term care, and ambulatory care, can enhance continuous quality improvements and patient outcomes in the future. Further development of team effectiveness evaluation tools specific to the health care setting can help further enhance continuous quality improvements and clinical outcomes in the future.