Team Functioning And Beliefs About Team Effectiveness In Inter-Professional Teams: Questionnaire Development And Validation.

PURPOSE: Few validated instruments are available to measure team functioning in acute and primary care teams. To address this, we developed a questionnaire measuring healthcare provider perceptions of team effectiveness (Provider-PTE) and assessed its psychometric properties. PATIENTS AND METHODS: Empirical evidence and a conceptual model were used for item generation. The 41-item self-completed questionnaire was developed. A cross-sectional survey of healthcare providers (n=283) across a range of settings was performed. Psychometric properties were assessed for French and English language questionnaires using Cronbach alpha (α) for reliability, the feedback form for face validity, expert opinion for content validity, and the known-group technique for construct validity. Responsiveness was examined by comparing scores in high and low functioning teams.
RESULTS: The mean time needed to complete the questionnaire was less than 9 mins. Respondents were typically female (84%), and employed full time (80%) in urban settings (82%). Cronbach α values were as follows: Team Processes = 0.88; PTE-Overall = 0.91; Outcomes = 0.72. Significant differences were found by professional group (p = 0.017), length of time in the team (p = 0.025), and presence of nurse practitioners. Responses to Outcomes varied by employment status (p = 0.017). Differences were identified in high and low functioning teams (p<0.001). Feedback indicated that two questions related to team meetings needed to be added.
CONCLUSION: The study produced evidence of validity for English and French language Provider-PTE questionnaires. The revised 43-item instrument represents an important contribution by providing a validated questionnaire to measure team functioning across a range of settings that is consistent with a conceptual framework.

Background

Inter-professional healthcare teams unite providers from different professions (eg, nurse, nurse practitioner, pharmacist, physician) to provide patient care.1 Inter-professional teams represent a safe and effective strategy to deliver care to patients and families because each professional contributes specific knowledge and skills.1,2 These teams were introduced in hospitals over 100 years ago but are relatively new in primary care.3 Team performance has been examined using system-level indicators including access to care, wait times, and screening rates.4,5 Yet, to better understand patient outcomes, it is essential to capture team-level processes.

Processes are defined as interactions that occur at the level of the patient and care provider.6 Team processes are the mechanisms that explain how actions unfold over time in response to what occurs in the surrounding context.7,8 Researchers have often compared processes to a black box where one can observe inputs and outputs without clearly understanding what is happening in between.8–11 Processes including teamwork, communication and patient engagement are cornerstones in the delivery of effective care.12–14 In primary care, Hofhuis et al15 found that team performance was influenced by how frequently team members interacted with each other. However, these authors were unable to measure team-level variance.

Studies have identified that provider judgements or beliefs about team functioning and effectiveness are influenced by processes that include decision-making, communication, cohesion, care coordination, problem-solving, a focus on the needs of patients and families, as well as role clarity and trust among team members.16,17 The measurement of these team processes has been a critical challenge because of the financial and human resources required and the lack of validated instruments. Thus, important gaps in knowledge related to team processes and team functioning remain.18

Greater understanding of how healthcare teams function is crucial given the negative consequences of poor team functioning. In the United States, miscommunication was identified as a key issue following a review of more than 23,000 malpractice claims in 4 years.19 The cost of the incidents was estimated at $1.7 billion US and 2000 lives were lost.19 Similar issues have been identified in Canada, where 37% of adverse events were deemed to be preventable with improved teamwork and communication.20

Team Functioning

A systematic review of the experiences of health professionals regarding teamwork and collaboration in primary care found that key attributes of teamwork focused on user needs, interdependent actions, negotiations between professionals, decision-making, mutual respect, trust among team members, and a recognition of the role and work of providers in different professional roles.13 These authors identified that providers in primary care faced enormous challenges to promote teamwork and collaboration. In primary care, Beaulieu et al21 examined organizational structures associated with the delivery of high-quality care and found that provider views of the team’s ability to innovate were moderately associated with outcomes. In acute care, Sidani and Doran22 identified positive associations between care coordination, counselling and education provided by nurse practitioners and patient outcomes related to symptom resolution, improved functional status and satisfaction with care.

Recent systematic reviews have examined the characteristics of instruments designed to measure team functioning and team effectiveness. Important gaps were identified,23,24 including the lack of inclusion of key dimensions of team functioning consistent with a conceptual framework and the need for patients and families as healthcare team members. Additionally, instruments have focused on specific clinical areas such as acute care or primary care but have not been validated across different settings. Validated questionnaires are needed to examine the inter-professional team members’ views of team processes in different settings.

Objectives

The study aimed to 1) develop a Provider-Perceptions of Team Effectiveness questionnaire (Provider-PTE) to examine processes in healthcare teams from the perspectives of inter-professional team members and to 2) assess the psychometric properties (ie, reliability, face validity, content validity, construct validity) of the Provider-PTE questionnaire.

Conceptual Framework

The study was guided by a conceptual framework developed to support decisions and research related to nurse practitioner roles.25 Nurse practitioners often work in teams that include members from more than one professional group, and the term inter-professional is used in the framework and the current study. The framework includes three central process dimensions: role enactment, boundary work, and perceptions of team effectiveness. Of particular importance to this study is the Perceptions of Team Effectiveness dimension. Key processes in effective teams include improved communication, involvement in decision-making, cohesion, care coordination, problem-solving, and a focus on patient and family needs.16,25,26

Methods

Research Design

A cross-sectional survey was undertaken. A self-administered questionnaire represents a cost-effective strategy to gather information from a large number of participants in distinct geographic locations.27

Recruitment

We recruited participants using a convenience sample. We aimed to recruit 250 care providers to assess the psychometric qualities of the instrument and identify differences in high and low functioning teams. Our sample size allowed us to detect medium correlations and a 95% confidence interval half-width of 0.1 with 80% statistical power and a 5% alpha level.28,29 We included in-hospital or specialized ambulatory care in our definition of acute care.30 Primary care was defined as comprehensive healthcare services for common health concerns at the point of entry to the healthcare system.31

Data Collection

Following Research Ethics Committee approval by the Comité d’éthique de la recherche du CIUSSS de l’Est-de-l’Île-de-Montréal (Project no 2018-1168, CIUSSS_EMTL_282), data were collected from December 2017 to June 2018. The survey included an information sheet, consent form, a feedback form and a questionnaire. Participants were under no obligation to participate in the study. They were advised that they could withdraw at any time. Online and paper copies of the documents were available to participants to enhance response rates.32 All documents were available in English and French.

Instrument Development

To address Objective 1, the questionnaire was developed using empirical evidence from multiple case studies of processes in inter-professional teams16,33–35 and a review of the literature describing processes in teams.36

Item Generation

Several steps were undertaken at this stage (Figure 1). To begin, items were generated using examples and descriptions of processes and actions identified by interprofessional team members in cardiac surgery and long-term care.16,33–35 (Figure 1). An initial list of 69 items was generated. Draft items were reviewed by members of the research team to ensure item clarity and legibility. Researchers (n=8) who specialized in questionnaire development and advanced practice nursing research reviewed the instrument. They made suggestions to clarify some items by providing examples and adding descriptions before the start of each section.

Figure 1

Flowchart of the Provider-PTE questionnaire development.

As a class exercise, 30 graduate-level nursing students attending a role development course worked in small groups to generate key contributions of advanced practice nursing to team functioning and patient care using relevant literature and their clinical experience. The contributions were mapped to the items in the questionnaire. All of the students’ ideas mapped to at least one specific item but some contributions were measured by up to five questionnaire items. This allowed the research team to identify redundancies. At the same time, our team was conducting a study to examine the psychometric properties of the Patient-PTE questionnaire.37 The items in both questionnaires were worded in a similar fashion for consistency. Items from the patient questionnaire were examined to determine if any were problematic. From these results, the number of negatively worded items were limited to three in the Provider-PTE questionnaire.

Instrument

The Provider-PTE questionnaire included 41 items divided into Work Setting, Team Processes, PTE-Overall, and Outcomes presented in Table 1, along with Socio-Demographic Characteristics. More specifically, eight items described the characteristics of the Work Setting (eg, type of healthcare setting, location, caseload, and team size). Team Processes included 14 items. One item specifically measured respondents’ judgements or beliefs about the team’s effectiveness (BE). This score was used to determine high and low functioning teams. Seventeen items measured PTE-Overall and included Team Processes and factors (eg, role clarity) known to influence PTE. Outcomes included seven items (eg, timely care, staff knowledge and skills). Response options ranged from 1 to 6 (ie, strongly disagree, disagree, disagree somewhat, agree somewhat, agree, and strongly agree). A mean value was estimated to obtain a score for each dimension. The average scores ranged from 1 to 6. Improved perceptions of team processes and better team functioning were indicated by higher scores for Team Processes, PTE-Overall, and Outcomes. The socio-demographic characteristics of providers included eight items (eg, age, gender, education, professional group, and employment status). Most response options in this section were categorical. Age and time in the professional role, the organization and the healthcare team were numeric values. One open-ended question gathered any additional comments from respondents. We used single-item measures to measure boundary work (ie, trust), belief about team effectiveness, and cohesion to reduce response burden.38 Two items measured decision-making, coordination, problem-solving, and role clarity. Three items were used to examine communication and a focus on the role of patients and families as team members.

Table 1

Preliminary Items Related To Healthcare Team Processes And Outcomes

Items
PTE-Overall
Role Clarity	The roles of members of the healthcare team are well defined
	I am happy with the way work is divided among members of the healthcare team
Boundary Work	I trust other members of the healthcare team
Team Processes	My healthcare team is effective to provide patient care (BE)
Decision-making	Team members share relevant information to inform patient care decisions
Decision-making	My ideas, information or observations are used to solve patient care issues
Communication	The plan of care and patient care objectives are clearly outlined in the patient’s health record
Communication	The patient’s health record is updated as required
Communication	The flow of information between team members and patients and families is constrained
Coordination	The healthcare team adjusts treatments according to changes in the patient’s condition
Coordination	The care provided by the healthcare team is well organized
Cohesion	Team members work together to solve patient care issues
Problem-solving	Disagreements among team members are dealt with fairly by team members
Problem-solving	Differences of opinion among team members are respected
Patient–Family Focus	I have a role to play in the team
Patient–Family Focus	My contributions are valued by my healthcare team
Patient–Family Focus	Working with families to solve patient care issues is not part of the team’s mandate
Outcomes
	Patient care is delivered in a timely manner
	The healthcare team can easily access a provider who can order tests or medication
	Potential or actual patient complications are dealt with quickly by the team
	Patients return home with as many unanswered questions about their medication
	Patient transfers to other care facilities include all relevant patient information
	Patients return home with all their questions answered about their care
	Members of the healthcare team possess in-depth knowledge and the skills required to provide care

Translation

The questionnaire was translated from English by a professional translator whose mother tongue is French. It was back-translated by a second translator whose mother tongue is English.39 Strategies to enhance the cross-cultural adaptation of the questionnaires included cognitive interviews with participants to ensure that items were consistently understood in the same manner and pre-testing of the instrument.40 The questionnaires were reviewed by bilingual members of the research team to ensure that the translations maintained semantic and conceptual equivalence with the framework.40,41

Pilot Testing

Research team members reviewed the draft items. English and French language versions of the questionnaire were pilot-tested with the same advanced practice nursing researchers to identify unclear wording. No revisions were needed following pilot testing. As proposed by Kilpatrick et al,42 the same respondents completed the questionnaire 2 weeks later to determine if their understanding of the items had changed. Participants’ responses and understanding of the questions were unchanged after 2 weeks.

To address Objective 2, cognitive interviewing, a feedback form, and statistical analyses were used to assess the psychometric properties of the questionnaire.

Cognitive Interviewing

Participants were asked to review instructions, questionnaire layout, procedures to complete French and English language questionnaires, and item and response options for clarity.43 Cognitive interviewing was used to examine the questionnaire with those who completed the pilot test and students in the graduate-level course.

Feedback Form

After the questionnaire was completed, a feedback form developed previously was administered.37,42,44 Participants were asked to provide feedback on typical items such as time needed to complete the questionnaire, clarity of instructions; comprehensiveness of the questions related to the topic; readability; and ordering of questions. Participants had the opportunity to provide suggestions to improve the Provider-PTE questionnaire via an open-ended question.

Analysis

Data were analyzed using the IBM Statistical Package for the Social Sciences (SPSS), version 25.45 Mean, standard deviation (SD), and range were generated using descriptive statistics. Overall Provider-PTE scores (PTE-Overall) and dimensional scores were calculated. Instrument performance was examined using Cronbach alpha (α) to assess reliability, item-total correlation of each item, and assessment of missing data. Responses did not follow a normal distribution (p values < 0.001),46 according to the Shapiro–Wilk test. For variables with two categories, we used the Mann–Whitney U-test, and for variables containing more than two categories, we used the Kruskal–Wallis one-way analysis of variance for nonparametric testing. A Bonferroni correction was applied to correct for multiple comparisons and decrease the likelihood of Type I error. Rank correlation was determined using Spearman’s rho for continuous data.47 Responses to open-ended questions were analyzed using content analysis.48

Instrument Performance

Reliability

Internal consistency of the subscales (ie, Team Processes, PTE-Overall, and Outcomes) was assessed using Cronbach alpha. Values ranging from 0.7 to 0.9 were considered acceptable to excellent.49 Because we did not have a sufficiently large number of participants from the same healthcare team, we did not compare the within-group variance to the between-group variance to determine the consistency of responses among participants in the same team.50

Validity

Face validity was assessed using the feedback form that examined the Provider-PTE questionnaire, formatting and instructions. Content validity was assessed prior to the questionnaire roll-out by expert groups including researchers and graduate students. Construct validity was examined using the known-group technique.51,52 We compared the scores between specific groups, and hypothesized a priori that no differences in scores would be found between 1) men and women; 2) English and French language respondents; and 3) respondents in urban and rural locations. Differences were expected according to these factors:

professional group (intra-professional (ie, within the same professional group), inter-professional and managers);

team size (small (less than 5 members), medium (5–10 members), large (more than 10 members);

teams with and without nurse practitioners

employment status (full time and part time).

Nurse practitioners can autonomously evaluate, diagnose, and treat patients for chronic or acute conditions as well as provide advanced nursing care to best meet patient healthcare needs according to the regulated scope of practice in their jurisdiction.53,54 Teams with and without nurse practitioners were identified in our study because the province of Québec Canada planned to implement 2000 nurse practitioners by 2024 to improve care quality and access to care.55

Streiner et al56 define responsiveness as a questionnaire’s ability to detect a meaningful change. However, several authors argue that there is no agreed-upon method to measure responsiveness.56–58 We compared processes in high and low functioning teams and determined a priori that the scores would be different.

In the following section, results including means, correlations, Cronbach α and p values statistically significant at the 5% level or α/n groups for post-hoc analysis are reported by measure and known group.

Results

Overall, 314 participants responded to the survey. Thirty-one questionnaires included only socio-demographic data and were excluded. Thus, 283 questionnaires were included in the analyses. Most questionnaires were completed online (75%, n=211). The mean time to complete the questionnaire was 21 mins. When three outliers were removed (ie, 243 min, 1447 min, and 1753 min), the mean time to complete the questionnaire decreased to 8.9 mins (SD: 8.75).

Socio-demographic characteristics of participants are presented in Table 2. A majority of respondents completed the questionnaire in French (85%, n=241). Most respondents were female (84%, n=216), and a majority worked full time (80%, n=205). The average age of respondents was 39 years of age (SD: 9.1, range 23–67). Providers had worked in their professional role for 13.3 years (ranging from 1 month to 45 years), in their current role for an average of 7.2 years (ranging from 1 month to 45 years), and in their current team for 6.4 years (ranging from 1 month to 45 years).

Table 2

Characteristics Of Respondents

	Variable		n	%	Mean	Standard Deviation	Min–Max
Respondent	Professional group	Intra-professional	217	84
		Inter-professional	31	12
		Management	9	4
	Language	English	42	15
		French	241	85
	Age (years)				39.2	9.1	23–67
	Gender	Male	42	16
		Female	216	84
	Job status	Full-time	205	80
		Part-time	31	12
		Temporary/contract position	7	3
		Leave	13	5
	Education	High school and below	8	3
		Above high school	250	97
	Length of time in current role (years)				7.25	7.8	0.08–45
		Below 2 years	46	19
		2–5 years	93	38
		>5 years	108	44
	Length with this team (years)				6.41	7	0.08–45
		Below 1 year	44	18
		1–5 years	106	42
		>5 years	100	40

Healthcare team characteristics are presented in Table 3. Respondents worked primarily in large teams (69%) located in an urban setting (82%). Specialties included primary care (56%), oncology (16%), Emergency/Intensive care unit (12%), and medicine (16%). Half of the respondents worked in specialized outpatient settings and in-patient units. Respondents worked in teams who cared for a wide range of patient populations.

Table 3

Characteristics Of The Healthcare Team

	Variable		n	%
Healthcare team	Province	Ontario	31	12
		Quebec	234	88
	Clinical setting	Primary care	132	56
		Medicine and surgery	39	16
		Oncology	38	16
		Emergency-critical care	28	12
	Location	Urban	229	82
		Non-urban	50	18
	Team size	Small	11	4
		Medium	76	27
		Large	193	69
	Nurse practitioner in team	Yes	167	60
		No/do not know	112	40

Reliability was examined using Cronbach α values for Team Processes, PTE-Overall, and Outcomes. Cronbach α values for Team Processes equaled 0.88; PTE-Overall equaled 0.91, and Outcomes equaled 0.72 (see Table 4).

Table 4

Mean Scores For Boundary Work, Perceptions Of Team Effectiveness, Role Clarity, And Outcomes

Dimension	Variable	Number Of Items	Mean	Standard Deviation	Min–Max	Cronbach Alpha	Spearman’s Item-total Correlations With Team Processes
							rho Coefficient	p-Value
Boundary Work	Trust	1	5.2	0.84	1–6
Team Processes		14	4.7	0.6	1.5–6	0.881
	Belief about team effectiveness	1	5.2	0.8	1–6		0.584	<0.001
	Decision-making	2	4.9	0.8	1–6		0.807	<0.001
	Communication	3	4.4	0.7	2.3–6		0.688	<0.001
	Coordination	2	4.9	0.7	1–6		0.691	<0.001
	Cohesion	1	4.8	1.0	1–6		0.695	<0.001
	Problem-solving	2	4.4	1.1	1–6		0.698	<0.001
	Patient focus	3	4.9	0.8	1.7–6		0.752	<0.001
Role Clarity		2	4.7	1.0	1–6
PTE-Overall		17	4.8	0.6	1.4–6	0.909
Outcomes		7	4.8	0.6	2–6	0.724

Face validity was assessed using responses (n=85) in the feedback form (see Table 5). Most (76%) believed that key questions to understand team functioning were included in the questionnaire and no question was unimportant (87%). Positive assessments were noted for the time needed to complete the questionnaire (96%), instructions (99%), formatting (100%), font size (100%), and logical ordering of questions (96%). Some respondents noted that it would be easier to see the negatively worded items if these expressions (eg, not) were bolded.

Table 5

Responses Related To Face Validity

		n	%
The time it took to complete the questionnaire was	Appropriate	81	96
	Too long	3	4
	Too short	–	–
Overall, the instructions contained in the consent were	Clear	83	99
	Not clear	1	1
Are there any questions NOT included in this questionnaire that you feel would be important to include to better understand how teams function effectively?	No	63	76
	Yes	20	24
Are there questions you feel are unimportant, irrelevant, or redundant and could be eliminated from the questionnaire without jeopardizing completeness of the study results?	No	74	87
	Yes	11	13
The format of the questionnaire was:	Easy to read	81	100
	Difficult to read	–	–
The font size made the questionnaire:	Easy to read	81	100
	Difficult to read	–	–
The questions were ordered in a logical manner that was easy to follow:	Agree	79	96
	Disagree	3	4

Content validity was assessed using cognitive interviewing and the responses from the open-ended question in the feedback form (n=85). No question was perceived to be redundant. A small number of respondents found it difficult to select a specific patient group if they worked with several patient populations.

Overall, trust and beliefs about team effectiveness (BE) were ranked highest (mean: 5.2/6±0.8, range 1–6). The dimension PTE-Overall rated highest (mean: 4.8/6 ±0.6, range 1.4–6) followed by Team Processes (mean: 4.7/6 ±0.6, range 1.5–6) and Role Clarity (mean: 4.7/6 ±1.0, range 1.0–6). Outcomes were rated 4.8/6 ±0.6 (range 2–6). The ratings for the variables included in the PTE-Overall dimension ranged from 4.4/6 ±1.1 for problem-solving to 5.2/6± 0.8 for the BE score. The rho coefficients (rs) for the variables included in the PTE-Overall dimension ranged from 0.58 to 0.81 (p value< 0.001) (see Table 4).

Construct validity was examined using known groups (see Table 6). As anticipated, no differences were found between men and women, English and French language respondents, and respondents in urban and non-urban locations. As hypothesized, differences were identified according to professional group (p = 0.017), length of time in the team (p = 0.025), and presence of NPs. We had anticipated differences according to team size but no differences were identified between these groups. Differences were noted in Outcomes according to employment status (p = 0.017).

Table 6

Differences In Team Processes And Outcomes By Known Group Comparisons

Dimension	Professional Groups	NP In Team	Length of Time In Team (Less Than Or More Than 2 Years)	Full vs Part Time
Boundary Work		0.027
Team Processes		0.034
Role Clarity	0.017*		0.025
PTE Overall		0.035
Outcomes		0.014		0.017

Notes: *Mann–Whitney U post hoc test £ Bonferroni correction significance α/3 (p<0.017); empty cells=not statistically significant.

Responsiveness was assessed using the scores in high and low functioning teams. Significant differences (p<0.001) were noted in all the scores for all the dimensions (see Table 7).

Table 7

Differences In Processes And Outcomes In High And Low Functioning Teams

Dimension	Belief About Team Effectiveness	n	Mean	Standard Deviation	Mann-Whitney-U test
					Median	p Value
Boundary Work	Low (scores 1–4)	29	3.86	1.217	4	<0.001
High (scores 5–6)	234	5.38	0.608	5	<0.001
Team Processes	Low (scores 1–4)	29	3.92	0.84	4.1	<0.001
High (scores 5–6)	235	4.83	0.52	4.88	<0.001
Outcomes	Low (scores 1–4)	29	4.2	0.755	4.29	<0.001
High (scores 5–6)	227	4.83	0.53	4.93	<0.001
Role Clarity	Low (scores 1–4)	29	3.5	1.35	3.5	<0.001
High (scores 5–6)	235	4.83	0.844	5	<0.001

Discussion

The Provider-PTE questionnaire is a self-reported instrument available in English and French that measures perceptions of team functioning and team outcomes. The questionnaire includes provider and team characteristics and three subscales (ie, PTE-Overall, Team Processes, and Outcomes). Psychometric testing (n=283) included construct validity using known groups. As anticipated, differences were identified by professional group, teams with nurse practitioners, and length of time in the team. Reliability was examined using Cronbach α. Subscale values ranged from 0.72 to 0.91. Rho coefficients (rs) for processes included in the Team Processes subscale ranged from 0.59 to 0.81 (p< 0.001), indicating that these processes were highly correlated with beliefs about team effectiveness. Responsiveness was assessed, and differences were noted in high and low functioning teams. The questionnaire fills important gaps identified in systematic reviews of instruments that measure team processes by integrating a conceptual framework and examining inter-professional team functioning across different settings.23,24

More recently, one member of the research team (LP) pilot tested several data collection tools, including the Provider-PTE questionnaire, for her doctoral study examining nursing role enactment in intensive care units (ICUs). Participants (n=8) in the pilot test highlighted that one item needed to be clarified and two items needed to be added to the Provider-PTE questionnaire to better capture effective team functioning in the ICU. The item about patient care goals “The plan of care and patient care objectives are clearly outlined in the patient’s health record” was changed to “Healthcare team members know the goals of patients’ plans of care”. This item was reworded because some respondents indicated that unlicensed care providers who do not have access to the patient’s health record would also need to have an understanding of the goals of care. In addition, participants noted the importance of scheduling regular team meetings to plan patient care. We added two items to measure the conduct of regular team meetings in addition to meetings scheduled for emerging issues “Regular inter-professional team meetings are scheduled to plan patient care” and assess if the team could meet quickly to address emerging issues “The healthcare team meets as needed to address complex care issues that arise”. These changes were consistent with the preliminary findings of a systematic review that our team is conducting to highlight the contributions of team meetings to improved team functioning.59

The current study generated preliminary data of the validity of the Provider-PTE questionnaire. Further testing is required for the revised 43-item questionnaire (see Table 8). The next steps include recruiting team members in large existing teams to examine whether perceptions within each team are less varied than between teams. This is important to determine our ability to aggregate survey findings from individuals to the team and organizational levels.50,60 In addition, repeated measurements with the same teams at different time points are needed to generate further evidence of the instrument’s responsiveness and allow for the determination of a minimally important difference in scores that can be used in different context including research (eg, a randomized control trial) or ongoing quality improvement initiatives.58,61 As proposed by Husebo and Akerjordet,62 measurements can be taken before and after a change in practice is undertaken to examine its impact on team functioning. Thus, it will be possible to tease out factors that are known to influence team functioning including trust, role clarity and the conduct of team meetings.59

Table 8

Revised Items Related To Healthcare Team Processes And Outcomes*

Items
PTE-Overall
Role Clarity	The roles of members of the healthcare team are well defined
	I am happy with the way work is divided among members of the healthcare team
Boundary Work	I trust other members of the healthcare team
Team Meeting	The healthcare team meets as needed to address complex care issues that arise
	Regular inter-professional team meetings are scheduled to plan patient care
Team Processes	My healthcare team is effective to provide patient care (BE)
Decision-making	Team members share relevant information to inform patient care decisions
Decision-making	My ideas, information or observations are used to solve patient care issues
Communication	Healthcare team members know the goals of patients’ plans of care
Communication	The patient’s health record is updated as required
Communication	The flow of information between team members and patients and families is constrained
Coordination	The healthcare team adjusts treatments according to changes in the patient’s condition
Coordination	The care provided by the healthcare team is well organized
Cohesion	Team members work together to solve patient care issues
Problem-solving	Disagreements among team members are dealt with fairly by team members
Problem-solving	Differences of opinion among team members are respected
Patient–Family Focus	I have a role to play in the team
Patient–Family Focus	My contributions are valued by my healthcare team
Patient–Family Focus	Working with families to solve patient care issues is not part of the team’s mandate
Outcomes
	Patient care is delivered in a timely manner
	The healthcare team can easily access a provider who can order tests or medication
	Potential or actual patient complications are dealt with quickly by the team
	Patients return home with as many unanswered questions about their medication
	Patient transfers to other care facilities include all relevant patient information
	Patients return home with all their questions answered about their care
	Members of the healthcare team possess in-depth knowledge and the skills required to provide care

Note: *Negative wording bolded.

The questionnaire makes an important contribution to the teamwork literature because it was developed to be consistent with a conceptual framework. The instrument was validated in English and French, in two different Canadian provinces, and across a range of clinical settings. We were unable to determine the extent of non-response bias because we did not have information on non-responders.63 Strategies to reduce the risk of response bias included the use of pilot testing and cognitive interviewing.63 Simple response options were used to reduce cognitive load when answering questions, and the questionnaire was kept as short as possible while cohering with the conceptual framework.56 We anticipate that the recruitment of healthcare providers from different professions, working in acute and primary care with different patient populations in two Canadian provinces, will enhance the generalizability of our findings to other inter-professional teams.

Previous studies have focused on teams in specific contexts, including primary care,64,65 and acute care62,66,67 or in specialized clinical areas including mental health68 and medicine.69 The Provider-PTE questionnaire can be administered in intra- and inter-professional teams to examine team functioning. It can be used to assess team functioning before and after changes in skill mix, in different settings from acute to primary care, and in-patient and out-patient settings. Given that higher scores indicate better team functioning, it will be possible to monitor team performance to identify processes that function well in addition to those in need of improvement. Such measurements will provide feedback to decision-makers, team members as well as patients and families.

Limitations

Our study included a convenience sample. Subsequent studies will need to be conducted using a random sample of providers. We did not recruit a sufficiently large number of providers working in the same team or a large enough sample of managers. Future studies will need to be conducted in larger inter-professional teams in different settings to accrue additional evidence of validity and responsiveness. In addition, we administered the questionnaire to members of one healthcare team without direct patient care activities. Some items were difficult to answer for team members without patient contact, which suggests that survey items would need to be adapted for teams not involved in direct patient care. Finally, we did not use factor analysis to examine construct validity because the proposed subscales had fewer than three items per scale.70 Future research is needed to test the questionnaire’s structure and confirm the proposed subscales.

Conclusion

The study produced evidence of validity for the English and French language versions of the Provider-Perceptions of Team Effectiveness questionnaire. The revised 43-item instrument is a self-completed questionnaire that takes less than 9 mins to complete. The questionnaire measures perceptions of team processes and outcomes of all members of the inter-professional team. Reliability indices were acceptable to excellent for the subscales. Face validity and content validity were examined, while construct validity was assessed using known groups. Responsiveness was examined by comparing high and low functioning teams. The questionnaire can be used to examine how inter-professional teams function in different contexts.