Italian version of the Assessment of Interprofessional Team Collaboration Scale II (I-AITCS II): a multiphase study of validity and reliability amongst healthcare providers.

OBJECTIVE: To develop and validate an Italian version of the Assessment of Interprofessional Team Collaboration Scale II (I-AITCS II).
METHODS: A multiphase validation study was conducted. The first phase was the AITCS-II translation from English into Italian to develop the first version of I-AITCS II for practitioners. The second phase was the study of I-AITCS II face and content validity, and the third phase was a cross-sectional data collection to provide evidence of construct validity using the psychometrics testing and the reliability assessment through the internal consistency study.
RESULTS: The agreement for the forward-translation among researchers was high. The face and content validity were satisfactory. The underlying constructs of I-AITCS II were partnership, cooperation and coordination. Internal consistency was good for both scale and domains level. There were significant differences related to partnership in the comparison between settings.
CONCLUSIONS: I-AITCS II showed evidence of validity and reliability. It will be useful to gather data to address programs aimed to enhance interprofessional team collaboration within the Italian healthcare contexts, and it could be used for cross-national researches.

Introduction

Interprofessional team collaboration (ITC) represents a key element for the safety and quality of every healthcare setting (11). ITC refers to a process where different professional groups are involved in working together (18). Currently, evidence from the healthcare field shows that effective ITC improves team meetings, team debriefings, and enhances case discussions to best manage patient issues (11, 23). Accordingly, ITC has an effect in improving patient health outcomes at organizational and individual levels (2, 7), and decreases counterproductive working behaviors and distress among health providers (3, 22). The professional groups involved in healthcare delivery are mainly represented by physicians and nurses, but also by clinical psychologists and all other healthcare providers involved in the patients’ care paths, such as technical professionals.

Some authors described the main constructs underlying ITC as partnership, cooperation and coordination (20). Partnership recognizes the role of patients and their families as partners with the interprofessional team to achieve the goals of their care (5). Cooperation relates to the mutual respect amongst team members and their patients’ opinions, viewpoints, beliefs or perspectives (1). Cooperation is the process where patients and healthcare providers consider their preferences together to achieve a mutual agreement and therefore is an important antecedent to shared decision making (10). Coordination is the ability of team members to work together harmoniously. To achieve this harmony requires the ability to suspend single preferences to reach shared results, especially around team working issues in high demand situations (17).

The proper measurement of ITC is critical to improve current knowledge about the evidence related to the effect of ITC on some organizational or health outcomes (23). Its measurement is also important for organizational administrators to address strategies aimed at improving ICT in different healthcare settings. For this reason, Orchard et al. (2012) developed the Assessment of Interprofessional Team Collaboration Scale (AITCS), recently modified in a shorter version (i.e., AITCS II) (19). AITCS II is a self-report tool to measure partnership, cooperation and coordination (ITC) in healthcare settings. It encompasses 23 items rated using a five-point Likert scale, from 1=never to 5=always. The cutoff score for the items is a mean of 4 to indicate an adequate level of ITC (19,20). AITCS II shows evidence of reliability and validity in its original version (19,20), and in the Swedish adaptation (13). More specifically, in the original study the tool was assessed for validity and reliability on a sample of 125 practitioners, coming from 7 different healthcare teams in Canada (20). In the Swedish version, the tool was tested on 349 participants working in team-based pain rehabilitation (19). In both studies, the validity was assessed through psychometric evaluation and the reliability through stability assessment (test-retest) and internal consistency evaluation. However, AITCS II is not available for an Italian context. For this reason, this study aims to develop a valid and reliable Italian version of AITCS II.

Methods

Study design

This was a three-phase validation study. The first phase assessed the methodological translation and cultural adaptation of the AITCS II from English into Italian resulting in the first version of I-AITCS II. The second phase studied the face and content validity of the I-AICTS II. Finally, in the third phase the construct validity and the reliability of the I-AITCS II were assessed.

Phase one - Translation and cultural adaptation

Phase one translation was carried out in March 2017 using the modified Brislin’s translation model for cross-cultural research (14). This methodology combines a translation technique that uses a group approach when applying the back-translation, and it ensures rigor for the translation into a target language. The setting of phase one was a hospital of Northern Italy. In the first step of the translation process, a project manager was identified (RC). This approach was carried out as follows. Initially a translation process was performed using a consensus discussion including all four authors, and four experts coming from the hosting hospital. During the consensus discussion, members discussed the nuances of the forward translation to find the most culturally suitable Italian wording of each item. At the end of this process individual members rated each translated item using a 5-point rating from 0=not clear to 4=very clear until the group consensus in expressing a “very clear translation” was reached. The lead research team member (RC) endorsed this first version of I-AITCS II (Appendix 1).

Phase two - Face and content validity

During April and May 2017, the second phase was carried out involving a panel of 16 experts including: nine nurses, three physicians, three psychologists, and one midwife to ascertain I-AITCS II face and content validity as described by Polit & Beck (21). In the face validity, panelists were asked to explore their understanding of the items and share their views about the overall concept (collaboration) that they purport to measure. The research team used the following open-ended questions to gain responses: “Can you express your thoughts about the items’ wording?”; “Can you express your thoughts about the overall clarity?”; “Can you express your thoughts related to possible issues in understanding the items’ wording?”. To gain agreement among the panelists regarding how pertinent each item was to measurement of collaboration, content validity was used. Three aspects were assessed: Content Validity Ratio (CVR), and Content Validity Index for scale’ (S-CVI) and items’ level (I-CVIs) (21). These indices could range from -1 to +1, and a value of ≥0.60 is considered adequate to retain the item in the translated version.

Phase three - Construct validity and reliability

Once the I-AITCS II’s face and content validity was obtained, a cross-sectional data collection was carried out from August to September 2017. The setting was a hospital in Northern Italy, all its healthcare professionals were invited to anonymously participate in this study. The eligible participants were contacted via their working mail, using a survey approach. Specifically, the authors disclosed the study invitation using the mailing list of eligible participants, who were able to print the study information sheet, the informed consent and the I-AITCS II to fill them in a paper-based format and re-send the documents back to the authors. To establish the adequacy of sample size the authors considered the item/participants Hair’s ratio of 1:10 (12), rather than the response rate. This approach was consistent with the main objective of this study, which would reflect sufficient power to perform psychometric analysis. In other terms, considering that I-AITCS II had 23 items, the minimum acceptable sample size to ensure sufficient power to perform the psychometric analysis to assess construct validity was equal to 230 participants.

Statistical strategies

For phase one, the translation agreement was assessed through the Fleiss’s kappa, considering that each individual in the translation discussion meeting rated each translated item using a 4-point rating from 0=not clear to 4=very clear. For phase two, the content validity was assessed through the computing of CVR, I-CVIs, S-CVI, and multi-rater kappa, calculating the 95% confidence interval (CI) for free-marginal kappa statistic.

For phase three, both descriptive and inferential statistical analyses were employed using the Statistical Package for Social Science version 22 (SPSS; Chicago, IL, USA) and MPlus 8,1. Initially, descriptive statistics were used to describe all data (demographic and I-AITCS II), which were checked for missing values, outliers or errors, and for their normality. Descriptive statistics for categorical variables were frequency and percentage, for continuous data non-normally distributed were median and interquartile range (IQR), for continuous data normally distributed were mean and standard deviation (SD).

An explorative approach to ascertain evidence of the I-AITCS II’s construct validity. Bartlett’s test and the Kaiser-Meyer-Olkin (KMO) index were then used to assess the sample’s adequacy for application of an exploratory factor analysis (EFA). An EFA was then used to assess the psychometric properties of the I-AITCS II, using the Maximum Likelihood Robust (MLR) estimator and a Geomin rotation to maximize the factor loadings on their latent dimensions. The number of factors to be extracted was determined firstly based on the theoretical structure of the original scale (3 dimensions), and secondly on the evaluation of the eigenvalues, the screen test, the model/factors explained variance, and the goodness of fit, considering the following indices: omnibus test (χ2), Comparative Fit Index (CFI; values ≥0.85 indicate a good fit), Tuker and Lewis Index (TLI; values ≥0.85 indicate a good fit), Standardized Root Mean Square Residual (SRMR; values ≤0.08 indicate a good fit), and Root Mean Square Error of Approximation (RMSEA; values ≤0.06 indicate a good fit). Finally, the reliability of the I-AITCS II was assessed using Cronbach’s α at a scale and domain level. Each domain was computed using the mean of the items that predicted the same domain. Then, a one-way ANOVA with Tukey’s post hoc test was used to compare ITC between professional groups and settings. All statistics used an α=0.05.

Ethical considerations

This study obtained approval from the involved Research Review Board. The study was conducted in full accordance with international ethical principles, as well as Italian legal and research ethics requirements for non-interventional studies. All participants (i.e. patients, nurses, physicians, translators) were informed about the aims and the method of the study, and were asked to provide written informed consent, as required by the Italian Legislative Decree n. 196 of 30 June 2003. Participants in each phase were also informed about the confidentiality of their answers.

Results

Consensus for the choice of the translated items was high, showing Fleiss’s kappa equal to 0.84.

Panelists were mainly females (68.75%) with a median (IQR) age of 36 (31.5-56) years. The thematic analysis of the open-ended questions to explore the panelists’ understanding and views of the items showed that the items’ clarity was the main theme. CVR (M 0.90; range 0.63 to 1.00), I-CVIS (M 0.99; range 0.88 to 1.00) and S-CVI mean of 0.98 are shown in table 2, all the indices were higher than 0.60.

Table 2

Content Validity (Phase 2)

Panelists (N=16)	CVR*	I-CVIS°	Multi-rater Kappa (95% CI for free-marginal kappa statistic)¥	S-CVI°
Item 1	0.88	1
Item 2	1	1
Item 3	1	1
Item 4	0.63	1
Item 5	1	1
Item 6	1	1
Item 7	0.88	0.94
Item 8	0.75	1
Item 9	0.88	1
Item 10	0.75	1		0.98
Item 11	0.75	1	0.74 (0.64-0.83)
Item 12	0.75	1
Item 13	0.88	0.94
Item 14	1	1
Item 15	1	1
Item 16	1	1
Item 17	0.75	1
Item 18	0.75	1
Item 19	0.75	1
Item 20	0.63	0.88
Item 21	0.88	1
Item 22	1	1
Item 23	0.88	1

Legend:

* Content Validity Ratio (CVR): to assess the pertinence through a three-point ordinal scale (1=not pertinent; 2=useful but not pertinent; 3=highly pertinent)

° Content Validity Index (I-CVIs; S-CVI): to assess the relevance, through a four-point ordinal scale (1=not relevant; 2=somewhat relevant; 3=quite relevant; 4=highly relevant)

¥ Multi-rater Kappa was computed considering two categoris (Pertinence versus no/low pertinence), 16 raters and 23 items. Multi-rater Kappa equal to 0.74 represent the 86.81% of overall agreement among panelists

The eligible participants were 688 and the responders to the study invitation (response rate) were 253 (36.8%). The demographics of the enrolled sample are shown in table 1. The majority of responders were females (71.1%), nurses (64%), employed in clinical wards (68.2%). Their average age was 43.76±10.31 years, and the mean of their working years was 19.55±11.16.

Table 1

Phase 3 sample descriptive statistics (N=267)

N	%
Gender
Male	73	28.9
Female	194	71.1
Marital Status
Married	162	60.7
Unmarried	105	39.3
Education level
High school	86	32.2
Degree	85	31.8
Post graduate*	96	36.0
Professional groups
Physicians	52	19.5
Nurses	171	64.0
Others§	44	16.5
Wards
Clinical wards	182	68.2
Ambulatory	37	13.9
Operating Room	36	13.5
Other#	12	4.5
	Mean	SD
Age (years)	43.76	10.31
Working time in the same ward (years)	11.26	9.68
Total years of working	19.55	11.16

Legend:

* Post graduate includes: master’s degree, residencies

° Clinical wards are: medicine (general and specialized), surgery department (i.e., urology, vascular surgery) pediatrics, intensive care unit and emergency department

§ Others are given by midwifes, psychologists and technicians

# “Other” refers to all those outpatient services which were out of the hospital settings (e.g., the psycho-social center to support patients with psychological disorders or substance addictions)

Internal assess reliability showed the following Cronbach’s α: Partnership=0.923; Cooperation=0.944; Coordination=0.923. The overall scale had a Cronbach’s α equal to 0.968. Overall, the categorization as “adequate” or “inadequate” level for each scale domain is shown in table 3, where the level of adequate partnership, cooperation and coordination were respectively 36.4%, 33.6%, 25.7%.

Table 3

Frequency of Adequate versus Inadequate ITC

	Adequate		Inadequate
	N	%	N	%
Partnership	92	36.4	161	63.6
Cooperation	85	33.6	168	66.4
Coordination	65	25.7	188	74.3

Comparison between groups

When the domains’ were compared between professional groups (figure 1), there were not significant differences and the lowest mean was found for coordination.

Figure 1

I-AITCS Comparison between Professional Groups

When comparison between the domains and settings was assessed (figure 2), partnership was the only domain which differed in the comparisons (F=6.13; df=2; p-value <0.001). Specifically, there were differences in partnership between settings comparing clinical wards versus other (p-value <0.001), and between ambulatory versus other (p-value=0.002), where “other” encompassed all those outpatient services which were outside of the hospital settings, such as the psycho-social center to support patients with psychological disorders or substance addictions.

Figure 2

I-AITCS Comparison between settings

The preliminary assessment on the correlations matrix given by the I-AITCS II answers showed a significant Bartlett’s test (χ2 =136.266; d.f.=88; p-value >0.001) and KMO=0.91. The authors explored the three dimensional model proposed by the theoretical structure of I-AITCS II, using an EFA. This model explained 51% of the total variance (rotated partnership=24%; rotated cooperation=14%; rotated coordination=13%), showing no cross-loadings and with all the factor loadings higher than 0.40 as shown in table 4. The model fit indices demonstrated an adequate goodness of fit: RMSEA=0.054 [95%IC=0.039–0.088]; CFI=0.960; TLI=0.948; significant test omnibus (χ2) = 1348.648, d.f.=136, P-value=0.000; SRMR=0.076. This model appeared the most suitable solution to explain the underlying dimension of I-AITCS II.

Table 4

Factor Loadings

	Loadings
Items	Mean (SD)	Partnership	Cooperation	Coordination
Item 1	3.66 (0.95)	0.802	0.061	-0.147
Item 2	3.62 (0.89)	0.708	0.161	-0.078
Item 3	3.57 (1.11)	0.604	0.337	-0.118
Item 4	3.69 (0.96)	0.475	0.347	-0.036
Item 5	3.55 (1.07)	0.505	0.239	-0.051
Item 6	3.06 (1.05)	0.621	0.251	-0.056
Item 7	3.35 (1.07)	0.582	0.272	0.111
Item 8	3.36 (1.06)	0.711	0.207	-0.137
Item 9	3.71 (0.90)	0.229	0.487	0.059
Item 10	3.53 (0.91)	0.166	0.634	0.096
Item 11	3.37 (0.99)	0.099	0.472	0.247
Item 12	3.17 (1.12)	-0.061	0.558	0.251
Item 13	3.26 (1.05)	0.064	0.749	0.091
Item 14	3.54 (1.04)	0.169	0.611	0.110
Item 15	3.60 (0.93)	0.078	0.833	-0.082
Item 16	3.56 (0.91)	0.206	0.836	-0.194
Item 17	3.47 (0.96)	0.201	0.091	0.746
Item 18	3.47 (0.97)	0.077	0.055	0.664
Item 19	3.19 (1.13)	0.102	0.137	0.419
Item 20	2.96 (1.11)	.116	-0.189	0.516
Item 21	3.34 (1.01)	0.106	-0.265	0.568
Item 22	3.25 (1.16)	0.225	0.025	0.657
Item 23	2.88 (1.16)	0.258	-0.049	0.598
Variance (%)		24	14	13

Discussion

While ITC has been shown to be measured using AITCS II in healthcare settings, it was not available for the Italian context, as it had not yet been translated nor validated. In this study we developed and validated the I-AITCS II. We found that the I-AITCS II showed evidence of validity and reliability, just as the versions in other languages. This study followed a high methodological design to avoid pitfalls that could threaten the overall validity of the translated tool. Specifically, the translation was performed using methodology to solve issues related to cross-cultural research, such as the adaptation of the items from English into Italian (14). A consensus building process was adopted to overcome the most common issues reported during the translation process, in particular cultural equivalence and topic-specific terminologies (6, 8). The overall process did not show problems with either of the above issues, and the involved researchers obtained a satisfactory agreement related to the final forward-translation (Fleiss’ K=0.84). Subsequently, the translated items were tested for face and content validity.

The response rate was limited (36.8%) among eligible participants of phase three, invited using the facility’s mailing list. However, this rate was considered acceptable, because it was in line with the survey response rate level in organizational research, which is generally lower than the data collected from individuals (4). In fact, literature showed that organizational response rate had a mean of 35.7%±18.8%, while individuals’ surveys reported a mean of 52.7%±20.4% (4). Thus, the response rate of this study (36.8%) was slightly higher than the mean rate described in literature (35.7%). For this reason, the authors did not use solicitations to improve the response rate. Further, the use of facility’s mailing list was not spread in the same way among all the eligible participants. Then, it is reasonable that some eligible participants were not reached through the study invitation. Overall, the sample size of 253 participants was consistent with the item/participants Hair’s ratio of 1:10 to ensure sufficient power for the psychometric testing (12).

The I-AITCS II’s construct validity, was established using an explorative approach to identify the possible underlying factor structure. We found that the three-dimensional structure of the original scale was maintained in the Italian version. De facto, EFA model showed a clear factor loading structure, and an adequate model goodness of fit. We also found support for the three-dimensional structure in assessing its internal consistency. Furthermore, all the domains showed no problems related to ceiling effects or cross-loadings that could undermine the interpretability of the EFA model.

In this study, the findings for ITC and its domains of partnership, cooperation and coordination were lower than reported studies conducted in Canada and Sweden (13, 19, 20). However, this topic needs to be more broadly investigated at a national level, considering that ITC is an important predictor of overall organizational wellbeing and better patients’ clinical outcomes (9, 18). At the same time the inferential results of this study have to be interpreted with caution, since the main purpose of this study was to validate the I-AITCS II, and not to describe ITC within teams within their respective settings in the Italian context. Interestingly, the domains’ description comparison between professional groups and settings showed a preliminary picture of the Italian scenario, which may be used for future investigations in the near future.

While the ITC domains between professional groups showed no significant differences, the domain of coordination showed the lowest values in all the investigated professional groups. This aspect needs to be more deeply studied to understand why, since coordination is a key ITC element within team collaboration (20). In a Cochrane review to enhance coordination, it has been suggested that strategies such as interprofessional rounds, multidisciplinary team meetings, and interprofessional audits may be used as research interventions (23). However, the paucity of interventional studies focusing on collaboration limits the level of evidence on effectiveness, due to the limited availability of studies, and limited cross country comparisons.

Secondly, the comparison between settings seems to show that only ICT’s partnership differed among in-hospital clinical wards or ambulatory and outpatient settings. However, this difference is subject to the unbalanced frequency distribution, especially for outpatient setting where only 12 participants were enrolled. For this reason we used only a non-parametric approach for the analysis, and the inferential consideration should be interpreted with caution. Specifically, this finding may be related to the type of patients (those with psychological disorders or substance abuse) care was provided for in the outpatient setting where the respondents worked. This result could be linked with the peculiarities of the involved outpatient settings, due to the recognition of patients with psychological disorders or substance abuse and family roles that can be difficult to manage. In fact, these types of health problems are largely described as barriers to achieving patient empowerment (16). Hence, structural, ideological, and cultural factors may well be the main barriers to improvement in partnership levels, through empowering patients and their families. It is recommended that barriers interfering with empowerment need to be specifically addressed through tailored continuous professional education. The focus needs to be on interprofessional team building where professional groups work with patients and their families to gain an enhanced understanding of how to achieve shared decision making around agreed upon goals (15, 16).

This study had some limitations. Firstly, the information coming from phase three must be interpreted with caution, due to the monocentric sampling and the cross-sectional data collection. Secondly, the response rate of phase three was limited, even if acceptable. Moreover, there are some limitation related to the nature of I-AITCS II, which is a self-report measure. In fact, the responses could be influenced by social desirability. However, this issue appeared unlikely in our study, due to the descriptive statistics of each item showing that responses captured sufficient variance without polarization or ceiling effects, as shown by skewness and kurtosis indices which were present very slightly in the answering. Conversely, the main study strength relates to the high methodological design, which was aimed to decrease the risk of validity pitfalls for the I-AITCS II.

Conclusions

The I-AITCS II shows good evidence of its validity and reliability. Currently, there is a paucity of available national data on ITC. The I-AITCS II will be useful in gathering Italian data to address programs aimed to enhance ITC within healthcare contexts. This study is important due to the fact that it makes possible the measurement of ITC in Italy, using the same metrics and tool used in other countries, such as Canada and Sweden. This implies the current possibility to deeply assess ITC in relation to national contextual factors, giving a clearer interpretation and description of ITC among different healthcare systems. For this reason, cross-national investigations are needed to deeply understand the relationship between ITC and contextual factors.

No potential conflict of interest relevant to this article was reported by the authors