Preliminary testing of the Swedish version of the Assessment of Interprofessional Team Collaboration Scale (AITCS-S).

Interprofessional collaboration might improve healthcare processes and outcomes; however, it has been found that most instruments that aim to measure collaboration have undergone limited testing. The assessment of interprofessional team collaboration scale (AITCS) is one questionnaire that aims to evaluate collaboration, but it has not yet been extensively tested. The aim of this study was to translate and to cross-culturally adapt the AITCS for use in Sweden, to describe floor and ceiling values, and to investigate the AITCS in terms of reliability, face, and content validity. The study included a total of 349 participants working in team-based pain rehabilitation. The participants were asked to fill in the Swedish version of the AITCS (AITCS-S) at baseline. Of these, 73 participants also completed the AITCS-S two weeks later. The results showed that the content and face validity were good. Internal consistency varied from 0.79 to 0.96 and judged to be acceptable to excellent. Test-retest stability showed excellent stability with intraclass correlation values above 0.75 for all subscales. This study concludes that the Swedish version of the AITCS is a reliable and valid questionnaire. Further psychometric investigations might be undertaken in order to attempt to develop shorter versions of the AITCS-S.

Introduction

Interprofessional team collaboration is described as the process in which different professional groups work together (Momsen, Rasmussen, Nielsen, Iversen, & Lund, 2012; Zwarenstein, Goldman, & Reeves, 2009). In reviews focusing on interprofessional collaboration, it was concluded that such collaboration could improve healthcare processes and outcomes (Zwarenstein et al., 2009). It is thus important to focus both research and practice on how to measure and facilitate interprofessional team collaboration in healthcare and rehabilitation. A systematic review has identified a number of instruments available to measure various dimensions of team structures and functioning that influence healthcare outcomes (Brennen, Bosch, Buchan, & Green, 2013). To date, it has been difficult to draw any general conclusions regarding the key components in, and effectiveness of, interprofessional collaboration. This is due to such things as the small number of studies and problems with conceptualising and measuring collaboration (Zwarenstein et al., 2009). A further issue has been limited testing of most instruments measuring interprofessional collaboration (Brennen et al., 2013).

Previous research has focused on solitary factors that hinder and promote effective team work. A literature review showed that promoting factors were related to team size, premises, meeting frequencies, shared goals, and stable staff groups (Cartmill, Soklaridis, & Cassidy, 2011; Howarth, Warne, & Haigh, 2012; Xyrichis & Lowton, 2008). In addition, several studies identified communication, both formal and informal, as an important factor for effective team work (Cartmill et al., 2011; Croker, Trede, & Higgs, 2012). Yet another important factor in improving healthcare processes and outcomes is the involvement of patients/clients in interprofessional collaborative teamwork. For example, both team-based pain rehabilitation and stroke rehabilitation strive to equally involve the patient and the rehabilitation team in order to coordinate the rehabilitation towards a shared goal (Guzman et al., 2001; Lundgren & Molander, 2008; Rosewilliam, Roskell, & Pandyan, 2011). In order to measure interprofessional collaborative teamwork, it is necessary to apply a definition which clearly includes the client/patient as an equal partner. Orchard, Curran and Kabane (2005) defines interprofessional practice as “a partnership between a team of health professionals and a client in a participatory, collaborative and coordinated approach to shared decision-making around health issues” (p.1). Orchard and colleagues (2012) have specifically designed and developed the Assessment of Interprofessional Team Collaboration Scale (AITCS) to evaluate collaboration within teams that involve the patient as part of team practice, across various practice settings. The instrument was developed based on previous literature focusing on interprofessional collaboration and the conceptual model for patient-centred collaborative interprofessional practice (Orchard et al., 2005). The final Canadian version of the AITCS consists of 37 items divided into three subscales measuring (1) partnership/shared decision-making, (2) cooperation, and (3) coordination. The subscale partnership/shared decision-making includes 19 statements, the subscale cooperation includes 11 statements, and the last subscale measuring coordination within the team includes 7 statements. Respondents are asked to rate each of the statements in relation to how they currently feel that the respondents and their team work and act within the team on a five-point scale, ranging from 1 (never) to 5 (always). The scores in each subscale are summed and maximum scores on the subscales (per respondent) are 95 on partnership/shared decision-making, 55 on cooperation, and 35 on coordination. The mean score for each subscale is calculated and a cut-off score of 4 is used to determine when collaboration within each subscale occurs (Orchard et al., 2012). The AITCS has been psychometrically tested and the internal consistency for the subscales ranged from 0.8 to 0.97, with an overall reliability of 0.98 (Orchard et al., 2005). However, the AITCS has not yet been extensively tested and further psychometric analysis is required to establish test–retest reliability, responsiveness to change, and construct validity. The aim of this study was to:

translate the AITCS and to cross-culturally adapt the AITCS for use in Sweden;

describe floor and ceiling values and descriptive statistics for the Swedish version of the AITCS; and

investigate the Swedish version of the AITCS in terms of its reliability, content, and face validity.

Methods

Design

This study is one part of a larger study focusing on identifying factors in team-based pain rehabilitation that contribute to successful rehabilitation measured by decreased sickness absence rate. Data were collected by interviews focusing on experiences of working with return to work (RTW) in team-based pain rehabilitation (Hellman, Jensen, Bergström, & Busch, 2014); register data on sickness absence; and a questionnaire including questions on the characteristics of the rehabilitation programmes, and the health professionals’ attitudes to RTW, improved health, evidence-based practice and interprofessional practice (AITCS-S). For this larger study, 558 individuals were identified from registers by administrative staff in three county councils and 533 of these fulfilled the inclusion criteria (i.e. working with team-based pain rehabilitation and being able to understand Swedish). Study participants were invited to participate by email. The email included a short text explaining the aim of the study. If the participants were interested in participating, they had to click on a hyperlink to gain online access to the questionnaire. Participants gave their informed consent to participate by logging into the questionnaire.

To be eligible for the present study, each of the participants from the larger study had to respond to 70% of the items in each subscale of the AITCS. This criterion was set to reduce the number of missing values in the study population. By this action, it is possible to specifically describe the population that is included in our actual analyses.

The translation and adaptation process

The first step of our adaptation process was to translate the Canadian version of AITCS into Swedish according to guidelines by Beaton and colleagues (2000). During this procedure, validity aspects such as content and face validity were considered. Validity refers to the degree to which an instrument measures what it is actually intended to measure (Cohen, Swerdlik, & Philips, 2012). Content validity is one aspect of validity which pertains to how well the instrument covers the intended area. In this study, content validity is based on the information from the expert group responsible for translation and adaptation of the AITCS. This group consisted of representatives from various professions—psychologists, an occupational therapist, and a nurse. Care was taken to assess the wording both from a professional and semantic perspective.

In stage I in the translation procedure, two individuals whose mother language was Swedish and who were also skilled in the English language independently translated the AITCS into Swedish. Each person created two translations, one that was literally translated and one that captured the essence of the statements. In the second stage of the adaptation process, the various translations were discussed by the two translators and differences in the translations were resolved and agreed upon. Most statements in the AITCS as well as the short instructions in the instrument were translated without difficulty. In statements where it was difficult to come to consensus, the translation was further discussed with the developer of the original version of the instrument. The third stage of the translation—back translation—procedure started when the initial Swedish translation was agreed upon. In this Swedish version, we have consistently replaced the word “care” with “rehabilitation” due to the specific target group of our study. Furthermore, in statement 31 we replaced the concept of “interprofessional collaborative practice” with a more descriptive term. Overall, the statements were considered to be content-relevant. This translation was sent to two further independent native English translators for back translation into English. In stage 4, the research group, who were experienced in rehabilitation care and included the Swedish translators and the developer of the instrument, discussed all translations and produced a pre-final version of the AITCS-S. The AITCS-S is available from the first author.

Face validity

The pre-final version was tested for face validity by sending the instrument out to 37 people working in rehabilitation teams in a region in Sweden that was not included in the larger study. These respondents rated the statements and answered some additional open-ended questions. Examples of the additional questions are “do you consider the questionnaire to be relevant to your work?”, “were there any questions that you consider not to be relevant to your work?”, “were there any questions that were hard to understand?”, and “do you have any further comment on the statements?”. The participants commented on the difficulty of answering statements in which they were supposed to consider how they currently felt that their team members thought and felt about certain issues. However, this is the formulation used within the original Canadian version and thus we decided not to change this formulation. A few participants commented on the similarity between statement 1 (“Establish agreements on goals for each patient we care for”) and statement 10 (“Are involved in goal-setting for each patient”). Due to the perceived similarities, they wished for a clearer explanation of these statements. Furthermore, the statement about adjusting the care plan together with the patient and his/her relatives (statement 19) does not distinguish between the patient and his/her relatives and that made it difficult for a few participants to answer this item. Still, 81% answered that they perceived the questionnaire as a whole to be relevant to their work with team-based rehabilitation and the majority of the participants found the statements to be understandable.

Reliability

Reliability refers to the consistency of the instrument when testing procedures are repeated on a specific population of its focus. The reliability of AITCS-S was tested by analysing the instrument’s internal consistency and the test–retest stability.

Internal consistency

The internal consistency of a scale is the measure of the degree to which different items/statements intend to measure the same characteristic. Items/statements that are supposed to measure the same characteristic should be highly correlated with each other. Cronbach’s alpha coefficient is used to investigate the internal consistency and when above 0.70 is considered to be acceptable (Tavakol & Dennick, 2011). However, if Cronbach’s alpha is too high it may indicate that some items are redundant and a maximum alpha value of 0.90 has been recommended (Tavakol & Dennick, 2011).

Test–retest reliability

Test–retest reliability refers to the consistency of the measurement results over time and is assessed with intraclass correlation coefficient (ICC). The ICC is more advantageous than Pearson’s r since it accounts for the actual magnitude of scores and the agreement between ratings, not only the correlation and linear association among variables. A common cut-off value for the instrument to be considered as stable is set to 0.75 (Weir, 2005). For this analysis, the first 90 individuals who answered the questionnaire were asked to complete the questionnaire once again two weeks after completion of the first questionnaire. The sample size of 90 was based on previous experience from similar research.

Ceiling and floor effect

Floor and/or ceiling effects refer to when individuals are reporting the lowest or highest possible score of a scale or an item and is considered to be present when more than 15% of a group’s score is at the lowest or highest possible score (Terwee et al., 2007). Consequently, when floor or ceiling effects are found the detection of improvement or decline becomes difficult.

Statistics

The statistical methods used to assess validity and reliability are described in connection to each section above. All statistical calculations were carried out using the IBM SPSS Statistics, version 22.0 (Armonk, NY, USA). Among those responding to at least 70% of the items in the respective AITCS-S scales, any missing responses in each subscale were imputed using the individual’s mean score for the specific subscale.

Mann–Whitney U tests were used for evaluating differences on the subscale scores of the AITCS between gender and workplaces, whereas Kruskal–Wallis tests were used to test for differences between age groups. All these tests were two-sided with 95% confidence intervals.

Results

Study participants

The questionnaire was answered by 383 individuals (72%) who represented various professions; 80% were women and 48% were 51 years or older. For the non-responders, we had information regarding sex for 95%; among these 31% were men and 64% were women. In this sample, 349 individuals responded to 70% of the items in each subscale of the AITCS and were thus included in the present study. A description of the study sample is given in Table 1. A vast majority was women and worked in the public sector. Almost half of the study group was above 50 years. Among the 34 subjects that were excluded due to missing responses on >30% of the items, 59% were women and 38% were 51 years or older.

Table 1.

Description of the study population (n = 349).

	n = 349
Gender, n (%)
Female	281 (81)
Male	67 (19)
Missing	1 (0)
Age, n (%)
≤30 years	21 (6)
31–49 years	158 (45)
≥50 years	167 (48)
Missing	3 (1)
Employer, n (%)
Public sector	226 (65)
Private sector	120 (34)
Missing	3 (1)
Occupation, n (%)
Occupational therapist	63 (18)
Physician	44 (13)
Psychologist	40 (12)
Physiotherapist	105 (30)
Other	93 (27)
Missing	4 (1)

The results of the AITCS-S total and subscales among all participants as well as in subgroups regarding gender, age, and workplace are presented in Table 2. Statistically significant differences between participants working in the private and the public sectors were found for all the AITCS subscales as well as for the total score of the AITCS (p < 0.001). For gender and age, no statistically significant differences were found.

Table 2.

Subscale score and total scores for the AITCS.

	Partnership/shared decision-making	Cooperation	Coordination	AITCS-S total
Total group, mean (SD), n=349	4.43 (0.45)	4.46 (0.54)	4.10 (0.66)	4.38 (0.48)
Floor effect (%)	0	0	0	0
Ceiling effect (%)	4	18	8	2
Subgroup
Gender
Men, mean (SD), n=67	4.37 (0.48)	4.46 (0.57)	4.04 (0.73)	4.33 (0.51)
Women, mean (SD), n=281	4.44 (0.45)	4.46 (0.54)	4.12 (0.65)	4.39 (0.47)
Age
20–30 years, mean (SD), n=21	4.25 (0.53)	4.17 (0.75)	3.66 (0.90)	4.12 (0.63)
31–50 years, mean (SD), n=158	4.38 (0.48)	4.43 (0.55)	4.13 (0.64)	4.35 (0.49)
51 years, mean (SD), n=167	4.49 (0.40)	4.52 (0.50)	4.13 (0.64)	4.43 (0.43)
Workplace
Private sector, mean (SD), n=120	4.53 (0.43)	4.64 (0.48)	4.23 (0.64)	4.51 (0.44)
Public sector, mean (SD), n=226	4.37 (0.46)***	4.36 (0.56)***	4.04 (0.67)***	4.30 (0.48)***

Mann–Whitney U-tests were used for evaluating differences in AITCS scores between gender and workplaces, whereas Kruskal–Wallis tests were used to test for differences between age groups. No statistically significant differences (p < 0.05) were found between gender or age groups (although age groups approached statistical significance with figures between p = 0.054 to p = 0.067).

Due to missing information on the subgrouping variables, the total numbers of subjects in respective subgroup do not add up to 349.

SD: standard deviation.

*** p <0.001.

No signs of floor effect were detected for any of the scales. However, all of the scales were negatively skewed, that is, the scores tended to concentrate to the higher end of the scale interval. Consequently, small ceiling effects were detected in the subscales partnership/shared decision-making and coordination. In the subscale cooperation, a total of 18% of the respondents reported the highest possible score, that is, showed a ceiling effect.

Cronbach’s alpha coefficients for the three subscales and the AITCS-S total sum score varied from 0.79 to 0.96 (Table 3). As visible, except for the subscale “cooperation”, all scales showed an internal consistency above 0.90.

Table 3.

Reliability analyses.

	Partnership/shared decision-making	Cooperation	Coordination	AITCS total
Internal consistency*	0.91	0.95	0.79	0.96
Test–retest**	0.78	0.79	0.77	0.83

*Cronbachs’s alpha coefficient was used to investigate the internal consistency.

**Intraclass correlation coefficient was used to investigate test–retest reliability.

The test–retest analysis was based on the 73 participants who completed the AITCS-S on two occasions (2-week interval). A higher percentage of the non-responders (71%, n = 12) were below 50 years compared to 54% (n = 39) among the respondents. Among the non-responders 65% (n = 11) were women versus 74% (n = 53) among the responders.

The stability over time was assessed by using the intraclass correlation and showed excellent stability with values above 0.75 for all subscales (Table 3).

Discussion

This study shows that the AITCS-S is a reliable and valid self-administered questionnaire. Our results regarding the reliability of the AITCS-S were comparable to those from the testing of the original Canadian version of the AITCS (Orchard et al., 2012). In addition, we tested the test–retest reliability for the AITCS-S which has not previously been carried out and found excellent stability over time.

The face validity assessed by persons working in team-based pain rehabilitation teams was considered generally good. Some comments regarding the statements were brought up when the scale was piloted. However, due to the small number of respondents who raised these issues and due to our ambition to keep the Swedish version as close as possible to the Canadian version we did not change these statements. In order to further investigate the face validity and revise the statements, it might be appropriate to perform a think-aloud study of the AITCS-S in order to examine what the respondent is thinking when they fill in the questionnaire (Collins, 2003).

The mean item score for each subscale was higher in the Swedish version of the AITCS, with values above four in all subscales, compared to the results based on the questionnaire as originally developed (Orchard et al., 2012). One possible interpretation of these results might be the differences in the samples included in the two studies. Orchard et al. (2012) included 127 participants representing seven teams from various healthcare settings, such as orthopaedic, general surgery, acute mental health, and palliative care. The teams were quite large with an average of 18 team members in each team. The sample in the present study comprised of rehabilitation teams with 3–7 team members in each team. Furthermore, analysis in the main study revealed that the teams were often located in the same accommodation and they described that they worked very closely with each other in the interviews. Thus, the rehabilitation teams consist of a small group in which patients are supposed to be included in the rehabilitation planning and that might be one possible interpretation for the high mean item scores in the present study. Still, some differences in the results were seen in the subgroups analyses in the present study. The participants working in the public sector reported significantly lower on the AITCS-S than those working in the private sector. This result indicates that not only the healthcare setting is influencing the interprofessional teamwork but also that the premises around the team might influence the clinical practice. It is known that organisational support is one important factor influencing interprofessional teamwork (Reeves, Lewin, Espin & Zwarenstein, 2010) and that might be one possible interpretation of our results. However, these results must be interpreted with caution since statistical significance is not equal to clinical meaningfulness and the AITCS has not yet been evaluated concerning how large a difference between AITCS scores must be to actually be meaningful. This needs to be further investigated in the future.

All subscales showed a ceiling effect, and the subscale measuring cooperation was scored at the maximum by 18% of the respondents. This might make the cooperation subscale less suitable for evaluation purposes (Terwee et al., 2007).

The high mean item scores in the present study might also be an expression of social desirability, that is, the participants might be reporting answers that conform to norms of how interprofessional team collaboration in team-based rehabilitation should be performed (Tourangeau & Yan, 2007). The participants in this study are all involved in a governmental financial investment for team-based rehabilitation. The standards for their professional practice are clearly outlined, and the interprofessional team collaboration is one important aspect that is emphasised. These circumstances might affect how respondents answered questions as they might be eager to show that they are working according to the directives formulated for the rehabilitation. Thus, the ceiling effect that is shown for the AITCS-S may be related to the above. This study needs to be replicated in other settings to gain further evidence about the above effects.

Concluding comments

The Cronbach’s alpha resulted in high values for the subscales and for the total score indicating a high internal consistency. Our results show that there could be redundant items and it might thus be considered whether some items can be excluded from the AITCS-S without threatening its reliability, the comprehensiveness of the scales, or the comparability with the original AITCS. A factor analysis needs to be done to further examine the factorial validity of the AITCS-S and perhaps develop a shorter version of the measure. However, before doing this analysis additional data need to be collected in order to increase the sample size and include samples not only drawn from people working with team-based pain rehabilitation. Furthermore, the validity of the Swedish version of the AITCS needs evaluation concerning its construct and prognostic validity, for instance, if patients receiving rehabilitation from a more collaborative team according to the AITCS-S also have better rehabilitation outcomes. In conclusion, the Swedish version of the AITCS-S is a reliable and valid measure to use when assessing team practice in rehabilitation settings.