Has "The Ceiling" Rendered the Readiness for Interprofessional Learning Scale (RIPLS) Outdated?

PURPOSE: The present study aimed to investigate the preliminary effects of collaborative learning and simulation on readiness to engage in and attitudes toward future interprofessional learning activities. We translated into Norwegian and validated the original Readiness for Interprofessional Learning Scale (RIPLS) (part 1) to measure the efficacy and feasibility of a structured collaborative learning activity (part 2).
MATERIALS AND METHODS: Undergraduate social and health care professional students from five Norwegian universities (n = 307) participated in the validation stage of this study (part 1). A Norwegian version of the RIPLS was developed using forward and backward translation. An expert panel discussed discrepancies between the translations and professional concepts. We planned to conduct a principal component analysis to evaluate the structure, reliability, and internal consistency of the Norwegian version of the RIPLS, after investigating the skewness, kurtosis, and range of items included. One hundred fifty students participated in collaborative learning activities; 72 (48%) of these individuals answered the translated RIPLS questionnaire.
RESULTS: We found a substantial ceiling effect in the majority of items in the RIPLS, making it difficult to use the instrument as a measure of change. We evaluated the efficacy and feasibility of the collaborative activities based on the changes in the single items that had sufficient univariate normality and ultimately confirmed positive changes in two of these items.
CONCLUSION: Norwegian students appear ready for interprofessional learning; however, due to significant ceiling effects, the majority of items in the RIPLS no longer seem suitable for measuring and evaluating the effects of interprofessional learning (part 1). Single-item analysis revealed a potential effect of collaborative learning (part 2). A new questionnaire is needed where readiness is instead understood as self-efficacy in areas such as role awareness and interprofessional communication. Researchers should be aware that even previously validated questionnaires may lose their applicability over time and require revision. Demands for interprofessional learning and practice are continuously evolving, and evaluation methods should be adjusted accordingly.

Introduction

The growing complexity of care and rising rates of morbidities necessitate the introduction of multidisciplinary health care practices with good collaboration between health professionals to ensure effective and safe patient care is provided.1–3 The World Health Organization emphasizes the need to learn collaborative care needs through interprofessional education,4 an initiative supported by many governmental policies—Norway included.5 In this context, interprofessional learning (IPL) and competencies develop through interactions between professionals.6

Learning with, from, and about members of other health care professions has been suggested to improve interdisciplinary collaboration and healthcare delivery quality.7 Several publications support the notion that IPL improves the attitudes and perceptions of other professions, along with strengthening collaborative knowledge and skills.8,9 To this end, several assessment tools have been developed to assess different aspects of IPL.10 The Readiness for Interprofessional Learning Scale (RIPLS), developed by Parsell and Bligh,11 is one of the most commonly used options at this time.12 An accurate understanding of the individual’s attitude toward interprofessional collaboration and prejudices against other professions is essential for IPL to be effective. Specifically, Parsell and Bligh11 stated that students’ attitudes toward shared learning activities must be assessed to evaluate their readiness to participate in shared learning since the outcomes of teamwork and collaborative skills depend greatly upon their attitudes regarding IPL. The RIPLS is therefore designed to explore undergraduate students’ perceptions and attitudes towards IPL.

Originally written in English, RIPLS has been translated into different languages, including Swedish,13 French,14 German,15 Danish,16 Dutch,17 Japanese,18 Chinese,19 and Persian,20 but not yet into Norwegian. Questionnaires need validation before their use in any new language and culture.21 The purpose of this study was therefore to investigate the preliminary effect of a structured interprofessional collaborative learning activity. To do so (part 1), we translated the RIPLS into Norwegian and validated it for use among Norwegian-speaking health care students and professionals. Second (part 2), the validated questionnaire was used in a pre/post design intervention to evaluate the readiness for and attitudes toward IPL among medical and nursing students. The intervention was performed in the 2018–2019 academic year, involving 140 bachelor’s degree students in nursing and 10 medical students participating in interprofessional education for seven days over seven months. The collaborative activity consisted of five days of interprofessional simulation and two days of lectures and team-based learning. Simulation is a well-known collaborative activity in IPL,22 while team-based learning is a purposeful method for collaborative learning.23 This project’s uniqueness lay in extending the training over the entire academic year to give the students repeated experience in IPL.

Methods

In part 1 of the project, we translated and validated a Norwegian version of the RIPLS for use in part 2, where we sought to investigate the feasibility and efficacy of a series of collaborative learning activities. Translation of the instrument was conducted initially, while both the data sampling for validation and pretest measurements were conducted simultaneously (based on different samples). To present how these parts of the study were conducted with regard to data sampling and analysis, results, discussions, and decisions of how to proceed with the study, we herein present information on the study methods, results, and discussion relating to each of the two parts.

Ethics

The questionnaire for the validation study (part 1) was made available for targeted health care students through their web message board, together with information about the purpose of answering the questionnaire. The questionnaire did not contain any personal data that could identify the students and was administered through the university’s secure database for surveys, with respondents’ Internet protocol addresses hidden from the researcher to prevent any direct or indirect identification of the students; thus, the study (part 1) did not require approval from an ethics committee. The intervention study (part 2) was approved by the Norwegian Data Protection Centre. During this part of the study, the students were informed by written information about the interprofessional education that was implemented in the curriculum and compulsory for all students, but were reassured that answering the questionnaires was voluntary.

Part 1

Materials and Methods

The Norwegian translation was based on the adapted RIPLS retrieved from Latrobe Community Health Service and the Health and Socialcare Interprofessional Network.24

Expert Panel (for Translation, Adaptation, and Revision)

Four professionals in nursing, medicine, and psychology with strong knowledge of both the English and Norwegian languages were invited to be the expert panel. Each participant conducted their forward translation, while a native English–speaking translator who also spoke Norwegian fluently did the backward translation. The panel discussed discrepancies in the translations and any variations in professional concepts that arose between before and after the initial translation or after the backward translation of the instrument.

Pretesting

In addition to the expert panel of four professionals, a panel of three experts in nursing and nursing education reviewed the Norwegian translation and the back-to-English translation. Comments from this panel were included when drafting the final version of the translated questionnaire. The main concerns regarded uncertainty in the phrasing of questionnaire items, which the expert committee took into consideration while attempting to retain the original meaning from the English version.

Data Collection

The researchers distributed the questionnaire to undergraduate students at five universities in Norway. In total, a sample of 307 students responded to the questionnaire. Table 1 displays their demographic characteristics. The sample included mostly women (86%), while most respondents were in their second (42%), first (31%), or third (19%) year of professional education, respectively. In Norway, medicine is a six-year program and pharmacy, nursing and social studies are bachelor’s degree programs of three years. Bachelor’s degree students were requested to answer at all academic years, but for medical students, only those from the third year were invited to answer the questionnaire since they start with clinical practice in their third year of study.

Table 1

Demographic Characteristics of Students’ Participation in Interprofessional Education

Characteristics	Profession
	Pharmacy	Medicine	Nursing	Social Studies
Number of respondents, n (%)	13 (4.2)	30 (9.8)	253 (82.4)	11 (3.6)
Sex, n (%)
Female	11 (84.6)	24 (80.0)	221 (87.4)	9 (81.9)
Male	2 (15.4)	6 (20.0)	32 (12.6)	2 (18.2)
Year of study, n (%)
First	6 (46.2)		89 (35.6)	11 (100)
Second	4 (30.8)		125 (50.0)
Third	3 (23.1)	7 (23.3)	36 (14.4)
Fourth through sixth		23 (76.7)

Statistical Analysis

Data were analyzed using the Statistical Package for the Social Sciences version 23.0 (IBM Corporation, Armonk, NY, USA). Descriptive statistics were conducted to assess the properties of included items in the translated RIPLS. Principal component analysis, with varimax rotation, was performed to assess the factor structure in the translated RIPLS.11

Part 2

Part 2 of the project evaluated the efficacy and feasibility of interprofessional collaborative learning activities among students.

Content of the Structured Collaborative Activities

In the 2018–2019 academic year, 140 second-year bachelor students in nursing and 10 third-year medical students participated in an interprofessional education initiative for seven days, which consisted of two days of thematic teaching based on team-based learning and five days of simulation.

Participants and Data Collection

Participation in the learning activities was integrated as a compulsory part of the curricula of both the medical and nursing students, but participation in answering the pre- and postinterventional questionnaires (RIPLS) was voluntary. One hundred twenty-eight students answered the questionnaire, of whom 16 only answered the preinterventional questionnaire and 40 only answered the postinterventional questionnaire. Thus, 56 participants were excluded, leaving a sample of 72 participants composed of eight medical students (response rate: 80%) and 64 nursing students (response rate: 45.7%).

Data were analyzed using the Statistical Package for the Social Sciences version 23.0 (IBM Corporation, Armonk, NY, USA). Mean and standard deviation values were calculated for all items of the RIPLS questionnaire and compared between before and after the intervention by paired-sample t-tests and Wilkinson’s nonparametric test. Internal consistency of the instrument was measured with Cronbach’s alpha. The significance level was set at p < 0.05 for all tests.

Results

Part 1

Table 1 shows the demographic characteristics of the 307 students who answered the questionnaire for the validation study (part 1).

Table 2 presents the mean, skewness, and kurtosis values for all items of the questionnaire. The results indicate a substantial ceiling effect exists for most items, except for items 12, 18, and 19. A principal component analysis with varimax rotation of the original 19-item questionnaire resulted in four subscales with an overall Cronbach’s α of 0.85 and loading factor α-values varying from 0.32 to 0.85 (Table 3).

Table 2

Descriptive Statistics for Each Item in the Norwegian-Language Version of the RIPLS Questionnaire (n = 307)

Question	Mean (SD)	Median	Mode	Skewness	Kurtosis
Q1	1.27 (0.56)	1	1	2.50*	8.63*
Q2	1.18 (0.50)	1	1	4.05*	22.57*
Q3	1.39 (0.64)	1	1	1.85	4.40
Q4	1.44 (0.71)	1	1	1.96	4.56
Q5	1.24 (0.55)	1	1	2.81*	10.23*
Q6	1.54 (0.79)	1	1	1.61	2.73
Q7	1.37 (0.61)	1	1	1.77	4.11
Q8	1.50 (0.72)	1	1	1.49	2.46
Q9	1.14 (0.43)	1	1	4.11*	23.79*
Q10	1.71 (1.03)	1	1	1.65	2.23
Q11	1.54 (0.82)	1	1	1.83	3.64
Q12	2.11 (1.11)	2	1	0.87	0.00
Q13	1.55 (0.72)	1	1	1.72	4.68
Q14	1.44 (0.68)	1	1	1.58	2.63
Q15	1.60 (0.84)	1	1	1.41	1.68
Q16	1.53 (0.76)	1	1	1.57	2.79
Q17	1.38 (0.65)	1	1	2.07*	5.91*
Q18	2.39 (1.07)	2	2	0.39	−0.76
Q19	2.96 (1.00)	3	3	−0.12	0.11

Note: *Violation of univariate normality (ie, skewness > 2 and/or kurtosis > 7).

Table 3

Principal Component Analysis of the Translated RIPLS in the Norwegian-Language Version of RIPLS

	Factor Loading
Item No.	I	II	III	IV
Cronbach’s α = 0.85	α = 0.83	α = 0.62	α = 0.85	α = 0.32
1	0.73
2	0.82
3	0.63
4	0.42
5	0.53
6	0.42
7	0.69
8	0.59
9	0.53
10		0.74
11		0.66
12		0.71
13			0.74
14			0.70
15			0.76
16			0.70
17			0.77
18				0.78
19				0.69

Part 2

Table 4 shows the demographic characteristics of the sample of students participating in the interprofessional education program with simulation (part 2). The sample was saturated by nursing students and by female students in both professions.

Table 4

Demographic Characteristics of Participants in the Interprofessional Education Initiative

Characteristics	Medical Students	Nursing Students
Number of participants, n (%)	8 (11.1)	64 (88.9)
Sex, n (%)
Female	6 (75.0)	60 (93.8)
Male	2 (25.0)	4 (6.3)
Age
Mean (SD), years	24.6 (1.4)	21.9 (3.1)

Most of the questionnaire items were skewed, except for items 6, 8, 18, and 19 in the preinterventional survey and 12, 18, and 19 in the postinterventional survey, respectively. Paired t-tests were conducted to investigate changes in items that met the assumption of univariate normality. Out of 19 items, two (4 and 18) changed significantly (Table 5).

Table 5

RIPLS Before and After Interprofessional Learning Activities (n = 72)

	Before Intervention			After Intervention
Item	Mean (SD)	Skewness	Kurtosis	Mean (SD)	Skewness	Kurtosis	Cohen’s da
Q1	1.25 (0.47)	1.60	1.59	1.19 (0.43)	2.10	3.78
Q2	1.07 (0.26)	3.46	10.26	1.13 (0.33)	2.32*	3.46*
Q3	1.35 (0.65)	1.99	3.82	1.42 (0.60)	1.34	0.32
Q4	1.47 (0.56)	0.62	−0.69	1.32 (0.50)	1.14	0.09	0.26
Q5	1.33 (0.69)	2.86	10.73	1.29 (0.49)	1.31*	0.58*
Q6	1.42 (0.60)	1.14	0.32	1.43 (0.62)	1.52	2.91
Q7	1.22 (0.45)	1.83	2.52	1.34 (0.58)	1.99	5.17
Q8	1.46 (0.67)	1.17	0.16	1.44 (0.77)	2.12	5.65
Q9	1.08 (0.28)	3.08	7.70	1.14 (0.45)	4.30*	22.52*
Q10	1.33 (0.75)	3.47	14.51	1.18 (0.51)	3.49*	14.15*
Q11	1.24 (0.46)	1.71	2.02	1.19 (0.40)	1.58	0.50
Q12	1.85 (1.03)	1.03	0.22	1.78 (0.88)	1.23	1.78
Q13	1.54 (0.80)	2.37	7.88	1.54 (0.77)	1.58*	2.50*
Q14	1.44 (0.73)	2.21	6.99	1.28 (0.61)	3.60*	18.59*
Q15	1.57 (0.89)	1.98	4.64	1.60 (0.93)	1.98	4.19
Q16	1.43 (0.73)	2.27	7.24	1.46 (0.63)	1.40*	2.55*
Q17	1.31 (0.64)	3.23	14.78	1.29 (0.68)	3.42*	14.52*
Q18	2.50 (0.99)	0.18	−0.63	2.06 (0.92)	0.68	−0.22	0.41
Q19	2.74 (0.93)	0.13	0.26	2.78 (0.97)	0.16	0.32

Notes: aEffect size of paired t-test measured with Cohen’s d if the mean difference was significant (< 0.05). Missing values on distinctive variables did not exceed 3. *Violation of univariate normality (skewness > 2 and/or kurtosis > 7); t-tests were not conducted for these items.

Data were analyzed for possible differences in total sum scores and changes in sum scores between before and after the intervention. No associations were found for sex or professional education.

Discussion

In this study we planned to evaluate the efficacy and feasibility of a collaborative learning initiative (part 2). In order to do so, we translated and validated (part 1) one of the most established instruments used for evaluating interprofessional learning – the readiness for interprofessional learning scale (RIPLS). Due to substantial ceiling effects using the translated instrument, we need to discuss the potential reasons and implications for this, still in two sequential parts.

The percentage distribution of professions and sexes in our sample (of part 1) differs from the actual distribution seen in these health professional studies in Norway. Nursing students are overrepresented by approximately 18%, while medical students and social care students are underrepresented by approximately 10% and 12%, respectively. Also, female students are overrepresented by 5% to 10%, except among nursing students, with almost the same percentages as the actual distribution seen in nursing study programs.25

The principal component analysis with varimax rotation of the original 19-item questionnaire resulted in four subscales (Table 3), similar to those of McFadyen et al,26 except for item 17, which achieved better fit with subscale III (positive professional identity). The results for the Norwegian version of the questionnaire provoked two significant issues for discussion. First, subscale IV obtained an internal reliability α-value of only 0.32, which is considered low. However, this low internal consistency is consistent with the findings of Parsell and Bligh,11 who first reported an α-value of 0.32 for the roles and responsibility subscale. Later validation studies have discussed the same issue,13,19,26–28 and the validity of the instrument has been questioned.29

Second, another major problem was the skewness and the existence of a strong ceiling effect, which created a challenge in using the questionnaire for evaluation studies to measure changes in the readiness for IPL. This issue has been previously explored and suggested as being the result of selection bias.16 We instead assert that the ceiling effect could demonstrate that young health care students already have a high degree of readiness for IPL.30,31 We assume this is due to persistent changes in attitudes toward interprofessional collaboration in the last few decades.

We intended to translate the RIPLS into Norwegian to validate the instrument for use in efficacy and feasibility studies in IPL activities. Because of the skewness and ceiling effect in most of the items, however, we argue that the RIPLS seems less suitable for assessing IPL activities in a Norwegian context. The skewed data in our material distort the potential to use the scale to measure changes in evaluation studies.32 Here, we could have engaged in a long discussion about the differences in suggested subscale structures;29 for example, one might consider the use of a different point scale instead of the five-point scale. Studies have shown that the involvement of a broader scale does not limit floor or ceiling effects; instead, it has the opposite effect.33

Item 18, belonging to the role and responsibility subscale, was one of the few items without pronounced skewness. Although this item belongs to the subscale with unexcepted reliability, we agree with the opinion of Edelbring et al,34 who stated that, “given its importance, it is problematic that this dimension of IPL does not function well in the RIPLS.”

Since the RIPLS translation was conducted first, while data sampling for validation and pretesting the collaborative learning activity were performed simultaneously, we had no opportunity to make changes in the measurements of efficacy based on the results from this part of the project. We continued the study (part 2) using the Norwegian translation of the RIPLS, including items that had adequate univariate normality, and conducted a single-item analysis of data in the interventional study presented in part 2.

The results of the study with medical and nursing students (part 2) retained the same problem as for the validation study (part 1), with a high degree of skewness and a notable ceiling effect, making the questionnaire less viable for evaluating the effects of interventional studies. Two questions revealed significant positive changes: items 4 (“communications skills should be learned with other health and social care student/professionals”) and 18 (“I am not sure what my professional role will be/is”). Communication skills were a focused learning area in the collaborative learning activity using the ISBAR (Identify, Situation, Background, Assessment, Recommendation) communication tool for structured communication between the nurse and physician. Therefore, it was no surprise that item 4 achieved a change in evaluation, albeit of a minor effect. Only item 18 showed a moderate effect size, with Cohen’s d greater than 0.30.35 This question belongs to the subscale “roles and responsibilities,”11 which retained low internal consistency.11,13,14,19,26–28 Interestingly, this was almost the only question with any relevance to measuring change in the Norwegian-language translated version of the RIPLS. By face validity, role awareness should clearly be included in a questionnaire on IPL. Thus, it is a problem that the internal consistency reliability for this item almost excluded it from the instrument. Our findings revealed enhanced awareness of learning communication skills and of professional role awareness among respondents. We believe interprofessional simulation is an excellent method for developing interprofessional skills. Several studies have confirmed the same beneficial outcomes.36–38

The results indicate improved role awareness exists after IPL with team-based learning and simulation. We believe that interprofessional simulation helps students to be aware of their role and the roles of other professionals in the health care team.39,40 These results do not surprise us given the interaction between health care professionals and the patient. The role of the professional team members becomes evident through the process of assessing patients’ needs and evaluating problems and decisions regarding interventions. The same learning outcome is achieved through team-based learning programs, where heterogeneous groups working together allow a good understanding of the role differences and strengths of each profession in the group.41 Item 18 asks explicitly about professional role awareness, but item 6 also asks about role awareness in terms of “my own professional limitations.” We expected to find an association between these two items, but ultimately did not. The scoring did not change from before to after the intervention for item 6. Both of these items ask about self-awareness about the professional role. Still, the RIPLS contains no explicit question regarding the respondent’s understanding of the roles of other professionals in the health care team. Both concepts of understanding should have been included since they are essential knowledge for clarifying roles and responsibilities.

Numerous instruments for assessing IPL have been developed; however, one limitation for all these instruments is the lack of integrated theoretical and psychometric framework.10,42–44 Considering the analysis of our Norwegian-language translation, five of 19 items should be discarded because of skewness. The Norwegian translation needs to be further tested and evaluated for subscale structures if planned for use in evaluation studies of the readiness for IPL. We propose that the RIPLS is becoming outdated for assessing readiness for IPL since professional students, at least in Norway, seem to have acquired this readiness. We suggest looking beyond readiness for IPL and investigating the conceptual content of collaborative learning. More research is needed to develop scientifically based IPL curricula, along with instruments for evaluating interprofessional competencies. The Interprofessional Education Expert Panel45 has defined four core competencies for IPL: values/ethics for interprofessional practice, roles/responsibilities, interprofessional communication, and teams/teamwork. Work on developing instruments with sound psychometric properties in the framework of these concepts needs to be continued.

This study has some limitations. We performed a validation study of the translated Norwegian questionnaire simultaneously as the pretest of the intervention study, drawing the conclusion that the RIPLS is outdated for use in evaluation studies among health care professional students in Norway. Given this result, one might ask why we continued to use the instrument in the intervention study: we used it due to the lack of other validated instruments, but, additionally, the pre- and posttests served as a second evaluation method of the feasibility of the translated questionnaire.

The convenience sampling may have caused selection bias. Due to the cross-sectional design of our validation study, as part of an intervention process, we were unable to measure the test-retest reliability of our translated version of the instrument. We were also unable to test the sensitivity to change. Although the number of respondents was sufficient, the majority were nursing students. Our notion that attitudes toward IPL seem to have reached a threshold, which reduces the applicability of the existing RIPLS, should be interpreted with caution, both across different cultures and among the professions that were less highly represented and other professions not included at all in this study. Our study also boasted more female than male students, which represents the disproportion seen in health care education today. Importantly, this scale was validated for use by undergraduate students only and no professional health workers were included; involvement of these individuals might lead to different results.

Conclusion

Our evaluation of the feasibility and efficacy of a series of structured collaborative learning activities was somewhat derailed by problems with the translation and validation of our selected outcomes measurement (RIPLS). Notably, a ceiling effect greatly influenced the translated version. In the Norwegian context, students seem ready for interprofessional learning. Based on this, we argue that questionnaires evaluating attitudes and cultural dimensions of health care practice needs to be continuously revised, not only across cultures but also across time. In a continuously changing, professionalized, and collaborative health care industry, raising the bar of interprofessionalism should also include regular revision of evaluative questionnaires, both in education and practice, so as to further improve health care for patients of the future.

We acknowledge the impact that RIPLS has had, and will continue to have, on addressing attitudes towards IPL. However, to further professionalize interprofessional learning, an important question is: are the students prepared for interprofessional work? We argue that future studies should move away from exploring attitudes toward interprofessional learning to role awareness, specific skills, and the four core competencies put forth by the Interprofessional Education Expert Panel.