Improvement of quality and safety in health care as a new interprofessional learning module - evaluation from students.

BACKGROUND: Interprofessional teamwork is in many ways a norm in modern health care, and needs to be taught during professional education. DESCRIPTION: This study is an evaluation of a newly introduced and mandatory learning module where students from different health profession programs used Improvement of Quality and Safety as a way to develop interprofessional competence in a real-life setting. The intention of this learning module was to integrate interprofessional teamwork within the students' basic education, and to give students a basic knowledge about Improvement of Quality and Safety. This report focuses on evaluations from the participating students (n=222), mainly medical and nursing students.
MATERIALS AND METHODS: To evaluate this new learning module, a questionnaire was developed and analyzed using a mixed methods design, integrating both qualitative and quantitative methods. The evaluation addressed learning concepts, learning objectives, and interprofessional and professional development. RESULTS AND
CONCLUSION: A majority of students responded positively to the learning module as a whole, but many were negative towards specific parts of the learning module and its implementation. Medical students and male students were less positive towards this learning module. Improvements and alterations were suggested.

Introduction

Working in interprofessional teams has become more established in today’s health care.1 Therefore, future health care workers need to practice this during their education.2–5 The Faculty of Health Sciences at Linköping University, Sweden, has used problem-based learning (PBL) and interprofessional learning1,6 as cornerstones in its educational programs since 1986. Three interprofessional learning modules, designed to develop higher competencies over time, are mandatory for students in all health programs (medicine, nursing, physiotherapy, occupational therapy, biomedical analysts). This paper reports the students’ evaluations of a newly developed learning module in the students’ third year, when they work in interprofessional teams to practice Improvement of Quality and Safety (IQS) in clinical settings. Apart from its theme, the novelty about this learning module is twofold: 1) that it is conducted in a clinical setting, and 2) that it is a mandatory interprofessional module for all undergraduate students of the medical faculty.

Integrating quality improvement in professional health education, as done in this module, represents “a new way of viewing students as change agents in clinical practice” as they simultaneously learn about the systems and processes that improve safety for patients, and report back to the clinic.7 In this way, the students also function as “vectors” to transfer new knowledge about IQS into the clinic. Making all concerned professions cooperate is an effective way of performing IQS.8 In this learning module, this was simulated by asking students to work in interprofessional groups.

To be safe and effective future practitioners in health care, students need to take in critical competence of IQS.9 PBL, the traditional research process, and systematic IQS all resemble the knowledge improvement PDSA (Plan–Do–Study–Act) cycle.10 These processes start with a real problem, and involve similar questions and tools. Because of these resemblances, it should be easy for PBL-trained students to understand IQS.11 The new learning module evaluated here is tailored to train interprofessional cooperation, to prepare students for continuous improvement of quality and safety, and to enable critical thinking and lifelong learning.

A few studies already describe interprofessional education modules on IQS, but with mainly medical and nursing students, and usually as an elective course.12–15 In a study with voluntary students, Cusack and O’Donoghue showed that interprofessional education leads to changes in students’ knowledge, skills, attitudes and beliefs.12 Using self-reflection by a small number of students, Robichaud et al found that participating in a quality improvement project can be an excellent vehicle to promote interprofessional collaboration.13 Headrick et al described multicenter efforts to integrate quality and safety into curricula to foster joint learning, but found that few educational programs were able to measure changes in students’ behaviour, changes in organizational practice, or benefits to patients or clients.14 To our knowledge, large-scale student evaluations of IQS in a clinical setting, used as an interprofessional learning object, mandatory for all undergraduate students of a medical faculty, have not previously been reported.

Aim

The aim of this study is to describe, evaluate, and discuss a new interprofessional learning module on IQS within health care. Based on a student questionnaire, we wanted to know if the students experienced the module as an effective way of learning interprofessional teamwork and IQS. We also discuss the results of the evaluation in terms of which student groups were most satisfied with their learning, and how the course could be improved.

Description of the evaluated learning module

At the beginning of the two-week learning module on IQS, students from different health programs (medicine, nursing, physiotherapy, occupational therapy) are formed into small interprofessional groups, with an academic supervisor. Each group is allocated a clinical ward or primary health care center that has advertised an IQS project. A local facilitator introduces the group to the improvement area at hand, for instance to improve the discharge routines at a surgery ward, or to advance hygiene routines in primary health care. The student group discusses and decides on a focus for their improvement project.

The student group explores the problem area by interviewing the clinical staff involved and by investigating and measuring relevant routines and habits. The students then systematize their observations and gained knowledge, formulate general questions, and identify learning goals. In parallel with this, students participate in mandatory seminars and are offered theoretical lectures on IQS. After studying theories, students apply their gained knowledge on the chosen project. Their reports with suggestions for improvement and for measuring the outcome are assessed by the supervisor and then presented by the students to the staff of the clinic. The staff of the clinic are free to integrate the students’ contribution into their routines and their everyday quality improvement work.

Materials and methods

It is common practice within the faculty to distribute evaluation forms to students at the end of each learning module. A new evaluation questionnaire was developed and tested, and face validated in our research group during a 2-year pilot phase before the module was implemented for all students. This evaluation questionnaire was handed out to all students enrolled in the learning module in spring 2012 (N=248), and submitted voluntarily and anonymously. The questionnaire was answered by 90% (n=222) of the students. Of these, 53% were nursing students, 23% medical students, and 24% were students from other health science programs (physiotherapy, occupational therapy). Among respondents, 71% were between 21 and 25 years old, and 71% were female. The average response rate for the open-ended questions was 48%.

The questionnaire posed 19 statements to be answered on a six-point Likert scale, and three open-format questions. Themes addressed were: learning module concept and implementation, fulfilment of learning objectives, lectures given, professional and interprofessional development, and practice involvement. The open format questions asked the students to give examples regarding interprofessional group interaction and dynamics, and to give additional comments and suggestions.

A mixed methods design integrating qualitative and quantitative methods was used for data analysis and to compare the graded Likert scale answers with the textual data.16

First, we coded the textual answers using conventional qualitative content analysis,17 including structuring the data into themes and categories. We also established criteria and assigned negative/neutral/positive labels to the textual answers according to their content and tone. Analyses of the textual answers were done independently by two of the authors (KG and CJT) and then merged. To allow for comparison with the quantitative data, we divided our quantitative data into the same three categories accordingly. In this, we labeled points 1–2 on the six-point Likert scale as negative, 3–4 as neutral, and 5–6 as positive. We compared quantitative and textual answers regarding the same themes to scout for differences.

Second, the quantitative answers were simply dichotomized; ie, points 1–3 in the scale were regarded as negative answers, while 4–6 in the scale were regarded as positive answers. We analyzed questions regarding concept and implementation, learning objectives, reflection on professional roles, and perceived effects of interprofessional teamwork, and of the IQS project. We also analyzed our quantitative data for differences that could be attributed to sex or program affiliation. For this we used Pearson’s one-tailed chi-square test. A P-value of <0.05 was considered statistically significant. All quantitative data were stored in a database and statistically analyzed using SPSS 20 (IBM SPSS Statistics, IBM Corporation, Armonk, New York, USA).

Results

A majority of the students, 69%, reported that their improvement project had helped them reach the learning objectives of the module (Figure 1). A majority (82%) also stated that they were able to apply methods and tools for their improvement work (Figure 2). About two-thirds of the students (64%) believed their project would lead to better quality or safer care for the patient (Figure 3).

Figure 1

Responses to the statement, “Our improvement project contributed to us reaching our learning objectives.”

Figure 2

Responses to the statement, “I can apply methods and tools in practical improvement work.”

Figure 3

Responses to the statement, “Our improvement project will lead to better care/health for the patient.”

Answers about professional development in IQS showed mixed results. A majority (80%) stated that they had developed their professional knowledge and competence about improvement work (Figure 4). Sixty-seven percent said they had developed their ability to work together with other professions (Table 1).

Figure 4

Response to the statement, “During this learning module, I have developed my knowledge and competence about improvement work.”

Table 1

Improvement of Quality and Safety as interprofessional learning

Chapter/statement (abbreviated)	Answers (n)	1 not at all (%)	2 (%)	3 (%)	4 (%)	5 (%)	6 very much (%)	Difference between sexes	Program affiliation difference	Figure/note
Learning module concept
Our improvement project contributed to us reaching our learning objectives	216	3.6	11.7	13.1	35.6	26.1	7.2	NS	Medical < other studentsP=0.0135	Figure 1
Learning objectives
I can apply methods and tools in practical improvement work	221	1.4	5.0	11.3	34.2	32.0	15.8	Male < femaleP=0.0185	Medical < other studentsP<0.001	Figure 2
I can analyze complexity in change processes	219	2.3	6.8	19.8	25.2	36.0	8.6	NS	Medical < other studentsP=0.011
I can describe my own interprofessional competence	220	11.3	23.0	17.6	23.9	15.3	8.1	Male < femaleP=0.0185	Medical < other studentsP=0.0035	Figure 5
I can describe how my own and other’s professional knowledge and approach influences the organization’s outcome	222	11.7	13.1	22.5	27.0	18.5	7.2	Male < femaleP=0.043	Medical > other studentsP<0.001
Professional and interprofessional development
I had sufficient prior knowledge to carry out an improvement project	218	1.4	6.8	18.0	32.0	31.1	9.0	NS	Medical > other studentsP<0.001
I have developed my knowledge and competence about improvement work	221	3.6	5.9	10.4	33.3	32.4	14.0	Male < femaleP=0.0095	Medical < other studentsP<0.001	Figure 4
I have developed my ability to solve problems	222	5.4	9.0	18.9	32.0	29.7	5.0	Male < femaleP=0.003	Medical < other studentsP<0.001
I have developed my ability to work together with other health care professionals	221	9.0	10.4	14.0	24.3	24.3	17.6	Male < femaleP=0.016	Medical < other studentsP<0.001
Practice involvement
Our improvement project will lead to better care/health for the patient	193	6.8	8.1	8.1	24.3	24.8	14.9	NS	Medical < other studentsP=0.013	Figure 3

Notes: This table describes response distribution and significant differences in response distribution based on sex and educational program affiliation using the Pearson’s chi-square 1-tailed test. N=222 answers (total).

Abbreviations: n, number; NS, nonsignificant difference.

Analysis of our quantitative data showed differences between the students, attributable to program affiliation and sex. Medical students rated significantly lower on all the qualitative questions. There were no significant differences between the students of the other programs. However, on most questions, males rated lower than females (Table 1). Overall, 64% of all 222 respondents reported that they believed that their improvement project would lead to better care/health for the patient. Two-thirds of the students stated that they had developed their cooperative abilities; one-third stated that they had not.

Only 53% felt able to describe and evaluate how their own and other’s professional knowledge would influence the organizations’ outcome (Table 1). This aspect was confirmed by the open format answers. Only a few students gave examples of “how different professional competences contributed to the work”. Most students attributed the groups’ success to interpersonal dynamics rather than to professional competences. This also echoed the low percentage of students able to describe their own interprofessional competence and professional knowledge (Figure 5).

Figure 5

Responses to the statement, “I can describe my own interprofessional competence.”

When asked for examples of how the different professional competences in their group contributed to the project, 46% out of 125 answers were negative, 22% were neutral, and 31% were positive. This compares to the ratings on “I can describe how my own and other’s professional knowledge and approach influence the organization’s outcome”, where 25% were negative, 50% neutral, and 26% positive. Inability to describe how interprofessional work took place seems to echo inability to see how outcome was affected by improvement work as an interprofessional enterprise.

Sixty-six students (30%) wrote comments about the learning module, providing additional insight. About two-thirds of these comments were categorized as negative; the rest were neutral or positive. Main themes identified in these comments were frustration and anger. Many challenged the concept of the learning module and questioned whether this was a good way to gain interprofessional competence. Criticism about the practical implementation of the learning module, irrelevant improvement projects, and insufficient cooperation and coordination by the university and the involved clinics was also raised by the students.

To help interpret these responses, we compared the comments to students’ ratings on statements about the same themes, specifically “During this learning module I have developed my knowledge and competence about improvement work,” and “During this learning module I have developed my ability to work together with other professions”. Combined, these questions had 44% positive, 41% neutral, and only 14% negative ratings. The open format questions thus revealed a more negative attitude than the graded questions, although with lower response rate.

Discussion

Although IQS is a part of many curricula in health care education,12–15 not many of these courses have an interprofessional design that includes students from all health professional programs. Thus, the results and insights from this evaluation can be valuable for others designing similar learning modules. Our main result from the student evaluation of this learning module was that the students generally believed that they had increased their interprofessional competence and their competence to perform improvement work. However, we also found that many students could not describe how interprofessional teamwork occurred and how it contributed to the improvement project.

Although about two-thirds reported that they had developed their problem-solving abilities and ability to work in interprofessional teams, students reported that the projects did not seem to need all the participants’ professional competences. In both quantitative and open format answers, it appeared that the students did not know what professional competence they had and how it was used.

Several factors may explain these results. The students’ theoretical knowledge on IQS and their clinical experience might have been insufficient at the time they participated in the learning module. Medical students rated their prior knowledge higher than did other students, but they responded less positively on all analyzed statements. Female students responded more positively than male students on questions regarding the fulfillment of learning objectives and professional and interprofessional development. Our data seems to be in accordance with the findings reported by Wilhelmsson et al,18 who found that female nursing students were more ready for teamwork and interprofessional cooperation.

The projects were also perceived to revolve too much around the nursing profession and were therefore considered less relevant to students of other programs, especially of the medical program. This may have consequences for other students’ involvement and feeling of participation in this learning module, and can affect their attitudes to interprofessional work.

Imbalance in the number of students from the different programs gives an imbalance in group composition. This may play a role in how interprofessional learning modules are experienced as effective or not, and also affects the results of the quantitative analyses.

To resolve the “nursing focus” of the projects, many comments suggested the use of theoretical improvement projects, through multimedia techniques. No longer doing projects on site would mean the loss of the practical experience. On the other hand, theoretical projects could be constructed as more complex, and relevant for all participating student categories. Theoretical projects might also be easier to fit in tight schedules, making participation easier. Adequate practical implementation, relevant improvement projects, and better coordination and cooperation by the university and involved clinics/hospitals seem to be crucial for the success of the learning module.

Understanding the concept of the clinical microsystem19–22 is seen by many as essential in the planning and execution of improvement projects. A clinical microsystem, in short, can be explained as the interprofessional environment at the ward, clinic, or primary health care center, where the patient and family may also be part. Understanding of health care as a system, leadership skills, and methods and tools for improvement are also important to a successful improvement project,23 and were intended as learning outcomes from this learning module. For this, a short visit to the microsystem may be insufficient from an IQS perspective, as it does not allow full insight into the complexity and optimum function of microsystems.24

In the learning module, students were asked to choose from given problem areas and could not choose independently. This could reduce their sense of involvement and ownership to the projects.25 The short time frame also limits possibilities for data gathering and feedback.

Our respondents generally did not question the premises behind learning IQS. The students had positive responses about the intentions of the learning module despite some criticisms about its implementation. Positive as well as negative reactions and attitudes to the learning module can be of value for its further development, as “much can be learnt from disappointments”.26

Limitations

This is the first evaluation of this new learning module, and further follow-ups of its strengths and weaknesses are necessary to get more stable results over time. Our questionnaire was designed to assess the learning outcomes specific to this learning module. Our question about how professional competence contributed to the work assumed that there were specific profession-related contributions to be recognized, which might not be the case. The open format questions had modest response rates, and it may be argued that only the students with the strongest opinions answered.

Furthermore, it was not the intention to measure the exact knowledge gained by the students. No common postcourse knowledge test was conducted, thus we cannot quantitatively measure the impact made. Rather, this evaluation measures their attitudes and experiences to a course that integrates interprofessional learning and IQS.

Our findings might be useful for learning module designers of similar interprofessional education learning modules. Our use of mixed methods, we believe, extends our understanding and increases this study’s reliability and usefulness.

Conclusion and implications

Our health science students were positive towards learning IQS through interprofessional learning. Most students reported they had developed their knowledge about IQS as well as their interprofessional competence. Some criticism was directed towards the nursing focus of many projects and insufficient practical implementation and coordination. In general, medical students were less positive than other students, and female students were more positive than male students.

Practices and clinics engaging students in improvement projects may use our results to make their projects relevant and challenging to the students. The insights from our study indicate that there is a need to think through the complexity of the situations presented and the level of interprofessional teamwork required to solve the problems. The available time frame as compared to the expected outcomes should also be assessed before the improvement project is handed to the students.

Students’ views on this learning module may serve as a basis for the continued improvement of this module, the quality improvement scenarios selected, and the evaluation questionnaire. Students and their views should be involved when designing similar educational modules.