Effect of role-playing on learning outcome of nursing students based on the Kirkpatrick evaluation model.

BACKGROUND: Evaluation of educational courses is important for estimating the achievement of learning goals and identifying the best way to learn. The present study is an attempt to assess the effectiveness of education through role-playing on the learning outcomes in nursing students based on the Kirkpatrick's evaluation model.
MATERIALS AND METHODS: A quasi-experimental study was conducted with participation of 74 nursing students at Iran University of Medical Sciences in 2016-17. The participants were selected through census and were randomly allocated to control (n = 35) and experimental (n = 39) groups. The common method of education was implemented for the control group, and the experimental group experienced role-playing educational method. In the next semester, each student was assigned to educate two patients at the hospital. The knowledge level of the participants at the end of the semester and patients' satisfaction with the educations by students in the next semester were measured as the outcomes of learning. Kirkpatrick's model was used to assess the learning outcomes.
RESULTS: The mean score of students, at the second level of Kirkpatrick's model, in the experimental group (63.85 ± 13.88) was significantly higher than that of the control group (46.41 ± 16.22, P < 0.001). The mean score of patients' satisfaction with patient educational performance, at the fourth level of the model, in the experimental group (73.26 ± 3.47) was significantly higher than that of the control group (47.32 ± 6.83, P < 0.001).
CONCLUSIONS: The evaluation by the Kirkpatrick's model showed that use of role-playing method improved learning outcome of nursing students. Copyright:

RCT Entities:   Population Interventions Outcomes

Introduction

Nurses comprise the largest component of the health-care teams,[1] and they play an effective role in patient's education.[2] Patient's education leads to higher quality of care and health preservation and improvement. In addition, it yields economic benefits as well, so that each dollar spent on patient's education leads to 3%–4% saving in health costs. As suggested by the statistics published in the USA, about 69–100 million dollars is spent in the USA to deal with the problems caused due to failure to educate the patient.[3] In Iran, Khezerloo et al. stated that only 31.7% of educational activities of nurses were at desired level.[4] The results of a study by Ranjbar Ezzatabadi et al. showed that nurses’ lack of knowledge about educational methods (48.3%) and educational needs (45%) of the patients was the main obstacle of patient's education.[3] Because skill development in nursing profession takes place in undergraduate program,[5] it is essential to improve nurses’ skills in patient education during their undergraduate program.

One of the main principles in education is to use a proper teaching method.[6] The reason for this is that choosing a proper teaching method in nursing education can add to the appeal and effectiveness of education.[7] Today, nursing students prefer interactive teaching methods that reflect the actual nursing world.[8] Therefore, to improve skill acquisition and capability in implementing the skills, experts recommend education in a controlled environment. That is, the closer the education environment to the reality, the more efficient is the learning.[8]

Role-playing is one of the novel and effective education methods[9] that trigger active learning.[10] In this method, the learner has the chance to deal with a structured clinical setting, whereas in traditional methods, the trainee encounters random opportunities to gain experience.[8] According to the new curriculum planning approved by the Supreme Council of Programming, role-playing method should be emphasized in “patient education” course.[11] Role-playing brings in several advantages such as time and cost-saving in the program,[12] higher performance of students,[13] improvement of decision-making skills,[14] and promotion of critical thinking.[15] Such method can be used at all levels of nursing education.[15]

In addition, evaluation is another main pillar of any educational program, which can lead education from a static mode to a dynamic path. In fact, education and evaluation are two interwoven processes.[16] Evaluation uncovers the extent of realization of learning goals by the students.[17] Taking into account that nursing is a performance-based profession,[1817] measuring the realization of educational goals as acquired skills in the students is essential. Kirkpatrick's model is one of the renowned methods of evaluation in this field. The model has been used for 40 years for the evaluation of educational programs in health profession.[16] It facilitates the complicated evaluation method and demonstrates how skills and knowledge mutually affect each other.[19]

Kirkpatrick's evaluation model was first introduced by Kirkpatrick in the 1960s.[12] It comprised four levels viz., reaction, learning, behavioral change, and organizational performance. The first level (reaction) refers to the level of reaction by the learners to all effective factors in an education course. In fact, reaction measures how the learner feels about the program. The second level (learning) refers to the nature and volume of changes in the learners caused by participation in the program. Behavioral change indicates whether or not the program has created a desired change in the learners’ behavior. Finally, organizational performance indicates if the program is successful in meeting the organization goals.[2021]

Educational course evaluations are normally performed simply at the first level or the second level of the model at best. Such evaluations indicate the performance and efficiency of the model at two first levels. In most of the cases, efficiency evaluation at the third and fourth levels is neglected due to the complicacies.[19] The authors believe that using Kirkpatrick's model adds to the efficiency of learning the principles of patient education in nursing students, so that they could demonstrate a better performance as nurses, i.e., one of the main goals of medical sciences education, which leads to a higher quality of care services in the country. In light of this, the present study is an attempt to determine the effects of role-playing education on the learning outcomes in nursing students based on the Kirkpatrick's evaluation model. This study is the first of its kind in Iran.

Materials and Methods

The subjects were selected from the nursing students in their 2nd semester who had taken “patient education” course and the patients hospitalized at Firouzgar and Hazrat Rasoul educational hospitals receiving cares from the nursing students. Inclusion criteria for the students were taking “patient education” course in the spring semester of the academic year 2016–2017 and adult/elderly training[1] in the hospitals in the fall semester of the academic year 2017–2018. Inclusion criteria for the patients were ability to communicate; hospitalized in internal, surgery, and orthopedic wards; consciousness; and awareness of one's situation. Sampling method in the case of students was census, which needs no specific formula. To this end, all the students in the spring semester of the academic year 2016–2017 were selected and allocated to control (n = 35) and experimental (n = 39) groups through a simple random method. To prevent information bias, the author attended the first session of the course and briefed the students about the objectives and procedure of the study. Demographics and consent forms were filled out at this stage. Student's evaluation was based on active participation in the classroom (10% of total score), group project (40% of total score), and end of term exam (50% of total score).

There was no specific ratio of the number of patients to students recommended in the literature. Only Can et al. adopted a specific ratio of the number of patients to students (1:1) in their study on satisfaction of patients.[22] To achieve results with higher accuracy, two patients were selected per students by the instructor. Therefore, each student trained two patients, so that 74 patients were in the control group and 68 patients were in the experimental group. Teaching at the clinical and theoretical stage was done by the same instructors.

Teaching method in the control groups was the standard method of the school, so that the approved contents of the course were instructed in theoretical and practical sections through eight 2 h sessions at the school. Theoretical instruction consisted of giving lectures and asking/answering questions, and the practical instruction consisted of asking the students to select topics for training a patient and design a teaching and evaluation plan to orally deliver the trainings to the patient. The theoretical education for the experimental group was the same method used for the control group (eight 2 h sessions at school consisting of giving lecture and asking/answering questions). The practical section, however, consisted of different stages of role-playing as follows.

Stage one

Determining subject and problem statement. Students were grouped into groups of 5–6 members. Then, they selected subjects of interest about educating patients based on consultation with advising instructors in the area of their course. The available subjects to choose from were orthopedic, digestion, and water and electrolyte disorders. Then, scenarios were developed by the students and reviewed by the instructor and researchers to make the required modifications. Mental and physical conditions of the patients were also taken into account in the design of scenarios.

Stage two

Selecting the role players and their roles.

Stage three

Practicing

The groups practiced in a practice room and the researcher observed the practices. Based on the observations, new members were added to some of the groups and a few modifications were made in the contents of scenarios.

Stage four

Preparing stage equipment.

Stage five

Preparing students for the observation

To engage the students, the researcher assigned different tasks to them such as determining the obstacles of patient education, facilitators, physical and mental condition of the patient, the way of interaction of the medical team with the patient, and the role of the patient in the education process.

Stage six

Performing the show.

Stage seven

Discussing and evaluating the show.

Stage eight

Sharing experiences and generalization

The researcher and instructor led the discussion to enable the performers to generalize the situations and outcomes after gaining experiences, so that they could use the experiences in practice.[23]

The second and fourth levels of the Kirkpatrick's model were measured as the outcomes of short- and long-term learnings. To assess the short-term outcomes, the students answered questions about the knowledge at the end of 8th week. To assess the long-term outcomes, the students’ performance in the next semester was measured in the form of patients’ satisfaction with the educations. Each student in the experimental group trained two patients who met the inclusion criteria. Afterward, the patients filled out the satisfaction scales of patient education performance. The collected data were analyzed in SPSS v16.0 IBM Corporation, Somers, NY, USA using descriptive and inferential statistics (Chi-square test, independent t-test, Mann–Whitney test, and Fisher's exact test) (P = 0.05).

Data gathering tools

Demographic questionnaire

The questionnaire was filled out by the students before implementation of the intervention. The content validity was used to determine the validity of the tool. To this end, it was provided to ten faculty board members of Iran, Tehran, and Shahid Beheshti universities of medical sciences. The experts supported the validity of the tool.

Knowledge evaluation questionnaire

This researcher-designed questionnaire consisted of twenty questions (four alternatives). The questions are about the stages, elements, patient's education method, evaluation method, and documentation of the education process. Only one of the four alternative answers of each question is the correct answer and the other three are wrong. Each correct answer is considered as one score and wrong answers are considered as zero. The maximum score is 20, and the scores were reported based on 100% at three levels of > 50%, 50%–74.99%, and < 75%. The higher the score, the higher the knowledge of students about education. The questionnaire was filled out by the participants after the intervention (at the end of spring semester). Validity of the questionnaire was supported by ten faculty board members of Iran, Tehran, and Shahid Beheshti universities of medical sciences based on the content validity. Internal consistency was determined using Kuder–Richardson[21] equal to 0.83 with participation of thirty students.

Patients’ satisfaction scale with education performance of nurses

The scale was first used in urology ward of a hospital in Norway to measure patients’ satisfaction with education performance of nurses. The primary scale consisted of 21 statements.[24] It was validated by Golaghaie and Bastani in Iran and four statements were removed. Therefore, the Farsi version of the scale contains 17 statements. Maximum score of the scale is 170 and based 100; three levels, >50%, 50%–74.99%, and <75%, were defined. The higher score means higher satisfaction. Validity of the Farsi version of the tool was supported by Golaghaie and Bastani using content validity method. Internal consistency was also measured using Cronbach's alpha equal to 0.85.[25] Here, the internal consistency was measured through Cronbach's alpha with thirty patients equal to 0.85. It is notable that these thirty patients were not among the participants.

Results

The data collected from 74 students at short-term outcome evaluation stage (the spring semester) and 71 students at long-term outcome evaluation stage (the next fall semester) were examined (two students in the experimental group left the school and one student in the control group was transferred to another school).

The results of the Fisher's exact test and Chi-squared test showed that there was no significant difference between the control and experimental groups in terms of gender, marital status, domicile (dormitory, with family), number of failed credits in the past semester, job (student, health assistant, and freelancer), and interest in nursing profession (Likert's five-point score; very high and very low) (P > 0.05). Based on the independent t-test, Chi-squared test, Fisher's exact test, and Mann–Whitney test, there was also no significant difference among the patients in terms of gender, age, education level, marital status, history of disease, history of hospitalization, and length of hospitalization (P > 0.05). Demographical information of the subjects is listed in Table 1. There was a significant difference between the control and experimental groups in terms of knowledge score (P < 0.001) after the intervention. The knowledge score in the control and experimental group were 46.41 ± 16.22 and 63.85 ± 13.88, respectively [Table 2]. There was also a significant difference between the control (47.32 ± 6.83) and experimental (73.26 ± 3.47) groups in terms of patients’ satisfaction scores with the students’ education performance (P < 0.001) [Table 3].

Table 1

Demographics of the subjects

	Variable	F (%)
		Control group	Experiment group
Students	Gender
	Female	21 (53.8)	18 (51.4)
	Male	18 (46.2)	17 (48.6)
	Age (years)
	17-20	35 (89.7)	20 (57.1)
	21-25	3 (7.7)	10 (28.6)
	26-30	1 (2.6)	3 (8.6)
	>30	0	2 (5.7)
	Marital status
	Unmarried	38 (97.4)	30 (85.7)
	Married	1 (2.6)	5 (14.3)
	Employment
	Positive	5 (12.8)	6 (17.1)
	Negative	34 (87.2)	29 (82.9)
	Failed credits in the last semester
	None	35 (89.7)	32 (91.4)
	1-3	2 (5.1)	2 (5.7)
	3-6	1 (2.6)	0
	>6	1 (2.6)	1 (2.9)
Patient	Gender
	Female	34 (45.9)	23 (33.8)
	Male	40 (54.1)	45 (66.2)
	Age (years)
	<29	9 (12.2)	9 (13.2)
	30-39	18 (24.3)	15 (22.1)
	40-49	18 (24.3)	11 (16.2)
	50-59	17 (23)	10 (14.7)
	>60	12 (16.2)	23 (33.8)
	Marital status
	Unmarried	14 (18.9)	10 (14.7)
	Married	60 (81.1)	58 (85.3)
	Education
	Illiterate	12 (16.2)	17 (25)
	Below high school	28 (37.9)	27 (39.7)
	High school	17 (23)	15 (22.1)
	College degree	17 (22.9)	9 (13.2)
	History of hospitalization
	Positive	23 (31.1)	19 (27.9)
	Negative	51 (68.9)	49 (72.1)
	Receiving education on the current issue
	Positive	1 (1.4)	2 (2.9)
	Negative	73 (98.6)	66 (97.1)

Table 2

The nursing students’ score in patient education at the second level of the Kirkpatrick’s model

Knowledge (out of 100)	Group, F (%)
	Control	Experiment
<50	21 (53.8)	3 (8.6)
40-75.99	18 (46.2)	23 (65.7)
>75	0	9 (25.7)
Total	39 (100)	35 (100)
Mean±SD	46.41±16.22	63.85±13.88
Maximum-minimum	15-70	15-85
Independent t-test	t=5.691, DF=72, P<0.001

SD=Standard deviation

Table 3

The nursing students’ score in patient education at the fourth level of the Kirkpatrick’s model

Satisfaction (out of 100)	Group, F (%)
	Control	Experiment
<50	50 (67.6)	0
40-75.99	24 (32.4)	42 (61.8)
>75	0	26 (38.2)
Total	74 (100)	68 (100)
Mean±SD	47.32±6.83	73.26±3.47
Maximum-minimum	36.47-62.65	66.47-79.71
Independent t-test	t=6.58, DF=69, P<0.001

SD=Standard deviation

Ethical concerns

The study is part of an MSc dissertation, and it is approved by the Ethics Committee of Iran University of Medical Sciences under No.: IR.IUMS.REC.1395.9411686011. The study is also registered in Iran Trial Center under No.: IRCT2017091636215N1. All the participants signed a written letter of consent and principles of unanimity and confidentiality were respected.

Discussion

The evaluation by Kirkpatrick's model showed that use of role-playing method improved learning outcome of nursing students. As the findings showed, the mean score and standard deviation of knowledge of students in the experimental group was higher than those of the control group after the intervention. Therefore, one may say that role-playing was effective in knowledge of the students about patient education at the second level of the Kirkpatrick's model. Moreover, the results showed that the score of none of the students in the control group was higher than 75%, whereas 9 (25.7%) students in the experimental group obtained scores above 75%.

The results also showed a significant difference between the control and experimental groups in terms of the mean score of satisfaction in the patients with the education provided by the students (P < 0.001). Therefore, the role-playing method was effective in the education provided by the nursing students to the patients at the fourth level of Kirkpatrick, i.e., the patients’ satisfaction with education performance of the students.

Patients’ satisfaction with nursing services is one of the indicators of quality of nursing cares, and it can create a positive attitude and reaction in the society toward nursing profession. In addition, nurses are the main group of care providers in hospitals, and their role has a profound effect on the quality of care and patients’ satisfaction consequently. Therefore, empowering nurses seems to be an imperative need. In the same sense, Pour Rabari et al. showed in their study to determine the effects of educating patients by nurses that the mean score of patients’ satisfaction with education changed significantly after the intervention. Therefore, improvement of nurses’ knowledge about patient education leads to a higher performance and higher satisfaction in patients in return.[26] Mourad et al. showed in their study that educating patients improved their level of satisfaction and consequently improved patients’ quality of life.[27]

Choosing an appropriate teaching method to enhance the learning and performance of nursing students is important, because for nursing education, having only theoretical knowledge is not enough.[28] Experts have suggested training in reality-like environments to acquire skills.[8] Role-play is one of the most appropriate methods to do this. In a study that compared the two methods of role-play teaching and group discussion on the performance of 30 interns in the field of transmitting bad news, it was found that, in the role-play group, the mean score of knowledge and performance of individuals was significantly higher than discussion group.[29] Perhaps one of the reasons for role-playing method is more effective than group discussion is the greater participation of learners and the realization of the learning process for them. Baghdari et al. also showed that the students’ knowledge score in transmitting bad news increased after using role-play method. There was also a significant difference in the mean scores of knowledge and attitude between the role-play group and the control group after education.[30]

Role-playing guides the learners toward understanding their social behavior and role in social interactions, developing empathy with others, and finding better problem-solving methods.[31] Ward et al. showed that role-playing improved the empathy felt by the students toward patients and their understanding of the patients’ concerns and physical and mental problems.[32] Van Winkle et al. showed that empathy was improved immediately after role-playing.[33] One probable reason for the improvement of empathy in the participants after role-playing could be the fact that role-playing engages the actors and observers in the action and the emotions within the role so that they are motivated to analyze the roles and situations. Indeed, through actual visualization of roles and different subjects, the practitioners can better understand the needs and condition of patients. This leads to a higher empathy with patients, so that the students try to solve the physical and mental problems of patients using more effective methods.[32]

Another reason for the better performance of students in the experimental group in terms of patient education comparing with the control group was the fact that role-playing, to some extent, fills the gap between theory and practice. Role-playing is one of the novel education methods that are used to teach theoretical concepts and bring them to the real world.[6] Therefore, it can be helpful in improvement of clinical skills. Robinson-Smith et al. conducted a study in a psychology ward and showed that 45% of nursing students found role-playing similar to the real world. In addition, role-playing improves self-confidence and creative thinking in students and improves satisfaction with learning in the students.[34]

Role-playing method mentally prepares the students for learning as the students are given the chance to demonstrate their education technical skills in quasi-clinical setting and control their stress. On the other hand, role-playing gives the students not only a chance to practice their clinical skills, but also have the opportunity to polish their communicational skills and experience the patient's reaction to the care. All these lead to positive effects on the cares provided by students to patients. Role-playing is widely used for educating communication skills. It is a useful method for repeating, observing, and discussing the roles and directing the roles toward other educational programs.[35] Bosse et al.[36] and Burns et al.[37] showed that role-playing method improved communicational skills of subjects.

In addition, role-play method enables a nursing student to practice the role of a nurse in an environment similar to clinic. Therefore, the student is prepared to face professional situations and by increasing self-esteem, enables the student to accurately address the patient's problems.[38] Hunter and Ravert argue that the practice of skills with role-play method leads to better performance, and such students gain more confidence in the clinical setting.[13] The study by Hermanns et al. has shown that role-play is a valuable method for teaching clinical skills and reduces anxiety and improves learning because students are not under pressure.[39] Similar to this study, the results of research findings by Martínez Riera et al. (2011) showed that role-play helped students encounter real situations and reduce their concerns.[40]

Conclusions

One of the weaknesses of nursing education is that clinical skill education methods are not comparable with actual clinical setting. However, students need actual clinical settings to practice problem-solving skills. Such an opportunity is given to students through role-playing as this method gives the student a chance to put themselves in real situation and behave as required. Through this, students learn how to deal with real situations and problems. By improving students’ knowledge about different situations and also decreasing their stresses, role-playing can improve students’ performance in clinical environment. This method improves self-confidence and motivation for learning in students and consequently leads to better outcomes for patients. The results showed that role-playing was effective in the learning outcomes of nursing students based on the Kirkpatrick's evaluation model. Based on the evaluation model, role-playing improved the knowledge of nursing students, which, in turn, leads to higher satisfaction of patients. Literature review showed that this study is the first study on the effects of role-playing education on the learning outcomes of nursing students based on the Kirkpatrick's model.

Limitations

Because of low education level of some of the patients, there were problems in filling out some of the questionnaires. Such situations were handled by asking the closest family member to help the patients with filling out the questionnaire. Another limitation was heterogeneity of students in the two groups in terms of age. Given that the students were randomly allocated by the education department to two classes, it was not possible to homogenize the students in terms of age. Knowing that improving knowledge of nursing entails several studies in different areas, the present study can be taken as the first step of many steps that need to be taken in the area of educational method and evaluation models with control of these limitations.

Financial support and sponsorship

The study was financially supported by Iran University of Medical Sciences, Tehran, Iran.

Conflicts of interest

There are no conflicts of interest.