The effect of direct observation of procedural skills method on learning clinical skills of midwifery students of medical sciences.

INTRODUCTION: Clinical education is one of the most important parts of medical students' education, and it is a major part of the education of qualified and professional people. Therefore, this study was conducted to determine the effect of applying Direct Observation of Procedural Skills (DOPS) on midwifery students' clinical skills.
MATERIALS AND METHODS: This is a quasi-experimental, two-group study conducted as a pre- and post-study on midwifery students in 2017-2018. Cluster and randomized sampling method was used. The processors involved in this study were three main skills of vaginal examination, pelvic examination, and vaginal delivery. The DOPS method was used to assess the practical skills in the interventional group during three times the process at day, 1 day, and at least 1 week later; the usual logbook method was used in the control group. Two groups were evaluated at the end of the midwifery course by Comprehensive Final Midwifery checklist. The tools were checked for validity and reliability, and data were analyzed using descriptive and analytical statistics.
RESULTS: There was no statistically significant difference between the two groups in terms of important demographic variables such as age, grade, marital status, and initial assessment score (P > 0.05). The mean of final scores in the normal delivery, vaginal examination, and pelvimetry was statistically significantly higher in the interventional group (P < 0.001). On the other hand, the functional field of the students in the interventional group was statistically significantly improved in normal delivery and pelvimetry (P < 0.05), and this difference was not significant in the vaginal examination. In addition, the mean scores of students before and after the DOPS method were statistically significantly different in every skill in Comprehensive Final Midwifery checklist (P < 0/05).
CONCLUSIONS: The DOPS assessment methodology is not only a useful tool of clinical evaluation, but also an effective tool for clinical learning of students. For this purpose, it is suggested that educational members of midwifery take enough time to design DOPS method in the same process. Copyright:

RCT Entities:   Population Interventions Outcomes

Introduction

Clinical education is one of the most important parts of education for students of nursing and midwifery sciences[1] and is a vital part of training competent and professional people. The value of ideal clinical education is such that its role in individual and professional development as well as the clinical skills of students is undeniable.[2] To this end, promoting clinical competence and education has always been one of the major concerns in the education of medical sciences.[3] On the other hand, the results of many studies show that evaluation is the most important measure to improve effectiveness in this field. The real purpose of evaluation is monitoring and gathering data to improve the educational status.[4] With proper evaluation, the strengths and weaknesses of education can be identified and stepped up in developing and reforming the educational system[56] by reinforcing the positive aspects and eliminating inadequacies. Effective evaluation not only enhances students' motivation but also helps teachers to evaluate their activities,[7] and if this evaluation is also accompanied by appropriate feedback, it can enhance the learning skills of the learner.[89]

Due to the increasing changes in clinical education approaches, the necessity of using appropriate new evaluation methods is becoming increasingly evident. In the study conducted in the nursing colleges in the southern states of America, it was found that 45%, 35%, 17%, and 3% of colleges have had no revision in their clinical evaluation methods for 5, 6–10, 11–15, and over 15 years, respectively.[10] In addition, a study conducted in the Tehran School of Nursing and Midwifery showed that 62% of students believed that the conditions and cases of clinical evaluation were not the same and satisfactory for all students.[11] Specialists have been searching for validated methods for years to effectively evaluate students' clinical performance.[11] Clinical evaluation methods that come with feedback, in addition to evaluating difficult cases in traditional student assessment, also promote learning.[1112]

Currently, the use of performance-based tests (such as Direct Observation of Procedural Skills [DOPS] and Objective Structured Clinical Examination Mini-Clinical Evaluation Exercise) for the measurement of clinical and practical skills is emphasized.[13] As midwifery is a practical profession, direct observational assessment, in practical and real situations, provides the examiners with confidence in the student's ability to anticipate and predict clinical changes and events in a particular patient's condition and helps determine her ability.[14] Therefore, evaluation through direct observational assessment of practical skills seems necessary. On the other hand, DOPS is one of the new methods of clinical education as well as a good way to provide an opportunity to provide constructive feedback and student attention and focus on what is needed to accomplish the desired skill because evaluation requires timely and specific feedback to improve performance.[15]

Considering the researcher's experience of the current problems in the clinical evaluation of midwifery students in current methods, and the future working conditions of these students in maternity hospitals that require high skill and speed in performing the techniques, the use of an accurate evaluation method to ensure good clinical competence seems essential; in addition, as midwifery is a stressful job and midwifery students need to be proficient in a variety of areas, including childbirth and physical examinations, and their clinical competence will lead to greater self-efficacy, the purpose of this study was to determine the effect of DOPS evaluation method on midwifery students' clinical skills learning.

Materials and Methods

This is a quasi-experimental, two-group study conducted as a pre- and post-study in 2017–2018. After obtaining permission from the Ethics Committee of Isfahan University of Medical Sciences, cluster random sampling was done by explaining the purpose and methodology and satisfying the last semester midwifery students who were undergoing pregnancy and childbirth internship. Each internship group was considered as a cluster and randomly divided into experimental (n = 30) and control (n = 30) groups. Inclusion criteria were theoretical pregnancy and childbirth courses and practicing the relevant course.

The procedures used in this study were three basic skills of vaginal examination, pelvic examination, and vaginal delivery. DOPS and log book (according to the routine of the university) evaluation methods were used for intervention and control groups, respectively. The students of control and experimental groups were evaluated on the basis of a checklist of relevant clinical skills. In the conventional method in the control group, students' skills during the internship period were measured with a log book so that on the 1st day of the internship, the log book was read by the instructor and justified to the students. Then, it is scored in a phase. In the experimental group, the interventional steps were as follows:

Step 1: Observing the desired skill within the prescribed time (1 min for vaginal examination, 15 min for delivery, and 15 min for episiotomy repair) and giving feedback in 5 min (while reviewing the items in the evaluation)

Step 2: Repeating the desired skill after 1 day and emphasizing the strengths and weaknesses of the student in that skill

Step 3: Repeating the desired skill after at least 1 week and emphasizing the strengths and weaknesses of the student.

The student training tool in the experimental group was a skill-based checklist; in the vaginal and pelvic and delivery examination skills, it included eight areas of communication, namely, pre examination preparation, sterile conditions, technical ability in examination, judging and reporting skills, and a general skill in the technique. In the study of Kuhpayehzade et al., reliability and validity of the checklist were reported to be 0.98 and 0.95, respectively.[15] However, the content validity was confirmed by four members of Isfahan Medical Sciences Faculty.

Both groups were then assessed at the end of semester eight by the Midwifery Comprehensive Examination Evaluation Checklist on the three skills listed; at the end, each skill was scored as poor, improper performance, and in need of full guidance and supervision; moderate and in need of relative guidance and supervision; good; in need of minimum guidance for the proper and great performance; and proper performance without the need for the slightest guidance. The checklist was standardized and approved by the Midwifery Board of the country. The content validity of the instrument was confirmed by the Midwifery and Reproductive Health Board and the faculty members of the Midwifery Department and had a reliability coefficient of 0.99. Data were then analyzed using SPSS-22 software and descriptive statistics (produced by IBM United States), Chi-square, independent t-test, paired t-test, and Pearson's correlation.

Results

There was no significant differences between two groups in homogeneity in term of age, average grade point by using independent t-test and pre intervention skill and marital status by using Chi-square test [Table 1].

Table 1

Frequency distribution of demographic quantitative variables by study groups

Variable	Mean±SD		Statistical test result
	Control group	Experimental group	T	P
Age (year)	23.1±35.04	23.1±5.09	0.88	0.3
GPA	15.1±3.2	15.1±9	−1.7	0.09
Points before intervention
Vaginal examination skills	15.2±3	15.1±9.5	−1.05	0.29
Points before intervention
Vaginal delivery skills	15.2±6.8	14.3±5	1.2	0.23
Points before intervention
Pelvimetry skills	13.1±8.3	14.1±2.8	1.22	0.3

SD=Standard deviation, GPA=Grade point average

At Kirkpatrick's level, at the results' level, there was a significant difference between the final evaluation scores in the two control and experimental groups in all the three skills of vaginal delivery, vaginal examination, and pelvic examination [Table 2].

Table 2

Comparison of the skills final scores in the control and experimental groups after evaluation

Variable	Mean±SD		Statistical test result
	Control group	Experimental group	T	P
Final score of the skill
Vaginal examination	16.1±3.5	17.1±3.03	−2.36	0.023
Pelvimetry performance	11.3±6.4	14.2±3.9	−2.6	0.01
Vaginal delivery	2±15.5	17.1±8.5	−4.1	0.00

SD=Standard deviation

In addition, the performance areas of each student were categorized according to the areas available in the instrument. Students in the experimental group had higher performance in the skills of delivery and pelvimetry compared to the control group, and this improvement in performance made a significant difference. However, this difference was not significant in vaginal examination skills [Table 3].

Table 3

Comparison of skill performance areas in the control and experimental groups after evaluation

Skill area	Control group				Experimental group				Statistical test result
	Poor	Moderate	good	Very good	Poor	Moderate	good	Very good	χ2	P
Vaginal delivery	0	2	10	8	0	0	2	18	10.16	0.006
Vaginal examination	0	0	11	9	0	0	6	14	1.61	0.2
Pelvimetry performance	4	11	4	1	0	9	3	8	9.7	0.02

While in the control group after the end of semester, there was no statistically significant difference in the scores obtained in delivery skills (P = 0.1), vaginal examination (P = 0.08), and pelvic examination (P = 0.2) in comprehensive midwifery test.

There was also a statistically significant difference between the mean of the control and experimental groups both in the evaluation of the same internship and at the end of the eighth semester in the comprehensive midwifery test, showing the students' proficiency and certification of clinical competency (P < 0.05).

At Kirkpatrick's levels, at the performance level, students' clinical competence was assessed by student self-assessment as well as evaluation by the relevant personnel [Table 4].

Table 4

Comparison of students’ clinical performance scores in control and intervention groups by students and personnel

	Control group	Intervention group	T	P
Students’ self-evaluation	15.1±9.7	18.2±1.1	−2.2	0.003
Personnel’s evaluation	16.1±2.4	19.0±3.57	−3.1	0.000

Discussion

This study was conducted to determine the effect of DOPS method on learning clinical skills of midwifery students. The results of this study showed that the DOPS evaluation method is not only a good way to evaluate students in practical skills, but also an acceptable way to learn and overcome students' weaknesses in practical skills [Table 5].

Table 5

Delivery, vaginal and pelvic examinations skills scores before and after direct observation of procedural skills in experimental group

Variable	Mean±SD		Statistical test result
	Before evaluation	After evaluation	T	P
Vaginal examination	15.1±9.5	17.1±4	−6.1	0.000
Natural childbirth	14.3±51	17.1±88.57	−5.5	0.000
Pelvimetry performance	11.3±6.4	17.1±8.5	−9.6	0.000

SD=Standard deviation

As stated in the study by Habibi et al., the DOPS evaluation method significantly enhances students' clinical skills.[13] Furthermore, in a study conducted by Bagheri et al. on medical emergency students, this method was found to be appropriate for learning emergency skills.[16] In addition, the study carried out by Nazari et al. acknowledged that the DOPS method in the intensive care unit nursing group was highly effective.[17]

On the other hand, in the study conducted by Nooreddini et al. by applying the DOPS method in the control group, the mean scores of students did not change and no improvement was observed in students' clinical performance.[18] However, in the present study, the mean scores before and after the DOPS method were significantly different. Moreover, even the skill area of students has been improved significantly in an emergency process, such as natural childbirth. Studies show that students' psychological satisfaction with workplace education is much higher[19]. These results are in line with the results of the study by Tsui et al. in Taiwan, who concluded in their study that using DOPS evaluation and feedback from teachers can enhance the skills of medical students.[20] It is recommended to perform this procedure in other clinical processes and review the results; further studies can also be carried out in other clinical groups.

Conclusions

According to the present results and the mentioned studies, it seems that DOPS method has a great impact on learning and understanding of students' weaknesses in both short- and long-term practical and clinical processes. Therefore, it is recommended that the executives of this test pay special attention to the time required for each practical skill. It is advisable for faculty and staff at each institution to spend time in evaluating knowledge, and this should be included in the human resources development program. The important point is that there is not much attention paid to the evaluator or observer training. This may be due to cost, time constraints, or ignorance. However, evaluators need to be adequately trained to understand the difference between various levels of students' performance.[21] Although evaluation training may seem costly and time-consuming at the first glance, the benefits of doing so are considerable in improving the quality of education.[22] According to the results of the study, it is recommended that teachers use this method to evaluate students' performance in clinical practice.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.