Designing and Executing a Checklist for Evaluating Midwifery Students' Practical Skills.

BACKGROUND: This study aimed to design and execute a reliable checklist for evaluating midwifery students' clinical skills.
MATERIALS AND METHODS: In this cross-sectional study, 8 checklists were designed and validated for midwifery procedures. The students' performances were compared using this method and two other evaluation methods.
RESULTS: The face and content validity of all questions were approved (content validity ratio and content validity index >0.80, inter-rater reliability >0.50, and Cronbach's alpha >0.70). The rate of the students' satisfaction was 19%, 14%, and 76% with the nonchecklist method, the general checklist, and the designed checklist, respectively. Furthermore, both students (F2,60= 107.07, p < 0.004) and evaluators (F2,9 = 152.23, p < 0.001) gained significantly higher attitude scores towards the new checklist compared to the two other methods.
CONCLUSIONS: The designed checklist was quite reliable and valid for evaluating the midwifery students' clinical skills. Copyright:

Introduction

Midwives play an essential role in reducing maternal and infant morbidity and mortality and enhancing access to high-quality care. Midwifery education plays a vital role in the development of competent midwives.[1] Evaluation is a way to measure students' learning.[2] The educational assessment aims to guarantee the promotion of the quality of educational programs.[3]

The most popular evaluation techniques include checklists and rating scales. The checklist method is a systematic way of reporting observer judgments. In the global rating system, a student's clinical performance is evaluated as a general scoring scale. Although the worldwide rating method is the most common evaluation method, using a checklist is better to judge the learners' performances.[4] Using an appropriate and reliable evaluation checklist is one of the main components of student evaluation tests. This study aims to design and execute a checklist for evaluating midwifery students' practical skills.

Materials and Methods

This cross-sectional study was conducted at Hamadan University of Medical Sciences in 2018. The study population consisted of 21 undergraduate midwifery students at the end of the first semester who were selected via the census method. The inclusion criteria of the study were having passed the theoretical and practical courses of “The Principles and Techniques of Nursing and Midwifery”. The exclusion criterion was not taking part in the practical examination. The participants' demographic data were obtained using a demographic questionnaire.

At first, checklists were designed for 8 applied procedures in midwifery (serum therapy, catheterization, drawing up and injecting drugs, controlling vital signs, oxygen therapy, baby care principles, wearing protective equipment, and perineal prep and drape before vaginal delivery) based on the educational goals of “The Principles and Techniques of Nursing and Midwifery”. To assess the quantitative face validity of the checklist, the impact score was computed for each item. In case the impact score was above 1.5, the subject was considered to be appropriate for further analyses. The quantitative content validity of the checklist was assessed using the Content Validity Ratio (CVR) and Content Validity Index (CVI). The items with the CVR score of more than 0.80 were accepted.[5] The acceptable threshold for CVI has been considered to be 0.79. To determine the reliability of the checklists, the inter-rater reliability method was used. The correlation coefficient was found to be >0.50 and Cronbach's alpha >0.70.

In the next step, four stations were designed. The students were collected in a class before the examination. Therein, a code was assigned to each student. Accordingly, the students were entered into the four stations. In each station, the students were requested to perform one of two procedures randomly within 5 minutes. At each station, the evaluator assessed the student using 3 methods (the designed checklist, the general checklist, and without any checklists). The general checklist contained 12 general questions about dealing with the patient, patient privacy, proper communication with the patient, giving medication, injections, dressing, sterile tips, vital signs, writing a report, familiarity with operating room equipment, interacting with colleagues, and interacting with other wards of the hospital, which were not specific for a procedure. In the nonchecklist method, only the student's overall performance was assessed. All analyses were done using the Statistical Package For the Social Sciences (SPSS) 18 software (version 18, SPSS Inc., Chicago, IL, USA).

Ethical considerations

This study was approved by the Ethics Committee of Hamadan University of Medical Sciences (IR.UMSHA.REC.1397.27). The purpose of the study was explained to the students and their written informed consents were obtained.

Results

The mean (SD) age of the students was 18.90 (1.04) years. All students were single, and none of them was employed. The results revealed a significant difference among the students' performance scores in the three methods (p = 0.004). The highest and lowest mean (SD) scores were related to the designed checklist 16.60 (1.60) and the nonchecklist method 14.96 (1.75), respectively [Table 1]. Tukey's post hoc test revealed a significant difference between the students' mean scores in the nonchecklist method and the designed checklist (p = 0.003).

Table 1

Comparisons of the students’ mean scores in the three evaluation methods

Evaluation method	Number	Minimum score	Maximum score	Mean (SD)	One-way ANOVA
Without checklist	21	11.00	18.50	14.96 (1.75)	F(2, 60)=5.93, p=0.004
General checklist	21	12.75	18.00	15.67 (1.24)
Designed checklist	21	13.50	19.00	16.60 (1.60)

ANOVA: Analysis of variance; SD: Standard deviation

The rate of the students' satisfaction was 19%, 14%, and 76% with the nonchecklist method, the general checklist, and the designed checklist, respectively. Additionally, the four evaluators were completely satisfied with the designed checklist.

The results indicated a significant difference among the students' mean scores of attitude in the three methods (p < 0.005). Moreover, the evaluators' mean (SD) score of attitude towards the designed checklist was significantly higher compared to the two other methods 28.25 (1.25), p < 0.001 [Table 2].

Table 2

Comparison of the students’ and evaluators’ mean scores of attitude in the three methods

Evaluation method	Students’ attitude scores			One-way ANOVA
	Minimum	Maximum	Mean (SD)
Students’ attitude scores
Nonchecklist	9.00	24.00	17.52 (4.00)	F(2, 60)=107.07, p<0.004
General checklist	6.00	14.00	9.61 (1.96)
Designed checklist	17.00	29.00	24.71 (3.68)
Evaluators’ attitude scores
Nonchecklist	14.00	19.00	16.25 (2.21)	F(2, 9)=152.23, p<0.001
General checklist	6.00	9.00	7.25 (1.50)
Designed checklist	27.00	30.00	28.25 (1.25)

ANOVA: Analysis of variance; SD: Standard deviation

Discussion

In this study, up-to-date references were used to prepare the checklists. Rowan (2016) has referred to the necessity to consider accurate values and principles for designing an evaluation method so that it would measure what it has been intended to measure.[6] Using a standard checklist is one of the criteria for the quality of the objective structured clinical examination.[7] Mansourian et al. (2017) reported that the content validity of the checklist for evaluating dental students was 0.8 and its reliability coefficient was 0.9.[4]

In this study, the students gained higher scores in the designed checklist, while the lowest scores were related to the nonchecklist method. In Mansourian's research, the total mean score was 89.9 for the checklist method and 86.2 for the non-checklist method.[4] Moreover, both students and evaluators were satisfied with and had a positive attitude towards the designed checklist. However, Mansourian et al. reported no significant difference between the two methods of evaluation (using and not using a checklist) regarding the students' and evaluators' satisfaction. Yet, the students were more satisfied with the checklist method.[4] Arfaie (2018) also explained that from the students' perspective, their awareness of the goals and procedures of the clinical evaluation was the main priority associated with this process.[8]

This study findings indicated that the designed checklist increased the quality of assessment. According to McGill et al. (2015), evaluation methods based on judgment do not have excellent reliability and validity. Therefore, the validity and reliability of evaluation methods can be enhanced using standard instruments.[9] One of the limitations of this study was its small sample size.

Conclusion

The designed checklist was sufficiently reliable and valid for evaluating the midwifery students' clinical skills. Furthermore, both students and evaluators were satisfied with and had a positive attitude towards the designed checklist.

Financial support and sponsorship

Research Council of Hamadan University of Medical Sciences

Conflicts of interest

Nothing to declare.