The Barriers and Challenges of Applying New Strategies in the Clinical Evaluation of Nursing Students from the Viewpoints of Clinical Teachers.

BACKGROUND: Numerous studies have emphasized the use of new approaches in clinical evaluation. However, there are some challenges and barriers to applying these new approaches. The aim of the present study was to investigate the barriers and challenges of applying new strategies in the clinical evaluation of nursing students from the viewpoints of clinical teachers.
MATERIALS AND METHODS: This cross-sectional study was conducted among 151 clinical teachers. A researcher-made questionnaire was used to collect data. The questionnaire was validated using library references and a variety of texts, as well as thorough consulting with 15 clinical teachers. The questionnaire's reliability was approved with a Cronbach's alpha of 78%. Data analysis was conducted using Pearson's correlation coefficient, one-way analysis of variance (ANOVA), and descriptive statistics in SPSS software.
RESULTS: The highest score was related to the "students and clinical environment" domain [24.05 (8.10)], and the lowest to the "facilities" domain [13.31 (1.50)]. One-way ANOVA results showed a significant relationship between the mean scores of academic degree and the two domains of "tests" (F = 9.66, p < 0.001) and "facilities" (F = 8.26, p < 0.001).
CONCLUSIONS: The implementation of a new approach for evaluating clinical training requires infrastructure and overcoming executive obstacles. Educating students and clinical teachers on new evaluation methods requires their familiarity with the implementation process as well as encouragement and support by their educational institution and administrators.

Introduction

With the rapid development of information and caregiving techniques presently required by nurses, nursing education has drawn more attention to itself than ever.[1] As a result, the focus of nursing education is on training nurses who have clinical skills and knowledge.[2] This reveals the fundamental importance of clinical education, which plays an important role in nursing education.[3] The importance of clinical training, as a fundamental part of nursing education, is acknowledged by all educational planners and administrators. Moreover, one of the most significant and challenging issues in clinical training is evaluation. Evaluation is the systematic process of data collection, analysis, and interpretation that determines how close we have gotten to our predetermined goals.[4] This is truly an inseparable part of clinical nursing education, and the two complement one another. Evaluating students’ clinical qualifications is one of the most complex tasks of instructors and teachers.[5] Through appropriate evaluation, the strengths and weaknesses of the educational system can be identified, and thus, measures can be taken to transform and modify the educational system. In clinical evaluation, it must be ensured that students are using critical thinking, have professional conduct, interact well with patients, and can prioritize problems by using caregiving methods in clinical environments.[6]

Due to changes in clinical education approaches, the need for new evaluation methods is more apparent than ever. Because clinical evaluation is an essential tool, which is used to measure the qualifications and capabilities of nurses, in nursing instruction, it is vital to use different methods for evaluating various aspects of their performance. The American Association for Medical Education guidebook has recommended a variety of tools and methods for clinical evaluation including objective structured clinical evaluation (OSCE) and direct observation of procedural skills (DOPS). These new methods include feedbacks to improve students’ clinical proficiencies and do not have the disadvantages of traditional methods.[7]

Despite emphasis on using new methods in clinical evaluation, evidence shows that students usually relay on traditional or subjective methods, rather than accurate evaluation of clinical skills. Procedural skill plays the main role in clinical evaluation and subjective knowledge is of secondary importance.[38] A study conducted on nursing colleges in southern USA showed that most colleges had made no revisions in their clinical evaluation methods for a long time and they continued to use traditional methods.[9] Imanipour and Jalili found that the majority of students and teachers found the current (traditional) method for clinical evaluation unsuitable and emphasized on using new approaches.[10] Furthermore, Abotalebi et al. concluded that new methods should replace traditional approaches in evaluating the curriculum.[11] The study by Khosravi et al. in Gonabad, Iran, showed that most nursing students assessed traditional evaluation as weak and asked for a new method.[12] In addition, Eldarir et al. showed that most nursing students considered traditional evaluation techniques incapable of accurately assessing their clinical skills and demanded novel methods.[13] Although new evaluation methods in clinical education are quite effective in evaluating the qualifications of students and most students are satisfied with them, there are some obstacles and challenges such as being time consuming and the need for an experienced manpower, facilities, and resources. These practical issues could cause reluctance in educational institutions for using the new methods.[14] The study by Troncon (2004) in Brazil showed that issues such as lack of support from educational institutions, lack of facilities, being time and money consuming, and the need for trained and experienced teachers affect the implementation of new evaluation methods.[15]

Even though numerous studies have stressed the use of new approaches in clinical evaluation, most teachers in Iran commonly use traditional methods such as a checklist that results in discontent among students. In previous studies, a limited number of barriers and challenges in clinical evaluation were discussed, but the views of nursing instructors on the new methods were not studied and small sample sizes were not used. Thus, the aim of the present study was to investigate the barriers and challenges in applying new strategies in the clinical evaluation of nursing students from the viewpoints of clinical teachers.

Materials and Methods

This cross-sectional study was conducted in the Shiraz University of Medical Sciences, Iran, from January to November 2016. It was carried out on the procedural challenges and obstacles in applying new approaches to clinical evaluation of undergraduate nursing students from clinical teachers’ viewpoint. Sampling was performed using a census. The participants consisted of 151 clinical teachers from the School of Nursing and Midwifery of Shiraz University of Medical Sciences. Because there was no sample loss, no questionnaire was excluded. The inclusion criterion was a minimum of 1 year of experience in clinical teaching. In this study, the unwillingness to participate was considered as the exclusion criterion. Before the commencement of the study, written informed consents were obtained from all the participants.

A researcher-made questionnaire was used for data collection. The questionnaire consists of two parts. The first part consisted of a demographic characteristics form with 4 questions (age, gender, clinical teaching experience, and academic degree). The second part included questions related to the procedural challenges and obstacles in using the new methods for clinical evaluation (such as OSCE and DOPS). The second part of the questionnaire included 20 questions in 4 domains; 6 questions were related to the “tests” domain, 3 to “facilities,” 4 to “teachers,” and 7 were related to the “students and clinical environment” domain. The questions were scored according to a Likert scale ranging from 1 to 5 based on importance (very low impact, low impact, moderate impact, strong impact, and very strong impact). The scores for each domain were obtained from the sum of the questions related to the same domain. Accordingly, the minimum score of the tests, facilities, teachers, and students and clinical environment domains was 6, 3, 4, and 7, respectively. The highest score for the tests, facilities, teachers, and students and clinical environment domains was 30, 15, 20, and 35, respectively. The validity of the questionnaire was verified using library references and a variety of texts, and through consulting with 15 nursing instructors in different fields of nursing, such as medical surgical, pediatric, community health, and psychiatric nursing, from the School of Nursing and Midwifery. Notably, qualitative face validity was also performed. To determine the reliability of the questionnaire, 20 clinical teachers filed out the questionnaire and Cronbach's alpha coefficient was obtained at 78%. Data were analyzed using the statistical tests for frequency distribution, mean and standard deviation, one-way analysis of variance (ANOVA), and the Pearson correlation coefficient in SPSS software (version 22, IBM Corporation, Armonk, NY, USA). All p values of less than 0.05 were considered significant.

Ethical considerations

This study was approved by the Research Council and the local Ethics Committee of Fasa University of Medical Sciences, Fasa, Iran (Code: 95109). Prior to the study, necessary permissions were obtained from the administrators. Furthermore, all the participants were informed about the study objectives and were assured of the confidentiality of their information. In addition, a written informed consent was obtained from each participant.

Results

In this study, a total of 151 teachers participated, among which 119 were women (78.80%) and 10 had a bachelor's degree (6.60%), 119 had a master's degree (78.80%), and 22 had a PhD in nursing (14.70%). The participants’ mean and standard deviation of age and work experience were 41.34 (7.39) and 13.16 (6.66) years, respectively. The highest score was related to the “students and clinical environment” domain [24.05 (8.10)], and the lowest score was related to the “facilities” domain [13.32 (1.50)] [Table 1].

Table 1

Mean, standard deviation (SD), and range in each domain

Kolmogorov–Smirnov test was used to assess the normality of variables. According to the results, the assumption of normality was confirmed for all variables. According to Table 2, in the “tests” domain, being time consuming, difficulty in design and implementation, and lack of suitable and standard evaluation tools achieved the highest scores. In the domain of “facilities,” lack of facilities and executive features, being money consuming, and the need for a high number of personnel obtained the highest scores. The highest scores in the “teachers” domain belonged to unfamiliarity of teachers with the implementation of new evaluation methods, educational administrators’ lack of interest, lack of reward and encouragement by the system, and the teachers’ inclination toward traditional methods. In the “students and clinical environment” domain, stress and anxiety in students, lack of adequate knowledge about new methods, the number of students, inconsistency with the training period, and clinical staff's lack of interest and cooperation obtained the highest scores. According to Table 3, ANOVA results showed a significant relation between the mean score of academic degree and the two domains of “tests” (F = 9.66, p < 0.001) and “facilities” (F = 8.26, p < 0.001). Post-hoc analysis (LSD) was used to determine the differences between two groups. The results showed that “test domain” score in group 3 (Philosophiae Doctor) was significantly different from group 1 (bachelor's degree) and 2 (master's degree) (p = 0.011, p < 0.001, respectively). Moreover, “facilities domain” score was significantly different in group 2 (master's degree) from groups 1 (bachelor's degree), and 3 (Philosophiae Doctor) (p = 0.005, p = 0.002, respectively). Concerning other variables (age, gender, and clinical training), the Pearson correlation coefficient showed no significant relation between them and the other four domains (p > 0.05).

Table 2

Mean score and standard deviation (SD) of each question in the four domains

Table 3

Comparison of mean scores of bachelor's degree, master's degree, and Philosophiae Doctor in the four domains

Discussion

In this study, the procedural challenges and obstacles in using new methods for clinical evaluation were examined in four domains. The highest to the lowest scores were, respectively, related to the “students and clinical environment,” “tests,” “teachers,” and “facilities” domains. Hence, the highest score was related to the “students and clinical environment” domain, and the lowest was related to the “facilities” domain. In this study, a significant relation was observed between the variable of academic degree and the two domains of “tests” and “facilities. The new approaches to clinical evaluation are very effective in assessing students’ qualifications, which results in the satisfaction of students. Nonetheless, there are some challenges and obstacles that prevent educational institutions from welcoming these new methods. Therefore, the purpose of this research was to study the obstacles to institutionalizing new approaches to clinical evaluation from the viewpoints of clinical teachers.

Results showed that, in the “tests” domain, factors such as being time consuming, difficulty in design and implementation, lack of suitable evaluation tools, and unequal reliability among tests were the most important procedural obstacles. The results of studies by Katowa-Mukwato et al.[16] and Nkeiruka et al.[17] showed that being time consuming, problems in design, lack of suitable evaluation tools, and lack of uniform reliability between tests were problems in implementing OSCE method in the clinical evaluation of nursing students. These results were consistent with the present study findings. In the study by Hasan et al., lack of suitable evaluation tools and test inconsistency were identified as procedural obstacles in using new methods for evaluation of nursing students in clinical environments.[18] Noohi et al. also reported that being time consuming and difficulty in design were the main challenges in using new methods such as OSCE.[19]

In the domain of “facilities,” results showed that the identified obstacles of lack of facilities and executive features, being money consuming, and the need for a large number of personnel obtained the highest scores. These findings were consistent with that of Pishkar Mofrad et al.[20] Palese et al. reported the same factors found in the present study as the executive obstacles to using new evaluation methods.[21] Mcwilliam and Botwinski also found lack of facilities, high costs, and the need for a large number of personnel to be the executive challenges for using new methods such as OSCE, which is consistent with the present study findings.[22] Even though significant costs and resources may be associated with the development and implementation of OSCE, these need not be a barrier for using OSCE, and it is possible to adopt the approach in resource-limited environments.[23]

In the present study, teachers’ unfamiliarity with the implementation of new methods and lack of interest or support from the administrators of educational institutions were determined as the executive challenges in the “teachers” domain. The results of studies conducted by Imanipour et al.[24] and Bourbonnais et al.[25] show that lack of knowledge, insufficient knowledge, and lack of encouragement and support of the educational institution were the main obstacles and the reason to teachers’ unwillingness to welcome the new teaching methods. This finding was also consistent with that of the current study. To succeed in implementing a new evaluation method, it is essential to prepare workshops and to institutionalize that method, it is also essential that management provides the necessary substrates and support.[26] Bindal et al. stated that lack of adequate knowledge and discouragement by the training institute in using these methods are the reasons why educational instructors do not use new evaluation methods.[27]

The results showed that in the “students and clinical environment” domain, stress and anxiety in students, lack of adequate knowledge about new methods, the high number of students, and inconsistency with the training period obtained the highest scores. They were identified as the executive obstacles in this domain. Several studies have suggested that the use of new evaluation methods such as OSCE and DOPS causes stress and anxiety among students.[282930313233] They have also stated that the students did not have enough information about these evaluation methods, which was consistent with the findings of this study.[282930313233] For students to welcome the new methods of evaluation, they need to be completely familiarized with how they are implemented and must receive enough training in this regard. Lack of sufficient information about assessment methods is one of the most important reasons for students’ stress and anxiety. Another factor was inadequate time between each station; consequently, stress and anxiety can be reduced by allocating enough time to each station.[34] Eman Ali Moselhi Mater et al. reported that 65% of students experienced stress and anxiety when using OSCE method.[35] In their study, students reported inadequate time in each station and too many stations as a reason for their anxiety. Consequently, by allocating sufficient time to each station and designing appropriate number of stations, stress can be reduced.[35]

Moreover, the results of the study by Brand and Schoonheim-Klein↱ showed that new evaluation methods cause anxiety and stress among students, which is attributed to their lack of knowledge; therefore, educating students could reduce their anxiety. Furthermore, the indirect implementation of these methods would also reduce stress.[36] In his study, Allan focused on the reduction of anxiety and stress during OSCE due to indirect monitoring and found it to be the reason behind the evaluation method's success and the students’ satisfaction.[37]

Because standard tools in the field of research were not found in Iran or abroad, a researcher-made questionnaire was used. The content validity and reliability of the questionnaire were verified and approved. However, due to time limitation and lack of sufficient number of samples, the construct validity of the questionnaire was not verified. Thus, it is suggested that the construct validity of this questionnaire be examined in future studies.

Conclusion

According to the results, implementing new strategies to evaluate clinical education requires infrastructure, as well as overcoming executive obstacles. Appropriate training plus new evaluation methods and familiarity with the implementation process can encourage and support educational institutions and administrators to help implement these methods. Even though there are executive obstacles to the use of these new methods, due to their properties and advantages in evaluating clinical skills such as validity, reliability, and integrity, further studies in the implementation and development of these clinical evaluations are recommended. None of the clinical evaluation methods have been fully and comprehensively verified in terms of validity and reliability, and each one has advantages and disadvantages. Therefore, it is suggested that different evaluation methods be combined.

Financial support and sponsorship

Fasa University of Medical Sciences, Fasa, Iran.

Conflict of interest

Nothing to declare.