Evaluating validity of current criteria for judgment passing ER rotation among internee medical students.

BACKGROUND: Passing the emergency medicine internship is an essential process for all graduates of medical schools. The main purpose of this study was to evaluate validity of current criteria for judgment passing ER rotation among internee medical students.
METHODS: In this cross-sectional study, a total of 200 students in the emergency departments (ED) of the teaching hospitals at Tehran University of Medical Sciences (TUMS) were included. The data were gathered using by a valid self-assessment questionnaire including demographic information and 12-items about the students' level of confidence with the skills and procedures under study. Statistical analyses were done using SPSS (version 22).
RESULTS: 200 medical students participated in this study. A Univariate analysis showed a significant correlation between successfuly completions of training in emergency medicine (passing emergency medicine in past rotations) with some items in self-assessment questionnaire same as knowledge of interns in managing emergency situations (p = 0.009). The Spearman test demonstrated a significant correlation between the duration of emergency training (number of months which interns spent on the emergency training) with the some items in self-assessment questionnaire same as importance of emergency medicine training (p = 0.019).
CONCLUSIONS: According to association between successfully completions of training in emergency medicine and self-assessment questionnaire, it seems current criteria for judgment passing emergency medicine rotation is valid as well as we recommended using self-assessment for evaluating validity of such testes.

Background

It has been emphasized that medical students should be competent in the procedural skills considered crucial to handle emergency situations [1]. Also, there is a need for every medical school to provide the valuable experience for medical students who handle emergencies as they arise in daily practice [2]. Since, the main goal of teaching emergency medicine (EM) is to increase medical students' knowledge and expertise about important skills such as resuscitation, general procedural skills and to improve the perception of the emergency environment as a specialty [3]. In medical schools, students are required to demonstrate clinical proficiency in their educational experience in EM. hence, there are several tests currently in use for assessing students' clinical skills in the EM internship [4] however, previous studies have shown that the results of these tests and passing the EM internship do not correlate accurately with a student's degree of confidence in performing tasks or their readiness for their future career [5], [6], [7] Accordingly, validity and authenticity of assessment tools must be evaluated to ensure that these examinations predict whether the students who pass have greater EM skills and confidence when compared to the ones who fail the rotation [8]. It seems that the self-assessment method is a useful tool for predicting the performance and self-confidence of medical students [9] Studies indicate that self-assessment has many potential benefits, including improving the academic achievement, reflecting on the learning process and promoting a deeper learning approach and encouraging medical students to become life-long learners [10], [11]. Meanwhile, it's important to have multiple sources of evidence same as self-assessment results for supporting score-based inferences and for validity of the criteria used to judge whether a medical student who pass have greater ER skills and confidence when compared to the ones who fail the rotation. As mentioned in the literature review, many studies have addressed medical students' self-confidence, experience and attitude toward the EM rotation [1], [5], [12], [13], [14], [15] however, to the best of our knowledge, very little was found in the literature regarding EM medical students' perceived mastery and self-confidence of their EM skills by self-assessment technique to realign and enhance an emergency training course. According to the importance of the subject, the main goal of this study is evaluating validity of current criteria for judgment passing ER rotation among internee medical students by self-assessment method. Our assumption in this study was that if medical students pass an EM rotation exams and spent more time in EM training that they will have better confidence and skills when it comes to managing emergency situations when compared to their peers who failed the rotation or spent less time on training in self-assessment test. As a result of this research, it is possible that the curriculum for the emergency medicine course in Tehran University of Medical Sciences, one of the largest medical universities in Iran country, was redesigned and improved.

Methods

Study design

This was a cross-sectional study.

Study setting

The study was set in the emergency departments (ED) of the teaching hospitals at Tehran University of Medical Sciences (TUMS). The ED of these hospitals is staffed by a team including faculty, residents, and interns. The faculty members supervise the care of patients, who are assigned to residents and interns. They receive an approval by the faculty before ordering any tests or performing any procedures. Interns must take at least a mandatory four-week EM internship in their fourth year of clinical phase.

Participant

The study population consisted of all emergency internees of the two educational hospitals. A list of students was prepared for this purpose and participants were selected randomly.

Instrument

The instrument was a questionnaire which has previously been shown to be valid and reliable (r = 0.85). The questionnaire included two sections: (1) a demographic information and (2) 12-items about the students' level of confidence with the skills and procedures under study. Likert scales ranging from 1 to 5 were used to score questions in self-assessment questionnaire.

Ethical considerations

Survey participation was voluntary and the anonymity of the respondents was considered. The Ethics Committee of the Tehran University of Medical Sciences approved the study.

Statistical methods

A descriptive analysis was carried out to provide the demographic characteristics. We performed a Univariate analysis to assess the correlation between succefuly completions of training in emergency medicine (passing emergency medicine in past rotations) with mean scores of the self-assessment questionnaire. The variables with normal distribution were analysed by a two-sample t-test, while other variables with non-normal distribution were analysed using the Mann–Whitney test. In addition, the Spearman test was used to assess the correlation between the duration of emergency training (number of months which interns spent on the emergency training) with mean scores of the self-assessment survey.

Results

200 students completed the questionnaire. Of those, 55.3% were women and 44.7% were men; the mean age was 26.2 ± 1.5 years. The mean number of months of the internship training was 9.1 ± 3.5.18. 5% of students (n = 37) would consider a career in acute or emergency medicine in the future.

Overall, eight items had normal distribution, which was analysed by using the t-test, and four items had non-normal distribution, which was analysed using the Mann Whitney test. Univariate analysis showed a significant relationship between passing the emergency medicine rotation with emergency medicine training (p = 0.007), knowledge of interns in managing emergency situations (p = 0.009), and the ability of interns in performing clinical procedures (p = 0.002). The group who had passed the EM rotation had a higher rating than the other group (Table 1). In relation to the non-normal distribution questions, there was a significant relationship between passing the EM rotation with mean scores of the importance of emergency medicine training (p = 0.011), the ability of interns to identify patients in an emergency condition (p = 0.032) and initial assessment of the patient's ABCs (p = 0.007). In general, the group who had passed the EM rotation had a higher rating than the other group on the self-assessment survey (Table 2). In Table 3 distribution of students in each group were reported.

Table 1

Correlation between passing emergency medicine in past rotations with mean scores of the self-assessment survey using the t-test.

Questions	Mean (Group 1: not passed the ED rotation)	Mean (Group 2: passed the ED rotation)	P-value
How much exposure have you had to emergency medicine during your undergraduate training?	3.11 ± 0.71	3.05 ± 0.93	0.63
How much teaching have you had in emergency medicine during your undergraduate training?	2.85 ± 0.68	2.57 ± 0.69	0.007
How challenging do you feel emergency medicine is compared to other clinical specialties?	3.38 ± 0.78	3.56 ± 0.91	0.15
How stressful do you feel emergency medicine is compared to other clinical specialties?	3.76 ± 0.79	3.98 ± 0.89	0.07
How challenging do you feel are these areas of emergency medicine?	3.21 ± 0.65	3.44 ± 0.65	0.01
How would you grade your knowledge on the principles of management of the emergency conditions?	2.35 ± 0.62	2.59 ± 0.611	0.009
How confident do you feel in performing the clinical procedures in the emergency situation?	3.19 ± 0.79	2.84 ± 0.75	0.002
How confident do you feel in interpreting the following clinical investigations?	3.11 ± 0.79	3.05 ± 0.65	0.62

Table 2

Correlation between passing emergency medicine rotation with mean scores of the self-assessment survey using the Mann Whitney test.

Questions	P-value
How important do you think emergency medicine is as part of your training?	p = 0.011
How confident do you feel about managing patient in emergency conditions?	p = 0.032
How confident do you feel about recognizing acutely ill patients?	p = 0.15
How confident do you feel in performing the ABCs?	p = 0.007

Table 3

Distribution of students in each group.

Group	N	Percent
Group 1: Students who were in the first six-month internship	56	28%
Group 2: Students who were in the second six-month internship	103	51%
Group 3: Students who were in the third six-month internship	41	21%

In addition, the Spearman test demonstrated a significant correlation between the number of months of the internship period with the mean score of the importance of emergency medicine training (p = 0.019), knowledge on management of emergency conditions (p = 0.01), ability to manage emergency conditions (p = 0.02), ability to recognize acutely ill patients (p = 0.02), confidence in performing the ABCs (p = 0.007), and confidence in performing the clinical procedures (p = 0.001) (Table 4).

Table 4

Correlation of the duration of emergency training (number of months which interns spent on the emergency training) with mean scores of the self-assessment survey using the spearman test.

Questions	R	P-value
How much exposure have you had to emergency medicine during your undergraduate training?	0.07	0.3
How much teaching have you had in emergency medicine during your undergraduate training?	0.17	0.01
How important do you think emergency medicine is as part of your training?	0.09	0.019
How challenging do you feel emergency medicine is compared to other clinical specialties?	0.007	0.92
How stressful do you feel emergency medicine is compared to other clinical specialties?	−0.049	0.486
How challenging do you feel are these areas of emergency medicine?	−0.322	<0.001
How would you grade your knowledge on the principles of management of the emergency conditions?	0.17	0.01
How confident do you feel about managing patient in emergency conditions?	0.16	0.02
How confident do you feel about recognizing acutely ill patients?	0.15	0.02
How confident do you feel in performing the ABCs?	0.18	0.007
How confident do you feel in performing clinical procedures in an emergency situation?	0.283	<0.001
How confident do you feel in interpreting the following clinical investigations?	0.059	0.4

Discussion

In this study, a self-assessment questionnaire was used to investigate the self-confidence level of interns in several performance areas that were required for emergency medicine. Students in this study reported significantly greater confidence in areas such as emergency medicine training, the importance of EM rotation, knowledge and skills about managing emergency situations, the ability to identify patients with acute conditions, ability to perform clinical procedures, and the proper steps involving ABC after completing an EM rotation. This finding is in agreement with Avegno [1] study, which showed that medical students reported high levels of perceived skills and knowledge after finishing the EM internship [1]. In 2001, Johnson et al. also reported that participating medical students in EM internships provided unique clinical experience and that some of the shortcomings of other internships may be compensated [15]. In general, therefore, it seems that, after passing the EM internship, the undergraduate medical student will be able to apply prior knowledge and skills to the management of a Patient's Problem in an emergency situation [16].

Based on the results of our study, the students' self-confidence in their ability to perform clinical procedures was improved as a result of the EM training rotation. As the evidence shows, working in an EM setting provides an opportunity to learn the most important clinical procedures [14]. The Society for Academic Emergency Medicine identified 26 important procedures with which a physician should be familiar [17].

In addition, we found that exposure to EM internship does not increase medical students' interest in pursuing this specialty. This may be due to the fact that medical students have already decided their specialty prior to this time. This finding is in agreement with the results of a study by Zun and Downey [18], in that internship in EM did not correlate with medical students' decisions on their selection of emergency medicine specialty in respect of their future careers [18]. Based on the results of Reed et al. [19], many factors have been associated with medical students' career choices [19]. The evidence shows that, in recent years, there has been a growing demand for an increased number of EM physicians, since one of the objectives of an EM internship should be to introduce EM specialty to all medical students [1].

While previous studies of EM residents have identified many stressors in the emergency ward, including care responsibilities, patient mortality, peer competition, long hours, night shifts and sleep deprivation [20]. We found that passing the EM internship has not been perceived by medical students as a stressful or challenging event. However, we measured students' self-confidence in terms of stress and challenge in the emergency department, rather than using a specific scale for assessing the stress level in EM internship. Further work is required to establish this. In 2006, Sari Kaya et al. stated that sources of anxiety might be different among medical students who have been exposed to different preclinical phases and different educational settings [21].

In our study, the relationship between numbers of internship months with mean score of knowledge on the management of emergency conditions, the ability to manage emergency conditions, the ability to recognize acutely ill patients, confidence in performing the ABCs, and confidence in performing the essential procedures, was significant. The present findings seem to be consistent with other research, which found that, after spending a few months in the internship period, interns are better able to understand the main issues and skills for better performance in an emergency situation. Vosti et al. (1997) reported that a significant correlation was found between the number of months of clinical training and the attainment and application of clinical knowledge. Therefore, authors recommended that decreasing the duration of clinical training should be approached with caution [22].

It is believed that an increase in clinical knowledge and skills is related to the number of months in internship [22]. Indeed, the medical students will gain clinical knowledge and skills during the undergraduate medical programs such as continuous clinical rotation and quality in medical education, which are related to growth of student knowledge [23]. However, it seems that the adequate growth of clinical knowledge and skills may or may not happen without systematic planning. Mehmood et al. [24], who reported how the required training and core competency is not incorporated into the curriculum, stated that the expectation that all medical students are capable remains unfulfilled [24].

A limitation of this study is that the number of samples is relatively small. This limitation means that the study findings need to be interpreted with caution. Another limitation is the reliability on self-reported ratings by medical students. Thus, results may also be subject to bias. It is unfortunate that the study did not include medical graduates; therefore, other studies with a focus on this particular group are suggested.

Conclusion

According to association between successfully completions of training in emergency medicine and self-assessment questionnaire, it seems current criteria for judgment passing Emergency Medicine rotation is valid as well as we recommended using self-assessment for evaluating validity of such testes. We hope that the results of this study will provide some useful information about the design of the EM internship curriculum and, hence, benefit the administrators in better decision-making.

Ethical approval

The patient didn't involve in this research.

Sources of funding

No Funding.

Author contribution

Hooman Hoseinnejad: Study Design, Data Analysis, Writing the Paper.

Noushin Kohan: Study Design, Data Analysis, Writing the Paper, Data Collection.

Akram Mirzaee: Study Design, Data Analysis, Data Collection.

Conflicts of interest

The author declare no conflict of interest.

Guarantor

Noushin Kohan.