Evaluation of Educational Environment for Medical Students of a Tertiary Pediatric Hospital in Tehran, Using DREEM Questionnaire.

BACKGROUND: Tertiary pediatric hospitals usually provide excellent clinical services, but such centers have a lot to do for educational perfection.
OBJECTIVES: This study was performed to address under-graduate educational deficits and find feasible solutions. PATIENTS AND METHODS: This cross-sectional study was done in a target population of 77 sixth year undergraduate medical students (response rate = 78%) who spent their 3-month pediatric rotation in the Children's Medical Center, the Pediatrics Center of Excellence in Tehran, Iran. The Dundee ready educational environment measure (DREEM) instrument was used for assessing educational environment of this subspecialized pediatric hospital.
RESULTS: Among 60 students who answered the questionnaires, 24 were male (40%). Participants' age ranged from 23 to 24 years. The mean total score was 95.8 (48%). Comparison of scores based on students' knowledge showed no significant difference. Problematic areas were learning, academic self-perception, and social self-perception.
CONCLUSIONS: Having an accurate schedule to train general practitioner, using new teaching methods, and providing a non-stressful atmosphere were suggested solutions.

1. Background

Medical education aims to prepare medical students for clinical practice. Educational environment is the most important factor effecting on students’ training and achievements (1-3). Therefore, evaluating the educational setting can show defects in medical education and help to solve them (4).

This study was designed for evaluating medical educational environment in a tertiary pediatric hospital in Tehran, using an international questionnaire known as Dundee ready educational environment measure (DREEM). This instrument was used in several medical schools of different countries, such as Sri Lanka (5), Saudi Arabia (4, 6, 7), United Arab Emirates (8), Turkey (9), Germany (10), Sweden (11), United Kingdom (12-14), and Canada (15, 16) for different types of undergraduate educational studies.

In spite of excellent clinical services, there were many educational problems in this subspecialty referral pediatric center which was similar to other tertiary centers around the world (17, 18). Thus, we tried to address under-graduate educational deficits in this hospital and find feasible solutions.

2. Objectives

In order to investigate whether these educational problems were student dependent, we examined the relation between students’ knowledge and their perception of this educational environment.

3. Patients and Methods

3.1. Participants

This cross-sectional study was done in the Children’s Medical Center, the Pediatrics Center of Excellence in Iran affiliated to Tehran University of Medical Sciences during 2012. Participants were 6th year undergraduate medical students (n = 77). Seventeen students did not fill the questionnaire (response rate = 78%); and therefore were excluded from the study. There were no conflicting ethical issues.

3.2. Medical Setting

In the Iranian medical schools, knowledge of students is best determined by cumulative grade point average (cGPA), medical students’ comprehensive basic sciences exam (CBSE) score, and national university entrance exam (known as Konkoor) rank (19).

Clerkship period (before internship) in Tehran University of Medical Science lasts about 2.5 years; during this period, 3 months are devoted to pediatrics. The curriculum of pediatric course started with 2 weeks of classes and lectures; then students are divided into 10 groups of 7 or 8 and each group spent a week in one specialized ward, including emergency department, infectious disease, neurology, immunology and rheumatology, gastrointestinal, hematology, cardiology, endocrinology, and neonatology. During each week, students spent one or two days in related subspecialty clinics. Each day was planned for one teacher to train the group of students in ward or clinic.

3.3. Instrument

We used DREEM questionnaire for assessing the educational environment of this sub-specialty pediatric hospital. Its origination and use is extensively explained elsewhere (20). It consists of 50 items divided into 5 categories: student perception of learning, 12 questions, student perception of teachers, 11 questions, student academic self-perception, 8 questions, student perception of atmosphere, 12 questions, and student social self-perception, 7 questions.

Items were scored with Likert scale as follows: 4 = strongly agree, 3 = agree, 2 = unsure, 1 = disagree, and 0 = strongly disagree. Some items were negative questions; thus they were coded reversely. Two questions (Numbers 10 and 26) were changed slightly according to particular condition of this pediatric course.

Items scored above 3.5 have good condition; those scored between 2 and 3.5 can be improved, while those less than 2 show areas of shortage and deficit. Guideline for interpretation of each subscale score is summarized in Table 1 (21). As most of the items had low scores and were placed in the weak category (< 2), we used first and third quartile as a cut-off (1.44, 2.32) to select the most problematic and strongest items.

Table 1.

Guide for Interpretation of DREEM Scores

Subscale	Score	Interpretation
Learning	0 - 12	Very poor
	13 - 24	Teaching is viewed negatively
	25 - 36	A more positive approach
	37 - 48	Teaching highly thought of
Teachers	0 - 11	Abysmal
	12 - 22	In need of some retraining
	23 - 33	Moving in the right direction
	34 - 44	Model teachers
Academic self-perception	0 - 8	Feeling of total failure
	9 - 16	Many negative aspects
	17 - 24	Feeling more on the positive side
	25 - 32	Confident
Atmosphere	0 - 12	A terrible environment
	13 - 24	There are many issues that need changing
	25 - 36	A more positive atmosphere
	37 - 48	A good feeling overall
Social self-perception	0 - 7	Miserable
	8 - 14	Not a nice place
	15 - 21	Not too bad
	22 - 28	Very good socially

Translated version of DREEM questionnaire was prepared previously by other researchers (22). Construct validity was checked by confirmatory factor analysis which moderately fit with five predefined subscales. The reliability of this questionnaire in our population was calculated with Cronbach’s alpha method. The internal consistency of total items was 0.93 and of subscales 1 to 5 were 0.89, 0.76, 0.84, 0.89, and 0.7, respectively.

The forms were anonymous; also, there were five questions in advance of main questions, including gender, cGPA, CBSE score, and konkoor rank. Sheets with open questions were also spread among students separately and comments were extracted and classified.

Before distribution of questionnaires the aim of the study was explained in a session and the students were asked to answer the questions according to their general view of this course rather than individual events.

3.4. Statistical Analysis

Nonparametric tests were used to analyze the data. Kruskal-Wallis H test was used to compare students based on their GPA, CBSE, and Konkoor ranks. Missing values of each variable were replaced with its mean. Data were analyzed in SPSS version 20. P value considered significant, if it was less than 0.05.

4. Results

Among 60 students who answered the questionnaire 24 were male (40%) and 36 were female (60%), aged from 23 to 24 years. Forty two (70%) students had cGPA between 16 to 17.9 out of 20; nineteen (32%) students gained CBSE score between 120 to 139.9 out of 200, and 23 (38%) of them had konkoor rank of less than 50 (Table 2).

Table 2.

Comparing Total Mean Score According to Gender, cGPA Score, CBSE Score, and Konkoor Rank [a]

	No. (%)	Mean Score ± SD
Gender
Male	24 (40)	91.3 ± 33.1
Female	36 (60)	98.8 ± 18.6
cGPA score
18 - 20	11 (18)	98.2 ± 33
16 - 17.9	42 (70)	95.6 ± 23.9
14 - 15.9	7 (11)	93.2 ± 24.0
12 - 13.9	0	0
10 - 11.9	0	0
CBSE score
180 - 200	0	0
160 - 179	11 (18)	104.6 ± 33.7
140 - 159	18 (30)	90.5 ± 22.3
120 - 139	19 (31)	94.3 ± 25.1
100 - 119	12 (20)	98 ± 22.2
Konkoor rank
1 - 49	23 (38)	99.1 ± 28.6
50 - 99	9 (15)	83.4 ± 28.6
100 - 149	9 (15)	83.4 ± 27.1
150 - 199	7 (11)	104.2 ± 16.4
> 200	12 (20)	98 ± 23.1

a Abbreviations: CBSE, comprehensive basic science exam, cGPA, cumulative grade point average; SD, standard deviation.

The mean total score was 95.8 (48% of total score). Table 3 demonstrates the mean score for each subscale. A comparison between proportionate score of each subscale is revealed in Figure 1. Items scored above third and below first quartile are shown in Table 4. The least (L) and most (M) scored items in each subscale are as follows:

Table 3.

Total Mean Score and Mean Score of Each Subscale

Questionnaire	M ± SD [a]	Cronbach’s α
Total score	95.8 ± 25.4	0.93
Learning	18.4 ± 8.6	0.88
Teacher	26.2 ± 5.6	0.76
Academic self-perception	13.6 ± 5.3	0.84
Atmosphere	23.5 ± 8.8	0.89
Social self-perception	13.8 ± 4.4	0.70

a SD = standard deviation.

Figure 1.

Box and Whisker Plot for Score of Each Subscale

Table 4.

Items Less Than First and More Than Third Quartiles

Items	Mean Score ± SD [a]
Items more than third quartile (≥ 2.32)
2. The course organizers are knowledgeable	3.08 ± 0.74
15. I have good friends in this course	2.98 ± 1.02
18. The course organizers appear to have effective communication skills with patients	2.71 ± 0.97
37. The course organizers give clear examples	2.68 ± 0.7
6. The course organizers espouse a patient centered approach to consulting	2.61 ± 0.88
49. The registrars irritate the course organizers	2.59 ± 0.99
50. I feel able to ask the questions I want	2.56 ± 0.99
39.The course organizers get angry in teaching sessions	2.53 ± 0.91
17. Cheating is a problem in this course	2.43 ± 1.04
19. My social life is good	2.43 ± 0.90
8. The course organizers ridicule their registrars	2.41 ± 0.99
33. I feel comfortable in teaching sessions socially	2.37 ± 1.06
5. Learning strategies which worked for me before continue to work for me now	2.32 ± 0.87
Items less than first quartile (≤ 1.44) qua
14. I am rarely bored on this course	1.2 ± 0.99
22. I feel I am being well prepared for my profession	1.24 ± 0.94
26. Last year’s work has been a good preparation for this year’s work	1.32 ± 0.91
25. The teaching overemphasizes factual learning	1.33 ± 1.05
13. The teaching is registrar centered	1.34 ± 1.11
3. There is a good support system for registrars who get stressed	1.34 ± 0.95
7. The teaching is often stimulating	1.36 ± 1.09
44. The teaching encourages me to be an active learner	1.37 ± 1.09
24. The teaching time is put to good use	1.39 ± 1.16
43. The atmosphere motivates me as a learner	1.41 ± 1.03
41. My problem solving skills are being well developed here	1.42 ± 1.024
12. The course is well timetabled	1.44 ± 1.168

a SD = standard deviation.

Learning category: L: “The teaching is too teacher centered.” was weakest. No item was above the third quartile.

Academic self-perception category: L: “I feel I am being well prepared for my profession.” M: “Learning strategies which worked for me before, continue to work for me now.”

Atmosphere category: L: “The atmosphere motivates me as a learner.” M: “I feel able to ask the questions I want.”

Social self-perception category: L: “I am rarely bored on this course.” M: “I have good friends in this course.”

Teacher category: No Item was below the first quartile. M: “The course organizers are knowledgeable” was the strongest.

Comparison between different cGPA, CBSE, and konkoor groups showed no considerable difference (P value > 0.05). Extracted students’ comments are gathered in Table 5.

Table 5.

Summarized Comments of Students for Improvement of Educational Environment

	Suggestions for Reform
Lectures
	Subjects should be related to common pediatric diseases (those will be necessary for general practitioners (GP))
	Lectures should be changed to group discussion and case-based learning instead of conveying large amount of facts and information to students
	Attending the class should not be obligatory
	Evaluation of teacher’s skills at the end of each lecture
Wards
	Introducing the semiology of related organ system at the beginning of each ward
	Introducing Important issues that is necessary for GP at the beginning of each ward
	Clinical training is better to be taught by only those teachers with high skills and high enthusiasm
	Clinical evaluation of students is better to be done at the end of each week instead of end of the whole course (which makes the evaluation more valid)
Clinics
	Students should have the role of history taking and physical examination instead of just being an observer
	Students should attend afternoon clinics (which have more opportunities for learning)
Educational office
	Omitting the attendance checking
	Respectful interaction with students
	Accepting feedbacks from students
Theoretical assessment
	Omitting the midterm exam (due to incompatibility of wards and exam subjects)
	The exam should have harmonic distribution
	The exam should contain only necessary subjects for under-graduate student
New offers
	Training pediatric physical examination in skill lab
	Preparation of guideline as reference for clinical learning according to national epidemiology of pediatric diseases

5. Discussion

Although there is no published study comparing clinical skills of medical students in Tehran University of Medical Sciences with other universities, according to konkoor ranks and CBSE scores, it could be noted that they are the best among their peers in Iran. Considering that, educational environment of this subspecialty pediatric hospital obtained a mean total score of 95.8. As said by McAleer and Roff (21) this score enlightens plenty of problems in educational environment. Besides, we found that both weak and strong students with low and high cGPA, CBSE, and konkoor rank were dissatisfied with this educational environment; emphasizing that these problems were not related to students’ incompetency.

In comparison to most of studies surveying whole courses of undergraduate education, our study was about only a single course. Therefore, comparing it with other studies in different countries is roughly inferable; nevertheless they are summarized in Table 6.

Table 6.

Comparison of DREEM Mean Total Scores of Different Studies in Various Countries

Country	Year of Study	Number of Participants	Mean Total Score
Asia
Iran (22)	2010	210	99.6
Saudi Arabia (6)	2004	450	102
Saudi Arabia (7)	2008	500	89.9
Saudi Arabia (4)	2012	358	108.28
United Arab Emirates (8)	2011	190	120
Kuwait (23)	2009	202	105
Turkey (9)	2008	553	117.63
Malaysia (24)	2009	71	133.12
Sri Lanka (5)	2005	339	108
Japan (2)	2010	6725	112
Europe
Germany (10)	2011	1119	109.75
Sweden (11)	2011	503	145.2
United Kingdom (25)	2009	216	143.3
United Kingdom (14)	2005	206	139
United Kingdom (12)	2007	130	143
Ireland (18)	2010	199	149.47
North America
Canada (15)	2004	407	97
South America
Chile (26)	2009	328	127.5
Australia
Australia (27)	2009	143	62

By means of McAleer and Roff guideline (21), students’ scores in each subscale showed that Learning “is viewed negatively”, Teacher “moving in the right direction”, Academic self-perception has “many negative aspects”, Atmosphere has “many issues which need changing”, and Social self-perception is “not a nice place”. In order to discuss about the strengths and weaknesses, we used the individual item scores in each subscale.

Learning: Consecutive lecture-based classes conveying large amount of information, made the theory classes unsatisfactory, as was shown by low scores in items 7, 13, 25, 44, 48 and found in students’ comments (Tables 4 and 5). Regarding that all items in this section scored below the third quartile (less than 2.32), fundamental reforms should be made.

Teacher: High scores in items 2, 8, 18, 37, 39, 49, and 50 (Table 4) elucidate that there are knowledgeable and experienced pediatricians in this center, practicing scientifically and making moral interaction with students and patients.

Academic self-perception: There was lack of an explicit plan to prepare students for general practice. Also most of the lectures and rounds were pertained to subspecialty issues and rare cases, as low scores in 22, 26, and 41 items (Table 4).

Atmosphere: Items 33 and 50 (Table 4) showed that students were able to interact with their teachers easily; however, the atmosphere of classes and rounds were not motivating, as shown by item 43 (Table 4). Other important de-motivating factors were stressful attendance checking and authoritarian educational office, as was also noted in students’ comments (Table 5). This fact was also discussed in another study (2). Numerous tasks of attending caused interference between their educational activity and the written timetable, explaining low score in item 12.

Social self-perception: Because of 3 months of tight classes, large amount of tasks and two heavy exams with limited time to prepare, students became jaded, as shown by item 14 (Table 4). Low score in item 3 is due to lack of friendly relationship between teachers and students out of teaching classes, which led students to feel ineffective and having no social supports (Table 4).

We believe that most of these mentioned educational problems are due to subspecialty nature of this center. Many studies showed advantages of medical education in general practice setting versus tertiary referral hospitals (18, 28-30). Learning medicine in general view, engagement in medical team, doing some clinical tasks and following patients are explained reasons for this preference. We think that training students for general practice needs separate setting in this tertiary hospital; for instance, making 1 or 2 general pediatric wards may be helpful.

At last, the results were discussed with manager and attending of this hospital and few changes were made in the curriculum.

Limitation of our study was small number of participants, short period of this course and inability to reevaluate perception of this group of students after modification of curriculum. Precise distribution of questionnaire, non-parametric analysis of our data, good cooperation of students for giving comments, and discussing it to hospital authorities were the strengths of our study. Further studies on subsequent semesters and modifying the environment and re-evaluation of the program can improve this educational environment.

In spite of excellent pediatric health care, several educational problems could be noted in this tertiary pediatric center. The most problematic areas were learning, academic self-perception, and social self-perception. Feasible solutions for improving this educational setting are having an accurate schedule to educate general practitioner by emphasizing on the most prevalent necessary subjects, using new teaching methods that involve students, and providing a non-stressful and friendly atmosphere. This study can be used as a baseline to be compared with new curriculum and educational reforms.