Does curricular change improve faculty perceptions of student experiences with the educational environment? A preliminary study in an institution undergoing curricular change.

PURPOSE: College of Medicine, Gulf Medical University, United Arab Emirates, underwent a major curriculum change from a discipline-based to an organ system-based integrated curriculum. However, it was not known how the faculty perceived the changes in the educational environment as experienced by the students. In this context, we aimed to compare the faculty perceptions of the student experiences in the discipline-based curriculum with those in the organ system-based integrated curriculum.
METHODS: The Dundee Ready Education Environment Measure (DREEM) questionnaire was modified to assess faculty perceptions of the student experiences, pilot-tested, and administered to all faculty members (n=28) involved in the discipline-based curriculum (FDC) in January 2009. In the subsequent year, data were collected from the same faculty involved in the new integrated curriculum (FIC). Collected data were transferred to Predictive Analytics Software version 18. Total, domain, and individual statement scores were assessed with the Wilcoxon signed rank test. Percentage agreement, disagreement, and uncertainty were assessed by the McNemar's test for proportion.
RESULTS: The mean total DREEM score was significantly higher (P<0.001) for FIC (139/200) as compared to FDC (119/200). The FIC perceived significantly more positive student experiences with the educational environment as indicated by the domain scores and statement scores. The differences in proportions of agreement between FIC and FDC also reinforced that the FIC perceived more positive student experiences with the educational environment.
CONCLUSION: The study showed that the faculty perceived the organ system-based integrated curriculum as providing a better educational environment for the students than the discipline based curriculum.

INTRODUCTION

The importance of gauging the educational environment of a medical school has been well recognized. Positive environment and student achievement, satisfaction and success are intimately linked. Learning environment and curriculum are also closely associated [1]. The College of Medicine, Gulf Medical University, United Arab Emirates, had a traditional discipline-based undergraduate curriculum for over 10 years. However, reviews of the traditional curriculum, guidelines from Ministry of Higher Education (accreditation body in United Arab Emirates) and a desire to keep up with the global changes in medical education resulted in a major curricular change from the traditional curriculum to an organ-system based integrated curriculum with elements of problem based learning. A more student-centered approach using less didactics and more interactive sessions (small group learning, computer aided learning, case based learning, problem based learning) was the focus of the new curriculum. The new curriculum, in place from 2008, is imparted in three phases. The first phase (of one year duration) involves an introduction to medical sciences with courses like Cells, Molecules and Genes, Tissues and Organs, Embryogenesis and Life Cycle, Metabolism and Nutrition, Internal and External Environment, Language and Communication Skills, and Psychosocial Sciences. The next two years of Phase II are devoted to the integrated study of all organ systems and the final Phase III (of two years duration) includes the clinical clerkships. Subsequently, the students undergo one year of compulsory rotating internship.

While a lot of attention has been given to the students’ perspective about the educational environment [2-5], very few studies have looked into perspective of the other stakeholders such as the teaching staff. Previous studies comparing the faculty perceptions with those of the students revealed that the faculty and students had similar perceptions about the student experiences in an integrated curriculum [6,7]. The perceptions of the educational environment by medical teachers are significant not only for themselves but also for the students. The environment of any medical school will be a determinant of the behavior of the students and teachers of the medical school [1]. Hence, the quality of the educational environment in any institution can be improved by studying the teaching faculty’s perceptions. The faculty viewpoints about the student experiences in the educational environment prior to and after the curricular change at College of Medicine were not explored in depth. Hence, the objectives of our study were to measure and compare the faculty perceptions of the student experiences with the educational environment in the discipline-based curriculum with that in the newly introduced integrated curriculum at College of Medicine.

METHODS

Instrument

The validated and highly reliable Dundee Ready Education Environment Measure (DREEM) has been used by a number of medical colleges to assess the learning environment as perceived by the students. DREEM has been used to diagnose the positive or negative aspects of individual institutions; to identify the perceived weaknesses of a new curriculum and to compare perceptions of medical students in discipline-based curriculum with those in innovative curriculum [2-5]. A modified version of the DREEM inventory also assessed the perception of teachers and compared it with the students' perceptions [6,7].

The DREEM inventory involves 50 statements divided into 5 domains which are:

1. Students' perceptions of learning (SPL): 12 statements, maximum score is 48; 2. Students' perceptions of teachers (SPT): 11 statements, maximum score is 44; 3. Students' academic self-perceptions (SAP): 8 statements, maximum score is 32; 4. Students' perceptions of atmosphere (SPA): 12 statements, maximum score is 48; 5. Students' social self-perceptions (SSP): 7 statements, maximum score is 28.

The total score for all domains is 200. Each statement is scored from 0–4 with 4, strongly agree; 3, agree; 2, unsure; 1, disagree; 0, strongly disagree. Nine negative statements are scored in reverse for analysis.

The main aim of this paper was to assess the faculty perceptions of students’ experiences with the educational environment in a curriculum. However, to the best of our knowledge there is no appropriate instrument available to assess the views of the faculty. As the DREEM questionnaire was originally developed to give only the students’ perceptions of the educational environment, hence, in our study it was modified so that certain statements in the first person (e.g., I am encouraged to participate in class.) were rephrased to allow the faculty to give their opinions about the student experiences in our institution. (e.g., The students are encouraged to participate in class.) Permission to modify the instrument was obtained from the authors of the original DREEM. The content validity of the modified DREEM questionnaire was obtained through a review process with a medical education expert and a socio-psychologist. The questionnaire was pretested on a group of faculty members, to ensure face validity and this data was excluded from final analysis. Following the pilot study, descriptive phrases were added to some statements of the questionnaire for better understanding. In addition, open-ended responses about the students’ experiences were also encouraged.

Sample

The 30 faculty members of College of Medicine were contacted personally by the authors. Explanations of the objectives and the method of filling out the questionnaire were given. The faculty was specifically instructed that their perceptions should be about the students' experiences in the previous academic year. Any statements dealing with students’ personal aspects of the environment (e.g., Students have good friends in this school.) or outside of the context in which some of the staff encountered the students (e.g., Teachers are patient with the hospital patients.) were encouraged to be responded as unsure. Voluntary participation was stressed upon with informed consent obtained from the faculty.

DREEM questionnaire was answered anonymously by the faculty involved with students of the 2007 batch (last batch of discipline-based curriculum; n=28; FDC) in January 2009 and in the subsequent year (January 2010), by the same faculty who were now teaching students of the 2008 batch (first batch of integrated curriculum; n=28; FIC). Data from those staff involved in teaching in both the consecutive academic years (2009 and 2010) were selected for final analysis.

The study was approved by the Ethics Review Committee of the institution.

Statistical analysis

Data was analyzed using the PASW ver. 18.0 (SPSS Inc., Chicago, IL, USA). The mean total, domain and individual statement scores were expressed as mean±SD. Comparison of scores was done using Wilcoxon signed rank test. P-value less than 0.05 was considered as statistically significant.

Due to the anticipated large number of unsure responses, the faculty responses to individual statements were also classified into three categories: agreement (strongly agree or agree), disagreement (strongly disagree or disagree), and unsure. The McNemar’s test for proportion was used to compare the percentage agreement of staff giving responses in each category using a significance level of 0.05.

RESULTS

A total of 28 questionnaires were returned out of 30 subjects. Majority of the faculty were female (64%), senior faculty (61%), those with teaching experience of more than 10 years (68%), and from pre-clinical/para-clinical departments (82%). The faculty had multiple teaching roles as lecturers/seminar leaders and as tutors for tutorials, case-based learning (CBL)/problem-based learning (PBL) groups.

The mean total DREEM score for FDC and FIC was 119/200 and 139/200, respectively (P<0.001). The mean domain scores obtained by FDC and FIC are shown in Table 1. Interpretation of domain scores revealed that both groups perceived moving in the right direction for the teachers as the scores for the domain Students' Perceptions of Teachers ranged from 23 to 33; feeling more on the positive side for the academic self-perception as the domain scores for Students' Academic Self-Perceptions were between 17–24 ; a more positive atmosphere for the atmosphere as scores for domain Students' Perceptions of Atmosphere were between 25–36 and not too bad for the social self-perceptions with scores of Students' Social Self-Perceptions ranging from 15 to 21. FDC perceived a more positive approach whereas FIC perceived teaching highly thought of for the learning as the scores for domain Students' Perceptions of Learning were 28.3 and 36.9, respectively. However, all mean domain scores for FIC were significantly higher as compared to FDC.

Table 1.

Difference of scores on five domains of the modified Dundee Ready Education Environment Measure questionnaire by faculties who participated not only in faculty teaching in discipline-based curriculum in 2009 but also faculty teaching in integrated curriculum in 2010 in College of Medicine, Gulf Medical University, United Arab Emirates

Domain	FDC (%)	FIC (%)
Students' perception of learning	28.29±5.54 (58.9)	36.89±4.34 (76.9)***
Students' perception of teachers	28.43±5.37 (64.6)	31.68±4.60 (72.0)***
Students' academic self-perception	18.29±3.39 (57.2)	21.25±3.34 (66.4)***
Students' perceptions of atmosphere	29.00±5.37 (60.4)	32.57±4.80 (67.9)***
Students' social self-perceptions	15.18±2.70 (54.2)	16.57±2.86 (59.2)**
Total DREEM score for the group	119.18±18.76 (59.6)	138.96±15.51 (69.5)***

Values are presented as mean±SD domain scores (% of maximum score).

FDC, faculty teaching in discipline-based curriculum; FIC, faculty teaching in integrated curriculum; DREEM, Dundee Ready Education Environment Measure.

P<0.01.

P<0.001.

FDC identified Students' Perceptions of Teachers as the domain with highest mean score, whereas FIC gave highest scores to the domain Students' Perceptions of Learning. Both groups gave lowest mean scores to the domain Students' Social Self-Perceptions. On analysis of the mean scores of individual statements, mean scores 3 and above were considered as areas of strength; mean scores greater than 2 and less than 3 were considered as areas that could be improved and mean scores equal to or less than 2 were areas of weaknesses. FDC and FIC gave the highest score for the statement the teachers are knowledgeable. FDC did not feel that the teaching is student centered and so this statement had the lowest score. FIC gave the lowest scores to the statement stating the students’ accommodation is pleasant (Tables 2, 3).

Table 2.

Each statements of the modified Dundee Ready Education Environment Measure questionnaire by faculties, with significantly different scores, who participated not only in faculty teaching in discipline-based curriculum in 2009 but also in faculty teaching in integrated curriculum in 2010, in College of Medicine, Gulf Medical University, United Arab Emirates

No.	Statement	FDC	FIC
Students’ perception of learning
1	The students are encouraged to participate in class.	2.68±0.905	3.39±0.786**
7	The teaching is often stimulating.	2.62±0.832	3.11±0.629*
3	The teaching is student centered.	1.25±0.752	2.96±0.881***
16	The teaching is sufficiently concerned to develop the student’s competence.	2.68±0.819	3.11±0.629*
22	The teaching is sufficiently concerned to develop the students’ confidence.	2.68±0.772	3.21±0.630***
25	The teaching gives too much of importance to factual learning.	1.82±1.020	2.54±1.071**
38	The students are clear about the learning objectives of the course.	2.36±1.096	3.25±0.752***
44	The teaching encourages the students to be active learners.	2.21±0.876	3.18±0.476***
47	Long term learning is given importance over short term learning.	2.54±0.881	3.21±0.568***
48	The teaching is too teacher-centered.	1.54±1.036	2.75±1.005***
Students’ perception of teachers
8	The teachers ridicule the students.	3.11±0.875	3.39±0.685*
9	The teachers are authoritarian.	2.00±0.943	2.46±0.744*
32	The teachers provide constructive criticism.	2.54±0.744	3.07±0.716**
39	The teachers get angry in class.	2.00±0.816	2.57±0.790**
50	The students irritate the teachers.	2.04±0.881	2.54±0.962*
Students’ academic self- perception
21	The students feel they are being well prepared for their profession.	2.61±0.786	2.96±0.838*
27	The students are able to memorize all they need.	1.79±0.738	2.07±0.766*
31	The students have learned a lot about empathy in their profession.	2.21±0.568	2.71±0.763**
41	Students’ problem solving skills are being well developed here.	1.89±0.786	3.11±0.416***
45	Much of what students have to learn seems relevant for a career medicine.	2.79±0.738	3.32±0.612***
Students’ perceptions of atmosphere
11	The atmosphere is relaxed during the hospital ward teaching.	2.11±0.567	2.25±0.645*
23	The atmosphere is relaxed during lectures.	2.61±0.832	2.93±0.539*
30	There are opportunities for the students to develop interpersonal skills.	2.46±0.744	3.11±0.786***
33	The students feel comfortable in class socially.	2.57±0.634	2.86±0.591**
36	The students are able to concentrate well.	2.32±0.723	2.79±0.787**
43	The atmosphere motivates the students as learners.	2.29±0.763	2.86±0.651**
49	The students’ feel able to ask the questions they want.	2.75±0.799	3.29±0.600***
Students’ social self-perceptions
14	The students are rarely bored in this course.	1.75±0.752	2.39±0.875**
15	The students have good friends in this school.	2.50±0.694	2.86±0.756*
19	The students’ social life is good.	2.25±0.752	2.39±0.786*

Values are presented as mean±SD scores of statements with significant differences. Negative statements are in italics with the reversed scores inserted into the Table. Higher a score for these statements, the more the faculty members disagreed to the statement.

FDC, faculty teaching in discipline-based curriculum; FIC, faculty teaching in integrated curriculum.

P<0.05.

P<0.01.

P<0.001.

Table 3.

Each statements of the modified Dundee Ready Education Environment Measure questionnaire by faculties, without significantly different scores, who participated not only in faculty teaching in discipline-based curriculum in 2009 but also in faculty teaching in integrated curriculum in 2010, in College of Medicine, Gulf Medical University, United Arab Emirates

No.	Statement	FDC	FIC
Students’ perception of learning
20	The teaching is well focused.	3.00±0.816	3.18±0.723
24	The teaching time is put to good use.	2.93±0.663	3.00±0.667
Students’ perception of teachers
2	The teachers are knowledgeable.	3.36±0.826	3.43±0.690
6	The teachers are patient with the hospital patients.	2.61±0.832	2.71±0.854
18	The teachers have good communication skills with hospital patients.	2.54±0.838	2.71±0.763
29	The teachers are good at providing feedback to students.	2.50±0.839	2.79±0.917
37	The teachers give clear examples.	2.79±0.787	2.96±0.637
40	The teachers are well prepared for their classes.	2.96±0.744	3.04±0.881
Students’ academic self-perception
5	Learning strategies which worked for the students before continue to work for them now.	2.46±0.793	2.21±1.067
10	The students are confident about passing this year.	2.14±0.591	2.29±0.854
26	The students’ work last year has been a good preparation for this year’s work.	2.39±0.737	2.57±0.742
Students’ perceptions of atmosphere
12	The timetable of this school is appropriate.	2.64±0.870	2.64±0.911
17	Cheating is a problem in this school.	2.07±0.940	2.21±1.031
34	The atmosphere is relaxed during seminars/tutorials.	2.75±0.645	2.86±0.891
35	The students find the experience (of studying here) disappointing.	2.36±0.780	2.54±0.962
42	The enjoyment outweighs the stress of studying medicine.	2.07±0.858	2.25±1.041
Students’ social self-perceptions
3	There is a good support system (help) for students who get stressed.	2.50±0.839	2.6±0.956
4	The students are too tired to enjoy this course.	1.96±1.105	2.04±1.036
28	The students seldom feel lonely.	2.32±0.723	2.43±0.742
46	The students’ accommodation is pleasant.	1.89±0.629	1.86±0.651

Values are presented as mean±SD scores of statements without significant differences. Negative statements are in italics with the reversed scores inserted into the Table. Higher a score for these statements, the more the faculty members disagreed to the statement.

FDC, faculty teaching in discipline-based curriculum; FIC, faculty teaching in integrated curriculum.

For FDC, there were 10 statements (20%) indicating the areas of weaknesses; three statements in the positive area (6%), and the remaining statements were in the areas that could be improved (74%). However, FIC showed a more positive perception as only 1 statement had a mean score of 2 or less (2%); 17 statements had a mean score above 3 (34%), and the rest were in the areas that could be improved (64%) (Fig. 1). On comparison of percentage agreement, FIC tended to agree significantly more than FDC for 4 statements (Table 4). Analysis of percentage disagreement identified one statement with which FIC significantly disagreed. (The teaching is too teacher-centered.)

Fig. 1

Mean statement scores for faculty teaching in discipline-based curriculum (FDC) and faculty teaching in integrated curriculum (FIC).

Table 4.

Number of faculties with agreement of statements out of 28 respondents

No.	Statement	FDC (%)	FIC (%)	P-value
Students’ perception of learning
13	The teaching is student centered.	2 (7.1)	24 (85.7)	0.0001
38	The students are clear about the learning objectives (goals) of the course	11 (39.3)	25 (89.3)	0.0303
44	The teaching encourages the students to be active learners.	10 (35.7)	27 (96.4)	0.0085
Students’ academic self-perception
41	The students’ problem solving skills are being well developed here.	6 (21.4)	27 (96.4)	0.0005

FDC, faculty teaching in discipline-based curriculum; FIC, faculty teaching in integrated curriculum.

The unstructured qualitative data from the open-ended responses also provided insight into the respondents' views. Most of the faculty supported the curricular change. However, one of them felt that there was less emphasis on basic sciences knowledge in the new curriculum.

DISCUSSION

The success of an effective curriculum depends on a positive educational environment [1]. Any change made to the curriculum will invariably result in changes to the environment. To obtain the best outcomes, teachers must also be committed to any changes in the curriculum. Hence, we have studied the changes in the educational environment, as perceived by the teachers, after a shift to a more student centered curriculum. Since we could not find an appropriate instrument to gauge the faculty perceptions of the changed educational environment, we had modified the DREEM questionnaire and used it to assess the perceptions of the faculty about the student experiences in our school. This would indirectly allow us insights into the faculty perceptions of our curricular change. The findings from this study will also help in ascertaining the weaknesses and strengths of our curriculum and in turn in improving our newly introduced curriculum.

We had an acceptable response rate of 93%. We found that the mean total DREEM score of our faculty (139 /200) was lower than that (144/200) previously reported [7]. The positive response observed by us is similar to studies which show teachers’ views on integrated; problem-based medical curricula to be positive [8]. The positive response by FIC to the student centered curriculum, which emphasizes long term learning and problem solving skills, are reflected in the higher mean scores in the domains Students' Perceptions of Learning and Students' Academic Self-Perceptions. Low scores in domains Students' Social Self-Perceptions may not reveal perceptions of weaknesses, instead could be due to staff unfamiliarity with certain aspects of the students' experience since faculty were asked to select unsure when the statements dealt with aspects with which they were unfamiliar.

The individual statement analyses identified the teachers' medical knowledge and competence as the strength of our college. The FIC perceive that significant endeavors have been made to encourage student participation in class, make teaching stimulating, develop the students' competence, confidence and interpersonal skills; clarify learning objectives to students, encourage active and long term learning and encourage the students to ask questions they want. The relevance of learning to a career in medicine (No. 45) has also been perceived to be enhanced which may lead to enhanced motivation and retention of learning. The significant positive responses to the student-centeredness of teaching and the development of problem solving skills in the students reassure the curriculum planners that the curricular change has been well discerned by the faculty. This is vitally important as faculty contributions and positive views of planned curricular changes lead to successful curricular renewals [9].

The highlighting of the teachers strengths such as well focused teaching, teaching time put to good use and teachers being well prepared for their classes are common to both the groups and this reflects the self-confidence of the teachers. The FIC perceived positive changes with regard to constructive criticism and teachers’ behavior towards the students as significant. However, scores of statements teachers are authoritarian (No. 9); teachers get angry in class (No. 39), teaching is too teacher-centered (No. 48), and students irritate the teachers (No. 50) reveal FIC’s perception of being unsuccessful in relinquishing their conventional roles. FIC also perceive that there is room for improvement with regard to feedback by the teachers and teachers giving clear examples. Though literature reviews have revealed that there are substantial differences between the students' perceptions about their lecturers when compared with the perceptions of the faculty about their colleagues [10], a previous study comparing faculty perceptions with those of the students showed similar perceptions of the teachers by both the groups [6].

The traditional medical curriculum has been deemed to be overloaded with information and previous studies and meta-analytical reviews have shown PBL based curriculum associated with less factual knowledge [8]. However in our study, scores of statements associated with importance to factual learning, inability to memorize, appropriateness of the timetable of the school, stress of studying medicine and weariness of students may indicate that the teachers still perceive a curriculum overload. Though significant endeavors have been made to prepare the students for the next level, there is still room for improvement as indicated by the scores being less than 3 for all above statements. Cheating has also been identified as an area requiring intervention. These common issues of concern have also been identified by studies conducted in other institutions [2,5,11] as the areas warranting remediation. The rest of the statements for which the faculty were probably unsure have garnered scores below three.

As the DREEM was not created to acquire other stakeholders’ opinions of how the students experience their educational climate, it was expected that the faculty would not be able to respond to certain statements. The respondents, therefore, were encouraged to mark these statements as unsure. Consequently, the comparison of percentage agreement (faculty who agreed/strongly agreed), identified significant differences between FIC and FDC which reinforced the positive perceptions of FIC about the student-centered curricular change. The fact that FIC tended to disagree that the teaching is too teacher-centered (No. 48) significantly more than FDC, has also supported the previous findings of this study. We were unable to identify any major problem areas with the new curriculum based on the percentage analysis.

The faculty strength of College of Medicine is rather small and most faculty members are well informed about the methods used in the curriculum due to the continuous faculty development programs. Faculty buy-in is essential for any successful major curriculum reform effort [9]. While many of the faculty teaching in the integrated curriculum were not involved in the developmental stages, they have been actively involved in the implementation of the curriculum. These may have contributed to most FIC teachers finding the experiences of the students in the new curriculum more favorable. Faculty perceptions reveal the need for undertaking certain interventions to improve our curriculum. Substantial reduction of core curriculum, introduction of study guides, curriculum maps and providing conceptual frameworks for learning [12] may circumvent curriculum overload. An institutional culture of integrity can be established with clear-cut institutional policies: introduction of medical ethics, formative self-assessment, and newer strategies in assessment like portfolio assessment. These may help in thwarting the problem of cheating encountered. A collegiate, cooperative staff student relationship will reduce the authoritarian leaning of the faculty [1]. Constructive and effective feedback techniques should be reinforced through our faculty development programs [13].

We are aware that our sample size is too small to generalize the results of this study to other schools, and that a modification of the instrument may have had an effect on its reliability. Moreover, the method we have used is indirect, subjective and dependent on teachers' recollection of students' experiences from the previous year. However, faculty members' perceptions are frequently dependent on these subjective factors and may influence the faculty's resultant behavior. More structured qualitative analysis conducted in focus groups and using open-ended questions can supplement our findings [14] and deal with the limitations of this questionnaire based data [15].

In conclusion, our study provides preliminary data about the faculty perceptions’ of the student experiences in the new curriculum in the College of Medicine. The modified DREEM questionnaire has helped in identifying the strengths and weaknesses of the new curriculum from the faculty’s point of view. The faculty teaching in the integrated curriculum perceived the organ system-based integrated curriculum as providing a better educational environment for the students than the discipline based curriculum. However, authoritarian leaning of the faculty, lack of effective feedback, curriculum overload and cheating have been identified as areas of concern. Consequently, a need to fine tune the new curriculum in these areas surfaces as a means of improving the new curriculum and the learning environment in our institution.