Evaluation of the Institutional Educational Environment by using the Dundee Ready Educational Environment Measure.

BACKGROUND: Evaluation of the educational climate has been highlighted as key to the delivery of high-quality medical education. Health educators across places and educational settings have widely used the Dundee Ready Educational Environment Measure (DREEM) to appraise their institutions' educational climate.
OBJECTIVES: The aim is to assess the educational environment of our Institution and to find the association with the related variables.
METHODOLOGY: Institution-based cross-sectional study was conducted using the DREEM questionnaire enrolling 527 MBBS undergraduate students.
RESULTS: An inclination toward a positive environment was found in our institution, which has statistical significant association with the phase of training of the students.
CONCLUSION: Positive educational environment was found in our institution. This suggests reinforcement of positive practices and scope for improvement in certain other areas. Copyright:

Introduction

Educational and professional environments for medical students across the world are considered highly stressful.[12] Education environment is a complex entity to explain, though carrying great importance in the life of the students, especially in the life of the future health-care providers. Medical professionals dealing with the toughest syllabus, stressful environment, changing demands and expectations of the patients, newer technologies and the endless list, need a productive environment to flourish and excel academically and professionally. The medical education environment is constituted by all the areas affecting the performance of the students, including faculty, wards, classrooms, hospital, teaching skills, demonstration, and the last but not the least peers of the individuals. This educational environment has a personality which can be described or categorized, so it can be stressed or relaxed, strict or lenient, competitive or even hostile.[3] It seems reasonable to assume that stress may be due to a number of factors; and this issue must be addressed with concern and sensitivity as stress may be inversely associated with learning.[4]

According to Harden, measurement of the educational environment for learning acts as a foundation for the analysis of prevailing practices within the institution. The measurement may be the basis for making required adjustments for state of art education practices in alignment with the goals of the institution.[5] It is widely agreed among medical educators that an optimal educational climate is an important factor for effective learning. Accordingly, identifying the prevailing educational environment of any institute must be the focus of administrative stakeholders. Indeed, evaluation of the educational climate has been highlighted as key to the delivery of high-quality medical education.[6] Based on these evaluations; attempts are being made by the medical educators to reform the educational environment making it more student-friendly along with keeping the standards of the learning as per norms.[7]

Evaluation of learning environment helps in identifying strengths and weaknesses of the educational program by obtaining learner's perspective, establishing the impact of change in curriculum, comparing educational programs of different institutes, establishing differences between observed and ideal learning environments, determining the effect of the learning environment on learner's achievements and self-efficacy.[89] Dundee Ready Educational Environment Measure (DREEM) has been recommended as the most suitable tool for measuring the educational environment in undergraduate medical institutions.[910]

Over the past 15 years, medical and allied health educators across places and educational settings have widely used the DREEM to appraise their institutions' educational climate.[111213] This valuable tool was originally designed in English by Roff et al., in 1997.[5] Studies have shown that DREEM is internationally accepted as a useful tool to provide feedback on the strengths and weaknesses of the educational climate at particular educational institutions. One of the important implications of DREEM is that it provides a standardized way for international comparisons between medical schools as well as allowing them to benchmark their educational climate and is claimed to be the cultural-free instrument. In addition, it may locate areas of concern shared by the majority of students that might be unintentionally neglected by educators.[14]

In our institute, no such study has ever been conducted since its inception 15 years back. The current study has been conceptualized with the objective of assessing the educational environment in our Institute and this is a maiden effort in this regard.

Methodology

Institution-based, cross-sectional, convenient sampling study was conducted in Adesh Institute of Medical Sciences and Research, Bathinda. Self-administered, anonymized DREEM questionnaire was used to assess the student's perception regarding their education environment. The questionnaire was administered to the students after taking oral consent from each individual. All MBBS students except interns were enrolled in our study. There are 150 MBBS students in each phase of the MBBS, making a total of 600. Using convenient sampling 527 students were enrolled in the research. Students who were not willing to participate and those who submitted incomplete forms were excluded from the study.

In DREEM, there are 50 items measuring five aspects of the educational environment based on students' perception, which include students' perception of learning (SPoL) – 12 items, students' perception of teaching (SPoT) – 11 items, students' academic self-perception (SASP) – 8 items, students' perception of the atmosphere (SPoA) – 12 items and students' social self-perception (SSSP) – 7 items. Each item was rated based on five-point Likert-scales ranging between 0 and 4 (0 = strongly disagree, 1 = disagree, 2 = unsure, 3 = agree, and 4 = strongly agree). Nine negative items, namely items 11, 12, 19, 20, 21, 23, 42, 43, and 46 were scored in a reverse manner before analysis and interpretation. The maximal global score for the questionnaire is 200, and the global score is interpreted as follows: 0–50 = very poor; 51–100 = many problems; 101–150 = more positive than negative; 151–200 = excellent.[15]

Statistical analysis

The collected data were analyzed using Microsoft Excel (version 2016). Mean and standard deviation along with the 95% confidence interval was calculated for the domain scores and for every question in the questionnaire, the median score was calculated. Frequencies and percentages were calculated for the interpretation of the domain scores and overall scores. Independent t-test and one-way ANOVA was used to find out the associations between the domain scores and related variables.

Ethical consideration

Informed oral consent was obtained from students after briefing about the nature of the research work, making sure that the anonymity of the data will be maintained and will only be used for the research purpose. The study was initiated after taking the permission of the Institutional Ethics Committee. The permission for using DREEM inventory was obtained through E-mail from the original copyrighted author.

Results

In this study, a total of 527 MBBS students were evaluated using the DREEM Questionnaire. The number of females (55.6%) outnumbered the males (44.4%). Almost equal number of the students participated from each MBBS prof-Prof-1 (148), Prof-2 (138), Prof-3 (138), and Prof-4 (103). The reliability coefficient calculated using Cronbach's alpha for the total 50 items was 0.931, indicating high internal consistency. Descriptive analysis of the mean scores showed that overall score and domain scores interpret the good perception of the students regarding the institution's education environment [Table 1].

Table 1

Descriptive analysis of total and domain scores

Domain	Number of items	Maximum score	Mean±SD	95% CI
SPoL	12	48	24.44±6.88	23.85-25.03
SPoT	11	44	24.58±5.56	24.10-25.06
SASP	8	32	18.90±5.41	18.44-19.37
SPoA	12	48	26.00±8.53	25.27-26.73
SSSP	7	28	14.73±4.781	14.33-15.14
Total	50	200	108.65±25.67	106.46-110.85

SPoL: Students’ perception of learning; SPoT: Students’ perception of teaching; SASP: Students’ academic self-perception; SPoA: Students’ perception of atmosphere; SSSP: students’ social self-perception; CI: Confidence interval; SD: Standard deviation

Using the McAleer and Roff scoring system of the DREEM questionnaire,[16] data of the current study was analyzed [Table 2]. In this study, the majority of the students had overall score of being more positive than negative (62.8%), SPoL score – a more positive approach (50.6%), SPoT score - moving in the right direction (60.9%), SASP score - feeling more on the positive side (59.2%), SPoA score – a more positive atmosphere (54.6%) and SSSP score – not too bad (49.9%).

Table 2

Interpretation of total and domain scores (n=527)

Domain	Score	Interpretation	Frequency (%)
Total	0-50	Very poor	8 (1.5)
	51-100	Plenty of problems	172 (32.6)
	101-150	More positive than negative	331 (62.8)
	151-200	Excellent	16 (3.1)
SPoL	0-12	Very poor	26 (4.9)
	13-24	Teaching is viewed negatively	220 (41.7)
	25-36	A more positive approach	266 (50.6)
	37-48	Teaching is highly thought of	15 (2.8)
SPoT	0-11	Abysmal	10 (1.9)
	12-22	In need of something retraining	168 (31.9)
	23-33	Moving in right direction	321 (60.9)
	34-44	Model teachers	28 (5.3)
SASP	0-8	Feeling of total failure	25 (4.7)
	9-16	Many negative aspects	130 (24.7)
	17-24	Feeling more on positive side	312 (59.2)
	25-32	Confident	60 (11.4)
SPoA	0-12	A terrible environment	43 (8.2)
	13-24	There are many issues that need changing	156 (29.6)
	25-36	A more positive atmosphere	288 (54.6)
	37-48	A good feeling overall	40 (7.6)
SSSP	0-7	Miserable	48 (9.1)
	8-14	Not a nice place	187 (35.5)
	15-21	Not too bad	263 (49.9)
	22-28	Very good socially	29 (5.5)

SPoL: Students’ perception of learning; SPoT: Students’ perception of teaching; SASP: Students’ academic self-perception; SPoA: Students’ perception of atmosphere; SSSP: Students’ social self-perception

Of all the 50 items, only question no. 20 had a median value of one, while 25 questions had a median score of 2 and rest of the question had a median value of 3 [Table 3]. After reversing the sequence of negative questions, it was found that question –20 had a median score of 1 and showed strong agreement that teachers got angry in the class. In the list of questions having a median score of 2, question number - 11, 12, 19, 21, 23, 42, 43, and 46 showed agreement and other questions in the same group have disagreed with the concerned statement. All questions scoring the median score of 3 have shown the unsure/neutral state of the students regarding the question/situation asked.

Table 3

Question wise median score

Question number	Median score
20	1
2, 5, 7, 8, 11, 12, 18, 19, 21, 23, 24, 26, 27, 28, 31, 33, 37, 39, 42, 43, 45, 46, 47, 48, 50	2
1, 3, 4, 6, 9, 10, 13, 14, 15, 16, 17, 22, 25, 29, 30, 32, 34, 35, 36, 38, 40, 41, 44, 49	3

Using the independent t-test, the association of the mean total and mean domain scores with the gender of the participants was calculated, however, the association was found to be statistically insignificant [Table 4].

Table 4

Distribution of total and domain scores according to the gender of the participants

Domain	Gender	Mean±SD	T	P*
Total	Female	109.68±25.05	1.022	0.307
	Male	107.38±26.43
SPoL	Female	24.63±6.77	0.706	0.480
	Male	24.21±7.01
SPoT	Female	24.80±5.28	1.022	0.307
	Male	24.30±5.91
SASP	Female	19.08±5.21	0.823	0.411
	Male	18.68±5.66
SPoA	Female	26.34±8.29	1.028	0.304
	Male	25.57±8.82
SSSP	Female	14.83±5.03	0.510	0.610
	Male	14.62±4.44

*P<0.05 was taken as statistical significant. SPoL: Students’ perception of learning; SPoT: Students’ perception of teaching; SASP: Students’ academic self-perception; SPoA: Students’ perception of atmosphere; SSSP: Students’ social self-perception; SD: Standard deviation

Association of the means of total scores and domain scores with different MBBS professionals was calculated using one-way ANOVA test. Results were found to be highly significant in final year students (Prof-4) in all domain scores including total score [Table 5].

Table 5

Distribution of domain and total scores according to different MBBS profs

Domain	Prof	Mean±SD	F	P
Total	1	108.49±25.78	28.996	0.000**
	2	111.81±25.69
	3	94.83±21.85
	4	123.17±20.67
SPoL	1	24.43±6.83	16.939	0.000**
	2	25.60±6.50
	3	21.35±6.99
	4	27.05±5.73
SPoT	1	24.24±5.00	31.178	0.000**
	2	24.63±5.21
	3	22.01±5.45
	4	28.45±4.78
SASP	1	18.63±6.37	9.275	0.000**
	2	19.88±5.20
	3	17.14±4.81
	4	20.33±1.19
SPoA	1	25.66±8.55	30.971	0.000**
	2	27.12±8.47
	3	21.42±7.56
	4	31.12±6.32
SSSP	1	15.53±4.35	12.211	0.000**
	2	14.58±5.22
	3	12.91±4.35
	4	16.23±4.54

**P<0.05, showing statistical significant association. SPoL: Students’ perception of learning; SPoT: Students’ perception of teaching; SASP: Students’ academic self-perception; SPoA: Students’ perception of atmosphere; SSSP: Students’ social self-perception; SD: Standard deviation

Discussion

In this study, 527 MBBS students were evaluated using DREEM questionnaire, females (55.6%) being more in number than males (44.4%). However, in other similar studies conducted on the undergraduate students using the DREEM questionnaire, it was found that the number of males outnumbered the females.[171819] Improved access of the medical education to the female gender in our part of the world is truly reflected in these figures.

Cronbach's alpha in the current study was 0.931-indicating high internal consistency. In another study conducted by Khan in 2018, Cronbach's alpha has the value of 0.78.[20] It indicated that the questionnaire is highly validated for different populations in different scenarios.

The mean overall score in our study was found to be 108.65, with domain mean scores of-SPoL – 24.44, SPoT-24.58, SASP-18.90, SPoA – 26.00, and SSSP – 14.73. Similar results were found in a study conducted by Phadke et al. in 2020, with mean scores as-Total – 121.47, SPoL - 29.27, SPoT - 27.42, SASP - 20.78, SPoA - 27.83, and SSSP - 16.17.[21] In another study conducted by Naik and Singh in 2017 in Northern India, higher mean scores were observed (SPoL - 31.51, SPoT - 28.02, SASP - 21.5, SPoA - 32.46, and SSSP - 17.12), with a total mean score of 130.63-indicating higher quality of the educational environment in their institution.[18] This difference in the scores can be attributed to the government and the private institution's learning and teaching differences.

In this study, the median score of the individual question was used to interpret the agreement of the students regarding the different situations. Median scores interpreted that students have agreed to the zones such as teacher-centered teaching, over emphasizes on factual learning, teachers ridicule the students, students irritate the teachers, disappointing experience, cheating is the problem and admitting that they were too tired to enjoy this course. On the other hand, students showed disagreement with some of the spheres which need amendments like teaching should develop confidence and competence, stimulating teaching, teaching time should be used for good, providing constructive criticism, development of problem-solving skills, entertaining the questions of the students, providing good support to the stressed students and good accommodation to live at. Similar problems/low scoring questions were also observed in a conducted by Phadke et al.[21] Teaching should be such that students should be able to memorize the subject, well prepared for their profession, new learning strategies, motivation to concentrate, and providing support to the lonely and bored students.

However, Bavdekar in 2019 in his study, used the median scores of each domain for the interpretation-with values of median scores as – SPoL - 28, SPoT - 24, SASP - 20, SPoA - 29, SSSP - 17 and overall median score-119.[17] On the other hand, a study conducted by Shah et al. in 2019 used the mean scores of individual questions to find out the lacking areas.[19] It used a scale that items with a mean score of 3.5 and above are the strengths, between 2 and 3 score needs improvement and <2 scoring items are problem areas of that institution.

Majority of the students (62.8%) have marked that the institutional environment is inclined more toward positive side, which was similar to the study conducted by Shah et al.[19] In this study, domain scores showed no statistically significant association with the gender of the participants. Similar findings were found in other studies also.[1718] However, the final prof MBBS students have shown statistically significant domain scores, indicating that these students with time-lapse had adapted in better to the institution's surrounding and learning skills. In contrary to this, study conducted by Naik and Singh did not report significant improvement of domain scores with the advancement of the students through the various years.[18]

The study helped in identifying the various areas of strength and weakness in the educational environment of the institution. The institute has to improve on its teaching and learning methodologies and involvement of the learners in the planning. The student–teacher relationships need to be worked on and made more flexible. Certain administrative measures regarding the fairness of examinations and infrastructure improvement are also required. The students experienced positive academic and social self-perception as well as positive atmosphere which implies good effort on the part of the institution which needs to be reinforced in future for better results.

Conclusion

In this study, more findings toward positive side were reported, indicating a good education environment in our institution. However, still many areas need amendments for further improvement to have better future prospects of the students. More friendly and better ways of teachings should be introduced, along with increase in interactions of the students with teachers are recommended.

Limitation

As the data collected were based on the individual perception, so the subjective variations as well as intra-individual variations depending on the state of mind of that person at that moment are a possibility.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest