Teachers' perceptions on student evaluation of teaching as a tool for faculty development and quality assurance in medical education.

BACKGROUND: Student evaluation of teaching (SET) is considered to be one of the most important as well as inexpensive resources for sustaining professional development in medical teaching. The aim of this project was to improve the quality of education in our medical college by using student feedback as a tool for faculty development. However, it is also important to obtain teachers' perceptions on SET.
MATERIALS AND METHODS: This was an educational research study conducted on a single practical/tutorial batch of fourth-semester students in pathology chosen by random selection. Feedback regarding teaching was collected for all the teachers in the department of pathology where the students had to rate the teachers on a scale of 1-5. Teachers' perceptions on students' feedback were gathered with the help of another structured prevalidated questionnaire containing 15 questions/items. The feedback data were obtained using a 5-point Likert scale. The scores obtained from the students' evaluation data and the teachers' perception data on the different items were analyzed using SPSS software version 20.0.
RESULTS: Significant findings from students' feedback were that 80% of teachers had a median score of >4 in explicit curriculum, indicating that most of the students agreed that the teachers teach their core subject well. However, only 20% of teachers had a median score >4 in implicit curriculum. Teachers, in their feedback, fully agreed that students should be involved in the evaluation of teachers and that student feedback ensures the overall faculty development in the institute and can be used as a tool for quality assurance in medical education.
CONCLUSION: Students' feedback is one of the best methods of evaluation of teachers to ensure the overall faculty development and quality assurance in medical education. Thus, a regular feedback mechanism should be in place for the entire institution. Copyright:

Introduction

Evaluation is an integral part of medical education. It is a well-accepted fact that obtaining feedback about our own teaching is an inexpensive and invaluable way of improving the quality of our teaching.[1]

Quality assurance in medical education is the need of the hour, and accreditation is a process, widely used to evaluate the quality of educational programs.[2] Quality can be assured by transparent selection procedures, well-established entrance examinations, centrally regulated curricula, self-evaluation and academic audits conducted by the institutions themselves, appointing external examiners, and requirement of national examinations before licensure.[3] In the Indian scenario, the initiatives taken by the Medical Council of India (MCI) recently in their Vision 2015 document[4] meet most of these criteria for quality assurance. The MCI has proposed common entrance examinations, has regulated the curricula, and has also proposed a national common licensure examination to be implemented. The final university examinations are conducted by using external examiners. One aspect that is partly addressed is the self-evaluation and monitoring. Although the MCI conducts periodic inspections, they are limited to verification of resources in terms of workforce, infrastructure, and hospitals' bed strength. Curriculum implementation, innovations in teaching–learning processes, and assessment are rarely looked into. If our educational standards have to meet the global standards, quality assurance measures need to be put in place.

While there are a large number of possible sources of feedback and evaluation data on teaching, the most common source of input for teaching evaluation is feedback from the students. Medical students play a pivotal role in faculty development and quality assurance in medical education. During accreditation, independent reports of the students get considerable weightage.[5] Validated and regular feedbacks from students on the course and teaching are considered an essential and integral part of quality assurance.[6]

As a center that aims to promote good quality teaching and support academic staff in their challenging and complex role as teachers, our particular focus would be on a formative and diagnostic feedback mechanism generated from the students, in efforts to improve teaching. This feedback mechanism will serve as a quality indicator tool in medical education and help in the development of faculty members. By obtaining feedback, we can identify the areas to improve our acceptability as a teacher.

The aim of this project was to improve the quality of education in our medical college by using student feedback as a tool for faculty development. There have been a few extensive studies on faculty perceptions on student evaluation of teaching (SET),[7] as most studies are typically localized within one university or even one faculty within a university. Thus, even the published research on faculty perceptions is largely anecdotal in nature.[8] Therefore, the index study also aims to analyze the teachers' perceptions on SET as a tool for faculty development and quality assurance in medical education.

Aims and objectives

Aim: To study teachers' perceptions on SET as a tool for faculty development

Objectives:

To gather students' feedback on every teacher in the department of pathology regarding the quality of teaching

To provide every teacher with his/her individual feedback

To analyze the teachers' responses and their perceptions on the effectiveness of students' feedback

To suggest appropriate recommendations to the institution based on the study findings to incorporate students' feedback mechanism as a tool for quality assurance in medical education.

Materials and Methods

The study was carried out on the effectiveness of student feedback for teachers' evaluation in a recently established medical college by analyzing fourth-semester students. A structured feedback form for teacher's evaluation was developed; face validation and content validation of the same were done by the Institutional Research Committee in collaboration with the Medical Education Unit [Annexure 1]. Reliability analysis was applied to determine the internal consistency of the questionnaire. Internal consistency of the items was evaluated by Cronbach's alpha values. The items were considered to represent an acceptable level of internal consistency if the Cronbach's alpha value was within 0.5–0.7 and a good level if it was >0.7. It was then given to the undergraduate students during the start of their fourth-semester classes. Teachers were also sensitized about the importance of students' feedback through a discussion before starting the study.

Single practical/tutorial batch of fourth-semester MBBS students was chosen by random selection. Complete enumeration of one randomly selected batch (practical/tutorial) of MBBS students was done.

Feedback regarding teaching was collected for all the teachers involved in teaching–learning process in the department of pathology (total faculty strength in the department = 10). The students had to rate the teachers on a scale of 1–5, on 15 different parameters, of which the first 10 concerned with explicit curriculum (how well does the teacher teach the core subject) and the next 5 dealt with implicit curriculum (how well does the teacher model the core values through how he/she behaves with students and with other staff persons).

Feedback questionnaires were distributed to the students by nonteaching staff of the department in the absence of any faculty member which were evaluated (to avoid bias) and collected back in boxes kept in the department office (to ensure confidentiality and anonymity). These feedback questionnaires collected from the students were distributed to the respective teachers in sealed envelopes for their self-evaluation and suggestions, if any.

Teachers' perceptions on students' feedback were gathered with the help of another structured prevalidated questionnaire containing 15 questions/items. The feedback data were obtained using a 5-point Likert scale (1–5) where 5 indicates “Strongly Agree” and 1 indicates “Strongly Disagree” [Annexure 2].

Statistical methods used

The scores obtained from the students' evaluation data and the teachers' perception data on the different items were analyzed using SPSS software version 20.0. IBM Corporation, USA. On applying Kolmogorov–Smirnov test, the P value was found to be statistically significant; therefore, the distribution was skewed. Thus, central tendency and dispersion of data were expressed in median and interquartile range (IQR), respectively.

Observations and Results

Students' evaluation of teaching

Out of a total of 75 students (one randomly selected practical/tutorial batch of fourth-semester MBBS students), 60 students who were present for the class on that day filled the feedback pro forma. The feedback form had the following two broad domains:

How well does the teacher teach the core subject? (explicit curriculum)

How well does the teacher model the core values through how he/she behaves with students and with other staff persons? (implicit curriculum).

Table 1 summarizes the median and IQR scores of all the ten teachers on these two domains (explicit and implicit curricula).

Table 1

Median and interquartile range scores of teachers based on students’ evaluation of teaching

Teachers→	1	2	3	4	5	6	7	8	9	10
Explicit curriculum, median (IQR)	4 (1)	4 (2)	5 (1)	4 (2)	4.5 (1)	3 (0)	3 (1)	4 (2)	5 (1)	4 (1)
Implicit curriculum, median (IQR)	2 (1)	3 (1)	4 (2)	3.5 (1)	3 (2)	4 (1)	3 (1)	3 (1)	2 (1)	2 (1)

IQR=Interquartile range

The significant findings were:

Eight out of the ten teachers had a median score of >4 in the explicit curriculum, indicating that most of the students agreed that the teachers teach their core subject well

However, only two out of the ten teachers had a median score >4 in the implicit curriculum. This indicates that the students felt that 80% of the teachers did not give importance to the core values

Only one teacher had a median score of >4 in both explicit and implicit curricula.

Teachers' perceptions on student evaluation of teaching

The questionnaires were distributed to the ten teachers after they had gone through their individual students' feedback forms. All of them returned the completed questionnaires with their suggestions.

Table 2 summarizes the median and IQR scores of the different items as perceived by the teachers.

Table 2

Median and interquartile range scores of the student evaluation of teaching parameters as perceived by teachers

Serial number	Statements	Teachers→	1	2	3	4	5	6	7	8	9	10	Median	IQR
1	Students should be involved in the evaluation of teachers		5	4	5	5	5	4	5	5	4	4	5	1
2	I am satisfied with the present format of teaching evaluation system		4	5	4	4	4	4	2	4	4	5	4	0
3	Feedback from students should be taken after every semester		5	4	4	4	5	4	3	4	4	2	4	0
4	Students are casual about filling up the questionnaire		2	5	4	5	4	5	3	4	3	5	4	2
5	Designation of the teacher influences the students’ response		4	4	3	4	2	4	4	3	3	2	3.5	1
6	Gender of the teacher influences the students’ response		4	3	3	4	3	4	3	3	1	1	3	1
7	A strict teacher may get poor response as compared to a lenient teacher		5	5	5	4	4	4	3	1	3	1	4	2
8	Students’ feedback as a teacher evaluation system makes you uncomfortable		2	2	4	2	2	2	4	2	3	3	2	1
9	Teachers’ evaluation system helped you to overcome your weaknesses		5	4	5	5	4	5	4	3	3	2	4	2
10	The evaluation system made you more aware of the students’ needs		5	5	4	5	4	5	5	4	4	4	4.5	1
11	Students’ feedback ensures overall faculty development		5	5	5	4	5	5	5	4	5	4	5	1
12	Students’ feedback can be a tool for quality assurance in medical education		4	4	5	5	5	5	4	4	5	5	5	1

IQR=Interquartile range

Statistical analysis of teachers' perceptions

The median and IQR values of item nos. 1, 11, and 12 [Table 2] are 5 and 1, respectively. This indicates that all the faculty members “Strongly Agree” on these points, namely,

Students should be involved in the evaluation of teachers

Student feedback ensures overall faculty development in the institute

Student feedback can be a tool for quality assurance in medical education.

The faculty members did not agree with the fact that this kind of feedback system makes them uncomfortable (median [IQR] = 2 [1])

The faculty members also did “Agree” on the following points:

Teachers' evaluation helped them to overcome their weaknesses (median [IQR] = 4 [2])

The evaluation system made them more aware of students' needs (median [IQR] = 4.5 [1]).

Teachers, however, also agreed on the fact that students are casual about filling up the questionnaires (median [IQR] = 4 [2])

An interesting finding was that the teachers felt that a strict teacher may get poor response as compared to a lenient teacher (median [IQR] = 4 [2]).

Qualitative analysis of teachers' perceptions

The feedback questionnaire had scope for suggestions on the following two points:

To suggest for any other method of feedback which they feel would be more appropriate

What was not correct in the present system of evaluation by the students?

Regarding point number 1,

70% of the teachers felt that this method of feedback was good enough

25% of the teachers felt that peer evaluation may be a better method

Only 5% felt that evaluation by senior teachers or administrators would be better.

Regarding point number 2,

Most of the teachers (70%) were of the opinion that the students were very casual about filling up the questionnaire and many of the feedback forms were not completely filled up

40% of the teachers felt that the process of mid-semester feedback evaluation was not correct and it should have been done at the end of the course for better evaluation of the teachers.

Discussion

Evaluation of teaching processes concentrates on many aspects of teaching, covering areas such as planning and preparation of the class, knowledge of the subject, the classroom environment, and instruction of teaching. These constitute the explicit curriculum. On the other hand, aspects such as importance to core values, sensibility of the teachers to students, and behavior toward students constitute the implicit curriculum.

The primary purpose of SET is to help the faculty to identify the strengths and weaknesses of their teaching and evaluation methods.[91011] However, one of the key issues in the SET is the question of how competent students are to make judgments on teaching and course quality. It should be remembered that students may not be competent enough to evaluate teaching roles such as those involving course design (objectives, content, methods, and assessment), or grading practice in assessment. However, in terms of the quality of the delivery of lecture or instruction, behavior of the teachers toward students, sensibility toward students, etc., it is generally agreed that only students are in a position to provide a good feedback.[12] Therefore, it is important that we take into account both explicit and implicit curricula when analyzing SET.

Our study shows that although 80% of teachers had a median score of >4 in the explicit curriculum, only 20% could score >4 in the implicit curriculum. This indicates that, according to the students, although the faculty members had knowledge about their core subject and taught their subject well, they somehow lagged behind in their moral and core values toward students.

Teachers' perceptions toward SET showed that the teachers strongly agreed (median score 5) on the facts that students should be involved in the evaluation process and that these kinds of evaluation system will ensure overall faculty development and quality assurance in medical education.

In general, SET has been found to be reliable. Research has shown that certain teacher variables (such as gender, age, teaching experience, personality, and research productivity), student variables (including gender, age, level, grade average, and personality), course variables (class size and time of day of class), and administrative variables (time of module during the term) generally do not impact upon the evaluations given by students on teaching quality.[13] In our study, most of the teachers mentioned strictness of the teacher as an important variable affecting the students' response.

The teachers agreed that students' feedback made them more aware of the students' needs (median score 4.5) and helped them to improve their weaknesses (median score 4) and did not agree on the fact that feedback makes them uncomfortable (median score 2). This is in contrast to the study by Hussain and Khan, which found faculty members to be uncomfortable with such a system of evaluation.[1]

On being asked to suggest alternative methods of evaluation, some of the teachers (30%) had suggested methods, namely, peer evaluation and administrator observation as better methods. However, peer evaluation, self-evaluation, and administrator observation have questionable reliability due to a small number of raters. This is probably one of the important reasons which has led to the institutions using SET.[6]

Seventy percent of the teachers felt that the students were very casual about filling up the questionnaires, and many of the forms were incomplete. This observation has led us to recommend that students should be verbally made aware of the importance of such feedback systems and that they would be the ultimate beneficiaries from such an evaluation system. Such kind of awareness workshops in future would probably take care of this issue, and students would probably act more responsibly.

Forty percent of the faculty members felt that this evaluation should have been done at the end of the course. It may seem logical to evaluate the teachers after students have had the opportunity to experience the complete course, but if one considers ethical issues, it could be argued that those students would not be in a position to benefit from any changes made in response to the feedback.[14] Therefore, a mid-semester feedback, similar to the one in the index study, seems to be more reasonable.

Conclusion

We conclude our study with the following observations:

Students' feedback is one of the best methods of evaluation of teachers to ensure overall faculty development and quality assurance in medical education. Thus, a regular feedback mechanism should be in place for the entire institution

However, students should be made aware of the importance of such a feedback system and asked to share their feedback in a more serious and responsible manner

Teachers should focus not only on the explicit curriculum but also on the implicit domains of the curriculum in order to implement effective teaching.

However, the limitation of this study is that it was done only on a single batch of students and limited to faculty members of one department only. This kind of feedback evaluation needs to be implemented to the institute as a whole as it will help the faculty members to further improve their skills and academic performance, ultimately leading to overall faculty development. This will, in turn, ensure quality in the field of medical education.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.

Explicit curriculum How well does the teacher teach the core subject?							If you circled #1 Write why
1	Teacher is prepared for class	1	2	3	4	5
2	Teacher knows his/her subject	1	2	3	4	5
3	Teacher is organized and neat	1	2	3	4	5
4	Teacher plans class time and assignments that help students to solve problem and think critically. Teacher provides activities that make subject matter meaningful	1	2	3	4	5
5	Teacher is flexible in accommodation for individual student needs	1	2	3	4	5
6	Teacher is clear in giving directions and on explaining what is expected on assignments and tests	1	2	3	4	5
7	Teacher manages the time well	1	2	3	4	5
8	Teacher gives me good feedback on homework and projects so that I can improve	1	2	3	4	5
9	Teacher is creative in developing activities and lessons	1	2	3	4	5
10	Teacher encourages students to speak up and be active in the class	1	2	3	4	5
Implicit curriculum How well does the teacher model the core values through how he/she behaves with students and with other staff persons?							If you circled #1 Write why
11	Teacher respects the opinions and decisions of students	1	2	3	4	5
12	Teacher is sensitive to the needs of students	1	2	3	4	5
13	Teacher is fun to be with	1	2	3	4	5
14	Teacher helps you when you ask for help	1	2	3	4	5
15	Teacher is fair and firm in discipline without being too strict	1	2	3	4	5

Annexure 2

Teachers’ feedback on the evaluation system

Serial number	Statements	Response of faculty
		Strongly agree	Agree	Neutral	Disagree	Strongly disagree
1	Students should be involved in the evaluation of teachers
2	I am satisfied with the present format of teaching evaluation system
3	Feedback from students should be taken after every semester
4	Students are casual about filling up the questionnaire
5	Designation of the teacher influences the students’ response
6	Gender of the teacher influences the students’ response
7	A strict teacher may get poor response as compared to a lenient teacher
8	Students’ feedback as a teacher evaluation system makes you uncomfortable
9	Teachers’ evaluation system helped you to overcome your weaknesses
10	The evaluation system made you more aware of the students’ needs
11	Student feedback will ensure the overall development of faculty members
12	Student feedback can be an effective tool in quality assurance in medical education