Comparison between student rating, faculty self-rating and evaluation of faculty members by heads of respective academic departments in the school of medicine in Birjand University of Medical Sciences in Iran.

INTRODUCTION: University teachers are one of the main pillars of university and the quality of their performance must continuously and systematically be evaluated. This evaluation can be carried out in various ways. The aim of the present study was to survey and to compare the evaluation of faculty members in the medical school in Birjand University of Medical Sciences by three different sources: Student rating, self-assessment, and evaluation by head of related department.
MATERIALS AND METHODS: This descriptive analytical cross-sectional study was conducted in the academic year 2009-2010. Sampling was drawn from all students studying basic science and clinical training in the first and the second semesters. All heads of departments in basic science and clinical training and their faculty members took part in this study. Means of data collection were four different questionnaires designed in the education development center (EDC) and their validity and reliability had been verified by the center. These questionnaires were based on student rating, self-assessment, and evaluation of faculty members by heads of clinical and basic sciences academic departments. After the questionnaires were filled out, the obtained data was analyzed by Statistical Package for the Social Sciences (SPSS) software (version 13), independent t-test, and Pearson's correlation coefficient at the significant level of α = 0.05.
RESULTS: In the present study, 2417 students completed the questionnaires regarding 63 faculty members, 87 faculty members completed the self-assessment form, and for 60 faculty members, 48 members in clinical and 12 members in basic science, the questionnaires were completed by heads of respective departments. Mean and standard deviation of student evaluation, self-assessment, and teachers evaluation by heads of departments were 3.23 ± 0.38, 3.51 ± 0.33, and 3.60 ± 0.32, respectively, and the difference between student rating and self-assessment was significant (P = 0.02). In comparing between managers scores with students evaluation, no significant difference was observed (P = 0.68). Comparison between self-assessment and teachers scores by managers showed a significant difference (P = 0.04). Mean scores of faculty members in clinical training and in basic science were 3.23 ± 0.73 and 3.31 ± 0.69 on the part of students, respectively; thus, the difference was significant (P = 0.004).
CONCLUSION: Since, the present study was inconsistent with similar previously carried out investigations, the observed difference among the three procedures was statically significant; hence, it can be suggested that student's scores of teachers evaluation, previously used as the only one of evaluation source is not enough and other sources such as assessment by the respective department heads, dean of faculty, and self-assessment must also be taken into consideration. This collection can definitely yield a more favorable evaluation of faculty members and feedback can be more acceptable to them and it will be more effective in improving their education.

INTRODUCTION

The education system in every country, as a dynamic and targeted institution has qualitative and quantitative dimensions and a balanced growth of these dimensions is one of the main objectives of this system that must always be taken into consideration by the planners.[1] Continuous improvement of the quality of education requires continuous evaluation of its functions. This evaluation can be carried out on the basis of each of its elements such as input, output, processes, and outcome.[2] Investigations have shown that efforts to reform the curriculum and teaching-learning methods without a synchronous effort in the evaluation system will be useless because the evaluation component is more effective in the qualitative and quantitative improvement of educational programs than instructional designs and teaching methods.[34]

Evaluation is one of the most effective things for quality assurance and continuous quality improvement process in the educational system.[56] And, its basic aim is to improve effective activities and to weaken or eliminate non-effective and undesirable ones.[6] The obtained information from evaluation can be used to improve and develop training programs, teaching methods, planning, etc.[7] Chickering believes that evaluation is the basis of perfection and development and regards it in many different aspects, such as personality development, morality, intellectuality, motivation, and knowledge.[8]

University teachers are one of the main pillars of university and their function plays a fundamental role in the overall efficiency of university educational system. Therefore, it is necessary that the quality of their performance be continuously and systematically examined and the results presented to them, punctually. Numerous studies have shown that the feedback of these results will enable teachers to identify their strengths and weaknesses in order to improve the former and overcome the latter. This process will improve the quality of education.[910]

The aim of teachers’ evaluation is to determine their success rate in achieving decisive educational goals. To do this, an appropriate means (valid and reliable) in the field of educational work must be provided for data collection at first. Then the gathered data must be compared to standard criteria. Finally, judging about the success rate of teachers in achieving the goals must be done.[11] Performance evaluation of University teachers is carried out in various ways; such as evaluation by students, self-evaluation, evaluation by the respective department heads, colleagues, dean of faculty, etc., However, one of the most common done in most of countries, including Iran, is the evaluation by students.[121314]

To do so, in each semester this evaluation is carried out and the results - as feedback - are presented to teachers and are sometimes used for educational training. There are different and sometimes conflicting views with this type of evaluation. However, despite all the difference, this process is used extensively in a growing number of Universities around the world, including Iran.[15]

In extensive investigations in some great university centers, 100% of students participated in this assessment. For example, in Indiana University a total of 133,000 students were questioned in the academic year 1993-1994.[7]

Another teacher evaluation method is self-assessment which is a powerful technique to improve the capacity of teachers and is relevant to the belief in learners’ ability to recognize their quality of training.[16] This process can also be used as a measure to be compared with students evaluation scores.[17] However, regarding this process a complete and serious attitude has not been adopted in an academic community yet.[18]

Regarding facts, including high-speed development of science, the increasing volume of human knowledge, changes in teaching and learning methods, and lifelong self-directed learning, self-assessment is necessary and unavoidable in an academic community, especially for teachers, as future architects of every country. Obviously, the basic requirement of this capability is self-assessment. In this way, teachers can recognize their shortcomings, attitude, and performance; therefore, they can develop their qualifications.[45]

The third reference that can be used in teachers’ evaluation is the query from the head of their respective department. Because they have the most accurate information about the teaching of subjects; such as simplicity and complication, allotting necessary time to teaching, using the course syllabus, and being skillful in the subject taught, etc.[19] However, this source of evaluation has not been applied completely.

Therefore, the present study was designed to survey and compare faculty members evaluation of the medical school in Birjand University of Medical Sciences by three different sources; namely, student rating, self-assessment, and evaluation by the respective department heads.

MATERIALS AND METHODS

This descriptive analytical cross-sectional study was conducted at the medical faculty in Birjand University of Medical Sciences. Sampling was carried out as census on all students who were studying basic science and clinical training in the first and the second semester in the academic year 2009-2010. Sampling from teachers’ evaluation by heads of departments in basic science and clinical training was also carried out as census. In self-assessment, all faculty members who were in conditions of employment were included.

Means of data collection were questionnaires designed in the education development center (EDC) of the university whose validity and reliability had been verified by the same center. These questionnaires are used for the assessment of faculty members across the country. The questionnaire assessing teachers by students (with Cronbach's alpha coefficient of 0.91) was used in a study in Kerman University of Medical Sciences[3] and in another study in the same university - with the coefficient 0.86.[19] This questionnaire has two sections.

In the first section, demographic information is recorded including the name of the faculty member, student's course of study and gender, and subject of study. The second part, consists of 17 questions about evaluating teachers in different aspects of teaching methodology (7 questions), scientific proficiency (5 questions) observing educational rules (3 questions), and finally adhering to a teacher's dignity (2 questions); each question having four options low, average, fine, and very fine given a score between 1 and 4. The total scores given to each teacher by students was divided by the number of questions, and eventually, a score between 1 and 4 was achieved. In order to fill out the questionnaire the executive of the project attended the final session of every course in the semester, and after justifying students, the questionnaires were distributed and collected after e-completion.

The self-assessment questionnaire had 24 questions and its Cronbach's alpha coefficient had been determined as 0.91 in Asghary et al. study in Hormozgan University of Medical Sciences[20] this questionnaire was given to faculty members and was collected after 2 days.

The evaluation questionnaire portraying department heads’ views had different questions regarding each curriculum 11 questions in basic science and 17 questions in clinical training.

Considering that no previous studies were done by means of this questionnaire, first its Cronbach's alpha coefficients was determined to be 0.80 and 0.76, respectively. These questionnaires were completed for all faculty members by heads of different departments of basic science and clinical training. Finally, the obtained data were analyzed by means of Statistical Package for the Social Sciences (SPSS) software (version 13), using independent t-test and Pearson's correlation coefficient at the significant level of α =0.05.

RESULTS

In this study, 63 faculty members of Medical School in Birjand University of Medical Sciences were evaluated by 2417 students. 87 faculty members completed the form of their self-assessment, and for 60 faculty members, 48 members in clinical training and 12 members in basic science, the evaluation form was completed by the heads of the respective departments.

Mean and standard deviation of evaluation scores allotted to the university teachers by all students was 3.23 ± 0.38 and the mean of faculty members’ self-assessment scores was 3.51 ± 0.33 out of 4, and the difference between them was statistically significant (P = 0.02).

Comparison between the mean of teachers’ evaluation by students, faculty members’ self-assessment, and evaluation by the heads of departments is shown in Table 1. As it is shown the differences were significant. The obtained scores of clinical faculty members on students’ part was (3.23 ± 0.73), and in the basic science training it was (3.31 ± 0.69); the comparison of the mean difference between them was statistically significant (P = 0.004). That is to say that the teachers’ scores in basic science were higher than teachers’ scores in the clinical training. Comparison of mean scores of faculty members in basic science and clinical training (out of 4) on the students’ part and the heads of respective departments is summarized in Table 2. Besides, Pearson's correlation coefficient between different variables is represented in Table 3.

Table 1

Comparison of mean and standard deviations cores in teacher evaluation based on different variables

Table 2

Comparison of mean and standard deviation of evaluation scores of basic science and clinical training teachers gained from students and heads of respective departments

Table 3

Estimates of Pearson's correlation coefficient of teacher's scores according to different variables

DISCUSSION

The purpose of this study was to survey and compare the results of teachers evaluation based on students’ perspective, self-assessment, and evaluation by the respective department heads. According to the results, comparison between mean of assessment scores by total students and mean scores of self-assessment of faculty members shows a statistically significant difference (P = 0.02). This finding is arguable for several reasons.

Firstly, there are different and sometimes conflicting perspectives on teachers’ evaluation by students. A group of researchers agree with it absolutely and account it as the most important and the most reliable source of evaluation. They believe that students are the best source of assessment and their evaluation is the best and the only tangible one, because students are the only people who are directly taught by teachers; therefore, they are the best reference to express their concerns about the education of teachers.[1115202122] And, it is necessary that their views and opinions, regardless of their level of knowledge should be considered.[1521]

Aultman believes that teacher evaluation by students can be valuable feedback to improve the quality of their education and provide a good chance to lead to the promotion of learning.[22]

In a research conducted in this regard in Jahrom Medical School, it was found that 70.9% of teachers generally agreed with teacher evaluation by students and 65.7% of them believed that awareness of the aims of evaluation increased their satisfaction from the performed evaluation.[11]

Cashin in a review article surveyed a large number of articles from 1971 to 1988 regarding teacher evaluation by students and concluded that this method is the first and the most important source of teachers’ evaluation among other sources. This investigator showed that this type of evaluation is statistically reliable and valid and is relatively free of error and; secondly, students’ educational performance has no effect on teachers’ evaluation.[23]

Another group of investigators disagree with the assessment of teachers by students and believe that students’ judgments are subjective and thus are not sufficiently valid.[2425] According to this group of researchers students lack sufficient information about teachers.[262728] They believe that this type of evaluation may become a threat to teachers and involve the risk that teachers use non-normative practices for students’ satisfaction instead of trying to improve the quality of their own education, trying to avoid seriousness and rigor in their job.[1224]

Based on a research on teachers evaluation by students at the University of Ahvaz 57.5% of teachers believed that student rating process had had an average effect on their education. They also believed that students due to lack of knowledge of the teaching process do not have a true judgment about evaluation.[24]

Greenwood is opposed to evaluation of teachers by learners and believes that perceptions and judgments of people are affected by general personality traits and by environmental characteristics; hence, these factors can affect students and they apply them in their teachers evaluation.[29]

Finally, a third group of researchers have postulated that teachers evaluation by students is important and necessary, but they believe that these judgments and decisions can only be used as one of multiple evaluation sources.[23] According to this group, since evaluation of faculty members is one of the most complex evaluations, and means employed and measuring methods are not of high credit, the obtained information cannot be accurate and without bias. Therefore, they have suggested that for the final judgment, a combination of assessment information must be used and cited.[1230] According to Jacobs, teachers evaluation by students should not be the only source of gathering information about their decision, but other methods like self-assessment must be applied.[7] Moreover, the difference between self-assessment scores compared and scores of teachers by students is statistically significant (P = 0.02). This means that the scores given by the teachers themselves were more than the ones given to them by students.

These findings are consistent with similar studies in Iran and other countries that are mentioned below:

Investigations of Ross et al. at the University of Toronto, Canada[31] and Reetz in Indiana have shown that the scores of teachers evaluation by students were lower than their own from self-evaluation.[32] Lane et al. in Pennsylvania,[33] Mattheos et al. in Sweden,[34] Cole et al. in Baltimore,[35] Houston et al. in Alabama[36] also have shown that self-assessment scores were higher than the scores given by students. These findings are consistent with ours. In a study by Shakurnia and Karami carried out about comparing scores of teachers evaluation by students with those of teachers self-evaluation at the University of Jundishapur in Ahvaz, Iran, it was found that the mean score of self-assessment on a scale of 5 was 4.52 ± 0.34 and the average score of students evaluation of teachers was 4.04 ± 0.39; the difference was statistically significant.[18]

In Goharian et al. study, in Isfahan University of Medical Sciences,[37] Asghari et al. in Hormozgan University of Medical Sciences,[20] Aghamollai et al. in the Health Faculty at Hormozgan University of Medical Sciences,[38] Jafari et al. at Mazandaran University of Medical Sciences,[6] it was found that between students rating and self-assessment of teachers, there was a significant difference, which was in agreement with the results of the present study. Of course, in the Allaei et al. study in Ilam University of Medical Sciences, it was found that in many domains, there was not a statistically significant difference between the scores of students evaluation and their teachers self-assessment that is inconsistent with the results of the present study.[17] In Barnett et al. research in the School of Pharmacology in Mercer University too, the difference between self-assessment scores and the scores allotted to teachers by the whole students was not significant, which is not consistent with the results of the present study.[39]

Comparison between mean of scores of teachers evaluation by heads of departments in basic science and clinical training curricula with mean scores of student rating, no significant difference was statistically observed (P = 0.68). However, in comparing the scores of heads of departments with those of teachers self-assessment the observed difference was significant (P = 0.04), that is the scores of department heads were higher than those obtained from their teachers’ self-assessment. Comparison of clinical directors given grades and basic science directors scores revealed no significant difference (P = 0.14); while, comparison between the mean of scores of clinical and basic science department heads, with those given by students, the difference was significant (P = 0.004). Unfortunately, to compare these findings with those of other similar studies, not many information sources were found. A study in Kerman University of Medical Sciences reviewed and compared various methods of evaluating teachers. In that study, for the evaluation of teachers by heads of departments some advantages had been listed: The appointment of every academic department head by the respective faculty members, his awareness of faculty members’ duties, his knowledge in the courses of study, and the relevant syllabuses, etc. These researchers believed that the directors of departments are aware of the teaching quality of faculty members and have a better understanding of the issues. Therefore, they can comment more accurately about the quality of education of teachers.[19] Another group of teachers in this study mentioned disadvantages for this method, e.g., they argued that there is the possibility of bias in the evaluation of department heads and they suggested monitoring of an especial committee for this purpose. They also held the view that these assessments may be used and cited as a backup on the decisions made. On the other hand, since the head of a department is not aware of the details of a teacher's teaching, his views may be influenced by students’ viewpoints.[19]

CONCLUSION

Since faculty members were assessed through various procedures, cited in this study, and a significant difference among them is observed as previously reported in other studies, it can be suggested that student scores of teachers evaluation can be used only as one of the evaluation sources and the other sources such as assessment by the head of the respective department, dean of faculty, and self-assessment must also be taken into consideration. Of course, this collection can definitely yield a more favorable evaluation result for faculty members and the feedback will be more acceptable to faculty members and thus will be more effective in improving their education.