Students' perceptions on feedback module in pharmacology.

CONTEXT: Feedback is an integral part of formative assessment though underutilized in medical education. The objective of this study was to review our feedback module through students' perceptions.
METHODOLOGY: We have developed a feedback module which is practiced by us for last 10 years for term ending examination that gives collective feedback to the whole class, followed by individual student-teacher interactions. Students were also exposed to 6-7 multiple choice questions (MCQs) based assessment during the course of pharmacology. Immediately after each MCQ test the answer keys is displayed along with an explanation. Two classes of students were requested to give their perceptions about the feedback by responding on Likert scale for the statements in the questionnaire. All the 206 students who volunteered for the study were enrolled in the study. Mann-Whitney test was used to calculate the difference in perceptions.
RESULTS: Of 278 students of two classes, 206 responded (74%). Students' agreement varied from 93% to 98% for 5 items in the questionnaire for the feedback after term ending examinations. Perception of students attending one or more than one feedback session did not differ significantly. For MCQs, tests agreement was 91% to 98% for the 4 items. There was no significant difference between two classes in their perceptions regarding feedback practices (P < 0.05).
CONCLUSION: Students gave a favorable opinion for our feedback module. In the medical colleges with a large number of students, this module is feasible for feedback in formative assessment in the form of written tests.

INTRODUCTION

Assessment is probably the most important thing we can do to help our students learn.[1] Assessment is an integral part of medical education. Assessment can be formative (guiding future learning, providing reassurance, promoting reflection, and shaping values) or summative (making an overall judgment about competence, fitness to practice, or qualification for advancement to higher levels of responsibility). Formative assessments provide benchmarks to orient the learner who is approaching a relatively unstructured body of knowledge. They can reinforce students’ intrinsic motivation to learn and inspire them to set higher standards for themselves.[2]

Feedback is defined as “Specific information about the comparison between a trainee's observed performance as a standard, given with the intent to improve the trainee's performance.”[3]

In a review feedback is described as information provided by an agent (e.g., teacher, peer, book, parent, self, experience) regarding aspects of one's performance or understanding. Feedback thus is a “consequence” of performance.[4]

Clearly, feedback is a core component of formative assessment, central to learning, and at “the heart of medical education.”[5]

A study of perceptions about the utility of feedback concluded that students require feedback if timely, specific and preferably delivered through individual tutorials. The creation of a “feedback chain” or overview of their overall progress was also suggested.[6]

In a randomized controlled study, the students receiving feedback achieved higher scores in the summative examination than the control group that did not receive feedback.[7]

As per regulatory requirements laid down by Medical Council of India, the internal examination can be considered summative rather than formative as its scores contribute to summative assessment in the form of internal marks.[8] However, it should be considered formative, for its essence is to effect improvement in learning. There is no reason why internal tests cannot be used for the twin purpose of providing feedback as well as assigning grades for a final pass-fail decision.[9]

This concept of dual utilization of internal examination is implemented by our department for over a decade even though it is not a mandatory requirement in our institute or as per Graduate Medical Education Rules of Medical Council of India. This indigenous module of assessment, followed by feedback has been utilized consistently.

In spite of feedback being so important in medical education, teachers shun giving feedback. Despite its central impact on learning, feedback is still relatively under-explored tool even in clinical teaching and assessment. While feedback is considered very important in clinical teaching, its importance in preclinical/para-clinical teaching and assessment, wherein there is a predominance of cognitive domain (knowledge) is not a routine practice in most medical institutes of India. In the latest document of Medical Council of India on Graduate Medical Education, there is no mention of feedback in the section referring to assessment. It mentions “Internal assessment shall relate to different ways in which students’ participation in learning the process during semesters is evaluated.”[8] There is an obvious lack of awareness among medical educators regarding the importance and techniques of giving feedback to the students. However, suitable methods of feedback need to be designed and evaluated for their effectiveness. Only a few studies report the impact of feedback practices on students learning or their perceptions regarding existing feedback practices related to formative assessment.[6710]

We at Department of Pharmacology, NHL Medical College, Ahmedabad, India under Gujarat University have been giving feedback after every formative assessment conducted as term ending written and practical tests or periodic multiple choice question (MCQ) tests for over 10 years. Hence, a questionnaire-based study was planned to evaluate the perception of the students toward the utility of the feedback module introduced into the subject of pharmacology and to review the students’ perception regarding utility of MCQ test feedback module.

METHODOLOGY

The study was carried out at Department of Pharmacology, NHL Medical College, Ahmedabad, India in November 2013, after obtaining the Institutional Ethics Committee approval. All the students who volunteered to take part in the study were enrolled in the study and students either not willing to participate or has not attended any feedback session were excluded (206 out of 278). The second year Bachelor of Medicine and Bachelor of Surgery (MBBS) curriculum is divided into three semesters (third, fourth, and fifth semester) as suggested by Medical Council of India.[8]

In our institute, formative assessment in Pharmacology Department in second MBBS takes place in two ways. At the end of each semester, internal assessment is conducted in the form of written and practical examination. Students are assessed for the course covered during the term. This is as per the requirement of Medical Council of India. As a part of the day to day assessment, MCQs test is conducted after completion of each major topic/system as lectures.

The term ending examination consists of one paper for the first and second term and two papers for the third (preliminary) examination. Each paper has four questions, the last containing 8–10 Short answer questions. The feedback session following term ending assessment is organized within 2–3 days of the declaration of the result of the examination. In the feedback session, model answers for each question (key points) are prepared by the examiner who has assessed that question. These answers are read aloud by the respective teachers to the whole class of students one by one. Any glaring misconceptions/mistakes are also pointed out and clarified. This is the presentation session of the feedback module. After the presentation session, corrected answer sheets are distributed to students so that students can review the comments written by the examiners. Students are also given chance to clear any doubts regarding assessment by discussing with the respective teacher who has assessed that question. During the tenure of second MBBS, students are exposed to a total of three feedback sessions for term ending assessment.

As far as feedback for MCQ test is concerned, the keys with explanation are displayed immediately once the test is over. During the tenure of second MBBS, if students have taken all the MCQ tests, they would be subjected to a total of 6–7 such sessions.

To find out the perception of students regarding our feedback module, a questionnaire was prepared with 5 items wherein each statement addressed an important goal of feedback. The students were asked to respond on a 4 point Likert scale from strongly agree to strongly disagree. The questionnaire was self-designed, with questions pertaining to important goals of feedback. The statements were pertaining to objectivity of information provided about students’ performance during the feedback session, ability to point out strong and weak areas of their performance, helping to improve their knowledge in the subject and feedback session acting as a motivation to work harder. The last statement asked their opinion regarding the continuation of feedback practices by the department.

For perceptions of students about formative assessment through MCQs there were 4 items which included role of MCQ test as a good motivator for learning in depth; display of key (answers) to MCQ test immediately after test helps in identification of their strong and weak areas in knowledge and MCQs discussed in the following class would be better than displaying the answers. The last statement asked their opinion regarding the continuation of such practices by the department. The students were asked to respond on a 4 point Likert scale from strongly agree to strongly disagree. At the end of the questionnaire, students were asked to give their comments and suggestions for improvement if any. Figure 1 depicts the feedback module practiced by us.

Figure 1

Feedback module scheme

The questionnaire was pre-validated by administering to ten students. The validity was assessed, and the internal consistency of the items in the questionnaire was assessed using Cronbach's alpha, which was 0.759.

As it has been an ongoing practice, it was decided to get a perception of two classes of MBBS. We selected the current batch of second MBBS students in their final term (5th semester of MBBS course) and the batch which passed the previous year (7th semester of MBBS course). Before giving the questionnaire, the students were explained about the study and were requested to fill up the questionnaire giving them the option of answering anonymously if they wished to do so.

Statistical analyses were done using SPSS V 21.0®[IBM, Corp.: Armonk, NY]. Mann–Whitney test was used to calculate the difference in perceptions between the two classes and also between male and female responders. P < 0.05 was considered statistically significant.

RESULTS

Out of 278 students of 5th and 7th semester, 206 responded making the response rate of 74%. From 5th semester, 97 out of 144 students (67.4%) responded while 109 out of 134 students (81.3%) responded from 7th semester as shown in Table 1. Out of 206 respondents, 184 (89%) mentioned gender with 94 male and 90 female students. Around 75% students mentioned the number of feedback sessions they had attended.

Table 1

Students’ perceptions about feedback after term ending examination-distribution of responses in percentage (n=206)

Feedback session after term ending tests

About 97% of the students perceived that the feedback session provided objective information about their performance and about 93% students agreed that it helped to identify strong and weak areas. About 95% felt that it helped as a guide to improving their knowledge in the subject while 96% students agreed that it motivated them to work harder. About 98% agreed for the continuation of this feedback practice [Table 2]. There was no statistically significant difference between the two classes for rating of the statements about the feedback sessions [Table 3]. There was no significant difference between students who had attended one session or more than one session. There was statistically significant difference between male and female students in their perception regarding the term ending feedback session helped in improving knowledge regarding the subject (P = 0.037).

Table 2

Semester wise distribution of responses in percentage (n=206)

Table 3

Students’ perceptions about feedback after MCQ test-distribution of responses in percentage (n=206)

Postmultiple choice question test feedback

About 98% of the students agreed that the MCQ test is a motivator for learning in depth and about 97% felt that the display of keys immediately after MCQ test helps to identify their strong and weak areas of knowledge. About 91% favored, in-depth discussion of MCQ in the following class, over displaying the answers, and 95% agreed that this practice should be continued for future students [Table 4]. There was no statistically significant difference between 5th and 7th semester in responses to the statements. In the MCQs test feedback, perception regarding questions discussed in the following class being better than displaying the answers, the difference was statistically significant (P = 0.018, U score = 3460, Z = −2.376). Male respondents showed greater agreement as compared to female counterparts for 3rd question of the term ending examination (P = 0.037) and MCQ test (P = 0.018).

Table 4

Semester wise responses regarding post-MCQ test feedback in percentage (n=206)

DISCUSSION

The study aimed at assessing the students’ perceptions regarding the feedback module which has been practiced by us for over 10 years along with formative assessment in the form of written tests and MCQ based tests.

This study is first of its kind where the utility of the feedback module designed by the authors was subjected to the students’ perception. The subject of pharmacology in its heart belongs to the cognitive domain, hence it becomes difficult to give feedback to the students as compared to clinical subjects were there is dominance of psychomotor domain and feedback can be imparted with respect to the skills achieved by the learner. Hence, this study becomes a landmark study in Medical Education.

Ideal feedback is provided one to one from the mentor to the trainee, but our format is different. Considering the feasibility in terms of time consumed in individual feedback for a class of about 150 students, this type of group feedback system was designed. It started with providing examined answer books to students for checking marks and going through written comments. One-way of increasing the effectiveness of external feedback, and the likelihood that the information provided is understood by students is to conceptualize feedback more as dialogue rather than as information transmission. Feedback as dialog means that the student not only receives initial feedback information but also has the opportunity to engage the teacher in a discussion about that feedback.[11]

Hence, we improved our module further by discussing model answers and giving a chance to students to interact with examiners to clarify their doubts and giving guidance for further improvement.

For the different statements pertaining to important aspects of feedback, the respondents’ agreement varied between 93% for providing information about their strong and weak areas to about 99% for their opinion regarding continuing this feedback practice in future. We included two classes in the study-one which was currently going through pharmacology training and the other that had recently passed out so as to find out if there were any differences in perceptions of those who had attended all sessions and were in a better position to opine compared to current class which had not been exposed to all the sessions. The past and current class did not differ in their perceptions in this regard. While most comments were positive in nature, there were suggestions like compulsory attendance of students in these sessions and individual approach for interested students. It was heartening to get a favorable opinion from the students. The most important factor contributing to high rating seems to be the perceived transparency in the assessment process due to clear, unambiguous, instructional and directive feedback which is generally welcomed by students; they know how to interpret it and apply it.[12]

There was no difference between the two classes as far as the perception was concerned. Also there was no significant difference between students who had attended one session or more than one session. Even those who attended a single session gave a positive feedback. Hence, even a single feedback experience had a positive impact on students.

As the post-MCQs feedback highlights the logic behind the answers to multiple choice tests it has a positive impact on students’ attitude to learning. Moreover, the reason for the higher percentage of strong agreement for MCQ test as an important tool of assessment as students felt that MCQ based assessment could be helpful for selection for higher courses in future. Perceived relevance or applicability of the feedback is particularly important for students.[12]

It has been shown by various studies that feedback should be timely as students dislike the long interval between assessment and feedback.[11] In our feedback practice, we ensured that the feedback is given within a week after a term ending exam and immediately after a MCQ test.

Our feedback module may not be considered ideal as it is not given to individual students. Hence in a pilot study, we also tried giving individual feedback but time constraint was felt both on the part of teachers and students and hence it could not be implemented. Our simple module of feedback given to a group as a whole followed by individual student-teacher interaction is objective, unbiased, helped as a guide to improve knowledge and future performance and acted as a motivator for better learning. It can be improved further by training all the teachers involved in assessment and adding specific feedback information regarding practical assessment.

To be effective, feedback needs to be clear, purposeful, meaningful, and compatible with students’ prior knowledge and to provide logical connections. It also needs to prompt active information processing on the part of learners, have low task complexity, relate to specific and clear goals, and provide little threat to the person at the self-level. The major discriminator is whether it is clearly directed to the task, processes, and/or regulation and not to the self-level.[4] In order to give effective feedback, different forms should be used and tailored to the needs and acceptability of the learners. This should be done repeatedly and incorporated within the curriculum.[13]

Highlighting the principle strength of this study, this module which has been in use over a decade, is convenient for faculty to deliver and is acceptable to the students. We feel that in the Indian scenario of medical education with large numbers of students in a class, this module of feedback is feasible in formative assessment as written tests.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.