A mixed method study to validate a two-way feedback between student and faculty to improve learning of anatomy.

Although the students are subjected to some formative exams throughout the problem based learning units, feedback is not given appropriately and timely. Students want to know and use the reasoning behind judgments and always complain that assessment criteria need to be explained. The aim of this project is to implement a two-way feedback delivery (TWFD), in which both faculty and students have an opportunity to discuss their reflections on learning and examination processes. An Anatomy formative assessment is introduced to 100 students followed by implementation of TWFD. Faculty members provided the students with a structured and timely feedback on their performance. Also, the students reflected on the whole learning process, including real examination experience. The reaction was measured using quantitative and qualitative instruments through a questionnaire, focus group discussion, and semi-structured interviews. Ninety students (90%) participated in the questionnaire with high satisfaction toward implementation of TWFD. Ninety-four percent (n=85) admitted that the time of the session was appropriate. Ninety percent (n=81) of the students demonstrated that the TWFD helped them to identify their strengths and weaknesses. Eighty-five percent (n=77) of the students admitted that TWFD promotes active reflection on the effectiveness of teaching. Most of the students and teachers' comments in the focus group discussions and the interviews supported these results. TWFD seems to be a good approach to implement an effective and timely feedback process between the faculty and the students. Students and the faculty recommended the implementation of this session in different courses and units.

Introduction

Many strategies are used in medical education to improve students' learning and hence the achievement of learning outcomes with the aim to produce a health workforce that is capable to promote the society health status. The process of formative assessment is among those strategies and specially if it is associated with feedback to students [123]. Effective feedback provides comprehensive data on student performance and could be linked to student self-esteem and motivation [45].

Traditionally, feedback was seen as a “as a gift” from teachers; with students play a passive role [6]. This is no longer the case; recent application of feedback requires interaction between faculty and learners to identify the weaknesses and achieve improvement, modifications, revisions or repetitions as needed, beside the great difference to the quality of student's learning as a result of the active participation in the process [78], in this instance a two-way feedback namely teacher feedback and learner feedback searching for areas of improvement will happens [910].

Anatomy is one of the oldest basic medical sciences, it is considered core to medicine as well as some associated and complementary health disciplines [11]. Anatomy teachers continually look for implementing effective teaching and learning techniques that give the students a more interesting and advantageous experience in the course. The aim of this research is to assess the satisfaction of students and faculty with a process of a two-way feedback delivery (TWFD) between faculty members and students after formative assessment to enhance students' learning of anatomy.

Materials and Methods

The research is a cross-sectional study using both quantitative and qualitative approaches during teaching anatomy for the second-year medical students in first semester. It was done in the College of Medicine, University of Sharjah where an integrated, semester-system based curriculum takes place. Each semester consists of 15 weeks teaching before final summative exam in the 16th week. One hundred out of one hundred and sixteen students were involved in the research with a participation rate of 86%.

A TWFD session was carried out during teaching anatomy and immediately after a formative assessment of the unit in the tenth week. The faculty members divided the session into two main components: The first part was to provide the students with a structured feedback about the details of their performance in the formative exam. The ideal answers have been demonstrated with full explanation of the difficult questions that showed high difficulty index. In addition, a power point showing the practical specimens that were used in the formative objective structured practical examinations together with the right answers have been presented, which stimulated the discussion with the students.

In the second part of the TWFD session, the students were allowed to reflect on the formative exam in particular and on the whole learning process in general. An open discussion with the whole participating students was permitted in the presence of the anatomy faculty. The students provided the faculty with their concerns about the teaching process as well as the examination procedure.

The extent of satisfaction of the students and faculty is identified and analysed by gathering data through an anonymous questionnaire (Fig. 1) and focus group discussions. Interviews with the anatomy faculty members also were performed to explore the impact of that feedback process on the staff.

Fig. 1

Questionnaire used to measure the extent of satisfaction of the students and faculty after the two-way feedback delivery session.

The questionnaire was developed from the literature available and modified to match the local situation after being piloted on a group of students and the faculty [12]. The items in the questionnaire form obeyed Likert-type rating scales in which the respondent is asked to show the level of agreement or disagreement according to five-point Likert scale, where (1, strongly disagree; 5, strongly agree).

The questionnaire has been designed to contain items to analyze in three main themes. The first thought was to explore the TWFD organization and timing. The second issue was dealing with the formative assessment and its usage as a tool to enhance the feedback delivery and whether these low-stake types of exams could be useful tools in motivating the students' learning. The third consideration was to measure to how extent the feedback process was conducted efficiently and how much the feedback provided by the faculty was constructive.

For the sake of any unanticipated findings and giving the students the space for description and clarification of their thoughts, some open-ended questions have been added at the end of the questionnaire. These inquiries included what did you like most in the session, and what is your suggestions for improvement in teaching, formative exam and/or TWFD.

The quantitative data was analyzed using SPSS ver. 21 (IBM Corp., Armonk, NY, USA); the data summarized with percentages, mean and standard deviation for responses of each item. The responses for agree and strongly agree were merged in “Agree” and for disagree and strongly disagree were merged in “Disagree”.

Qualitative evaluation has been conducted through focus group discussion with the students as well as semi-structured interviews with the anatomy faculty who were involved in the study. The focus group discussion has been carried out to capture the views of the students who had participated in the formative and the TWFD.

The researchers were responsible for facilitating the discussion of the focus group.

Ten students have been invited by e-mail to attend a 50 minutes' focus group discussion. The students participated in the focus group were coded from one to eight starting by S1 on the right of the facilitator and ending by S8 on the left and their consents for audio recording were taken. The students' general reaction towards the TWFD and the formative assessment has been investigated. Also, the students thought regarding the different components of the TWFD and the impact of feedback given by the faculty on their learning also was explored.

In addition to that, two individual semi-structured interviews with two-anatomy faculty who had participated actively in the project were conducted. The faculty were coded as F1 and F2. The purpose of these interviews was to explore their beliefs on the implemented TWFD and to approach any difficulties that may prevent future implementation.

All the audio-recorded interviews were transcribed and analysed verbatim manually through an iterative process of thematic content analysis to identify emerging themes. The study was approved by the research ethics committee at the college of medicine, University of Sharjah (ERC/27/10/15/43).

Results

Results of data collection

From the 100 students participated in the study, ninety students have responded to the questionnaire with response rate of 90%. The responses showed a great agreement with almost all the items in the questionnaire. Ninety-two percent (n=83) of the students agreed that the TWFD was enjoyable and 94% (n=85) admitted that the time of the session was appropriate. All responses were presented in Table 1.

Table 1

Students' response to the questionnaire

Questionnaire	No. (%)			Mean±SD
	Agree	Neutral	Disagree
I enjoyed the session.	83 (92)	6 (7)	1 (1)	4.467±0.674
The Time of the session was appropriate.	85 (94)	4 (5)	1 (1)	4.5±0.64
The session has helped me to identify my strengths and weaknesses.	81 (90)	5 (6)	4 (4)	4.289±0.811
The faculty has provided me with specific advice on how to improve my performance.	79 (88)	8 (9)	3 (3)	4.389±0.789
The session promotes active reflection on the effectiveness of teaching.	77 (85)	9 (10)	4 (5)	4.267±0.815
The session encourages feedback that enhances learning.	78 (87)	9 (10)	2 (3)	4.267±0.818
Exam without marks prevents motivation for students cheating.	78 (87)	9 (10)	3 (3)	4.489±0.851
It is difficult to motivate students' performance on exams without marks.	36 (40)	20 (22)	34 (38)	3.267±1.314

Some of the students' comments from the focus group discussions also supported these results.

S1: “the session was very well-organized and the timing was ideal as it was ok.”

Small number of students viewed different opinions about the enjoyment and the timing of the session. They mentioned that the formative exam stressed them and the session would be much better if it was at the end of the semester to include the whole objectives of the final exam.

S3: “I was so stressed to get the maximum benefits of the formative assessment.”

S4: “The timing would be much more beneficial if it was done at the end of the semester to include all the final exam contents.”

S5: “I think this is the first time we feel that we will get benefits from our own feedback because of the timing of the session. The staff might listen to our feedback and do some modifications in the final exam.”

Ninety percent (n=81) of the students demonstrated that the TWFD helped them to identify their strengths and weaknesses. In addition, some of the students' comments from the focus group discussions also supported these results.

S2: “I discovered that I am so weak in identifying some important structures in the OSPE and the embryology needs more attention.”

Eighty-three percent (n=79) of the students agreed that the faculty have provided them with specific advices on how to improve their performance. Students' comments from the focus group discussions also supported these results.

S3: “I discovered that I am so week in identifying some important structures in the OSPE and the embryology needs more attention.”

Eighty-five percent (n=77) of the students admitted that TWFD promotes active reflection on the effectiveness of teaching.

However, one of the students' comments from the focus group discussions indicated that it is sometimes difficult to mention negative feedback directly to the concerned faculty and it is much better to be through a written survey.

S4: Students' comments on questions was helpful.

S6: “I couldn't say any negative feedback in front of the faculty. I prefer writing my comments in an anonymous survey.”

Eighty-seven percent (n=78) of the students agreed that exam without mark prevents motivation for students cheating.

The students and faculty members' comments from the focus group discussions were supportive.

S2: “When the exams have no marks, it gives us the opportunity to think freely without being afraid of the mark and results.”

F2: “One of the great advantages of the formative exam is that its low stake nature which prevents motivation of the students cheating.”

Only 40% (n=36) of the students agreed that it is difficult to motivate students' performance on exams without marks.

Students viewed different opinions in the focus group discussion. They mentioned that getting marks is not as important as living the real exam setting and reflect on it.

S7: “I know many students came to formative without studying… there is no marks.”

S8: “Positive feedback given during learning is really encouraging; I think it deserves working hard.”

F1: “Intensive dedication is required from both learners and faculty to maintain the process.”

F2: “I think the feedback that the students received in the TWFD is helpful regardless the students' preparation level for the formative. When he listens to the general feedback provided by the faculty as well as the opportunity given to reflect on the questions and the exam setting.”

Here are some of the faculty comments:

F1: “TWFD had a significant impact on students' learning because it came following a live image of assessment. The students were learning and assessed at the same moment. The immediate feedback they got would help them to correct their mistakes before the final assessment.”

F1: “Giving an effective feedback needs special skills. I don't think that the entire faculty are skilful for that. It would be a great idea if we had a training in how to provide an effective feedback to the students.”

F2: “We have to consider the great effort needed to implement the TWFD after formative assessment. For a successful formative, we need to prepare high standard questions with exactly the same level of difficulty as the final exam questions.”

F2: “We will look at the students' feedback and do the necessary modifications before the final summative exam.”

Discussion

This research showed that students have perceived the TWFD session as effective teaching and learning process. The questionnaire analysis and the comments from the focused group have revealed that TWFD has helped the students to recognize some of their shortfalls, and the best way to deal with these defects.

These findings are in alignment with the literature; researchers stated that formative assessment followed by feedback help students to narrow their gaps, to self-regulate their learning, and to improve their self-efficacy [1314].

The outcomes of the TWFD were also analogous to other similar studies in giving effective feedback. Ramani and Krackov stated that formative assessment and feedback are crucial to the educational means and supporting students to approach their greatest potential. The procedures and circumstances for valuable feedback delivery are well reported and include a particular learning environment; a “two-way conversation”; and acknowledgement and reinforcement of good practice [15].

In this study, researchers assumed that introducing the TWFD in week ten of the semester would give the students the advantage of having five more weeks before the final summative exam, so that they could benefit from the feedback provided to them by the faculty. When both the students and the faculty had sufficient time after the session and before the final summative exam, they would be able to apply any needed changes or any valuable outcomes from the feedback gained during the session. Researchers agreed that the impact of high-quality assessments is disabled unless feedback is pointed and timely so that the knowledge obtained is helpful to the individual's needs [161718].

O'Farrell [19] concluded that good quality, complete and timely feedback is a very influential factor in driving student learning. Assessment should afford feedback to students on their advance towards the accomplishment of learning outcomes. Feedback will allow students to understand where they have done properly and shown what they could develop on, as well as explain the grade/mark of summative evaluations [19].

The faculty and the students agreed that the low stake nature of the formative experience prevents motivation of the students cheating which promotes the process of learning. It can be considered as an approach to prevent cheating by offering the students the exam experience without marking pressure. Many authors suggest more frequent, low-stake assignments to reduce the pressure on students [2021].

Although the faculty members want honest feedback on their performance as teachers and evaluators, it is believed that this feedback has to be given anonymously for students to feel safe [22]. Some students indicated that it was difficult to disclose negative feedbacks in front of their faculty and prefer writing their comments in an anonymous survey. This study challenges the belief that student to faculty feedback needs to be anonymous and suggests that open two-way discussion between the student and the faculty would provide real benefits to both of them. Dudek et al. [23] stated that many of the elements of effective feedback require or are promoted by a non-anonymous or “open feedback” process as protecting anonymity comes at the cost of timely feedback.

One of the most significant challenges that opposed the project is the construction of the formative exam in regarding time for preparation, resources, high standard questions needed and the efforts required of the participating faculty and the organizing staff. Convincing the staff to prepare high-quality questions and make this extra effort is a challenging. Therefore, it would be beneficial to develop methods that would overcome these hurdles. Browne et al. [24] have utilized senior students to facilitate the conduction of the formative exam, which may reduce the efforts needed by the faculty as well as its benefits for the students learning.

The interviewed faculty agreed that the immediate feedback is the excellent approach that would help students to identify their needs before the final summative assessment. They also expressed their willing to learn more about providing efficient feedback to the students and agreed to make any modifications indicated during the TWFD.

Many authors acknowledged that training programs to the faculty are beneficial, and most the teachers are willing to listen to assessment ideas and to learn [2526]. Many authors stated that most of the faculty are willing to change their teaching methods and their prospects about how students perform when a variety of assessment indicators show a problem [2728].

Another significant challenge was how the students will deal with the formative exam and will they take it seriously or not? In this study, only 40% of the students agreed that it is difficult to motivate students' performance on exams without marks. Several techniques have been tried by many authors to increase the students' motivation like digital formative assessments and changing classroom practice [2930].

The main limitation of this research is that positive response does not assure positive learning, but a negative reaction toward a program certainly decreases the likelihood of positive results in future [31]. The study needs more observation on the students' accomplishments and if they got direct benefits or not. It also needs to be more widely implemented on all subjects.

In conclusion, TWFD seems to be a good approach to implement an effective and timely feedback process between the faculty and the students. It has a positive perception from the students and the faculty and it might reflect on the students' achievement as well as faculty performance and so leads to more competent graduate.

Introducing and enhancing the process of feedback within the medical education is a challenge for all the people concerned with curricula preparation. Deficient timely and efficient feedback is notable within the researcher's organization and needs continuous monitoring and improvement. TWFD after formative assessment forms a start in the way of providing this timely and effective feedback, which demands the continuous engagement of the faculty and the students in the process.

The researcher suggested that each unit coordinator introduces TWFD in the middle of each semester after a well-prepared formative assessment. Although it is an additional task and extra work, but as mentioned before, it deserves this effort. This workload can be divided on the people from different disciplines and during the session all the faculty involved will be responsible for giving and receiving the feedback from the students. In the second half of the semester, it would be the time for analysis of the feedback achieved and implementing all the necessary changes that might come up. All the changes performed will be included in the units' reports submitted at the end of the semester.