The Implementation of Peer-Reflection to Improve Retakers' Achievement in National Medical Students Examination.

INTRODUCTION: Faculty of medicine plays a role to support the students' success in passing national exit exam through mentorship. Peer-reflection evaluates the students' learning process. We aimed to analyze the impact of peer-reflection to change the students' learning attitude and improve national exam score.
METHODS: Nine test-retaker students participated in mentorship program for three months. They took two parts of faculty-level examination. After the 1st and the 2nd part of faculty exam, there were peer-reflections I and II that argued about the characters, strengths and weaknesses, suggestions. Then, they did self-reflection to conduct responses about the mentorship including the impact of peers' input and finally took national exam. The examination score before and after peer-reflection were compared. We analyzed the statement of reflection by documents and content analysis. The progress of examination score was analyzed descriptively.
RESULTS: The students objectively understood the strengths and weaknesses of their way of learning, and then implemented the peers' advices. Peer-reflection method provided a feeling of same purpose, then developed ways of learning that promoted them to be higher motivated. Finally, 6 of 9 students passed the exam. DISCUSSION: Suggestions given by peers would be memorable and powerfully changed motivation. Peer-reflection explored non-academic problems that determined the pattern and the way of how students learned.

Introduction

UKMPPD (National Medical Students Examination) is an exam that functions as a national-scale exit exam. UKMPPD is held in order to ensure the quality of medical education from the academic/pre-clinical stage to the professional stage. This is strengthened by various regulations, e.g. Law No. 29/2004 concerning Medical Practice, Law No.12/2012 concerning Higher Education, and Law No. 20/2013 concerning Medical Education.1 For educational institutions, especially medical education, this exam is part of the evaluation of the learning process. The institution is responsible for the quality of the graduates, one of which is to provide supporting program to assist the students in passing UKMPPD. UKMPPD is a series of exam consisting of computer-based tests (CBT) and objective-structured clinical examinations (OSCE).

To be failed in passing UKMPPD occasionally becomes a traumatic factor for the students, especially if they failed several times. This condition is called “burnout” which is identified by the loss of learning enthusiasm, loss of self-confidence, and loss of ability to rise.2 The failures experienced by the students have delayed their graduation as a medical doctor. They were left behind from their friends, and no longer had learning community.

In 2017, a mentorship program was launched for 378 medical students throughout Indonesia who had not passed the UKMPPD for more than three times.3 The students joining in this program are called retaker. Based on the evaluation in early 2018 (after the end of the program), the highest pass rate was achieved by the Faculty of Medicine, Universitas Sebelas Maret. We tried to explore the methods used in mentoring the retakers and their effectiveness in improving the retakers’ success in the examination. One of the methods developed was peer-reflection.

Peer reflection was chosen due to the complexity of problems faced by the retakers. Cognitively, medical students have good cognitive abilities. The problems that caused failure came mostly from non-cognitive problems such as self-management, psychological problems, and problems in the family and living environment. Peer reflection was one of the sub-activities of the mentoring that aimed to evaluate the retakers’ learning processes more individually by actively involving fellow students with similar condition. Peer-reflection brought special impression and affected the students’ learning attitudes.

Materials and Methods

Based on the selection of all students of Faculty of Medicine Universitas Sebelas Maret in 2017, there were 9 students who did not pass the UKMPPD more than three times, consisting of 6 male students and 3 female students. They were grouped in a mentoring forum guided by one mentor. The procedure for conducting peer reflection and the method of evaluating the effects of peer reflection were carried out as the scheme in Figure 1.

Figure 1

The scheme of peer-reflection and the evaluation model.

The first round of faculty-level examination consisted of three exams every 2 weeks, namely examination A, B, and C. The faculty-level examination was a series of exam set in such a way as to resemble the national-level examination. The first faculty-level examination round was analyzed to assess the students’ results without peer reflection. The second faculty-level examination round consisted of two examinations, namely exam I and exam II. Peer reflection was carried out after each round of faculty-level examinations, before UKMPPD.

In one reflection session, peer-reflection was carried out by 8 students to 1 student in turn, both verbally (the writer did the recording) and written. Peer-reflection took place after the mentoring session. Mentoring was conducted 3 times a week. In one mentoring session, peer-reflection was carried out to two students. The rest of the students were done on the other day so that all nine students had the opportunity to do peer-reflection. Table 1 shows the stages and steps that must be taken to give peer-reflection.

Table 1

The Aspects of Peer Reflection in Each Stage (Research Document)

No.	Aspects of Reflection	Instruction
1.	Learning model	The retakers fill out a questionnaire to determine their learning model using the VARK (Visual, Auditory, Read, Kinesthetic) method which has been translated into Indonesian.
2.	Personality	The retakers describe the personality of the fellow who becomes the reflection subject.
3.	Strengths and weaknesses	The retakers explore the strengths and weaknesses of their friends during class/group mentoring.
4.	Main factors that contribute to failures in UKMPPD	The retakers predict the factors which according to them play a role in the failure of their fellow in UKMPPD.
5.	Suggestions	The retakers provide suggestions for their fellow regarding the learning process that should be done in order to pass UKMPPD.

Reflection documents were analyzed by content analysis and descriptive analysis. Content analysis was carried out by all researchers by reading the reflection’s documents of every student and hearing the recorded voice during peer-reflection. One person served as the main reviewer and 3 comparative reviewers. The results of each examination at the faculty level and the national examination were compared in descriptive form. The ethical clearance was issued by Health Research Ethics Committee Universitas Sebelas Maret No. 091/UN27.06.6.1/KEPK/EC/2020. Informed consent was obtained from all individual participants involved in the study. The consent also stated that they had agreed the publication of this study without mentioning their identity.

Results

The retakers who joined the mentoring programs were those who failed in UKMPPD three times or more. They were from several academic year of enrollment. The data are shown in Table 2.

Table 2

Detail Basic Information of the Retakers (Research Document)

Student	Year of Enrollment	Age When Joining the Mentoring Program	Failure in UKMPPD
1	2007	29 years old	8 times
2	2009	26 years old	5 times
3	2004	31 years old	5 times
4	2009	26 years old	4 times
5	2009	26 years old	4 times
6	2004	31 years old	4 times
7	2004	31 years old	4 times
8	2004	31 years old	4 times
9	1999	36 years old	4 times

Based on Figure 2, it was found that before peer reflection there were differences in achievement among each individual. By setting a pass limit of 65, on the A test, 3 retakers (33%) passed the examination. The difference between the highest and lowest scores was 22 points. In the B test, it was found that only 2 retakers (22%) passed it with the difference between the highest and lowest scores of 24 points. In the C exam, it was found that only 1 retaker (11%) passed the exam with the difference between the highest and lowest scores of 15 points. The average passing rate (in percent) on the exam before peer reflection was 22% with an average difference between the highest and lowest scores of 20.3 points.

Figure 2

Results of round 1 without peer-reflection (research document).

After peer-reflection was carried out, there was a change in the trend of the graph where the score of each retaker was almost evenly distributed and the difference between the highest and lowest scores became smaller. As seen in Figure 3, in the first test, 1 retaker (11%) passed the exam with the difference between the highest and lowest scores of 17.5 points. In try out II, 5 retakers (56%) passed it with the difference between the highest and lowest scores of 13 points. At UKMPPD, 6 students (67%) graduated with the difference between the highest and lowest scores of 21.5 points. The average passing rate (in percent) of the exam after peer-reflection was 44.7% with an average difference between the highest and lowest scores of 17.3 points.

Figure 3

Results of round 2 and UKMPPD with peer-reflection (research document).

Some of the reflections conveyed by the students are written in the following statements.

He was a bit stubborn, taking too long in arguments with group members. He likes to dominate in groups. But I like him because he often explains and makes difficult materials easier (simplifying complex material). The delivery is also interesting, but still likes to underestimate the materials. It is better if he review the material at home (reflection from retaker 8 to retaker 6).

He has good self-confidence, but sometimes becomes too confident in his own opinion. This factor is his weakness in facing UKMPPD. He is clumsy, so I can never interact much with him (reflection from retaker 4 to retaker 9).

He is too quiet and introvert, rarely chatting or discussing. I don’t know much about him and his problems. I have no suggestions (reflection from retaker 4 to retaker 7).

I think he has a psychological problem. He has difficulty accepting suggestions from others, too moody, physically weak. But he is well motivated, diligent, and always writes notes on the material. He must be more open and less selfish in discussions. He also needs to accept suggestions and arguments from others (reflection from retaker 9 to retaker 1).

Discussion

Students’ learning includes input, process, and output. Every medical student basically has a good cognitive modality due to the tight competitiveness in the students enrollment. For example, to pass the selection of a medical faculty with A accreditation located in Surakarta, an applicant must beat 15–20 competitors.

UKMPPD is the final-stage examination that every medical student must pass. The competencies tested in the UKMPPD have adjusted the competency level of a general practitioner based on the Indonesian Doctor Competency Standards.4 The process of preparing and reviewing questions involves all medical institutions in Indonesia. Thus, the type and quality of questions can describe the ability of medical students who have done 4 years of academic stage and 2 years of professional stage.5

Failures in UKMPPD are burden for individuals and are the responsibility of the medical faculty since they still belong to the faculty. More than 3 times failure in UKMPPD are not only caused by one factor. The main factors that contribute to the students’ failure include a competitive atmosphere that forces the students to obtain same target, tiring exams, and exam costs.6 In addition, factors originating from the process during the clinical stage at the Teaching Hospital also contribute to the failure,7 for example trauma in certain fields due to unpleasant events, and too high workloads8 that hamper effective learning did not occur. Failure to pass exams often has an impact not only on students’ mental health, but also on several physical disorders such as sleep disorders, memory disorders,9 and even the desire to commit suicide.

The benefits of the peer-reflection method are as follows:

(1) Finding students’ learning problems that have never been identified.10 With peer-reflection, a supervisor becomes aware that the dominant problems of his students and the determining factors for failure are things that are not always related to cognitive domain, but can also be non-cognitive. The feeling and sense of fate experienced by the retakers make them find a community to share any problems they experience openly. In addition, not all students are aware of the problems within themselves, so peer-reflection provides opportunity for the type of student with self-insight who does not know his own problem from the perspective of others.11 With this openness, problems that have not been identified by the mentor will emerge. Thus, the mentor is able to plan appropriate handling for each retaker.

(2) Building students’ motivation to change. Peer-reflection helps students identify their strengths and weaknesses from the perspective of others.12,13 In addition, students are also required to reflect on their friends, which in turn has an impact on themselves as reflectors. Related to this fact, the retakers were asked to do self-reflection on the peer-reflections from their friends in one group. Some examples of documented self-reflection are as follows:

It’s very touching. Yes, I understand my shortcomings. It is so touching. I am lack of discipline and too strict with my opinions and shortcomings. Writing down everything I learned actually helps me overcome my memory deficiency. Underestimating too much is my biggest problem. Thank you to friends who have given real reflection. I tried to be better and tried to overcome my weaknesses (Retaker 6).

Regarding my logical capacity, memory, and so on, I think it is too much. I was just lucky. In fact, I feel that my friends know much more. I came to understand by listening to the discussions from my friends. I really need to talk more, be more open, be more active, and be flexible. I have learned a lot from this mentoring process, which in my opinion is much more important than just graduation results (Retaker 7).

First, I feel touched and happy when I read the reflections for me. From the interactions so far, they still remember the things I have done during this mentoring process. They can see my shortcomings and give me advice. Overall, this reflection is very precise and I really feel that the suggestions are right from their hearts. I’m really thankful that I was finally able to find my main problem as well as the solution. Little by little I can overcome academic and non-academic problems. I became more confident. Thank you all (Retaker 8).

Successfully passing UKMPPD. Graduation in the competency test of UKMPPD is happiness for every student of medical professional program. This graduation is one of the factors that makes graduates more confident in healthcare practices. Failure in the test several times can have an impact on the incidence of burnout which will affect chronic fatigue, depersonalization, and loss of personal-efficacy.14 Furthermore, burnout that occurs in medical students has an impact on the potential for burnout when they become general practitioner in future.15 Mentoring for the retakers is basically an effort to regrow interest in learning, self-confidence, and professionalism in serving as a medical doctor in society.

Peer reflection not only built their enthusiasm to pass the national exam, but also gave them confidence to pursue their profession after graduation.16 From the 9 students who took part in the mentoring program, 3 students had not graduated yet in the same year as the other 6 students. However, one year later, they were declared to have passed the national exam. They are currently working as emergency doctors in hospital, 2 students work in private hospital and 1 student works in a government hospital. Meanwhile, the six students who had graduated earlier are currently pursuing activities in various fields. One student chose to develop a baby spa business which he practices there. One student chooses to continue his studies as a resident in obstetrics and gynecology department. Two students have worked in the 24-hour clinic. The remaining two students have worked in the emergency room of the regional hospital. The mentoring program accompanied by peer-reflection has proven to be able to regenerate the enthusiasm for learning to pass the national exam (UKMPPD) after experiencing several failures. In addition, the mentoring program is able to foster a lifelong enthusiasm for learning as a doctor that they pursue in the future.

Although peer-reflection is an effective method for improving students’ performance, there are several problems in implementing this method. Some of the problems are as follow:

Mentor has to be patience and has enough time to conduct the students who become his mentee. Mentor who is also the clinician often has limited time.17 However, the peer-reflection method requires the presence of mentor as a facilitator and a motivator.

The disclosure of some personal business, even things that have been closed tightly from the public.

Peer-reflection needs active role of students in every task during the process. The passivity of students causes the peer-reflection to be not optimal and fail to achieve its goals.

Conclusion

Learning process involves mentors (lecturers) and students. Each of them forms an efficient interaction in order to support the success of the teaching and learning process. The interaction patterns of lecturers and students, as well as students with their fellows, are essential for the success of the students in the national-scale competency test. Peer-reflection is an effective method that arises from interactions among fellow students. The results of the peer reflection are valuable for mentors/lecturers to do evaluation and set learning activity by applying individual approach. Besides being effective in increasing students’ graduation in the national competency test, the peer-reflection method is also useful to build students’ mentality as professional general practitioner candidates.