"Written Formative Assessments with Peer-Assisted Learning" an Innovative Teaching Program for Postgraduate Students in Community Medicine.

Introduction: There is a paucity of research on conducting written formative assessment with constructive feedback for theory paper writing for postgraduates of Community Medicine in India. The concept of "Written Formative assessments with Peer-Assisted Learning Program" was implemented to improve the first 2 levels of Miller's Pyramid and assess its impact on the summative assessment. Materials and
Methods: The program was conducted for 2 batches of postgraduate students in the Community Medicine enrolled for the academic session of 2016-2019 and 2017-2020. The written formative assessment was conducted every Saturday for 1 h from August to March month in 2018 and 2019. After each test, answer papers were evaluated by the peer and faculty from the department. Written and oral feedback was given by the peer. After IEC approval, we planned to assess the program's effect on level 1 and level 2 Kirkpatrick's framework. The data were analyzed using SPSS statistical package version 24 (SPSS Inc., Chicago, IL, USA). P < 0.05 was considered statistically significant.
Results: Total 23 formative written assessments were conducted per year. The proportions of knowledge, comprehension, and analytical type of questions asked were 47%, 32%, and 21%, respectively. The mean attendance rate was 76.28% ±16.4%. There was no statistically significant difference in the average percentage of marks in formative (weekly test) and summative assessment (university final examination). There was a statistically significant positive co-relation of projected mean marks and summative assessment marks with the co-efficient of the determination being 22.6%. There was overall positive feedback of the formative and peer-assisted learning (PAL) from post graduate students. Conclusions: Written Formative Assessment with PAL program is one of the effective programs for postgraduate students to gain confidence in writing and presentation skills and to score higher in theory examination. Copyright:

INTRODUCTION

Community Medicine is an academic subject, a branch of medicine which deals with the promotion of health and prevention of diseases, involving people's participation, utilizing professional management skills. It is a three-year post-graduate course. However, there had been growing concerns about the postgraduate curriculum.[1] To address this, in the last few years, NMC (earlier Medical Council of India) has brought the paradigm shift in postgraduate education by introducing competency-based curriculum. The main goal of this new curriculum is to make medical education in India outcome and context oriented. Thus, as per this new curriculum, postgraduate students in Community Medicine should be equipped with the knowledge, skills, competencies in primary, secondary and tertiary care, control and prevention of outbreaks/epidemics, community diagnosis, health needs assessment, epidemiological assessment, research, and planning evidence-based health policies and programs. With regard to achieving these competencies, NMC guidelines state that formative assessment of these postgraduates needs to be done to assess their medical knowledge, patient care, procedural and academic skills, interpersonal skills, professionalism, self-directed learning, and ability to practice in the system.[2]

Formative assessment refers to assessment that is particularly intended to provide feedback on performance to improve and augment learning. It is not the instrument that's formative, it is the utilization of knowledge gathered, by whatever means, to regulate teaching and learning that merits the “formative” label.[3456] There are various researches conducted to assess the practical or clinical skills of postgraduate students.[789] As per our review of literature, no research is done on implementing formative assessment for theory paper writing (knowledge component) for postgraduates of Community Medicine in India. Theoretical knowledge is an important aspect of medical education. Theory examination is a valuable instrument to assess the cognitive domain of the student. Without the cognitive domain, the student cannot excel in the psychomotor and affective domains.[10] It is well known that good writing skill is important in public health career. Writing practices are an essential part of preparing for theory papers, to put the thoughts into well-sorted content. Writing an answer requires articulation of thinking. Conventionally, postgraduate learning includes studying theory from textbooks and through lectures. It is widely known that postgraduate learning consists mainly of practical experiences and less of classroom learning. Theory exams are an important part of summative assessment and in the current curriculum, there is no formative assessment of postgraduate curriculum for theory knowledge. After 3 years of rigorous practical training, many of the postgraduates fail to perform well in theory examinations. Their theory performances may not be proportional to the practical knowledge. Moreover, in Maharashtra University of Health Sciences (MUHS) summative assessment, there is equal weightage given to theory (400 marks) and practical (400 marks) examination.

The concept of “Written Formative assessments with Peer-Assisted Learning (PAL) Program” is relevant in this context.

Peer is defined as “people from similar social groupings who are not professional teachers helping each other to learn and learning themselves by teaching.”[11] The PAL as an educational teaching method is as old since Socrates and Plato began questioning one another's ideas in small groups. It is an umbrella term encompassing all PAL programs.[12]

As per Miller's Pyramid, four levels of competencies are “Knows,” “Knows how,” “Shows how,” and “Does.”[13] The first two competencies can be assessed by multiple-choice questions, short answered questions, or long answered questions while the latter two competencies can be assessed by objective structured practical/clinical examination and other performance-based assessments tools. Through this program, we intended to improve the first 2 levels of competencies i.e. knows and knows how through written formative assessment with feedback and assessed its impact on the students by analyzing marks obtained in formative and summative assessments and through feedback from postgraduate students regarding its utility in performance in the university examination.

MATERIALS AND METHODS

The “Written Formative Assessments with Peer-Assisted Learning Program” was started in the year 2018. It was conducted for 2 batches of postgraduate students in the Community Medicine enrolled in the Municipal Corporation-run Medical College. The medical college is affiliated with MUHS and recognized by National Medical Commission. The students were enrolled for the academic session of 2016–2019 and 2017–2020. There were 17 students per batch per year. Thus, a total of 34 students enrolled in this program. The summative assessment is conducted by MUHS usually in May each year.

The written formative assessment was conducted every Saturday for 1 h from August to March month in 2018 and 2019. The structured timetable was planned and communicated to the students in July. One topic was assigned per week. All the topics for assessment were aligned to the MUHS syllabus. In the MUHS MD examination, a student must appear for four papers (Paper I to IV) with each paper of 3 h duration. In this program, it was ensured that all the topics were covered for written assessment. Moreover, no students could appear for the test who reported after 15 min of starting the examination. This ensured that students got adequate time to write answers to questions. The question paper was set by the faculty who had 10 years of teaching experience. Each test had one long answer question (LAQ) and 1 short answer question (SAQ). The LAQ and SAQ had 25 and 10 marks respectively since the same marks are allotted for LAQ and SAQ in the university examination. The questions were of knowledge, comprehension, and analytical type. The students were asked to bring the answer sheets like the dimensions of the answer sheet provided in the MUHS examination. This was to ensure that, students could plan while writing the answer and realize the pages needed to write one answer and space needed to draw a particular concept map/flow chart. Model answer in terms of the content/headings for each paper was discussed by peers and faculty. Peer first evaluated answer papers which was then counter-checked by the faculty. The marks given by the faculty was considered as final. Peer and faculty were the same throughout for each batch. This was to maintain uniformity and track the progress of each student. Written feedback on each answer paper was given by the peer. Apart from this, oral feedback was also given during the distribution of papers, during postworking hours in a regular place inside the campus by the peer. This was done in small groups of 5–6 students based on their availability. Students could freely discuss their doubts regarding the questions asked in the test. The peer was the one, who had recently passed the university examination and joined the department to complete the bond. Both the peers for each year were from the top 5 rankers of their MD examination. We purposefully kept the discussion of peers with the student as unstructured. Students were motivated and asked to approach the peer, in case they need any clarification for any concepts. This was to bring out the self-motivation and urge among postgraduate students to learn and improve themselves. Feedback written on the answer papers was supervised by the faculty. In case, peer faced any difficulty in finding any resource material, the faculty provided the same.

After IEC approval and permission from the head of the department, we planned to assess the impact of this new program based on level 1 and level 2 Kirkpatrick's framework. The attendance record was regularly maintained in the register. The marks obtained by each student in the weekly test were regularly entered in the excel sheet.

Level 1 of Kirkpatrick's framework (Students' reaction) was assessed by 20 items and 5-point feedback form with the inclusion of few open-ended questions. The form was distributed upon completion of the final examination. Inductive content analysis approach was utilized to analyze the open-ended question. During the first stage, we carefully identified the systematic recurrence of codes throughout the data series and grouped them together using content analysis, generating open codes. Subsequently, axial coding was followed and more similar-looking open codes were grouped under a subtheme. Following the constant comparison analysis among various sub-themes, we ultimately generated the major themes.

Level 2 of Kirkpatrick's framework (Students' learning) was assessed by comparing marks obtained in formative and summative assessment. The data were analyzed using SPSS Statistical Package Version 24 (SPSS Inc., Chicago, IL, USA). Categorical variables were assessed using the Chi-square test. P < 0.05 was considered statistically significant.

RESULTS

Total formative written assessments conducted were 23 per year. The proportions of knowledge, comprehension, and analytical type of questions asked in the weekly test were 47%, 32%, and 21%, respectively. The variety of written feedback given after each test was pertaining to presentation, content of answer, framework of answer, time allotment for each answer, online and textbook references, handwriting improvement, concept map drawing, etc.

The mean attendance rate was 76.28% ±16.4%. The minimum and maximum attendance rate was 43% and 100%, respectively. About 54.5% had more than 75% attendance rate. The mean of total sum marks obtained was 225.42 ± 79.04 marks. The mean marks obtained in each test were 12.65 ± 2.7 marks. There was no statistically significant difference in the average percentage of marks in formative (weekly test) and summative assessment (university final examination) as shown in Table 1. There was a statistically significant positive co-relation of projected mean marks and summative assessment marks with the co-efficient of the determination being 22.6% [Figure 1]. The themes generated from open-ended questions on the usefulness of formative assessment and feedback were improvement in writing skills, presentation style, confidence building, prior preparation of exam, practice of variety of questions, and motivation to study [Table 2]. There was overall positive feedback of the formative and PAL from postgraduate students [Table 3]. On average 60% of the students gave positive responses. Around 34% of them were not sure and 6% gave negative responses

Table 1

Comparison of percentage of marks in formative and summative assessment

Percentage of marks obtained in formative assessment	Percentage of marks obtained in summative assessment		Total (%)
	≤60	>60
≤60	20 (74.1)	7 (25.9)	27 (100)
>60	4 (66.7)	2 (33.3)	6 (100)
Total	24 (72.7)	9 (27.3)	33 (100)

χ2-0.136, P>0.05

Figure 1

Correlation of formative and summative assessment marks

Table 2

Analysis of open-ended questions on feedback on the program

Major themes	Verbatims
Improvement in writing skills	I have improved my neatness of writing answers through this weekly test
Improvement in presentation style	It also helped in improving my presentation of answers
Build confidence in appearing for the final theory examination	It gained confidence for the exam
Prior preparation for final examination	My exam preparation was always at the last moment. But this was the only time I started at least 10 months earlier due to this weekly test. It was very useful for me
	Weekly test helped to start early preparation for final exams
Practice of variety of questions	It covered all MUHS relevant questions and topics. So, it was very useful
Motivation to study	Marks motivated me to study more

MUHS: Maharashtra University of Health Sciences

Table 3

Feedback of the postgraduate students on formative assessment and peer-assisted learning

Statements	Strongly disagree (%)	Disagree (%)	Not sure (%)	Agree (%)	Strongly agree (%)
It generated awareness of exam questions	3	3	32	39	23
It provided the opportunity to clarify doubts regarding particular topic	3	0	42	23	32
Marks motivated me for further studies	10	0	32	29	29
I sometimes checked other books or on net to verify what is given in the standard books	0	3	39	23	35
Questions asked in weekly test had good standards	6	0	29	26	39
Questions were mix of knowledge and applied type	3	0	35	29	32
Questions were only of knowledge type	3	0	23	47	27
I followed the same timetable for studying the topic as per the weekly test timetable	3	3	29	16	48
I started studying with my batchmates in group	0	3	42	13	42
I could better do self-study	13	0	37	23	27
My analytical thinking improved	3	0	30	27	40
Weekly test gave ample of writing practice	0	0	39	10	52
It helped me in completing the full paper on time in university examination	7	0	43	10	40
Learning was enjoyable	3	0	37	33	27
Discussion with peer were useful for clearing the doubts	0	3	35	19	42
Written feedback on my paper by peer motivated me to study	7	0	37	27	30
Properly planned timetable for weekly test motivated me to perform well in the examination	0	0	30	40	30
I could freely discuss my academic doubts with peer	7	0	33	27	33
It helped in university practical examination to perform better	10	0	34	21	34
Looking at the peer, I started gaining interest in teaching	13	0	23	16	48

DISCUSSION

In this competency-based curriculum era, wherein we need to demonstrate the competencies achieved by each postgraduate student, academicians and medical teachers need to be innovative to develop program which can suite the needs of the students and serve the goal of NMC. As mentioned by Garg et al. in their article, there is a need for a paradigm shift in the style of postgraduate teaching.[1] With this regard, we initiated this innovative and novel “Written Formative assessments with Peer Assisted Learning Program” in the Department of Community Medicine in 2018 and 2019. As per our knowledge, no such extensive structured program for postgraduate students has been initiated in any of the medical colleges for postgraduate students in Community Medicine. In an era of competency-based driven curriculum, where formative assessment is the principal component of it, the findings of our paper will play a crucial role in initiating such program in other Medical Colleges.

Systematic review done by Lerchenfeldt et al., mentions that peer can provide valuable feedback.[14] To be specific we incorporated near-peer – tutoring type method. It was near peer since peer was one academic level higher than the postgraduate students and tutoring type since the ratio of peer to students was <10.[12] In our study, the peer was the one who joined the department to complete the bond. By involving the peer, it solved the dual purpose. It is always observed, that the recently passed out student has knowledge of many aspects of public health. They always are inclined to impart knowledge to their juniors. Juniors also are receptive to learn from them. Both peer and juniors do not have time constraints such as working hours as usually both stay in the same campus or locality and thus can freely discuss anytime. The main motivating factors for peer in PAL, are the possibility to simultaneously share and improve their knowledge and expertise.[15] We also got similar findings. An added advantage for the peer who was involved in this formative assessment was that they kept themselves updated about recent advances in the public health.

Learning is promoted when formative assessment is backed with well-designed constructive feedback.[16] In this written formative assessment, we also ensured that constructive feedback is given to each student. Our findings also showed good congruence with Carrillo-de-la-Peña et al. research. In their research, the authors concluded that students who carried out mid-term formative assessment got better marks and had higher success rates in the final summative assessment.[17] In our program also, there was no statistically significant difference in the marks obtained in formative and summative assessments. Near peer education also improve the performance of students in the summative examination.[18] Moreover, after initiating this program, number of postgraduate students from our college in top 10 ranking in the university theory examination increased as compared to previous years when this program was not initiated. According to the findings of Krasne et al. study, formative assessments can be used as effective predictive tools of summative performance in medical school.[19] We too got the similar finding. Those who were high achievers (>60%) in the written formative assessment were also high achievers in the summative assessment. Moreover, the coefficient of determination was 23%. In the MD curriculum, students from 1st year are exposed to a wide variety of learning experiences such as postings at Urban Health Training Centre, Rural Health Training Centre, field visits to various public health importance places, inter collegiate seminars, conferences, workshops etc. Apart from this, our college regularly conducts postgraduate teaching every day for 1 h wherein various topics are discussed in groups. These teaching programs must have overall contributed for students' learning and achieving final marks in summative assessment. It is always observed that in 3-year postgraduate course, students usually start their formal study for the final examination in the third year. They are therefore named as “exam going batch” - the batch that will be appearing for the final university examination in that particular year. Considering this, we purposefully started this program for “exam going batch” since they were the ones who could devote time for this program and can actively participate in it.

Overall, there was positive feedback on this program. Through the feedback, around one-third of them were not sure as they neither gave positive nor negative responses. The reason may be as this program was like self-directed learning. Self-directed learners need to show some characteristics such as, they should set clear goals for themselves, be self-motivated, be curious, and willing to learn. We observed that some were self-directed learners whereas some were not. Learning process and style differs from student to student. Brown et al. mention that understanding learning styles encourages students' participation in the programs and motivate them to gain professional knowledge.[20] Through our program, students found it more useful in improving their writing and presentation skills. It gained their confidence and motivated them to study.

The limitations of this research is that it is implemented in one medical college. Thus, findings cannot be generalized. This is the drawback of educational research as there many influential factors and barriers that can affect its validity and reliability. However, this is unique educational teaching program which we had initiated in our college for our postgraduate students. The results can guide other academicians to implement or experiment it in their medical settings.

CONCLUSIONS

Written Formative Assessment with PAL program is one of the effective programs for postgraduate students to gain confidence in writing and presentation skills and to score higher in theory examination.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.