Online viva voce as a formative assessment method in forensic medicine during COVID-19 pandemic.

BACKGROUND: Conducting online classes and assessment during the COVID-19 pandemic is not without challenges. The world of medical education is adapting online training and assessment because of COVID-19 pandemic restrictions. The present study was conducted to assess the students' perception regarding the process, difficulties encountered and perceived effectiveness of online assessment.
MATERIALS AND METHODS: Online viva-voce (theory and visual based) was conducted in a government medical college in Karwar, Karnataka, India using videoconferencing application (Google Meet) to 149 second MBBS students as a formative assessment in 2020 over 3 months. Ten students per day joined Google Meet, 10 questions were asked to each student and assessed using a tutor marking system (on-spot). A feedback questionnaire (Google Form) was administered to students who attended online Viva-Voce. Data was analysed using descriptive and inferential statistics (Student's t-test).
RESULTS: Out of 149 students, 132 participated and responded to a feedback questionnaire. Majority of the participants (91%) agreed that questions covered all topics kept for viva, 82% of them felt it would be helpful for performance in final examinations. Thirty percent of students faced network issues at their places, 45% felt nervous while facing viva in the presence of other students and 35% of participants preferred online methods over traditional viva voce. Online viva voce can be transparent (90%) and less biased (88%) if done in structured format.
CONCLUSION: Online viva-voce may become relevant and effective in medical education assessment with transparent marking system for students' performance. Copyright:

Introduction

Assessment is an important component of the education system and formative assessment is an integral part of the teaching learning system. Formative assessment is for assessment for learning and takes place during the course of learning. The relationship between formative assessment and learning is relatively transparent. Feedback provides an opportunity for students to correct their learning track.[1] COVID-19 has caused unprecedented disruption to the medical education process and to the health-care system worldwide.[2] The first COVID-19 positive case was detected in India on January 30, 2020. The Prime Minister of India declared lockdown from March 24, 2020.[3] From then onward, medical education in the country was affected to a large extent because of cancelled onsite classes, students stuck in their homes far away from medical colleges, medical teachers deployed for COVID-19 related work. Health sector was overburdened resulting in stopping of routine services all over India. The strange behavior of novel coronavirus made it difficult to continue onsite lectures as usual thus influencing the medical education process which is based on lectures.[4] The current pandemic is expected to last for at least more than a year with its attendant disadvantages of social/physical distancing and curbs on classroom teaching. Hence, it would be better to conduct online classes and assessment instead of routine onsite classes and assessment. Worldwide, medical education is adapting online learning and assessment. Conducting online classes and assessment during the COVID-19 pandemic is not without challenges.[5] In developing countries issues such as financial difficulty to purchase mobile or computers, network problem, voice audibility, technology induced stress makes it difficult to conduct online classes and assessment. The success of online teaching and assessment in developing countries depends on factors such as planning, technical support, students participation, financial conditions, feasible high speed internet connection.[6] In spite of inherent challenges with online teaching and assessment, the apex body for medical education in the country National Medical Commission and state medical universities stressed upon to continue medical education in the online mode.[7] However, there were no clear guidelines for adapting medical education to online mode especially assessment during the lockdown period in India. Hence, this study was conducted to find out feasibility of online assessment and to know students' perception with an online feedback questionnaire.

Materials and Methods

Study design and setting

This study was a cross-sectional study conducted over a period of 3 months (October 1, 2020–December 31, 2020) in the Department of Forensic Medicine, Karwar Institute of Medical Sciences, Karwar, Karnataka, India.

Study participants and sampling

The study included students studying in MBBS Phase II in Karwar Institute of Medical Sciences, Karwar, Karnataka, India.

Inclusion criteria

Student studying in MBBS Phase II

Students attended online Viva-Voce conducted by the Forensic Department.

Exclusion criteria

Students who refused to submit written informed consent administered through Google Forms.

Complete enumeration method was followed to include 149 MBBS Phase II students, among which 132 participated in the study after meeting the inclusion criteria, 11 students were excluded based on the exclusion criteria.

Data collection tool and technique

Students who had given informed consent through online Google Form and attended online viva voce in forensic medicine conducted as a part of Internal Assessment (Formative Assessment) were included in the study. Google Form had three sections:

Participant information sheet and Informed Consent Form

Feedback Questionnaire

Confirmation message for opting “in” and submission/Thanking for “opting out” and end the submission.

The first page of Google Form had a participant information sheet and informed consent form. If a student opted to give informed consent, he/she opted “Yes” and the section containing feedback questions was visible to the student for answering. If a student opted to refuse, he/she opted “No” and the page automatically went to the end without revealing any feedback questionnaire.

Feedback questions consisted of both open ended and closed ended questions. Most closed-ended questions have been framed using a five-point Likert Scale (Strongly agree = 5, Agree = 4, Neutral = 3, Disagree = 2, and Strongly disagree = 1). As a part of the questionnaire validation process, we invited three faculty and ten students to pilot test the initial survey draft. The questionnaire was modified based on their feedback and used for the collection of data.

Online formative assessment process

The videoconferencing app Google Meet was used to conduct the online viva voce. The topics for viva voce mode and time of viva-voce were informed well in advance (20 days before assessment) to MBBS Phase II students. Theory viva-voce and visual based viva-voce was conducted to all students by a single examiner who is subject expert and trained in medical education technologies and health professionals' education. Ten students per day appeared for viva online. All 10 students were supposed to join Google Meet, 10 questions were asked to each student. The questions were from 10 different chapters with various difficulty levels. A student giving viva-voce was supposed to unmute his/her voice and switch on his/her camera while other students were supposed to listen in mute to the questions asked by examiner and answers given by him/her. There were 10 different slots of prior prepared questions (each slot contains 10 questions) where each student was given a chance to choose his/her slot of choice. A tutor-based marking system (on-spot) was used to evaluate the answers. Complete and satisfactory answers were awarded one mark; incomplete/half-answers were awarded half marks. A zero mark was awarded to the wrong answer.. The total marks scored by each student were informed immediately after viva was over.

In visual-based viva voce, students were supposed to write answers on a white paper page. Total 10 Visuals (each carrying 1 mark) and related questions were shown one by one to all students using PowerPoint (PPT), with 30 s time for each visual. After completing answering all the visuals, each student captured the image of the answer page and sent it to the WhatsApp number of the examiner immediately. All students kept their camera ON during visual round viva. Marks were announced to the students on the same day after evaluation of written answers to the visual round.

Ethical consideration

After obtaining the Institutional Ethics Clearance (IEC/KRIMS/O/03/2020-21 dated October 22, 2020) a feedback questionnaire (Google Form) was administered to these students who attended online Viva-Voce during the study period and willing to give informed consent for participation voluntarily. The online questionnaire was kept anonymous which helped us in obtaining the students' perceptions.

Data management and statistical analyses

The data obtained were coded and entered into MS-Excel 2019. The data were analyzed using statistical software SPSS Inc. Released 2007. SPSS for Windows, Version 16.0. Chicago, SPSS Inc. The results were described using descriptive and inferential statistics. The descriptive statistics included mean, standard deviation (SD), proportion, and percentages. The association or difference between two continuous variables was done by using Students' t-test for independent samples and P < 0.05 was considered as statistically significant.

Results

Table 1 shows the demographic characteristics of study participants. Out of 149 students who took online viva voce, 132 students gave responses to feedback regarding online viva voce they underwent as a part of formative assessment. Out of 132 participants, 41.67% were from rural background, 58.33% were from urban areas. 50.76% were females and 49.24% were males. Majority used mobile (66.67%) for attending viva-voce, followed by laptop (27.27%) and tablet (6.06%). The mean age of students was 20 ± 0.49 (SD) years.

Table 1

Demographic characteristics of the study participants (n=132)

	Frequency, n (%)
Gender
Male	65 (49.24)
Female	67 (50.76)
Residence
Urban	77 (58.33)
Rural	55 (41.67)
Device used for attending online viva-voce
Mobile	88 (66.67)
Laptop	36 (27.27)
Tablet	8 (6.06)

Table 2 shows the frequencies of feedback response obtained from the students after attending the viva-voce. Most of the participants (95.45%) received prior information about online viva voce, syllabus and mode of viva in advance. Majority (90.91%) opined that questions were chosen from all the topics kept for viva voce and (88.63%) were easy to understand. 82% of participants felt it would be helpful for enhancing performance in final exams. 87.88% participants agreed that time allotted for each student was adequate and they faced network issues at their places (37.88%). Nearly half the total participants (45%) were nervous while facing viva in the presence of other students. Only 35% of participants preferred online methods over traditional viva voce. Online viva voce was transparent with respect to marking system (90.15%) and less biased (88.64%) if done in structured format. With regard to visual round viva voce, 71.21% participants felt that the visuals shown were clear and questions asked were appropriate (81.82%) for visuals shown. 85.61% participants agreed that listening to questions asked to others and their correct answers helped in their learning. 34.85% of participants preferred this method over traditional face to face viva voce and felt that this avoids favoritism (68.18%). They (84.84%) were comfortable to face viva voce when done in a group of ten members.

Table 2

Feedback response from participants (n=132)

Item-students’ perception	Strongly agree, n (%)	Agree, n (%)	Neutral, n (%)	Disagree, n (%)	Strongly disagree, n (%)
Information about online viva, syllabus, mode of examination was given well in advance	59.09	36.36	3.03	1.52	0
Questions covered all topics from the syllabus kept for viva	54.55	36.36	4.55	3.79	0.76
Time allotted for each student was adequate	40.91	46.97	8.33	3.03	0.76
Questions were easy to understand	40.15	48.48	10.61	0	0.76
This will be helpful in enhancing performance in final examination	46.97	35.61	11.36	4.55	1.52
Marking system was appropriate and transparent	58.33	31.82	5.3	3.03	1.52
There was no bias in assessment	53.03	35.61	8.33	0.76	2.27
There was no disturbance in voice audibility	25	43.94	20.45	9.09	1.52
There was network problem in my place	15.91	21.97	21.97	22.73	17.42
Conducting online viva for ten members at a time was comfortable	39.39	45.45	8.33	5.3	1.52
Visuals were clear and helped in recalling the subject	25.76	45.45	16.67	6.82	5.3
Questions were appropriate for visuals shown	28.03	53.79	14.39	1.52	2.27
Time given for each visual was adequate to write the answer	25.76	43.94	17.42	8.33	4.55
Facing viva in presence of others in online made me nervous	17.42	27.27	26.52	21.21	7.58
Listening to questions asked to others and their answers, helped in my learning	40.91	44.7	11.36	1.52	1.52
I feel this method (online group viva) avoids favoritism	31.82	36.36	29.55	0.76	1.52
I prefer this (online) method over onsite method (traditional face to face)	15.91	18.94	31.82	21.97	11.36

Table 3 shows the difference between male and female students' feedback responses. An independent-samples t-test was conducted to compare feedback responses of male and female students. There was a statistically significant difference only for the scores of prior information about online viva for males (M = 4.42, SD = 0.66) and females (M = 4.64, SD = 0.06); t (130) = −2.072, P = 0.040. For the rest of the feedback responses, there were no statistical differences for the scores of male and female students.

Table 3

Difference between male and female students feedback responses from participants (n=132)

Item-students’ perception	Mean±SD		t	df	P
	Male (n=65)	Female (n=67)
Information about online viva, syllabus, mode of examination was given well in advance	4.42±0.66	4.64±060	−2.072	130	0.040*
Questions covered all topics from the syllabus kept for viva	4.34±0.87	4.46±0.75	−0.881	130	0.38
Time allotted for each student was adequate	3.40±0.95	3.19±1.03	1.193	130	0.235
Questions were easy to understand	4.12±0.88	4.36±0.69	−1.717	130	0.088
This will be helpful in enhancing performance in final examination	4.17±0.99	4.27±0.86	−0.614	130	0.54
Marking system was appropriate and transparent	4.35±0.89	4.49±0.80	−0.939	130	0.35
There was no bias in assessment	4.29±0.80	4.43±0.89	−0.95	130	0.344
There was no disturbance in voice audibility	3.82±0.88	3.82±1.04	−0.33	128	0.974
There was network problem in my place	3.05±1.26	2.88±1.42	0.709	130	0.48
Conducting online viva for ten members at a time was comfortable	4.28±0.72	4.04±1.04	1.492	130	0.138
Visuals were clear and helped in recalling the subject	3.85±1.06	3.75±1.08	0.536	130	0.593
Questions were appropriate for visuals shown	4.08±0.83	4.00±0.83	0.529	130	0.598
Time given for each visual was adequate to write the answer	3.74±1.06	3.82±1.07	−0.443	130	0.658
Facing viva in presence of others in online made me nervous	3.26±1.20	3.25±1.20	0.037	130	0.97
Listening to questions asked to others and their answers, helped in my learning	4.11±0.95	4.33±066	−1.549	130	0.124
I Feel this method (online group viva) avoids favoritism	3.98±0.93	3.94±0.86	0.286	130	0.775
I prefer this (online) method over onsite method (traditional face to face)	3.23±1.26	2.90±1.18	1.577	130	0.117

*Statistically significant (P<0.05) obtained with student’s t-test for independent samples. SD=Standard deviation

Table 4 shows the difference between feedback from students' residing in urban and rural areas. An independent-samples t-test was conducted to compare feedback responses of students residing in urban and rural areas. There was a statistically significant difference for the scores of perceived helpfulness for enhancement of performance in final examination after attending the online viva for students residing in urban area (M = 4.25, SD = 0.89) and rural area (M = 4.18, SD = 0.98); t (130) = 0.395, P = 0.046. There was a statistically significant difference for the scores of comfortableness in attending online viva in a group of ten members at a time for students residing in urban area (M = 4.25, SD = 0.95) and rural area (M = 4.04, SD = 0.82); t (130) = 1.331, P = 0.000. There was a statistically significant difference for the scores of perceived helpfulness in their own learning, after listening to questions asked to others and their answers, for students residing in urban area (M = 4.16, SD = 0.81) and rural area (M = 4.31, SD = 0.84); t (130) =-1.056, P = 0.028. For the rest of the feedback responses, there were no statistical differences for the scores of students residing in urban and rural areas.

Table 4

Difference between urban and rural students feedback responses from participants (n=132)

Item- Students’ Perception	Mean±SD		t	df	P
	Urban (n=77)	Rural (n=55)
Information about online viva, syllabus, mode of examination was given well in advance	4.57±0.55	4.47±0.74	0.879	130	0.381
Questions covered all topics from the syllabus kept for viva	4.47±0.66	4.31±0.98	1.043	88	0.300
Time allotted for each student was adequate	3.32±1.01	3.25±0.99	0.398	130	0.691
Questions were easy to understand	4.30±0.78	4.16±0.81	0.965	130	0.336
This will be helpful in enhancing performance in final examination	4.25±0.89	4.18±0.98	0.395	130	0.046*
Marking system was appropriate and transparent	4.51±0.75	4.31±0.96	1.322	130	0.693
There was no bias in assessment	4.43±0.79	4.27±0.93	1.039	130	0.188
There was no disturbance in voice audibility	3.90±0.94	3.71±0.99	1.100	130	0.301
There was network problem in my place	2.55±1.28	3.55±1.20	−4.536	130	0.273
Conducting online viva for ten members at a time was comfortable	4.25±0.95	4.04±0.82	1.331	130	0.000*
Visuals were clear and helped in recalling the subject	3.83±1.01	3.75±1.16	0.453	130	0.186
Questions were appropriate for visuals shown	4.06±0.86	4.00±0.79	0.440	130	0.651
Time given for each visual was adequate to write the answer	3.69±1.13	3.91±0.97	−1.176	130	0.660
Facing viva in presence of others in online made me nervous	3.06±1.18	3.53±1.17	−2.223	130	0.242
Listening to questions asked to others and their answers, helped in my learning	4.16±0.81	4.31±0.84	−1.056	130	0.028*
I Feel this method (online group viva) avoids favoritism	3.99±0.91	3.93±0.86	0.381	130	0.293
I Prefer this (online) method over onsite method (traditional face to face)	3.04±1.26	3.09±1.19	−0.239	130	0.81

*Statistically significant (P<0.05) obtained with Student’s t-test for independent samples. SD=Standard deviation

Discussion

The use of E-learning and online assessment methods is increasing in medical education due to their unique values in teaching-learning and assessment. Online assessment enables the provision of continuous and real-time feedback; it can be delivered at a time and place that suits both the learner or the educator, and it can be combined with relevant learning.[27] It is currently dominated by closed-answer type questions (that is, multiple-choice questions) though there are few centers in India where open-answer type questions are being used.[8] As with any technological innovation, it is difficult to define exactly where it will lead us, but in some ways online assessment will remain similar to traditional assessment in that, ultimately, it will have to deliver the same outcomes: Assessment that is valid, reliable, acceptable, cost-effective, and that has a positive effect on learners' behaviors. Delivering these required outcomes while continuing to innovate will be the challenge for future practitioners in this domain.[910]

Online assessment is relatively new for medical undergraduates in India; however, it will become mainstream over the next decade in all territories; this includes the developing world, partly because the needs of developing countries will be too great to be satisfied by traditional means alone.[910] The success of online teaching and assessment in developing countries depends on factors such as planning, technical support, student's participation, financial conditions, and feasible high speed internet connection.[56] The connection and feeling of being part of learning and assessment is somewhat lacking in online education and it is not unusual to feel isolated by the learner.[6] Many participants expressed that they were not comfortable to face the viva in the presence of others and preferred face to face viva voce rather than online. Network problem was another issue where participants were unable to hear the questions properly. Many expressed that the time given for the visual round was not sufficient. The time frame of 30 s was given to avoid malpractice of seeking answers from other friends. Questions were framed from easy to moderate to difficult level covering all the topics to avoid the bias. The marks distribution on spot for correct, partially correct and wrong answers made it transparent where other nine students observed the marks awarded for each student. Questions were clear and easy to understand by most of the participants.

Our study results suggest that the gender of students does have an effect on online viva-voce except for prior information provided to the students. Specifically, our results suggest that both male and female students perceive the same about on-line viva-voce conducted in the department of forensic medicine.

Our study results also observed that rural students perceived that listening to questions asked to others and their answers in online viva voce helped in their own learning as compared with urban resident students. However, rural students were less comfortable attending the online viva in a group of ten as compared to urban students. Furthermore, urban students perceived that performance in final examinations after attending the online viva will enhance in comparison with rural students.

Online assessments have their own sustained and inherent advantages and challenges. Conducting online viva voce as a part of formative assessment is not without challenges with respect to feasibility, technology adoption, network issues, transparency, learner comfort, etc., Many teachers themselves are technophobic which may delay the adoption of technology enabled education.[1112]

However, higher education institutions have faced another additional challenge of academic dishonesty during the current unprecedented COVID-19 incident due to the lack of preparation of institutions, teachers, and students.[13]

Many studies on online education and assessment have expressed the same concern as well as opinion that the students perceived the online assessment as both advantageous and disadvantageous. In a study conducted by Kumar et al.,[14] the students were happy with online assessment as it was a new way of learning and assessment with immediate results. Similar feelings were expressed by students in another study conducted in Taiwan by Chang.[15] The high acceptance of online assessment was noticed in a study done by Petrisor Marius et al. in Romania.[16] In this study, we tried to incorporate easy method of formative assessment to minimize academic dishonesty and bring more transparency with less bias which may be relevant in the COVID-19 lockdown period.

Limitation and suggestion

This study was done in a single institute with smaller sample size including one phase of MBBS only; the results of the study cannot be generalized to a larger population. Expanding the study to other institutes with more participants may yield and support the conclusion of the study.

Conclusion

Online viva voce was transparent with respect to marking system (90.15%) and less biased (88.64%) if done in structured format. Thus, it may become relevant and effective in online assessment with transparent marking system for students' performance. However, it is not devoid of feasibility issues. Poor network was a major setback as two-fifths of the participants were from rural backgrounds where net connectivity is always a problem. Many participants were of the opinion that viva voce with five members at a time would be better and comfortable. Facing viva in a group may make students nervous and so they prefer traditional face to face viva voce. This method can avoid bias and favoritism when done in a group of 5 or 10 students. It also helps students learn by listening to other questions and answers asked to remaining participants in groups of ten.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.