Evaluation of e-learning in a department of Community Medicine as a response to COVID-19 pandemic.

BACKGROUND: COVID-19 pandemic pushed all educational institutions to rely exclusively on technology-based learning. As this was done for the first time, it is ideal to evaluate the e-learning program to refine and consolidate the learned experience. Hence, the current study was undertaken to evaluate the online learning and teaching experiences of students and teachers.
MATERIALS AND METHODS: This program evaluation on e-learning was carried out in the department of Community Medicine (DCM) in a private medical college using context/input/process/product framework among IV, VI, and VII semester undergraduate students and faculties in DCM who were exposed to e-learning for the period of 2 months since April 2020. Google Forms was used to design a survey questionnaire that was conceptualized as per the needs of the evaluation framework. Ethics Committee approval was obtained. Descriptive analysis was done for quantitative variables and manual content analysis using Lewin's force field framework was performed for the qualitative data.
RESULTS: Out of 301 undergraduates contacted, 196 (65.1%) responded to online survey. Their mean age was 19.9 years and 128 (65.3%) were females. Mobile phone was used by 93.4% to access e-learning. Combined modality of learning was preferred by 58.2% of them in future. Six "for" and "against" factors on e-learning emerged out of content analysis pertaining to three main stakeholders, namely administrator, faculty, and student.
CONCLUSION: Our evaluation conveys that for effective e-learning in any subject, the students, educators, and institutional factors that were identified need to be considered throughout all phases of program development with careful assumptions about its acceptance by the millennial. Copyright:

Introduction

The COVID-19 pandemic has launched unprecedented changes in all spheres of life including health professions education. Physical distancing, the prime preventive strategy in the absence of an effective vaccine, forced many countries to go for complete lock-down.[1] Closure of schools and universities including medical institutions made education uncertain at all levels.[2] This extraordinary situation has pushed medical educators inevitably to rely on distance learning that maintains physical distancing between the learner and educator in the study process using technology-based platform (E-learning). It could have happened either in synchronous or asynchronous mode.

A recent survey showed that 65% of the medical undergraduates who were in prefinal year perceived that traditional face-to-face instructional mode of teaching was boring and difficult to follow.[3] Online teaching has been initiated since the end of April 2020. Teachers were given a brief training on various online platforms such as Zoom and Google Meet. As most of us were doing online teaching first time, it was crucial to know what is going on from students' and faculty's point of view. This information is important to refine the program, consolidate the lessons learned, and plan for better online sessions. Hence, an ongoing evaluation of online teaching and learning had been carried out by exploring the perceptions and practices among students and faculty.

Materials and Methods

Study design and setting

This study was carried out at a private medical college in the Union Territory of Puducherry. The traditional curriculum recommended by National Medical Commission is followed till the introduction of competency-based medical education in teaching Community Medicine. Face-to-face instructional method comprising lecture classes, field visits, practical sessions, and clinical postings is practised before the onset of COVID-19 pandemic.

It is an educational evaluation that involved a cross sectional survey using an online platform. The context/input/process/product (CIPP) evaluation framework that focuses on program improvement in the given context was used for the evaluation.[4] The various components of CIPP model identified were context (characteristics of learners and teachers, their needs for e-learning, purpose of using online platforms); input (online platforms used for teaching, devices used for accessing e-learning, training of teachers, time, and resources used); and process (planning and implementation details, impediments to meeting the necessary needs, implementation problems of e-learning and suggested solutions to overcome them). Since it is an improvement-focused, ongoing formative evaluation, the first three elements of the CIPP model were given due importance.

Study participants and sampling

The study participants were 301 medical undergraduates who were presently in the 3rdyear (VI and VII semester – 147 in numbers) and the 2nd year (IV semester – 154 in numbers) that have been exposed to e-learning in the subject of Community Medicine for the period of 2 months since April 2020 and six faculties who taught them. Since it is an ongoing evaluation and to obtain a candid view, all students who were exposed to online teaching were universally contacted. Among them who did not consent to respond were excluded and those who provided incomplete information were not included for analysis.

Data collection tool and technique

Google Forms was used to design survey questionnaire. The survey comprised of both open-ended and close-ended questions. Separate forms for students and faculties were created. The questionnaire was conceptualized as per the needs of the CIPP evaluation framework. In addition to the close-ended questions, two open-ended questions on challenges faced and suggestions suggested for improvement were asked both to students and teachers. The link to Google survey questionnaire was sent to personal E-mail ID of study participants.

Statistical analysis

The online survey responses were extracted from the Microsoft Excel spreadsheet and the descriptive analysis was subsequently performed using, IBM SPSS Statistics for Windows, Version 24.0. (Armonk, NY: IBM Corp.). Frequency and percentages were used to summarize categorical variables. Mean with standard deviation (SD) and median with interquartile range (IQR) was used to summarize age and number of online classes attended by the students, respectively.

Manual content analysis was carried out on the stakeholders' responses about e-learning. First, the authors read through the entire responses of all participants to familiarize themselves with the data. Initial codes were then manually generated by highlighting the relevant aspects of the responses. Initially, this was carried out by the first author, and then, it was discussed with the second author. Any discrepancy that arrived was sorted out and final consensus was arrived to minimize bias. Similar codes were then collated under general categories for both students and faculty responses.

The Kurt Lewin's framework of force field analysis was used to analyze open-ended responses.[5] The authors assigned these categories to create the driving forces or “for” factors (perceived advantages, positive attitudes), and restraining forces or “against” factors (difficulties/challenges) related to e-learning and solutions for improvement of e-learning. Force field analysis is an effective qualitative research tool that can be used to carry out a systematic analysis of a wide range of factors affecting any problem.

Ethical consideration

Institutional Human Ethics Committee approval was obtained to access and analyze the data collected as a part of routine teaching activity from study participants (EC No: 16/2020). All principles of ethics were given due attention.

Results

The context and input (CIPP framework) aspect of e-learning is given in Table 1 and Figure 1. The stakeholder's narration of its process component is categorized in Table 2. Out of 301 undergraduates contacted, 196 (response rate – 65.1%) responded to an online survey. Among the total respondents, 126 (64.3%) belonged to prefinal batch and 70 (35.7%) were in IV semester. The mean (SD) age of students was 19.9 (0.8) years. The median (IQR) number of classes attended by them was 30 (20–50). Of them, 128 (65.3%) were females. Of the six faculties contacted, all responded to the evaluation survey. Of them, 50% were males. Four of the faculty (66.6%) were in the age group of 30–39 years and two (33.4%) belonged to 40–50 years age group.

Table 1

Use of online platforms, its purposes, and type of devices used among medical students (n=196)

Characteristics	n (%)
Social platforms used by students
WhatsApp	194 (98.9)
Instagram	77 (39.2)
E-mail	37 (18.9)
Facebook	11 (5.6)
YouTube (live)	5 (2.5)
Twitter	1 (0.5)
Others	10 (5.1)
Purpose of using Internet
Entertainment (music, movies, games, etc.)	158 (78.6)
Communication with friends and family	143 (73.0)
News (local, regional, international)	57 (29.1)
Undergraduate education	124 (63.3)
Postgraduate preparation	58 (29.6)
Device used for e-learning
Mobile phone	183 (93.4)
Laptop	35 (17.8)
I-pad	20 (10.2)
Desktop	11 (5.6)

Multiple responses allowed

Figure 1

Students perception on certain features of possible modes of teaching (N= 196)

Table 2

Force field analysis on e-learning by students and faculty

Stakeholder	“For” factors	“Against” factors	Suggestions for improvement
Student (n=196)	In-depth understanding	Network issues	Technical solutions
	Recording and replaying (6)	Poor coverage (62)	Stable network (37)
	No peer distraction (12)	Slow (15)	Comfortable devices (8)
	Active participation	Physical issues	Computer self-efficacy (3)
	No shyness (9)	Eye strain (9)	Institutional arrangements
	Ask doubt (8)	Hard to hear (4)	Well-aligned timetable (12)
	Readily available content	Technology related	Less duration (8)
	Recorded videos (14)	Unwanted notifications (5)	Spacing of sessions (10)
	Lecture slides (12)	Device factors (12)	User-friendly tools (4)
	Physical comfort	Poor institutional strategies	Multiple teaching strategies
	Visibility and audible voice (18)	Overlapping time table (22)	Live demonstrations (8)
	Comfortable posture (8)	Lengthy lecture (9)	Recorded interactions (5)
	Personalized learning climate	Inadequate skills training	Engagement strategies
	Comfort at home (11)	No hands on (9)	Assignments (22)
	No distractions (9)	No patients (8)	Quiz/MCQs (43)
	Appropriate for pandemic (21)	Poor engagement
		Not interacting with all (11)
		Urge to finish fast (6)
Faculty (n=6)	Student-friendly	Technical skills deficit	Interprofessional collaboration
	Access via mobile phone (5)	Consumes more time (2)	Technical training (4)
	Tech-savvy generation (4)	Learning multiple tools (3)	Timely support (2)
	Better involvement	Negative attitude (1)	Institutional support
	Use of chat box (4)	Lack of institutional support	Cost (3)
	Attempt to write answers (2)	E-platform subscription (4)	Direction (2)
	Physically appealing	Fast and stable Wi-Fi (3)	Encourage social learning
	Voice clarity (1)	Difficulty in monitoring students	Tools based on interaction (1)
	Visual clarity (2)	Students distracted easily (2)	Small group teaching (2)
	New learning experience (2)	No face-to-face interaction (3)	Formal formative assessment (4)
	Aids self-assessment (1)	Less engagement and feedback (2)	Multiple teaching strategies
	Appropriate for pandemics (6)	Inadequate skills training (3)	Live demonstrations (3)
			Simulated videos (1)

Figures within parenthesis indicate the number of students/faculty who made similar statements. MCQ=Multiple-choice question

WhatsApp was used by 194 (98.9%) students, followed by Instagram (39.2%) and E-mail (18.9%). Twitter (0.5%) and YouTube live (2.5%) were accessed minimally by them. Entertainment (78.6%) followed by communication with friends and family (73%) was the main purpose of using Internet. Out of 196, 124 (63.3%) and 58 (29.6%) used Internet for learning undergraduate education and postgraduate entrance examination preparation, respectively. Mobile phone was used by 183 (93.4%) to access e-learning followed by laptop (17.8%) [Table 1]. Among the six faculty, Zoom class was used by all (100%); Google Meet, Google Classroom, and WhatsApp by four (66.6%) each; and free conference call by one (16.6%) for online teaching.

Out of 196 students who responded, 66.8% mentioned that motivation to learn was high with a combination of e-learning and traditional learning and 59.7% had the opinion that combined mode of learning helps them to improve understanding, analyzing, and judgment skills. Of them, 58.2% preferred combined modality of learning in future [Figure 1]. All six faculties (100%) and five faculties (83.3%) replied that the best teaching strategy for lectures or tutorials and the skills training in future was a combination of both online and traditional methods, respectively.

Students and faculties mentioned the following features in favor of e-learning. They were in-depth understanding, active participation, readily available content, better physical comfort, personalized learning climate, student friendliness, and appropriate during pandemics. The “Against” factors emerged out were network issues, physical problems, technology related, poor institutional strategies, inadequate skills training, difficulty in motivating students, less engagement, and feedback. The suggestions given by students and faculties for improving e-learning were grouped under the following categories. They were technical solutions, institutional support, multiple teaching strategies, student engagement strategies, formal formative assessment, encourage social learning, and interprofessional collaboration [Table 2].

A conceptual model of effective e-learning was framed based on the various factors identified from the force field analysis carried out among students and faculties exposed to it. For it to be effective, there have to be smooth coordination and collaboration between three main stakeholders, namely institution, faculty, and student [Figure 2].

Figure 2

Factors related to the effective e-learning (A conceptual model). Note: CPD-Continuing Professional Development, IT-Information technology, LMS-Learning Management System

Discussion

The mean age of students was 20 years, nearly two-third of the students were studying 3rd year and were females. Three fourth of them used Internet mainly for entertainment and communication. Nine out of ten students used their mobile phones to access e-learning. Network issues, device-related problems, and overlapping timetables were the common hindering factors. Majority of the students and faculties were of the opinion that motivation to learn, in-depth understanding of subject, and better development of skills were better achieved by combined (online and face-to-face) modes of teaching. It was synthesized based on the qualitative analysis that effective e-leaning requires smooth coordination and collaboration between three main stakeholders namely administrator (institution), faculty, and student at college level.

Sudden change in teaching mode from traditional to online indeed places continuous challenges mainly in terms of time constraints and computer literacy behind others on students and educators.[6] Previous reviews and studies showed that skills deficit, resources constraint, poor institutional strategies, and support and attitude were the barriers identified for effective online learning.[678] Our evaluation confirms similar issues among students and faculty.

In the present study, students reported that there were technical glitches, network issues, and lack of institutional support related to the quality of delivery of e-learning. Faculties felt that there was a need for better interaction and social learning strategies during e-learning. The digital natives seek instruction that is technology enhanced, convenient personalized, and linked to relevance and societal meaning.[910] Our students preferred a combination of e-learning and face-to-face mode of learning to motivate their learning, to widen their cognition, and to enhance their skills acquisition in future that was in alignment with a review article on e-learning.[11]

The current study revealed that all our millennial generation students used one or more social platforms and Internet; however, they primarily used it mainly for communication and entertainment. Only six out of ten have accessed them for educational purposes. It is obvious that they maintained a clear demarcation between the use of technology for academic learning and leisure activity.[12] Hence, the assumptions that the techno-savvy millennial generation students have an aptitude for web-based, self-directed learning and media literacy and they possess unique learning priorities and preferences than the students of previous generation get questioned.[913] It discloses the fact that e-learning has to be carefully planned as not all students will accept and engage actively.

Nearly all students used their mobile phones to access e-learning in our study. “Net Generation” students commonly use mobile phones for playing video games, instant messaging, and social networking that could relieve their stress and anxiety.[14] Teaching them through the tool that entertains them has to be considered as a double-edged sword. In such a scenario, the success of e-learning is more likely when students have to collaboratively prepare a presentation to their group with their familiar and convenient technology such as chat rooms or social network sites.[12] The facilitator needs to oversee the assignments. Hence, the learning management system which functions optimally in smart mobile phones would be of use in the future for better utilization and access to online courses.

General perception of students about Community Medicine subject is that it is more theoretical, information dense, less engaging, and boring, so they prefer self-study.[3] While designing e-learning, these issues need to be kept in mind; otherwise, it might fail to achieve its purpose. Community Medicine has some unique challenges in teaching it through purely online mode. As the name suggests, most of the teaching and learning happens around “community and family.” It requires a sound understanding of community, family, culture, belief, and environmental factors including health system needs.[15] More innovations in E-learning are required to make it suitable to address the above challenges.[16]

Strengths, limitations, and recommendations

Strengths being a combination of quantitative and qualitative information, it helped us to offer the complete picture of the situation happening in our context. The questionnaire used was conceptualized based on the time-tested framework of evaluation. However, the limitations should be kept in mind, as it was a context-specific evaluation done in a private medical college, generalizability of the findings to other educational institutions might be reserved. Training of students and teachers in technology-assisted learning is mandate as both of them were new to this kind of teaching and learning. Online teaching provides opportunity for creating scholarships like developing e-module, and e-materials that motivate students to get actively engaged.

Conclusion

It is our initial step in exploring millennial medical undergraduates' and faculties' perception on strengths and weakness of e-learning. The millennial generation is selective in using Internet and most prefer mobile phones for accessing e-learning. Both students and faculty prefer combined (face-to-face and e-learning) mode of teaching in future. Student, educator, and administrator factors need to be considered in all phases of program development without any assumptions that it will be accepted by the millennial.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.