Online self-assessment tool in Biochemistry - A medical student's perception during COVID-19 pandemic.

BACKGROUND: Online self-assessment tools have become an important asset among current teaching, learning, and assessment methods, especially among medical students. Developments in information technology and recent changes in methods of undergraduate learning, especially with the current COVID-19 pandemic situation, will provide an impetus among students to uptake this assessment tool. The study is aimed at bringing out the perception of medical students in using an online self-assessment tool in Biochemistry on the topics "Liver Function Tests" and "Renal Function Tests."
MATERIALS AND METHODS: This study was an observational study conducted among 150 1st - year medical students employing an online self-assessment tool in Biochemistry, and the perception response toward the questionnaire was graded.
RESULTS: The participation was 100%, with the perception questionnaire having a good internal validity (α = 0.847). The students' response was maximum for questions 9 and 10, which favored for provision of instantaneous feedback and application of the same tool for other topics. A positive correlation was observed between questionnaires for perceived learning and perceived engagement using the online self-assessment tool (r ± 0.554, P = 0.001).
CONCLUSION: Self-assessment tools have an impact on the learning of students only when provided in a suitable environment such as immediate feedback and nonranking mechanisms. Copyright:

Introduction

Several diverse assessment methods in the field of teaching have been brought by competency-based medical education (CBME), which has seen a recent shift from just facts communication to the active involvement by medical students. There are two different rationales among assessment programs, namely “assessment of learning” and “assessment for learning.”[1] With the establishment of CBME, the focus has shifted from the assessment of learning to assessment for learning. An increase in student intake every year among Indian medical schools has led to the development of several new dimensions in teaching, learning, and assessment methods.[2]

The outlook of medical universities all over the world has seen a facelift following COVID-19 pandemic. The suspension of classroom teaching following the directives of higher officials led to the displacement of students to their hometown. With the view of continuing the educational progress of the medical students, alternatives have to be provided by medical educators. Online learning has become one of the primary modalities of imparting knowledge to medical students in the current scenario. If not for the pandemic, the introduction of online teaching and learning methods in many institutions of a developing country would have taken several years. Online teaching is being done through various methods, including virtual live classes, class recordings, etc., However, assessment of the students through online mode may pose a challenge as the feasibility and practicality of monitoring them may not match the supervision provided in person. One of the key components of adult learning, self-assessment, which has undergone a tremendous change following the imminent entry of the online teaching methods, should be given great emphasis.[3]

This study focuses on the use of a simple platform such as Google forms, which can be created and used as an online self-assessment tool for 1st -year medical students in Biochemistry during the current COVID-19 pandemic situation where online classes have become the dogma of medical education.

Materials and Methods

The study was conducted after obtaining approval from the Institutional Ethics Committee and the higher authorities of the Institute. Online written informed consent was obtained from the participants and procedures were followed in accordance with the ethical standards of the responsible committee on human experimentation and with the Helsinki Declaration of 1975, as revised in 2000. This is an observational study conducted by the Department of Biochemistry in a Medical College. All 150 1st -year MBBS students were given an option of participating in the study and all of them gave consent to participate in the study.

The topics “Liver Function Tests” and “Renal Function Tests” are included under core competency in the new competency-based curriculum, which has both knowledge domain and knows how level. Didactic lectures and small group teaching with case studies were used to teach these topics to 1st - year medical students. “Google Forms” a survey administration application that is a part of Google Drive Office suite, was used to create an online self-assessment tool. This assessment tool was designed for the topics “Liver Function Tests and Renal Function Tests” and comprised 25 questions. The questions were either multiple-choice questions (MCQs) or true/false statements and were designed by a team of Biochemists with >5 years of experience. The face-validity of the questionnaires was further improved by assigning a random set of questions to three groups of 2nd -year MBBS students, the validity of MCQs was further improved by applying item analytic principle involving difficulty index, discrimination index, and distractor effectiveness. Automated feedback was given, and the student could attempt the test any number of times. The score obtained was given at the end. The link to this tool was shared with the students through electronic media.

After all the students had used the online self-assessment tool, the student perception was obtained. For this purpose, another google form was created to get the perception of the students toward this online assessment tool. This questionnaire comprised ten questions under two categories. Six questions were under “perception of learning” and 4 questions were included under “perception of engagement” and the responses for these questions were based on the five-point psychometric scale, Likert scale, which comprises the following responses -strongly disagree, disagree, neutral, agree, and strongly agree. The link for the perception questionnaire was sent to the students, and all the 150 students responded. The response to these questions was grouped on the categories of perceived learning and perceived engagement. The results were then averaged to obtain learning and engagement variables. The relationships between the participant's perceived learning and perceived engagement levels in using the online self-assessment tools were calculated by employing a Spearman correlation test with a r > 0.5 for positive correlation and a P < 0.05 using the Statistical Package for the Social Sciences version 16.0. (IBM, Chicago, Illinois).

Results

All 150 students enrolled under the 1st - year MBBS program, which included 63 (42%) males and 87 (58%) females with a sex ratio of 1:0.724, participated in the study. The mean age of students was 18.75 ± 0.603 years, and the students participated in both the self-assessment tool and perception study voluntarily.

Moderate levels of perceived learning and high levels of perceived engagement were seen among students using the online tool [Table 1]. The mean values of perceived learning and perceived engagement among male students, female students, and both were different, and this difference was statistically significant (P < 0.05) [Table 2]. However, the mean values for both perceived learning and perceived engagement between male and female students were not statistically significant.

Table 1

Students’ response to the perception questionnaire

Perception Questions	SD, n (%)	D, n (%)	N, n (%)	A, n (%)	SA, n (%)
Questions about students’ perception of learning
This online self-assessment tool was easy to use	2 (1.3)	3 (2)	22 (14.6)	81 (54)	42 (28)
The questions in this tool were relevant to the topic	2 (1.3)	3 (2)	19 (12.67)	86 (57.3)	40 (26.6)
Do you feel the options for the questions in the tool were complex	5 (3.3)	19 (12.67)	68 (45.3)	38 (25.3)	20 (13.3)
The questions were time consuming	4 (2.67)	37 (24.67)	51 (34)	43 (28.67)	15 (10)
Feedback given along with the questions were relevant	1 (0.6)	3 (2)	22 (14.67)	90 (60)	34 (22.67)
I will recommend this tool to my peers	2 (1.3)	5 (3.3)	37 (24.67)	77 (51.3)	29 (19.3)
Questions about students’ perception of engagement
The questions were designed based on examination point of view	1 (0.6)	13 (8.67)	30 (20)	75 (50)	31 (20.67)
The use of this tool has improved my knowledge on this particular topic	1 (0.6)	2 (1.3)	28 (18.67)	75 (50)	44 (29.3)
I will use this tool for revising the topics	1 (0.6)	4 (2.67)	29 (19.3)	79 (52.6)	37 (24.67)
I would like to use online self-assessment tools for other topics in Biochemistry	1 (0.6)	1 (0.6)	14 (9.3)	84 (56)	50 (33.3)

SD=Strongly disagree, D=Disagree, N=Neutral, A=Agree, SA=Strongly agree

Table 2

Perceived learning and perceived engagement – descriptive statistics

Variables	n	Minimum	Maximum	Mean	SE	SD	P
Perceived learning	150	1.00	5.00	3.74	0.044	0.54	<0.0001
Perceived engagement	150	1.00	5.00	4.022	0.049	0.59
Male
Perceived learning	63	1.00	5.00	3.7302	0.077	0.61	0.0183
Perceived engagement	63	1.00	5.00	3.9921	0.078	0.62
Female
Perceived learning	87	2.50	5.00	3.75	0.053	0.49	0.0005
Perceived engagement	87	2.50	5.00	4.04	0.062	0.58

SD=Standard deviation, SE=Standard error

The reliability of the online self-assessment tool was done by employing “item analysis” for both the categories together as well as individually [Figure 1]. When the tool is taken as a 10-item questionnaire, the reliability coefficient Cronbach's alpha is 0.847, thereby indicating a good internal consistency of the items in the scale [Table 3].

Figure 1

Scree plot of responses of the online perception tool

Table 3

Reliability analysis employing item analysis

Variables	n	Cronbach’s alpha	Item mean	Item variance
Online self-assessment tool	10	0.847	3.85	0.113
Categories
Perceived learning	6	0.716	3.743	0.152
Perceived engagement	4	0.786	4.022	0.023

Furthermore, factor analysis was performed to determine the direction of the scale. Kaiser-Meyer-Olkin test has a value between 0.8 and 1, thereby indicating the narrow proportion of variance among variables. The Barlett's test for homogeneity of variances has a Chi-square value of 583.8 and is statistically significant (P < 0.001).

A Spearman correlation statistic was carried out between perceived engagement and perceived learning categories, and it showed a positive correlation with an r-value of + 0.553 and a statistically significant P = 0.001 [Figure 2 and Table 4].

Figure 2

Scatter plot showing correlation between perceived learning and perceived engagement

Table 4

Correlation results

Variables	Perceived learning	Perceived engagement
Perceived learning	1.00	0.554
Perceived engagement	0.554	1.00

Discussion

Students, especially those in their first years of medical education, have difficulties in coping up with the vast medical language and often develop their strategies to combat the same. Assessment methods that rely on the ability of the students to recall like essays, short notes, MCQs have given way to the newer methods of assessments like self-assessment and peer assessment.[2] Self-assessment has been introduced in higher education due to the increased demands of lifelong and reflective learning.[4] Self-assessment helps students to identify those areas in which they are weak and provides them with an opportunity for a scope of improvement. In other terms, self-assessment helps the medical student to identify the value of teaching experience, explore new fields, and learning experiences.[56]

In this study, there was no difference observed on statistical grounds for student demographics in terms of age, gender, language. These findings are comparable to studies by Martin et al., Richardson and Swan (2019 that in online courses, the demographic profile of students does not influence their perceptions of learning and engagement.[78] This suggests that 1st -year medical students will be more receptive to online self-assessment tools irrespective of their demographic characteristics.

Peat and Franklin have shown through their studies that the value of self-assessment tools improves, especially when the feedback is given to students' responses immediately, thereby improving their learning skills.[7] Some students used this tool more than once, enabling them to learn. This shows the scope of using this as a learning tool. The importance of self-assessment exercises with automated feedback designed using a simple platform to provide a learning tool to students is highlighted in this study. Since medical students have to use a variety of learning strategies and higher-order thinking skills to improve their learning of Biochemistry, designing such online self-assessment tools would help a long way in learning methods.[789]

This study has shown that medical students had a positive reaction to the introduction of the online self-assessment tool in their Biochemistry class. They also perceived that introduction of the tool enhanced their learning and engagement in the topics “Renal Function Tests” and “ Liver Function Tests.” The positive correlation between perceived learning and perceived engagement in this perception study shows that the students were able to use the self-assessment tool to learn as well as use it as a problem-solving application.

Self-assessment tests for 1st -year MBBS students enable them to improve their understanding of basic science concepts by employing a self-paced learning methodology. Although this study focuses on the learning context through the use of an online self-assessment tool, it does not take into account the teacher's teaching style. Several studies have shown that active and collaborative learning, including self-assessment tests, have played a major role in improving the undergraduate perception of learning and engagement.[1011] Creating such tools has given us the confidence and ease to routinely use this for both self-assessment and formative assessment as well.

To keep pace with the current generation, many universities provide e-learning resources using different platforms. Not many institutes can provide these because of financial constraints and devoid of resources.

There are many tools for receiving and sending feedback in online self-assessment, which is an important part of the e-learning process. This internet-based platform is designed in such a way that access to self-assessment exercises is provided to the students, and at the same time, the teacher can access students' results. In this self-assessment system, the evaluation is done on a real-time basis, thereby allowing instantaneous and nonbiased feedback to the student's responses.

Studies by Aschbacher show that self-assessment is being considered and adopted as an integral method of assessment not among students of other streams but health-care professions as well.[12] Self-assessment helps to measure the qualities of self-regulation and engagement. Engagement is being used as an important method of assessment in the new competency-based curriculum. Self-assessment tools enable the students to reflect on their performance and help to promote self-directed learning.

In this study, all questions were provided with immediate feedback for the results and had the highest response among all other questions (82.5%). Studies by Taras on interactive self-assessment tests have shown that these tools can improve learning when students receive formative feedback about the results, which permits them to rate their knowledge, fill in the gaps identified and thereby regulate their learning.[101113] In our study, no ranking was given to the students for taking the online self-assessment tool, and the students could use the tool any number of times. This stresses the fact that ranking would serve only to block their intended supportive function. Emotional interference would be minimal when grades refrained for a time until students finish the task, which will permit them to focus on their work.[14] Students should explore their field, commit errors, correct them, and fill in the deficits without being wary of the impact it would have on their final marks. Self-assessment without feedback from the teacher/tutor would be insufficient for the students to become aware of their errors, especially when a shift occurs in their learning methodology.[1113]

In view of assessing their knowledge on a given subject, students carry out a process of “self-testing” which implies the advantage of using self-assessment in effecting students' learning process. Besides, it can serve as a diagnostic tool for providing information about students' previous knowledge to the teacher when used before the study of a given topic or content area.[1516] When online-based self-assessment tests are used, several issues relating to student performance that arise should be considered carefully. Studies have shown that the acceptability of online tests by students is strongly influenced by the presentation of questions and their learning environment.[10]

Several theories also suggest that learners by making use of self-assessments, review their study strategies, change effort levels, and draw more attention to subjects of unfamiliar material, which thereby facilitates learning.[1417] Given the opportunity to self-assess and receive feedback on learner's self-assessments, they become strongly related to actual knowledge levels reinforcing the fact by several studies that emphasize self-assessment as a valuable skill.

We believe that the use of online self-assessment tests to 1st -year medical students has created several opportunities for both the sender and receiver ends. Teachers have a nonrank-based monitoring system for the topics attended by students, and students have the advantages that the self-assessment tests are web-based, can be self-paced, repeatability, and immediate feedback. The current pandemic situation resulting from COVID-19 has seen an increasing utilization of online resources by universities in India as well as throughout the world for learning. The reasons for such increase are manifold and include: students have no/reduced opportunity to attend face-to-face classes, less teacher-to-student ratio, the expectation of the students for a university with strong involvement in information technology and computer-based learning and the ability of information technology to provide simple solutions that often meet user expectations.

We encourage further research on the role of self-assessments as a learning tool and suggest that similar online self-assessments should be administered throughout the course for every subject in medicine with the objective of understanding their improvement in accuracy and influence on learning processes. The quick implementation of this initiative both for the students and for faculty members is the most important aspect of this study. However, the drawback lies in the fact that it was restricted to a single specialty and for a few topics. This experience of ours can serve as a model, especially during situations which could hinder the traditional teaching methods like the current pandemic situation for those institutes looking for continuing medical education.

Conclusion

Medical educational institutions have started to integrate self-assessment tools into their learning systems to enhance the learning process, and therefore, it becomes increasingly necessary to have a thorough understanding of the mechanisms behind the effect of such technological advancements on learning performance. The online self-assessment tool is a recent technological development in the student learning process; several gaps need to be addressed that often limit our ability to have a complete understanding of the learning outcome. The future scope of this study lies in the fact that further detailed research in the field of medical education employing online self-assessment tools as a part of students' formative assessment could open up several possibilities in online platforms which could be extended to clinical subjects even in demanding situations.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.