Nursing students' evaluation of a gamified public health educational webinar: A comparative pilot study.

AIM: To compare undergraduate nursing students' evaluations of a gamified educational webinar to a non-gamified version. Both webinars covered the topic "Determinants of health," part of public health education.
DESIGN: A post-test evaluation design with a comparison group, and an additional qualitative component.
METHODS: First-year students reading for a Bachelor of Science in Nursing at the Malta College of Arts, Science and Technology were randomly allocated into two classes of 26 students each, and to the gamified and non-gamified webinars. The revised "Students Evaluations of Educational Quality" questionnaire was sent to all participants. Participants were also asked to provide comments on their learning experience.
RESULTS: A total of 40 participants who attended the gamified and non-gamified webinars completed the questionnaire. Educational quality was perceived as "good" to "very good," in both groups. Most participants in the gamified webinar group remarked that gamification helped to increase their engagement and interaction.

INTRODUCTION

Following the declaration of the COVID‐19 pandemic, most nurse educators had to resort to online teaching and learning. Online learning or e‐learning provides online access to learning, thus eliminating distance or timescale limitations that are inherent with traditional class‐based teaching (Regmi & Jones, 2020). Nonetheless, poor levels of student interaction and engagement have been identified within the nursing literature (Patterson et al., 2012).

Gamification is a recent interactive and innovative teaching modality that can engage and attract nursing students in learning (Brull & Finlayson, 2016). Gamification, which is the use of game‐based techniques, has proven to be effective in the engagement of students in education, particularly in subjects typically perceived as boring (Kapp, 2012). While Gentry et al. (2019) found that the use of serious gaming/gamification in healthcare students' education could result in increased student satisfaction, knowledge and skills when compared to traditional education, the authors did not differentiate between serious gaming and gamification. Although these tend to be used simultaneously, these are quite different in concept. According to Fatta et al. (2019) in serious games, fully fledged games are developed to deliver curriculum or skills to enhance the learning experience and increase understanding. In developing serious games, the notion is to offer a richer concept of playing, such as the use of simulation in learning clinical skills. Conversely, in gamification, the educator does not carry out a fully fledged game and only utilizes gaming concepts/elements (e.g. awarding points for questions correctly answered) within non‐gaming elements, that is his/her teaching practices, to increase student involvement and motivation to learn (Fatta et al., 2019).

Public health education, covered in undergraduate nursing education, often proves to be a challenge for educators, especially for supporting and retaining engagement for learning (Marin‐Kelso, 2013). The use of online teaching modalities that enable real‐time interaction, such as webinars, with the addition of gamification, may be particularly useful in the delivery of such a module.

BACKGROUND

Underpinning gamification is the concept of motivation (Kapp, 2012). Motivation can be intrinsic, because the student may undertake the gamified learning process, for example participate in a quiz out of the enjoyment or the learning it provides, or the sense of achievement it brings; or it can be extrinsic, where the quiz is taken to obtain some sort of reward, praise or admiration from others (Kapp, 2012). According to the self‐determination theory, which has been widely used to differentiate between successful and unsuccessful gamified processes (Rutledge et al., 2018), the core elements that need to be considered in gamification are autonomy, competence and relatedness (Kapp, 2012). For a game to be motivating enough for its users, players must feel that they are in control of their actions, competent to carry out the tasks, and they need to feel related to the other players who are also playing (Kapp, 2012). As opposed to the self‐determination theory which is focused on intrinsic motivation, operant conditioning looks at extrinsic motivation and suggests that students’ motivation is more likely to increase every time they are rewarded, such as through the provision of points (Kapp, 2012). In fact, in their systematic review, Subhash and Cudney (2018) noted that the assignment of points and use of leaderboards were important game elements for successful gamification. To motivate player activity and action, the use of time‐constrained activities is also suggested (Kapp, 2012). The use of timed questions helps to activate player action and encourages students to work under pressure (Kapp, 2012).

Research on undergraduate healthcare students’ evaluations of gamified education is limited (Gentry et al., 2019). Nonetheless, students who have experienced gamification highlighted a positive educational experience (Aktekin et al., 2018; Kinder & Kurz, 2018; Pettit et al., 2015), remarking increased learning (Kinder & Kurz, 2018; Roche et al., 2018) and other good qualities such as increased engagement and interactions (Aktekin et al., 2018; Pettit et al., 2015). Students attributed the gamified component to the creation of a positive competitive environment, an increase in motivation and encouragement to follow the course, and a supporting tool for retaining knowledge (Aktekin et al., 2018). No studies on students’ evaluations which utilized gamification in challenging subjects such as public health were identified. Moreover, no studies which reported the delivery of nursing education using webinars with the addition of gamification were found. The recent closure of most educational institutions because of the COVID‐19 pandemic, where academics have been constrained to deliver exclusive online teaching and learning methods, emphasizes the need to address the above‐mentioned research lacunae.

The purpose of this pilot study was to identify undergraduate nursing students’ evaluations of a gamified educational webinar as compared to the evaluations of a non‐gamified version. Both webinars addressed the topic “Determinants of health,” an important concept within public health education. Information from this pilot study, collected from users’ perspectives, will inform the researchers on the use of gamification in online public health education for full‐scale evaluation.

Research questions

RQ 1.1: How do students perceive the educational quality of a gamified webinar, compared to the non‐gamified version?

RQ 1.2: What are the students’ views about participating in a gamified and a non‐gamified educational webinar?

THE STUDY

Design

Given that perceptions of educational quality are subject to the topic being covered, the use of a pre‐test/post‐test was not advisable. Use of a pre‐test/post‐test at specific points within the delivery of public health education, an extensive module consisting of different topics with varying degrees of interest/disinterest, would have confounded this pilot study's results. Instead, a primarily quantitative design that includes a post‐test evaluation of the educational quality of the two webinars (gamified and non‐gamified), with a qualitative component to explore the views of students participating in both webinars, was utilized.

Method

The study's participants were first‐year Bachelor of Science in Nursing Studies students at the Malta College of Arts, Science and Technology (MCAST), who had just started their public health module at the time of the study (March 2020). Any other nursing students were thus ineligible to participate. The first‐year students were already randomly allocated into two classes of 26 students each, on enrolment of their studies. To participate in this pilot study, the first‐year students had to attend the “Determinants of Health” webinar.

The psychometrically sound questionnaire, the “Students Evaluations of Educational Quality” (SEEQ), which looks at teaching from a multi‐dimensional aspect (Marsh & Hocevar, 1991; Marsh & Roche, 1992), was utilized. Although developed more than 35 years ago by Marsh (1982), this is one of the most widely used and universally acceptable tools for students’ evaluation of teaching methods (Grammatikopoulos et al., 2015). Constant high levels of validation and reliability scores have been highlighted in the literature in various countries (Balam & Shannon, 2010; Coffey & Gibbs, 2001; Marsh, 1982; Marsh & Hocevar, 1991; Marsh & Roche, 1992). The SEEQ was also used amongst nursing students (Balam & Shannon, 2010).

This tool has nine distinct components (consisting of 31 items) which assess: “learning”; “enthusiasm”; “group interaction”; “individual rapport”; “breadth”; “examinations”; “assignments”; and “overall”, by using a 5‐point continuous Likert scale, “very poor” (1), “poor,” “moderate,” “good” and “very good” (5), for each item (Marsh, 1982). Given that students were asked to evaluate their learning experience on a sole topic of the public health module, the latter three components of the tool were not assessed. This revision had already been done in a previous study, where confirmatory factor analysis showed that the psychometric properties of the revised six‐component scale (four items per component) were maintained, yielding an alpha value of 0.94 (Coffey & Gibbs, 2001). Given that there could be other factors that are unrelated to educational quality but could confound results such as, gender, age, nationality and the highest qualification attained, these were also collected. Students were also asked to provide comments on their views about participating in a gamified and non‐gamified webinar. To ensure the local validity of the tool, the questionnaire was assessed for face validity by three nurse academics and five second‐year nursing students. No modifications were required.

The allocation of classes to the gamified and non‐gamified webinars was done at random. Students were provided with e‐learning resources to read before the webinar as part of their usual independent self‐directed learning. These were three documents: one on the Health Map model (Barton & Grant, 2006); another on the impact of social determinants of health in Malta; and another on the role of health professionals in addressing social determinants of health. Providing additional reading resources beforehand helped ensure that all students would have the same level of knowledge on the subject area, thus feeling at par with each other while feeling competent enough to participate in the game (thus ensuring participants’ competence and relatedness as suggested in the self‐determination theory).

The webinars for both groups were held on Microsoft Teams® on the same day. Both webinars lasted two hours each. Use of Microsoft Teams® enabled the lecturer to use his camera and share a PowerPoint® presentation (or the presentation on Mentimeter® as outlined below). Students were encouraged to participate in the discussions by using their microphones or the chatbox. During both webinars, the academic provided an overview of the determinants of health, particularly the social determinants of health, and the role of nurses in addressing these factors. The webinars included the identified key characteristics of social determinants of health education as was mapped out in a scoping review, such as the definition of the social determinants of health and the interaction of these factors on the individual and the population's health (Doobay‐Persaud et al., 2019).

In the gamified webinar, the same presentation was uploaded on Mentimeter®, a real‐time application for creating presentations that enables interactivity through various means such as quizzes (Mentimeter, 2020), and delivered on Microsoft Teams®. A six‐question quiz, having a multiple‐choice format (two or three responses) covering basic and simple concepts about the determinants of health, was spread out within lecture content and delivery. To ensure student autonomy (as suggested in the self‐determination theory), students were invited to join the game as a fun way of learning, and not as a means of assessment, by going on menti.com and entering a unique code. Questions, which were allocated points as a means of extrinsic motivation (operant conditioning), had a 20‐s time limit. The students who gave the right answer fastest got higher points. After each question, after discussing the students’ responses, a leaderboard slide was displayed showing the students’ ranking. At the end of the lecture, another leaderboard slide displaying the top scores was shown. The same questions forming the quiz were also presented in the non‐gamified webinar to generate discussion.

Participants were unaware (blind) that webinars differed between groups. The online questionnaire was sent to the participants at the end of each webinar, accepting responses until that same evening.

Analysis

All quantitative analyses were done using IBM SPSS Statistics Version 26. Descriptive statistics (frequencies “n,” percentages, mean and standard deviation “SD”) were used to characterize the sample and the revised SEEQ variables. Given the small sample, non‐parametric tests were used to identify any statistically significant differences between the gamified and non‐gamified webinars samples’ characteristics and their mean revised SEEQ scores. Fisher's exact test was used to identify any differences for nationality, gender and highest qualification, while the Mann–Whitney U test was carried out to identify differences in the mean participants’ age. The Mann–Whitney U test was also utilized to identify any differences between the groups’ mean scores of each of the revised SEEQ’s items and their respective components.

The participants’ comments on their learning experience were objectively analysed (manifest analysis) by both authors (separately), to identify what was being said (Bengtsson, 2016). Qualitative analysis followed the steps outlined by Bengtsson (2016). First, the authors familiarized themselves with the data and coded the text inductively into meaningful units (decontextualization). Then, the text was checked again with the study's aim and the identified units, revising/adding units as necessary (recontextualization). Similar units were then condensed creating categories (categorization) and referred to objectively, providing participants’ excerpts to stay closer to original meanings and contexts as possible (compilation). In the analysis, each participant who provided comments was assigned a code. For each category, the frequency of mentions and some examples in the form of excerpts (with the respective participants’ code) were provided.

Ethics

Before carrying out the study, a research proposal listing all relevant ethical considerations was sent to the Institution Review Board at MCAST for approval. No ethical issues were envisaged, and the study was approved.

An information letter was attached to the questionnaire. To support participants’ self‐determination, students were informed that participation and their answers, or non‐participation, did not affect their grades whatsoever. Participation in the questionnaire was voluntary and actual participation implied informed consent. No names, email addresses or personal identifiers were collected thus ensuring anonymity.

RESULTS

Twenty‐four students participated in the gamified webinar, while 25 students attended the non‐gamified session. The report generated by Mentimeter® showed that all students who attended the gamified webinar (24) had also played the game. On average, 67% of the participants got a typical question right. Nineteen and 21 participants in the gamified and non‐gamified webinar groups, respectively, filled in the online questionnaires. The sample characteristics are displayed in Table 1. The statistical tests carried out showed that participants did not differ significantly by their characteristics. All participants in the gamified webinar group confirmed that they had participated in the quiz.

TABLE 1

Sample characteristics and the statistical tests results for significant differences

Variable	Response	Gamified webinar group	Non‐gamified webinar group	Significance
		n (%)	n (%)
Nationality	Maltese	15 (88.2)	20 (100)	p =.204, Fisher's exact test
	Pakistan	1 (5.9)	0
	Syrian	1 (5.9)	0
Gender	Males	3 (15.8)	2 (9.5)	p =.863, Fisher's exact test
	Female	15 (78.9)	17 (81.0)
	Prefer not to say	1 (5.3)	2 (9.5)
Highest qualification	MCAST Advanced Diploma in Health Sciences a	11 (68.8)	12 (66.7)	p = 1.0, Fisher's exact test
	MATSEC Certificate a	3 (18.8)	4 (22.2)
	Mature student	2 (12.5)	2 (11.1)

Both the Malta College of Arts, Science and Technology (MCAST) Advanced Diploma in Health Sciences and the Matriculation Certificate (MATSEC) are MQF (Malta Qualifications Framework) Level 4 qualifications.

Tables 2 and 3 display the participants’ ratings for each indicator of the six distinct components in the revised SEEQ (Coffey & Gibbs, 2001). When considering quality in educational methods as “good” or “very good,” the majority in both groups perceived their teaching and learning experience as so. This was noted particularly for the items falling under “Individual rapport” and “Group interaction” (mean values listed in Table 4), where almost all or all students in both groups gave high ratings. The lowest “good/very good” scores were identified amongst participants in the non‐gamified webinar group for “Instructor enhanced presentations with the use of humour” (66.7%) and for “Instructor's style of presentation held your interest during class” (71.4%), and amongst students in the gamified webinar group for the item, “Instructor gave the lecture that facilitated taking notes” (73.7%). As shown in Table 4, the mean scores of the six distinct components were not significantly different between groups. Furthermore, no statistically significant differences were identified for each item.

TABLE 2

Students’ evaluation of educational quality (gamified webinar group scores)

Educational quality indicators	Rating (%)					n
	Very poor	Poor	Moderate	Good	Very good
Learning
You found the lecture intellectually challenging and stimulating.	0	5.3	10.5	47.4	36.8	19
You have learned something which you consider valuable.	0	0	0	36.8	63.2	19
Your interest in the subject has increased as a consequence of this lecture.	0	0	15.8	63.2	21.1	19
You have learned and understood the subject materials in this lecture.	0	0	5.3	42.1	52.6	19
Enthusiasm
Instructor was enthusiastic about teaching the lecture.	0	0	0	36.8	63.2	19
Instructor was dynamic and energetic in conducting the lecture.	0	0	10.5	36.8	52.6	19
Instructor enhanced presentations with the use of humour.	0	5.3	10.5	63.2	21.1	19
Instructor's style of presentation held your interest during class.	0	0	5.3	63.2	31.6	19
Organization
Instructor's explanations were clear.	0	0	0	36.8	63.2	19
Lecture materials were well prepared and carefully explained.	0	0	0	42.1	57.9	19
Proposed objectives agreed with those actually taught so you knew where the lecture was going.	0	0	11.1	50.0	38.9	18
Instructor gave the lecture that facilitated taking notes.	0	0	26.3	63.2	10.5	19
Group interaction
Students were encouraged to participate in class discussions.	0	5.3	0	36.8	57.9	19
Students were invited to share their ideas and knowledge.	0	0	0	26.3	73.7	19
Students were encouraged to ask questions and were given meaningful answers.	0	0	0	26.3	73.7	19
Students were encouraged to express their own ideas and/or question the instructor.	0	0	10.5	15.8	73.7	19
Individual rapport
Instructor was friendly towards individual students.	0	0	0	10.5	89.5	19
Instructor made students feel welcome in seeking help/advice in or outside of class.	0	0	0	15.8	84.2	19
Instructor had a genuine interest in individual students.	0	0	0	21.1	78.9	19
Instructor was adequately accessible to students during office hours or after class.	0	5.3	0	36.8	57.9	19
Breadth
Instructor contrasted the implications of various theories.	0	0	5.3	73.7	21.1	19
Instructor presented the background or origin of ideas/concepts developed in class.	0	0	5.3	52.6	42.1	19
Instructor presented points of view other than his/her own when appropriate.	0	0	5.3	42.1	52.6	19
Instructor adequately discussed current developments in the field.	0	0	0	36.8	63.2	19

TABLE 3

Students’ evaluation of education quality (non‐gamified webinar group scores)

Educational quality indicators	Rating (%)					n
	Very poor	Poor	Moderate	Good	Very good
Learning
You found the lecture intellectually challenging and stimulating.	0	4.8	14.3	42.9	38.1	21
You have learned something which you consider valuable.	0	0	0	52.4	47.6	21
Your interest in the subject has increased as a consequence of this lecture.	0	0	19.0	57.1	23.8	21
You have learned and understood the subject materials in this lecture.	0	0	14.3	19.0	66.7	21
Enthusiasm
Instructor was enthusiastic about teaching the lecture.	0	0	4.8	28.6	66.7	21
Instructor was dynamic and energetic in conducting the lecture.	0	0	4.8	33.3	61.9	21
Instructor enhanced presentations with the use of humour.	0	9.5	23.8	14.3	52.4	21
Instructor's style of presentation held your interest during class.	0	4.8	23.8	23.8	47.6	21
Organization
Instructor's explanations were clear.	0	0	4.8	38.1	57.1	21
Lecture materials were well prepared and carefully explained.	0	0	0	38.1	61.9	21
Proposed objectives agreed with those actually taught so you knew where the lecture was going.	0	0	4.8	47.6	47.6	21
Instructor gave the lecture that facilitated taking notes.	0	0	19.0	42.9	38.1	21
Group interaction
Students were encouraged to participate in class discussions.	0	0	4.8	42.9	52.4	21
Students were invited to share their ideas and knowledge.	0	0	0	42.9	57.1	21
Students were encouraged to ask questions and were given meaningful answers.	0	0	0	42.9	57.1	21
Students were encouraged to express their own ideas and/or question the instructor.	0	0	0	47.6	52.4	21
Individual rapport
Instructor was friendly towards individual students.	0	0	0	33.3	66.7	21
Instructor made students feel welcome in seeking help/advice in or outside of class.	0	0	0	23.8	76.2	21
Instructor had a genuine interest in individual students.	0	0	4.8	28.6	66.7	21
Instructor was adequately accessible to students during office hours or after class.	0	0	0	38.1	61.9	21
Breadth
Instructor contrasted the implications of various theories.	0	0	4.8	57.1	38.1	21
Instructor presented the background or origin of ideas/concepts developed in class.	0	0	4.8	47.6	47.6	21
Instructor presented points of view other than his/her own when appropriate.	0	0	5.0	35.0	60.0	20
Instructor adequately discussed current developments in the field.	0	0	0	38.1	61.9	21

TABLE 4

Mean scores of the six components in the revised students’ evaluation of educational quality and the statistical test results for significant differences

Component	Gamified webinar group	Non‐gamified webinar group	Significance (Mann–Whitney U test)
	Mean (SD)	Mean (SD)
Learning	4.33 (0.41)	4.30 (0.50)	U = 195.0; p =.915
Enthusiasm	4.33 (0.50)	4.36 (0.71)	U = 180.0; p =.611
Organization	4.33 (0.38)	4.44 (0.52)	U = 164.0; p =.347
Group interaction	4.64 (0.50)	4.54 (0.50)	U = 180.50 p =.611
Individual rapport	4.75 (0.37)	4.67 (0.39)	U = 172.0 p =.469
Breadth	4.41 (0.43)	4.49 (0.43)	U = 179.50 p =.592

Fifteen students in the gamified webinar group and 12 students in the non‐gamified webinar group provided comments on their learning experience. Following initial analysis, comments were found to be related to positive feedback or suggestions for improvement.

Most participants (n = 10) in the gamified webinar group highlighted the benefits in having participated in the quiz, in terms of increased engagement and interaction; “Quiz game … made us more involved” (participant in the gamified webinar group [GG] no. 2), “The lecture was very engaging and fun … enabled students to discuss among ourselves.” (GG 9), “The quiz was nice as it kept us alert to the lecture.” (GG 10), and “quiz … I feel it encouraged us more to stay focused and concentrate” (GG 14). Four participants in the non‐gamified webinar group and one participant in the gamified webinar group (who did not attribute his/her comment to gamification; “I think it was a good lecture, the message of the lesson was sent and I understood everything that the lecturer said.” GG 6) remarked that it was a good lecture; “he is quite good in delivering the lecture” (participant in the non‐gamified webinar group [NGG] no. 5), and “everything was fine and up to date” (NGG 7).

Various suggestions were provided for improving educational quality (identified categories and participants’ excerpts listed in Table 5). Five students, four of whom participated in the non‐gamified webinar, suggested that the lecturer should interact more on an individual basis. While two students who participated in the gamified webinar suggested that more questions could have been included in the quiz, four participants suggested that time to respond to quiz questions should have been longer. The use of humour was encouraged by two participants in the non‐gamified webinar group and one in the gamified webinar group.

TABLE 5

Main categories identified from the students’ suggestions for improving educational quality

Categories	Excerpts a	Frequency a
Interact with individual students	"Be more interactive with the students as individuals" GG13	(n = 5) GG13, NGG6, NGG10, NGG11, NGG12
	"Include the students more" NGG6
	"I think by asking more individually the students, like we do in class" NGG10
Gamification—longer time limit	"more time to read the questions during quiz" GG3	(n = 4) GG1, GG3, GG4, GG11
	"quiz … I think that there was not enough time allocated to read and answer the questions" GG11
More humour	"Be more humorous" GG13	(n = 3) GG13, NGG3, NGG8
	"more humor" NGG3
Slower teaching pace	"More time could be given while explaining" GG7	(n = 2) GG7, NGG3
Gamification—more questions	"The lecturer could have added more questions in the quiz" GG7	(n = 2) GG7, GG12
Less busy slides	"more visual interpretations … especially when there was a lot of information on a slide" NG3	(n = 2) GG8, NGG3
Use of videos	"Maybe involving videos" NGG7	(n = 2) NGG4, NGG7

Participants from the gamified webinar group are denoted as “GG,” while those from the non‐gamified webinar group are denoted as “NGG.”

DISCUSSION

Overall, students in both groups considered the quality of their educational experience as “good” to “very good.” This was also remarked by some students when expressing their views on attending the gamified and non‐gamified webinars. Such findings relate to the literature, where the majority of nursing students (88%, 95% CI: 79%–97%) were satisfied or very satisfied with the use of webinars for learning (Williamson et al., 2009).

In our study, almost all students in both groups reported “good/very good” “Individual rapport” and “Group interaction.” Nonetheless, five students, four of whom were in the non‐gamified webinar group, remarked in their comments that the lecturer should interact more on an individual basis. This highlights the importance of real‐time instructor interaction in online nursing education (Seckman, 2018; Wolf, 2018), which students might miss within online education. Given that gamification was found to increase student interaction, this was perceived less by students in the gamified webinar group.

More students in the gamified webinar group than in the non‐gamified webinar group rated the item, “Instructor's style of presentation held your interest during class” as “good/very good,” suggesting that the use of gamification was a useful educational tool to gain students’ interest. As was identified in the literature on the use of gamification in healthcare education (Aktekin et al., 2018; Kinder & Kurz, 2018; Pettit et al., 2015) and remarked by most of the gamified webinar group’s participants within their comments, apart from increasing student interactions, use of gamification can also help increase students’ interest in the subject topic and subsequently increase their engagement, resulting in an overall positive experience. Nonetheless, some participants claimed that the there was little time to respond to the posed questions. This could have influenced the level of self‐determination of some students and hence their motivation to participate. Given that this was a new topic for most, if not all students, more time should have been allocated for the students to think through the questions and answers.

While three students from both groups pointed out the need of “more humour” when providing comments on their learning experience, less students in the non‐gamified webinar group found that the presentation was enhanced with the use of humour. It is likely that since students in the gamified webinar group found that the style of the presentation held their attention, they might have perceived less the need (or lack) of humour.

Participants in the gamified webinar group were less likely to rate the item, “Instructor gave the lecture that facilitated taking notes” as “good/very good.” In the gamified webinar, which also lasted two hours, the lecturer had to deliver topic content and allocate time for the quiz and a discussion of the students’ responses. This might have resulted in a faster‐paced lecture which created some difficulties for students who were taking notes.

Strengths and limitations

To our knowledge, this is the first study that compared students’ evaluation of a gamified webinar to a non‐gamified one amongst undergraduate nursing students following a public health module. Student evaluation was carried out after the delivery of one non‐gamified/gamified webinar thus ensuring that results and suggestions for improvement were noted before embarking on a full‐scale evaluation. Moreover, it is likely that a longer intervention, that is comparing gamified and non‐gamified webinars across the whole module, would have then been subject to bias. This is because over time, students in both groups, being from the same institution, were more likely to disclose their learning experience, becoming aware of the noticeable difference in educational delivery. Consequentially, participants in the non‐gamified group would have been more likely to give a lower rating of the educational quality of the sessions when compared to those in the gamified group, since they would be aware that they are missing the additional gaming experience. A full‐scale evaluation, by recruiting classes from different institutions, and evaluating the use of gamification in webinars throughout the public health module, is thus recommended. This would also make up for the small sample limitation of this study.

It is also noteworthy to mention that this study did not carry out pre‐tests as a means to ensure parity between groups. As stated earlier, given that perceptions of educational quality are subject to the topic being covered, use of a pre‐test/post‐test at specific points within the delivery of the public health module, which consists of different topics with varying degrees of interest/disinterest, would have confounded this pilot study's results. However, the statistical tests carried out on baseline characteristics did not identify any significant differences.

Students’ evaluation of teaching, such as through the revised SEEQ (Coffey & Gibbs, 2001), generally does not accurately measure educational quality (Vanacore & Pellegrino, 2019). Nonetheless, this study aimed to compare education with the addition of gamification and not to evaluate teaching practice. While the addition of interviews would have provided more qualitative detail, the researchers aimed to identify and compare students’ views soon after the delivery of the webinars, to avoid potential bias (disclosure of the educational delivery between classes). The addition of an open‐ended question, asking students their views on their learning experience, to the psychometrically sound questionnaire revised SEEQ (Coffey & Gibbs, 2001), has enriched the data collected providing more depth and flavour.

CONCLUSION

This pilot study found that the students’ evaluations of the educational quality of a gamified webinar were similar to those of the non‐gamified webinar, with both being perceived of good quality. While our findings highlight the importance of ensuring individual student interactions when developing exclusive online teaching and learning methods, such as webinars, this was perceived less by students in the gamified webinar group. Positive feedback was in fact mostly remarked by the students in the gamified webinar group, who highlighted that the use of gamification in online learning helped increase student engagement and interactions during the lecture.

Despite no statistically additional benefit in the use of gamification to online synchronous communication technology (a webinar), the use of gamification may help enhance nursing students’ engagement and interaction in challenging nursing modules such as public health, particularly amongst students who may be used to traditional classroom‐based learning. While more time should be dedicated to read and answer gamified questions while ensuring a normal paced lecture, the users’ perspectives have provided additional evidence as to the value of gamification in online nursing education, warranting full‐scale evaluation.

CONFLICT OF INTEREST

The authors declare no conflict of interest.

AUTHORS’ CONTRIBUTIONS

The first author was the leader in this research. He participated in all phases of the study. He also delivered the webinars and collected the data on his own and wrote the initial draft. The second author participated in the other phases of the study, including the conception and design of the study, the review of the literature, the interpretation of results and the drafting of the article. Both authors have approved the final paper submitted.