Assessing medical student satisfaction and interest with serious game.

INTRODUCTION: Serious games are interactive and entertaining digital software with an educational purpose, and they are increasingly being used in undergraduate medical education. Effective serious games attempt to form positive mood in order to encourage players to continue the play, leading to increased interest in gameplay and satisfaction as well as better academic performances. AIM: To determine  the medical students' satisfaction, situational and individual interest during a serious game.
METHODS: This was a prospective study performed during a 2-year period (2018-2019 and 2019-2020). A total of 108 third-year medical students participated in this study. Students were asked to play a serious game on a computer for 20 minutes. A set of questionnaires containing evaluation grids to measure the satisfaction and interest was given to students. The effectiveness of the game was assessed using pre and post-tests.
RESULTS: Following the exclusion criteria of students due to missing data, complete data were available for 97 students. Satisfaction and interest experienced by the students were high. The median of the game performance of students was 418, 04 points. There was a positive relationship between ease of use and game performance. There was a negative correlation between the three scales of interest and game performance. There was a significant difference between the mean scores of pre-tests and post-tests (p<0.01).
CONCLUSION: The results of this study suggest the potentials of serious game on medical student's satisfaction, interest and learning achievement.

Introduction

The repertoire of computer strategies for medical education is becoming wider with the introduction of e-learning applications, game-based learning, gamification, and mobile learning 1. A variety of serious games are ever more frequently used in medical education taking into account that medical students are younger and keen on technologies 2, 3.

Serious Games are defined as digital games that educate, train, and inform. 4. They have been identified as being able to develop important skills, cognition and behavior as well as affective and motivational status 5.The challenge of developing these serious games is balancing both the entertainment and educational values. One of the reasons for the effectiveness of serious games in education may be their influence on learners’ satisfaction. 6. Effective serious games attempt to form positive mood in order to encourage players to continue the play, leading to increased interest in gameplay and satisfaction as well as better academic performances.

The aim of this study was to determine the medical students’ satisfaction, situational and individual interest during a serious game.

Methods

Study design :

This was a prospective study performed during a 2-year period (2018-2019 and 2019-2020).

Study population :

Nine successive groups of twelve students each (a total of 108 students) in the third year of medical training participated in this study. The training period lasts four weeks, in accordance with the university’s recommendations.

Inclusion criteria :

Students who agreed to participate were included in the present study.

Exclusion criteria :

Incomplete data from the questionnaire

Methods :

All participating were required to take a 60-minute teaching session of pulmonary embolism at the medical university of Tunisia.

In the third week of the training period presentation of the serious game to the student and they were asked to play the game.

Description of the Serious Game :

Wecreated a simulation-based serious game with design elements consisting in avatars, scores and explanations. This game was performed using a free version of the "VTS Editor" software.

Characteristics of the game :

Health topic: Pulmonary Embolism (PE)

Targeted students: Third-year medical students

Learning objectives: The game aims to train student to assess the pretest probability of PE, these include the original Wells score and modified Wells score, revised Geneva score, Assess for hemodynamic stability, the interpretation of an electrocardiography, risk stratification of pulmonary embolism and patient management.

Language: we used the native language hint

Assessment: The game had 7 levels and the Students were evaluated using multiple-choice questions. Each player had to answer the questions before the pre-defined time limit. During each level, we aimed to test a specific objective.

Points were awarded on the learner’s performance and served to numerically represent the progression. The final score was 500 points.

Story :

The story follows an 80-year-old patient consulting the emergency department for an episode of dyspnea and haemoptysis.

Virtual environment :

The game took place into the doctor’s consultation. Game platforms needed to play Computer. Estimated play time 20 minutes.

Primary evaluation criteria :

- The effectiveness of the game was assessed using pre and post-tests.

- Gaming performance: Quality of the gaming performance was measured by the number of points students were awarded by the game ( total 500 points).

- At the end for the training, an anonymous self-assessment questionnaire composed of 17 items [All items were measured using Likert scales ranging from one (very much disagree) to seven (very much agree)], and was submitted to student in two parts :

The first parts of the questionnaire evaluate the student’s satisfaction: in order to determine students’ satisfaction, we have adapted the Technology Acceptance Model (TAM) 7. TAM is among the most widely used model for the validation of the information systems. TAM postulated that usefulness and ease of use are the main factors to predict behavioural intention. Specifically, students of both groups were requested to answer to a questionnaire validated in the literature. The questionnaire includes instances of the 4 items for Usefulness (U) and the 4 items for Ease Of Use (EOU). The questionnaire includes also instances of the 4 items for attitude toward using the system (ATT) and the 3 items for behavioural intention to use the system (INT). Averages and medians of the students’ responses to the questionnaire were calculated. An average near 7 expresses that, in average, the students are very satisfied when using the learning game. However, an average near 1 expresses that, in average, students are very dissatisfied when using the learning games. Furthermore, a median such as 7 expresses that most students are very satisfied.

The second part of the questionnaire evaluates the three models of interest using the individual and sustained scale for serious game 8; composed of 12 items subdivided into three sub-scales assessing the three types of interest: individual interest, sustained situational interest and maintained situational interest.

Statistical analysis

Data were analyzed using SPSS software version 19.0. Comparisons of 2 means on paired series were carried out by the nonparametric test of Wilcoxon for paired series. The links between 2 quantitative variables were studied by the Spearman rank correlation coefficient. In all statistical tests, the significance level was set at 0.05.

Ethical approval

We made every effort to comply with data protection rules and all data were anonymized prior to analysis. Study participation was voluntary and all participants signed an informed consent form before entering the study.

Results

Participant characteristics

A total of 108 students were enrolled. Following the exclusion criteria of students due to missing data, complete data were available for 97 students. The effective response rate was 89.8% (97out of 108 eligible students). A total of 72% (n=70) of the participants were female.

Results of interest dimensions

The reliability of the used interest scale was internally quite consistent (12 items; α = 0.83). Table 1 show that the means and standard deviations of the interest scale experienced by the students was high.

Table 1. Means, medians and standard deviations of interest scale (N = 97).

	Mean	Median	Standard deviations
Individual interest	6,2	6	0,4
Sustained situational interest	6,2	6	0,3
Maintained situational interest.	6,3	6	0,4

Results of the satisfaction scale

The reliability of the satisfaction scale was good (12 items; α = 0.79). We can see in Table 2 that all mean score of student’s satisfaction indicators (evaluated with 7-point Likert scale) are particularly high overall. Table 2 presents the averages and medians for the variables Usefulness (U), Ease Of Use (EOU), attitude toward using the learning game (ATT) and behavioural intention to use the learning game (INT).

Table 2. Means and standard deviations of satisfaction scale (N = 97).

	Mean	Median	Standard deviation
Usefulness (U)	6,1	6	0,6
Ease Of Use (EOU).	6,4	7	0,7
attitude toward using the system (ATT)	6,1	6	0,7
Behavioural intention to use the system (INT).	6,3	6	0,6

Game performance

The median of the game performance of students was 418, 04 points (minimum 300, maximum 500 with standard deviation 44,64).

Correlation between game performance interest and satisfaction

We examined the bivariate correlations between learners’ interest, satisfaction and game performance. Table 3 show significant and positive relationships between Ease Of Use (EOU) and game performance. There was a negative correlation between the three scales of interest and game performance.

Table 3. Correlation between game performance interest and satisfaction

	Game performance
Individual interest	-,113 ,269
Sustained situational interest	-,089 ,388
Maintained situational interest.	-,024 ,819
Usefulness (U)	-,099 ,334
Ease Of Use (EOU).	,203* ,046
Attitude toward using the system (ATT)	-,089 ,387
Behavioural intention to use the system (INT).	-,115 ,264

** Correlation is significant at the 0.01 level (2-tailed). * Correlation is significant at the 0.05 level (2-tailed).

Evaluation of student’s learning outcomes

Efficiency of the serious game: the pre-test score mean reached 5.2 (SD=1.6) versus 8.7 for the post-test (SD: 1.2). There was a significant difference between the means scores of pre-tests and post-tests (p<0.01) (Figure 1 ).

Figure 1. Evaluation of the effectiveness of the serious game onLearning outcomes.

The pre-test score mean reached 5.2 (SD=1.6) versus 8.7 for the post-test (SD: 1.2). There was a significant difference between the means scores of pre-tests and post-tests (p<0.01)

Discussion

This study highlights that serious games are able to enhance the learning process, the interest and the satisfaction of medical students.

Satisfaction and interest experienced by the students were high. The median of the game performance of students was 418, 04 points. There was a positive relationship between ease of use and game performance. There was a significant difference between the mean scores of pre-tests and post-tests (p<0.01).

Limitations

There are several potential shortcomings that limit the generalization of the results. For instance, the results may be influenced by the fact that participation in this study was on a voluntary basis and included mainly interested students. Another limitation of our study is the fact that we did not study any long-term effects. Because our study focused only on a single intervention, future studies are needed to investigate if the effects of playing serious games can also be found after a long period of time. The strong point of the study lies in the fact that it is a prospective study that evaluate effectiveness of serious games in medical education.

Medical education has recently witnessed clearly more studies on serious game assisted learning compared with other fields. Learning in serious games typically occurs through a gameplay that engages the learner in challenges adapted to his in-game skills. Challenges are defined as subjective experiences that solicit the learners’ skills 9.

When developing serious games, it is important to develop a tool to track gameplay or learning progress 10, 11. It has been widely accepted that serious games, as a tool integrated into many courses, are playing an important role in learning and helping learners focus on the target subject. Perceived usefulness, ease of use, and goal clarity were indicators of satisfaction and effectiveness in the use of serious games 4, 12, 13. When learners clearly predict the goals and ease of use, they tend to focus on the contents and enjoy themselves 14.

Our finding, that student’s satisfaction when playing the serious game was high was in according with these researches.

Serious games might also encourage learners to hold positive attitudes toward academic tasks with strong self-regulation if they were immersed in the gaming situation. Positive attitudes help learners to produce better academic achievements. Thus, it is reasonable to conclude

that serious gaming leads to significantly more positive attitudes than traditional learning.

They usually hold positive attitudes, especially toward this attractive learning approach that easily triggers their interest and motivation 15.

Our study indicates that the interest scale experienced by medical students was high. This finding strongly confirms the findings of other studies which state that serious games manage to trigger and maintain situational interest for a longer time, it might have positive effects on the subsequent individual interest 16.

Assessment of players’ performances is a difficult issue to address, which is drawing much attention. Scoring players' performances should include various factors such as type

of games, teaching objective, and gaming context.

Players’ performances are scored during the gaming process 6. In case players overcome a hindrance, they will obtain some awards such as scores, advancement, and power. Educational elements can be integrated into the gameplay, which will be subconsciously acquired by the players during the gaming process 17.

Student dashboards provide information on performance and in-game outcomes, allowing them to easily assess their strengths and weakness. Current solutions for learners’ dashboards present several issues that should be considered. It is common to compare the results of students with their class or with average results from their classmates; however, some researchers have pointed out that this may demotivate those students who do not reach at least average rankings 18, 19. In accordance with these researches, there was a negative correlation between the three scales of interest and game performance in our study.

Another important issue for using serious games in education was the increase of learner knowledge gains 20, 21, 22, 23. Serious games provide a scalable, convenient method for learners to practice skills in a safe environment while incorporating interactivity and competition in a format well-liked by millennial learners 24, 25. In our study, there was a significant difference between the means scores of pre-tests and post-tests (p<0.01) and this result was in line with the previous research.

Conclusion

The results of this study suggest the potentials of serious game on medical student’s satisfaction, it also indicates that serious game was successful in triggering satisfaction and maintaining situational and individual interest and learning achievement.

Acknowledgments

The authors thank all medical students who participated in this study and dedicated their time.