Evaluating e-learning on an international scale: An audit of computer simulation learning materials in the field of dentistry.

BACKGROUND/
PURPOSE: Global networking and e-learning courses are an effective strategy for sharing educational content and there is potential scope to use e-learning technology in dental education. Therefore, this study aimed to explicate the challenges encountered in international e-learning use and decipher optimum solutions for disseminating course/systems on an international scale.
MATERIALS AND METHODS: An e-learning course with four computer-assisted simulation materials developed among international faculties was provided to dental undergraduates at dental schools in Japan and Vietnam in 2020. A post-questionnaire survey plus pre- and post-tests were conducted to obtain learners' feedback and assess the success of the course's implementation.
RESULTS: Altogether, 41 students participated, with a response rate of 78.8%. The mean post-test score was significantly higher than that of the pre-test (p < 0.001). Students from both groups felt that e-learning was beneficial, useful for their future, and should be continued.
CONCLUSION: This study showed that e-learning/course provision could be achieved at an international level by using a common online system, which is beneficial for students to gain a wider perspective and global dental education. Learners indicated that they learned without any major problems in learning operations. This type of educational material creation and course implementation is extremely important in global networking and dental education in the present era; and communication and cooperation between the faculties was important for course provision. More countries should be included in future studies to ensure that it has global application.

Introduction

Global networking is considered an important strategy to meet ever-changing global challenges in the delivery of effective dental education, and services. Sharing educational content/curricula is a real possibility in this global era, and the implementation of technology could function to establish a higher standard of dental education. Technology in dental education ranges widely and includes various methods, but primarily, computer simulation e-learning usage via global/international networks could offer an optimum solution. Although there are some challenges in the use of technology, such as cost, IT knowledge/expertise, and equipment preparation, course provision could also be optimized using easier or inexpensive interfaces.4, 5, 6 e-Learning was listed among the technologies to prepare for students’ futures that dental educators might need to adapt. Further, the global/international implementation of e-learning modules might not only function to combat the scarcity of materials or limited classroom hours/teaching staff, but could also enhance dental education itself.1, 2, 3 In addition, it might be important and advantageous for future dentists to have an education including information from a worldwide healthcare team.

Widely accepted in many countries, e-learning has come under discussion in others,5, 6, 7, 8, 9 leading to the use of advanced technology to support dental curricula possibly being encouraged more in the near future, which may serve to effect education, both domestically and internationally, as well as further enhance students’ cognitive skills. Additionally, due to the current pandemic situation, education delivery has become a combination of on-site and online resources with these technology-based learning opportunities.

Our review of the relevant studies found that to date, most of the research has been conducted in a single country, and less investigation has been conducted regarding the international use of e-learning course/modules. Tokyo Medical and Dental University (TMDU) has been developing and providing e-learning materials and courses to learners in Japan and has seen favorable outcomes.12, 13, 14, 15 However, no investigation had been conducted in international course provision for undergraduate study, even though some of the materials were authored in English. Results pertaining to one country might not be equivalently applicable to other countries; thus, the assessment of educational efficacy, the challenges, and solutions encountered are critical in the international provision of e-learning. Therefore, this study aimed to explicate the challenges encountered in the international implementation of e-learning to decipher how best to disseminate e-learning education courses and systems on an international scale.

Materials and methods

This study was approved by the Dental Research Ethics Committee of TMDU and the University of Medicine and Pharmacy (UMP) at Ho Chi Minh City Ethics Committee (No. D2018-067 (TMDU) and No. 757/HĐĐĐ-ĐHYD 22/10/2020 (UMP)).

Study population

In total, 52 dental students in their pre-clinical or early clinical stage at the dental school of TMDU, Tokyo, Japan, and of UMP, Vietnam, were invited to participate in our study. At TMDU, fourth-year students had access to an e-learning course, which contained four interactive materials (in two e-learning units) for self-learning via a learning management system (LMS) (WebClass, Data Pacific (Japan) Ltd., Tokyo, Japan) from August 20 to 27, 2020. Fifth-year students had access to the materials from October 16 to 23, 2020, while fourth-year students at UMP had access from December 11 to 18, 2020. Participation was voluntary, and all participants were provided with an ID and password to ensure their anonymity. Consent to the research participation was obtained via a questionnaire in the LMS.

An e-learning course comprised of interactive materials

The e-learning course consisted of five units: pre-course level training (pre-test), a tutorial unit, an international e-learning course for clinical settings, an international e-learning course titled Gerodontology/oral health care basics for the elderly people requiring nursing/long-term care, and post-course level training (post-test).

Four modules were uploaded to the e-learning course: In the course entitled “Clinical Settings,” participants learned how to examine and diagnose an endermosis (Material 1), how to conduct minor surgery and provide emergency response during the surgery (Material 2), and how to administer prosthetic treatment for a discolored tooth (Material 3). In the course entitled “Gerodontology/Oral Health Care,” they learned oral care basics for older adults requiring nursing/long-term care (Material 4). The interactive materials provided a self-learning opportunity wherein learners encountered a step-by-step scenario according to the clinical decision-making process and were required to choose the correct answers from multiple-choice questions or areas, and/or by viewing videos and pictures (Fig. 1). Two questionnaires were included in the e-learning course units. All the study elements, such as data collection and the pre- and post-tests, were administered via the LMS.

Figure 1

Example of the learner's screen view. The three windows in the upper half could have audio/visual situational information. The lower half had instructions/questions and multiple-choice type answers. Once the learners selected and confirmed their choice, their answer and explanatory notes as to the appropriateness of that choice were shown on the next page. After finishing the material, learners could review all questions with explanations.

The interactive materials were developed by faculty members from TMDU, UMP, and the University of Montreal (UM), Canada, using an authoring tool (SIMTOOL), and Microsoft Windows Movie Maker version 2012 (Microsoft Corporation, Redmond, WA, USA). Faculty members from TMDU, UMP, and UM discussed the contents and the program's versatility for international use. Each school had one or more faculty members whose English skills were sufficient for communicating with the others, including native level/native speakers. Additionally, before the materials were provided to learners, the materials' contents, interactivities, online user-friendliness, and the efficacy of multimedia usage were assured in two stages: by the faculty of dentistry: the Educational Simulation Production Subcommittee (under the TMDU Dental Educational Committee), which consists of healthcare professionals covering most fields of dentistry and by two reviewers.

Questionnaire to evaluate the course and pre-/post-tests

Individuals who agreed to participate in this study consented via the LMS and agreed that their results, including pre-and post-test score data, could be used for research purposes.

Two self-assessment questionnaires were distributed via the LMS at the conclusion of each “international e-learning course” unit, which enquired about the learners' fields of study knowledge, the influence of the study materials on their other learning, the material's usefulness for their future, their levels of interest in the material(s), and the ease of operation. These questions (Q1-10) were answered using a Likert scale of 1–6 (1 = I do not think so at all; 2 = I do not think so; 3 = If I have to choose, I do not think so; 4 = If I have to choose, I think so; 5 = I think so; 6 = I definitely think so), along with general comments on Q11 and 12. Questionnaires also gauged respondents' English ability.

A pre-test to evaluate each participant's baseline skills was conducted right before the self-study materials were distributed. The pre-and post-tests (and instructions on how to use the materials (tutorial)) were conducted using either of the following two methods: 1) Participants gathered at one location with computers and completed a pre-test regarding the relevant field of study via the LMS. After a self-study period of one week, participants underwent a subsequent post-test, wherein they gathered again at a location with computers. 2) All the instructions were given via a web conferencing system (Zoom, Zoom Video Communications, Inc., San Jose, CA, USA), and the participants completed tests using their own computers.

The pre-and post-tests consisted of ten multiple-choice questions, with ten points for each question, totaling a maximum possible score of 100. The contents were generated by this research team, based mostly on the knowledge that the learners could obtain via the e-learning materials and partly on self-study of the materials’ topics. To keep the degree of difficulty between pre- and post-tests consistent, the tests consisted of the same questions and choices rearranged in a different order to decrease the possibility of any increased scores due to memorization.

Statistical analysis

Statistical analyses were conducted using IBM SPSS Statistics for Windows® (version 26.0. IBM Corp, Armonk, NY, USA). Cronbach's alpha was used to assess the internal consistency of the questionnaire results. The pre-and post-test scores were analyzed by the Wilcoxon signed-rank test.

Results

Altogether, 41 of the 52 students participated, with a response rate of 78.8%. A total of 17 TMDU students and 24 UMP students agreed to participate in the study, with a response rate of 77.3% and 80.0%, respectively. Three students used only one of the “international e-learning course” units.

Regarding English ability, 19 participants divulged their level of proficiency in the questionnaire. Among that 19, seven were at the B2 level or higher of the Common European Framework of Reference for Languages (CEFR).

Pre- and post-tests

Overall, the mean pre-test score was 29.0, and the mean post-test score was 41.0, and the mean post-test score increased significantly (p < 0.001). The mean scores in Japan were 27.6 (pre) and 34.7 (post), which did not constitute any significant increase (p = 0.054), while in Vietnam they were 30.0 (pre) and 45.4 (post) (p < 0.001). Changes in students’ test scores are shown in Fig. 2.

Figure 2

Changes in the participants' pre- and post-test scores.

Post-questionnaires (Figure 3, Figure 4, Figure 5, Figure 6)

The internal consistency (Cronbach's alpha) was 0.824 for the questionnaire. In both universities, 85.4% of the participants in the oral health care unit and 95.1% of them in the clinical setting unit felt that these materials were useful for their future careers. No participant from TMDU felt that the level of these materials was easy for the clinical setting unit, and 11.8% agreed/somewhat agreed that they already had the level of knowledge conveyed in the material (Fig. 3). In UMP, however, 50.0% felt that this material was easy and 66.7% strongly agreed/agreed/somewhat agreed that they already had the conveyed level of knowledge (Fig. 4). More than 90% of participants from both universities strongly agreed/agreed/somewhat agreed that their level of knowledge increased after self-studying with the materials (92.7% for the clinical setting unit and 95.1% for the oral health care unit). The feedback from all participants was positive toward these materials and the learning method. Their free comments are shown in Table 1, Table 2.

Figure 3

Questionnaire survey results of the clinical setting unit (TMDU).

Figure 4

Questionnaire survey results of the clinical setting unit (UMP).

Table 1

Free comments from TMDU participants. Some were originally written in Japanese and translated into English. English was revised to convey the students’ meanings. One- or two-word answers, instead of sentences, were counted as nonresponses.

Good aspects of the materials (clinical settings)
1	Through these materials, I could review the content I learned as a third-year student. The material was also available on-demand, so I could learn at home or at a café, where and when I wanted. I think these are really good aspects. The pictures were also useful during the learning process; it was effective and deepened my understanding.
2	It was easy to answer questions.
3	The photographs of the findings were comprehensive/helpful. The practice contents were really interesting.
4	I think that it is useful to study because I also studied knowledge related to the content that came out.
5	Using pictures and figures was an effective way to learn.
6	We don't have the opportunity to learn these kinds of things in English, but I also want to learn in English. So, it definitely helps dental students learn about both dental materials and English. I also think studying online with these materials is more effective than normal classes because we can learn at our own pace.
7	The materials were easy to use.
8	Since the units were short, I could study without getting bored or overwhelmed.
9	The materials provided very useful knowledge on clinical practice. I think the material was at an international level. There were reviews after completing the tests, which helped me recapture what I had learned.
10	I was able to study dentistry.
11	When student dentists see patients at a university hospital, we are notified as to which diagnosis patients have in advance because senior dentists must choose appropriate patients for clinical education. After graduation, we have to see patients without any prior information about them and diagnose by ourselves. These materials help us practice, thinking of possible diseases that patients might have.
12	The good aspects of these materials did not pertain to their difficulty, but they were easy to work on/use. The questions were concise, and I could proceed with a relatively good pace/tempo. We had some questions in English in our exams based on the university policy, but I thought that this style of questioning would be more motivational for tackling questions.
(5 Nonresponses)
Points for improvement concerning the materials (clinical settings)
1	I was honestly familiar with dental technical terms because I learned most of those dental technical terms in Japanese, not in English. So, I didn't understand the important words or basic words which would be easy for English native speakers to understand; for example, erythroplakia, lichen planus and so on. I had to look up words to understand their meanings, and it took so much time. That's why these materials were tough for me. (Conversely, it promoted my self-learning …) Also, the interface of these materials was really old, which did not motivate me and some parts of the system were not useful. For example, the section describing the explanation of the answer was too small and I had to scroll down many times to read the explanation.
2	It was difficult to read dental terms in English.
3	The contents were slightly complicated, even in Japanese; therefore, I would like to see more learning sections. I would like to know how many answers I should choose from the multiple-choices.
4	It may be my lack of knowledge, but there were some things I did not understand, even after looking up the technical terms.
5	The web page does not always work well on my smartphone.
6	Sometimes when I wanted to click on a picture to answer a question, the picture didn't respond.
7	The letters were too small to read.
8	The English was difficult, and I was not able to understand everything.
9	If a translation was available, it would help a lot. I think the videos could be longer to show more details. I found the text too small.
10	I had studied dentistry in Japanese. So, I learned the English words and sentences of dentistry.
11	It's useless that each question had points. The important thing is to know which question was correct or how we misunderstood.
12	I wanted to read the explanations well on the questions that I was incorrect in choosing, however, as I couldn't read the English. Japanese translation would be appreciated.
(5 Nonresponses)
Good aspects of the material (oral care)
1	This material contained many videos and pictures that promoted my understanding of the situations the explanations described.
2	It is easy to understand the questions.
3	Since it had pictures, it was easier to see. The questions in the scenario helped me to stay engaged.
4	Each set of material was short, and I could finish without losing my concentration.
5	The explanations were easy to understand. I could learn without any difficulty using my iPad.
6	The knowledge was new for me, and I'm very interested in it.
7	By using this, I can learn English sentences in dentistry.
8	I remembered that I've learned this topic before. Though I forgot the detailed procedure, this material reminded me of it. It's useful to prepare and review the topic.
(9 Nonresponses)
Points for improvement concerning the material (oral care)
1	I think there are no points for improvement concerning these materials because the contents of this material were not difficult for me.
2	If the keywords were shown in different colors, that would be helpful.
3	Explanation of the answers seemed like it could be improved more. I thought it would be easier to understand if there was a check mark for correct answers instead of just saying it was appropriate.
4	Each question had a score, but if the total score were shown at the end of the method, it would have been more useful. Review exams could also be helpful for understanding the contents deeper.
5	Nothing at all.
6	I don't think that the answers and explanations were good to read. It might better to emphasize ‘Appropriate’ and ‘Inappropriate’ at least.
7	The letters were small.
(10 Nonresponses)

Table 2

Free comments from UMP participants. English was revised to convey the students’ meanings. One- or two-word answers, instead of sentences, were counted as nonresponses.

Good aspects of the materials (clinical settings)
1	Video image questions.
2	The documentation was clear and easy to understand.
3	Easy to understand.
4	Easy to use and helped me a lot in improving my knowledge of dentistry.
5	Picture video.
6	The materials related closely to clinical reality. The explanations of questions were detailed and easy to understand.
7	It will be useful in the future.
8	Helpful material for practical clinical cases.
9	There was analysis on all the answer options, which gave me a more in-depth view. The cases were practical and interesting.
10	Since it's like a demo version, the content is good enough. The WebClass had a friendly UI, which was easy to get into using without having to install any other software besides the normal Internet browser.
11	Picture, video.
(13 Nonresponses)
Points for improvement concerning the materials (clinical settings)
1	Display.
2	Has more action, image has more sound.
3	The font needs to be nicer. More sound, videos, and images.
4	More lessons.
5	Design of the web.
6	The display looks a bit old and boring.
7	Nothing.
8	Confusing interface; no place for revising the answers.
9	The interface and experience were not so enjoyable. The content was good for self-assessment and improvement, but not suitable for learning from scratch.
10	The spaces between lines for both the questions and answers were a little small and sometimes made it hard to read, since the lines were overlapped.
11	Design of the web.
(13 Nonresponses)
Good aspects of the material (oral care)
1	Images, videos, questions.
2	Easy to understand clear objectives.
3	The documentation was clear and easy to understand.
4	Easy to use and understand.
5	Video and pictures.
6	It would be useful in the future.
7	Easy to understand. Vivid.
8	The case was interesting.
9	Very good content, easy to follow and remember within just 5 min.
10	Picture, video.
(14 Nonresponses)
Points for improvement concerning the material (oral care)
1	Display.
2	More photos, more sounds, more video.
3	Should upgrade for easy use on multiple devices.
4	No.
5	Design of the web.
6	Nothing.
7	Videos were hard to watch because everything was so small.
8	The interface and experience were not so enjoyable.
9	Design.
(15 Nonresponses)

Discussion

In this study, more than 90% of the participating undergraduate dental students, regardless of their country, felt that the developed materials were beneficial for acquiring knowledge and that this e-learning study style should be continued (Figure 3, Figure 4, Figure 5, Figure 6). TMDU students were familiar with this type of e-learning, while UMP students were not, as TMDU students had previous experience using the LMS. However, UMP students also returned positive feedback regarding the material operation. Therefore, we could conclude that this type of e-learning/course provision could be achieved at an international level by using a common online system with positive feedback. An online facility to operate, share, and develop resources globally is important, not only to support and enhance dental education, but also to cultivate a global mindset and human resources in dental schools. Thus, course provision, such as that discussed in this study, has been demonstrated to be beneficial not only for the students but also for international dental education.

Figure 5

Questionnaire survey results of the oral care unit (TMDU).

Figure 6

Questionnaire survey results of the oral care unit (UMP).

Collaboration and communication were necessary to facilitate global networking and common curricula, and faculty members’ intellectual collaboration across countries could facilitate more standardization in dental education., During the development of material, discussion between the faculties of TMDU, UMP, and UM was indispensable. There were some sections in which revision was necessary, for example, the concentration of adrenaline in local anesthetics, and protocol around medication prescriptions. Even though our goals were the same, slight differences in pedagogy and methodology were encountered during the material development process and cooperation was necessary among educators from different countries to overcome these differences. By using certain interfaces like this study, such as SIMTOOL, educators were able to perform necessary editing and revisions from their respective countries. We realized that such material provision was therefore very important as it gave a wider perspective to the students who would construct global networks in the future.

Among the 19 who divulged their level of English proficiency, seven were at the B2 level or higher of the CEFR. The mean scores were 38.6 (pre) and 51.4 (post) for those seven, while they were 20.8 (pre) and 34.2 (post) for those at the B1 level or below. The B2 level or higher student pre-scores were significantly higher than students at the B1 level or below, but not the post-scores (Mann–Whitney U test) (p = 0.028 and p = 0.261). Thus, it is possible that English proficiency had some impact on learning. However, it might also be suggested that English, and dental knowledge acquisition in English, fall under different categories. Therefore, acquiring dental knowledge in English is necessary regardless of their English proficiency. In addition, in Japan, even though a few participants reported that they had previously studied these fields in their classes, many felt that the questions, as well as understanding English, were difficult, while this was not the case at UMP. These discrepancies, according to students’ English proficiency, could be cause for investigating the outcomes of students, and further research is required. From the results, it is felt that provision of dental English education along with their regular curriculum is necessary, at least in Japan. The most common language for global networking is English; therefore, dental English education requires more attention.

Technological advances have always influenced dental education and healthcare services. They tend to provide more options for global networking strategies, and their incorporation into dental education is beneficial for all the stakeholders.1, 2, 3, Virtual reality could not replace the traditional teaching methods, and even for Millennial dental students, content mattered more than the interface., However, e-learning sharing could be a genuinely worthwhile endeavor for the global dental education network;, our results have shown that a certain degree of satisfaction with the outcome was obtained from the scheme discussed in this study. This type of e-learning could be applicable in any country, but previous reports underscore the cultural differences in pedagogy. Therefore, including more countries in this scheme may be necessary to ensure that it is applicable to a broader range of countries and to examine the global use of this method.

Online self-study course provision was beneficial for participants in Japan and Vietnam, without any major problems in learning operations. For course provision, communication and cooperation between the faculties were important. Further investigation of a larger cohort will be necessary to evaluate the global use of e-learning and investigate cultural difference/competence. However, within the limitations of this study, this type of educational material creation and course implementation has been demonstrated to be important in international interface and dental education in the present global era.

Declaration of competing interest

The authors have no conflicts of interest relevant to this article.