Development of e-learning in medical education: 10 years' experience of Korean medical schools.

PURPOSE: We aimed to explore medical students' online learning patterns and needs by analyzing data obtained from an e-learning portal of Korean medical schools.
METHODS: Data were obtained from learning resources and registered users of the e-learning portal by the consortium of 36 Korean medical schools, e-MedEdu (www.mededu.or.kr) over a period of 10 years. Data analytics were performed of its contents and usage patterns using descriptive statistics.
RESULTS: The website currently has over 1,600 resources, which have almost tripled over the past decade, and 28,000 registered users. Two hundred and twenty medical faculty have contributed the resources; a majority of them were clinical cases and video clips, which accounted for 30% and 27% of all resources, respectively. The website has received increasing hits over the past decade; annual website hits increased from 80,000 in 2009 to over 300,000 in 2018. The number of hits on resources varied across resource types and subjects; 90% of all website hits were on online videos, and 28% of them originated from mobile devices. Among the online videos, those on procedural skills received more hits than those on patient encounters and video lectures.
CONCLUSION: Our findings demonstrate the increasing use of e-learning in medical education in Korea over the past decade. Our study also shows a wide disparity in the frequency of use in learning resources across resource types and subjects, which have implications for improvements in the design and development of learning resources to better meet medical students' curricular needs and their learning styles.

Introduction

e-Learning has been a key feature of learning technologies in medical education and is expected to maintain its position of prominence [1,2]. e-Learning encompasses many technologies and activities that constitute it, where the learning process is supported or mediated using technologies [3]. A meta-analysis of research in medical education indicates e-learning is associated with positive outcomes across a wide variety of learners, learning contexts, clinical topics, and learning outcomes [4].

Still, studies have shown several significant barriers hinder the adoption and implementation of e-learning by medical schools [5]. Universities and colleges in South Korea have actively adopted e-learning since the early 2000s in parallel with rapid, nationwide developments in information and communication technologies (ICT). This was due, in large part, to a government initiative to become a global leader in ICT and to provide lifelong education. In 2006, about half of all colleges in Korea were offering e-learning, but relatively few of the contents of these courses were developed in medical schools [6]. This tardy adoption of e-learning by Korean medical schools was largely due to lack of resources for it [6]. To resolve this issue, Korean medical schools established a national consortium with the aim of offering quality e-learning resources developed by medical faculty to medical students. Currently, 36 of 40 Korean medical schools are participating in this consortium, which has grown from 25 schools since its inception in 2007.

Despite the wide adoption of e-learning in Korean medial schools initiated by the consortium, little is known about the current status of e-learning in Korean medical education. This paper reports the use of e-learning based upon the data from e-MedEdu accumulated over a decade to investigate medical students’ online learning patterns and needs. We sought to answer the following questions: (1) How many users have registered and how many faculties have contributed contents to e-MedEdu? (2) What types of resources are available at e-MedEdu and in what subjects? (3) How often these resources are accessed and which ones are the most frequently visited?

Methods

1. Development and organization of learning resources

The consortium offers an e-learning portal (www.mededu.or.kr), which is a repository of online resources contributed by faculty members from medical schools participating in the consortium. These resources are available free of charge to members of participating schools. An executive committee, whose members are representatives of member schools, runs the consortium. This committee meets regularly, reviews the status of content developments and sets goals for future activities. Member schools in the consortium have collaborated to develop various learning resources, which include clinical cases, clinical videos, clinical images, and quizzes, which are peer-reviewed before being made available on the website.

The learning resources at e-Mededu are intended to be used in various teaching and learning contexts. Most importantly, students can access resources at their convenience to meet their learning needs to supplement conventional instructions and for self-study outside of class. Thus, the availability of these resources aims to facilitate students’ self-directed learning. Faculty members can also utilize these resources in various instructional settings, as student-centered learning and active learning are of increasing importance in medical education [7,8]. For example, faculty members can ask students to pre-study e-learning resources before a class, or use these resources as teaching materials during lectures or for out-of-class activities in flipped classrooms [9].

Users can find resources in e-MedEdu using a search engine or by browsing them by the categories offered at the website. In addition to the website, mobile applications were launched in 2011 that enable users to use e-MedEdu in mobile settings in response to the growing demand for mobile learning [10]. Users can also browse through the resources by major learning outcomes in basic medical education. This organization of learning resources help students find resources in subjects of their interest with more ease. This also helps faculty members identify needs for content development by mapping them with learning outcomes.

2. Data analysis and ethical considerations

Data analysis was performed on data collected from e-MedEdu using the data analytics method. Data analytics for education involves the analysis of large datasets, typically concerning the use of e-learning technologies, to identify patterns in educational practice or performance in single or aggregate datasets to deal with educational issues and problems [11]. Data on registered users, contents, and website hits at e-MedEdu were collected from September 2009 (when the website was launched) to September 2018. Data on cumulative hits on contents were collected over a 7-year period from October 2011, when we started collecting log data for each resource from the database, until October 2018. Data analytics were performed of its contents and usage patterns. The data was obtained by running queries on the database regarding users, contents, and number of visits on them. The data were also stratified by resource types and subject categories. Descriptive statistics was performed for data analysis.

Institutional review board (IRB) approval was not requested for the present study, because it complied with the general exemption issued by our IRB, as the data utilized already existed and its acquisition did not involve any direct contact with any human subject. All data were collected anonymously.

Results

1. Resources and user statistics for e-MedEdu

Table 1 shows the number of resources and registered users at e-MedEdu. Currently, e-MedEdu has over 1,600 resources, of which 87 (5.3%) were on basic sciences, 7 (0.4%) were on medical humanities; others (94.3%) were on clinical sciences. When classified by content type, the majority of these resources were clinical cases and online videos, followed by clinical images, quizzes, and audio clips. Other resources included online modules and animations. There were currently 27,747 registered users; approximately 25,000 were students, 2,000 were faculty members, and the others were primarily administrative staff.

Table 1.

e-MedEdu Content and User Statistics

Variable	No. of contents or users
Content types
Clinical cases	497
Video clips	437
Clinical images	335
Quizzes	286
Audio clips	46
Others	35
Total	1,636
User types
Faculty	1,983
Students	25,571
Others	193
Total	27,747

The number of contents and registered users quoted were obtained in October, 2018. Other resources include online modules in various subject areas.

Fig. 1 shows screenshots of sample resources at eMedEdu. A repository of clinical cases were available in e-MedEdu to supplement students’ clinical experiences as increasing emphasis is being placed on learning and assessment in authentic contexts in an outcome-based education framework [12]. The cases were presented in texts accompanied by multimedia, which included clinical images (e.g., chest X-rays and computed tomography), sound clips, and video clips. These cases were originally categorized into 17 themes based upon organ systems. With the learning outcomes for basic medical education based upon clinical presentations developed by the Association of Korean Medical Colleges, the clinical cases were also mapped with the clinical presentations since 2014 so that users could browse through cases by clinical presentations as well as by organ-systems.

Fig. 1.

Screenshots of Sample Resources at e-MedEdu

The online videos available in e-MedEdu consisted of: (1) video clips that demonstrated procedural skills, (2) video clips on patient encounters, and (3) video lectures on various topics, which included basic and clinical sciences, and medical humanities. Online videos included all subjects covered in the clinical skills test in the Korean medical licensing exam. Other resources at e-MedEdu included clinical images (e.g., X-rays, magnetic resonance imaging, and pathological slides), audio clips of heart and lung sounds, online modules on various topics (basic science, clinical science, and medical humanities), and quizzes.

The contents of e-MedEdu have grown consistently over the past decade from 811 in 2009 to 1,636 in 2018. Fig. 2 shows changes in the number of contents over the past decade by resource types. As seen in Fig. 2, content development has been active on online videos and clinical cases, whereas it has been slow on image resources.

Fig. 2.

Changes in the Number of Learning Resources in e-MedEdu by Resource Types

The number of faculty members that have contributed to e-MedEdu has also grown from 80 in 2009 to 220 in 2018. The faculty members who contributed contents to e-MedEdu were from 29 medical schools. The number of resources contributed by a faculty member ranged from 1 to 334 (median=1); approximately half of them (n=112) contributed more than one resource.

2. Usage statistics on resources

Fig. 3 illustrates the growth of the website visits over the past decade. The annual website hits have increased consistently from 84,361 in 2009 to 508,857 in 2018.

Fig. 3.

Annual Website Hits at e-MedEdu

Table 2 shows the number of accumulative hits by resource types over the past 7 years. The contents that users visited mostly frequently were online videos, which accounted for 90% of all website hits. This was followed by clinical cases (5.8%) and clinical images (2.7%). Of all hits on online videos over the past 7 years, 72% (n=636,027) were from the Web and 28% (n= 243,499) were from mobile devices.

Table 2.

Number of Accumulate Hits on Contents by Resource Types over the Past 7 Years

Resource type	Accumulate hits (%)
Online videos	879,670 (89.6)
Clinical cases	56,832 (5.79)
Clinical images	27,026 (2.75)
Quizzes	15,581 (1.59)
Others (e.g., online modules)	2,673 (0.27)
Total	981,782 (100)

Table 3 presents the number of hits on clinical cases categorized by organ systems and clinical presentations. In terms of organ-systems, obstetrics & gynecology, gastroenterology, and oncology received most hits. e-MedEdu contained clinical cases for 80 of the 101 clinical presentations identified as learning outcomes for basic medical education in South Korea. Of these clinical presentations, users most frequently looked up cases of dyspnea, fever (of unknown cause), and cough. An average number of accumulated hits on clinical cases for the duration of 7 years were 243.41 per case. The number of hits on clinical cases differed across subjects; they ranged from 98.05 to 360.22 per case across different organ-based subjects and from 60.00 to 411.40 across different clinical presentations.

Table 3.

Usage Statistics of Clinical Cases at e-MedEdu

No.	Objects by organ systems	No. of cumulative hits	No. of cases	No. of hits per case	No.	Subjects by clinical presentations	No. of cumulative hits	No. of cases	No. of hits per case
1	Obstetrics and gynecology	10,786	110	98.05	1	Dyspnea	9,849	32	307.78
2	Gastroenterology	10,549	31	340.29	2	Fever, unknown cause	8,742	30	291.40
3	Oncology	10,188	44	231.55	3	Cough	6,569	18	364.94
4	Pulmonology, allergy	9,726	27	360.22	4	Acute abdominal pain	5,914	31	190.77
5	Infectious diseases	9,015	39	231.15	5	Weakness, paralysis	5,082	19	267.47
6	Neurology	5,708	18	317.11	6	Chest pain	4,905	15	327.00
7	Endocrinology	5,473	24	228.04	7	Headache	3,270	13	251.54
8	Cardiovascular	5,160	16	322.50	8	Abdominal mass	3,190	20	159.50
9	Pediatrics	4,633	22	210.59	9	Loss of consciousness, coma	2,796	10	279.60
10	Musculoskeletal	4,623	20	231.15	10	Chronic abdominal pain, dyspepsia, heartburn	2,455	12	204.58
11	Nephrology, urinary system	3,978	16	248.63	11	Nausea, vomiting	2,170	9	241.11
12	Hematology	3,229	14	230.64	12	Vaginal bleeding	2,164	19	113.89
13	Emergency medicine	2,070	10	207.00	13	Dizziness, vertigo	2,061	8	257.63
14	Otorhinolaryngology	1,528	6	254.67	14	Joint pain, joint swelling	2,057	5	411.40
15	Psychiatry	849	3	283.00	15	Dysphagia	1,960	3	653.33
16	Dermatology	247	1	247.00	…	…
17	Ophthalmology	194	1	194.00	80	Contraception	63	1	63.00
	Total	105,638	434	243.41		Total	105,638	434	243.41

Table 4 illustrates usage statistics on online videos. Of all video clips in e-MedEdu available, 65% (n=285) were on procedural skills, 19% (n=80) were on patient encounters, and 16% (n=68) were video lectures on various topics, including basic and clinical sciences and medical humanities. Among the online videos, those on procedural skills received the greatest hits for the duration of 7 years, where an average number of hits per video clip was 4,163.26. This was followed by video clips on patient encounters (1,043.47 hits per clip). The number of hits on video clips differed across subjects; they ranged from 2,505.00 to 6,717.86 across different subjects on procedural skills, and the number of hits on videos on patient encounter ranged from 681.89 to 1,594.50.

Table 4.

Usage Statistics of Clinical Cases at e-MedEdu

No.	Videos on procedural skills				Videos on patient encounter
	Category	No. of cumulative hits	No. of video clips	No. of hits per clip	Category	No. of cumulative hits	No. of video clips	No. of hits per clip
1	Physical examination	323,098	89	3,630.31	General symptoms	26,400	24	1,100.00
2	Vascular procedures	201,051	37	5,433.81	Respiratory and cardiovascular symptoms	17,961	13	1,381.62
3	Surgical procedures	172,486	39	4,422.72	Psychological, neurological, and musculoskeletal symptoms	14,884	21	708.76
4	Emergent procedures	143,277	35	4,093.63	Gastrointestinal symptoms	12,990	14	927.86
5	Miscellaneous procedures	119,882	34	3,525.94	Difficult communication	10,408	7	1,486.86
6	Neurologic examination	103,721	28	3,704.32	Women health	6,378	4	1,594.50
7	Obstetric and gynecologic procedures	68,475	13	5,267.31	Others	6,137	9	681.89
8	Urologic procedures	47,025	7	6,717.86	Genitourinary symptoms	4,151	4	1,037.75
9	Mental status examination	7,515	3	2,505.00	-	-	-	-
	Total	1,186,530	285	4,163.26	Total	99,309	96	1,034.47

Discussion

Our study demonstrates e-learning is being increasingly used for medical education in South Korea. We have seen that e-learning has increasingly become mainstream in the teaching and learning in medical education over the past decade. Our experience shows e-learning in this context is feasible, given collaboration among medical schools to develop strategic plans for e-learning and implement them. We believe the Korean e-learning consortium has demonstrated a successful model for implementing e-learning in medical education at the national level.

One of reasons for the increasing use of e-learning found in the present study was the consistent efforts made by medical educators to develop learning resources that meet the curricular needs in basic medical education. Due to the broad range of subjects that must be covered in medical education, it is a considerable challenge to develop resources on all subjects. In terms of the on-going development of e-MedEdu, we have matched existing contents with learning outcomes for basic medical education. In doing so, we were able to recognize that there were some clinical presentation topics in which clinical cases were not available. As a result of this process, we have built up a repository of contents that covers most of the basic learning outcomes in terms of clinical presentations and clinical skills. Having over 30 faculty members from various disciplines in the executive committee of the consortium and a large volume of medical teachers who have contributed contents to e-MedEdu, we were able to draw on these members for support and identify experts in subject areas on a nationwide basis. Expanding collaboration with faculties with expertise in other diverse disciplines will further develop our ability to promote advances in e-learning.

This study also shows the popularity of learning resources offered by e-MedEdu differs substantially across resources types. Our study illustrates the current popularity of online videos as learning resources among medical students. This finding concurs with that of Khogali et al. [13] that students valued animation and video demonstrations highest among e-learning resources. It is likely that the prevalent use of online videos continues in the foreseeable future as they play an important role in clinical education. It is also likely that the creation and use of educational videos becomes more widespread in medical education with the increasing popularity of social media. This trend may emulate the learning styles of today’s medical students, who are millennial learners living and learning with technologies [14].

Moreover, the effectiveness of using technology to enhance the learning of clinical skills using video clips appears to have motivated students to engage in e-learning. Our previous study showed medical student perceptions of clinical videos available at e-MedEdu that those resources were helpful in supplementing conventional face-to-face instruction on clinical skills [15]. This finding is also consistent with those of previous studies that showed student satisfaction with clinical videos for the learning of clinical skills [16,17]. This rise in popularity of clinical videos is also in large part due to the implementation of clinical skills test in the Korean licensing exam since 2009, which illustrates the medical student needs for e-learning that meet their curricular needs.

Despite the importance of clinical cases in medical education, our findings show they are not being widely used at e-MedEdu. This finding indicates our clinical cases need to be better integrated with medical school curriculum, which warrants more active use of these resources as teaching materials on the part of the faculty. In the previous study, we found the low use of learning resources at e-MedEdu by faculty members for their teaching and the main reasons were lack of awareness of these resources and of pedagogical understanding of their utilities [15]. The consortium may need to enhance its efforts to identify best practices in the use of clinical cases by faculty members and to disseminate them so that they make active and effective use of learning resources in their teaching.

Furthermore, our finding of the low use of clinical cases by medical students may suggest they are not suited for the learning styles of today’s students as well as online videos are. This indicates we need to make clinical cases more interactive and engaging for the students. This warrants future research on the design of clinical cases. There are various formats for presenting clinical cases, from a simple text-based format to promoting interaction with the virtual patient in a computer-based simulation environment [18]. It is also plausible to present patient encounters using virtual and augmented reality, which are likely to become a prevalent technology in the near future [19] and are increasingly garnering the attention of medical educators [20]. Still, research is scant on such learning environments. Future research is warranted on more effective design of clinical cases to investigate whether they are used more widely by medical students and how to design them to better promote students’ clinical reasoning skills.

Our study also found differences in the popularity of learning resources across subjects. Among online videos, clips on procedural skills were more frequently viewed by users than those on patient encounters and video lectures. Moreover, there were also differences in the frequency of use of online videos on procedural skills across content domains. Such differences might reflect the learning needs for medical students to supplement conventional instructions on clinical skills particularly on those not readily available for direct observation or those that demonstrate somewhat complex procedural skills. There were also differences in the frequency of use in clinical cases, and there might be several reasons for that. Some cases likely fit more closely with the core curriculum and some might be beyond the scope of basic medical education.

Our study has implications for future directions of e-learning in Korean medical education. First, there is a need to build up more multimedia resources in response to the trend of incorporation of multimedia items for student assessment to enhance its authenticity [21,22]. Our efforts to build up resources have been active on clinical cases and online videos, yet it has been slow in other multimedia resources such as images and sound clips. Second, our findings show our resource development has focused on the core curriculum pertinent to learning outcomes for basic medical education. There is a growing emphasis on “student-selected components” (SSCs) in medical school curricular to promote student-centered learning by offering subjects of more choice and depth of study than the core curriculum [23]. Therefore, there is a need to expand our resources to support such curricular components. Furthermore, e-learning has the potential to offer an effective learning environment for SSCs by offering diverse learning resources and the flexibility for the learning of such components. The inter-institutional collaboration model of the e-learning consortium offers valuable experience in such collaborative efforts to provide learning resources that cover a broad range of subjects for SSCs.

This study has some limitations. First, this study was limited to the format of e-leaning which offer online learning resources through an e-learning portal. As there are various formats of e-learning, such as online courses, and various forms of learning resources are available in medical education, cautions need to be taken in generalizing our findings to other e-learning contexts. Second, although this study found the use of learning resources differed across resource types and subjects, it does not offer direct evidence as to what caused such differences. This study calls for future research to explore what causes such differences to better understand medical students’ use of learning resources in their learning. Such a study will help us advance of our knowledge of more effective use of e-learning that meets the learning needs of today’s students and help improve medical education by supplementing face-to-face instructions. Third, our data was not sufficient to explore students’ activities at our website associated with their academic performance. Future study of learner analytics regarding differences in the use of resources at e-MedEdu between high-achieving and low-achieving students is recommended to advance our understanding of effective use of e-learning.