Developing the Korean Association of Medical Colleges graduate outcomes of basic medical education based on "the role of Korean doctor, 2014".

The Korean Association of Medical Colleges (KAMC) developed graduate outcomes based on "The role of Korean doctor, 2014" to serve as guidelines regarding outcome-based education in Korea. The working group in this study analyzed 65 competencies proposed in "The role of Korean doctor, 2014" according to the developmental principle that certain outcomes should be demonstrated at the point of entry into the graduate medical education. We established 34 competencies as "preliminary graduate outcomes" (PGOs). The advisory committee consisted of 11 professors, who reviewed the validity of PGOs. Ultimately, a total of 19 "revised graduate outcomes" (RGOs) were selected. We modified the RGOs based on opinions from medical schools and a public hearing. In November 2017, the KAMC announced the "graduate outcomes for basic medical education," which serves as a guide for basic medical education for the 40 medical schools throughout Korea. Medical schools can expand the graduate outcomes according to their educational goals and modify them according to their own context. We believe that graduate outcomes can be a starting point for connecting basic medical education to graduate medical education.

Introduction

The purpose of medical education is to equip physicians with competencies that will satisfy the reasonable expectations of patients and society. Recently, emphasis has been placed on outcome-based medical education (OBME), or competency-based medical education, which defines outcomes that students must achieve, the faculty designated to teach the material, how it must be taught, and how to evaluate what has been taught [1]. In fact, OBME has quickly spread throughout the United States and Canada, as well as England and other European countries, since 1980, and it has provided medical schools with an impetus to improve their educational methodology [2]. The General Medical Council in the United Kingdom provides guidance to medical practitioners through its standard entitled “Good Medical Practice,” which describes what it means to be a good doctor and defines outcomes that medical students must be able to demonstrate in order to graduate [3,4]. The Accreditation Council for Graduate Medical Education (ACGME) and the American Board of Medical Specialties in the United States proposed six core competencies that residents should achieve during their medical training [5]. The Association of American Medical Colleges developed the core Entrustable Professional Activities (EPAs) for Entering Residency in 2014 [6]. Canada also analyzed the competencies that doctors must acquire to meet the needs of society, and conceived the Canadian Medical Education Directions for Specialists framework (CanMEDS framework; 2005, 2015). The CanMEDS framework is being used in many other countries as a foundation for OBME curriculum development [7].

In Asia, the National Medical Undergraduate Curriculum Committee in Singapore presented the “Outcomes and standards for undergraduate medical education in Singapore” in 2014. The report contains long-term directions for the healthcare workforce, as well as outcomes and standards for medical education [8]. China also recognized that its medical education system needed to be improved innovatively to keep up with the rapid changes in the healthcare environment. Zhao et al. [9] conducted a large-scale survey to identify the role of Chinese physicians in the Chinese healthcare environment; their findings led to elucidation of a framework based on six competencies. Lio et al. [10] also proposed competencies for residency training programs.

In 2012, the Korean Institute for Medical Education and Evaluation introduced “Post-2nd cycle accreditation standards” for basic medical education in South Korea; these standards emphasize OBME. The announcement about the accreditation standards led to discussions on the development and implementation of OBME curricula [11]. Research on the roles of Korean doctors began in the early 2000s [12-14]. Many studies have reported on the roles of future physicians, but there has been a lack of research on the competencies that future doctors must demonstrate. There has also not been any agreement on a competency framework at the national level. Therefore, medical schools have begun defining competencies that their graduates should achieve during basic medical training. It should be noted that OBME in Korea is approached differently than it is in the United States, England, Canada, and other countries. Ahn et al. [15] conducted a study on the future roles of Korean doctors with funding from the Ministry of Health and Welfare. He and his colleagues proposed five domains (patient care, professionalism, social accountability, communication and collaboration, and education and research) and 65 competencies for Korean doctors. In 2014, the Council for Medical Education in Korea—the consultative committee of 13 organizations related to medical education (the Korean Academy of Medical Sciences, the Korean Society of Medical Education, the Korean Medical Association, the Korean Institute of Medical Education and Evaluation, the Korean Association of Medical Colleges [KAMC], the Korean Hospital Association, the Korean Medical Practitioners Association, the Korean Association of Basic Medical Scientists, the Medical Professors Association of Korea, the National Teacher Training Center for Health Personnel, etc.)—ratified that Korean doctors should have 65 competencies in five domains, in “The role of Korean doctor, 2014” (Appendix 1).

“The role of Korean doctor, 2014” is a proclamatory description of a doctor’s virtues and roles that professional community and our societies have agreed to and accepted [15,16]. There are difficulties in applying this standard to the basic medical education phase. Yang et al. [17] analyzed the applicability of 65 competencies stipulated by “The role of Korean doctor, 2014” to the curricula of various medical schools. They also proposed the need for the development of graduate outcomes of basic medical education to be used by medical schools based on “The role of Korean doctor, 2014” and reported that 38 of 40 medical schools insisted on being provided with the curriculum developer’s guidelines for OBME. Ultimately, the KAMC developed the “Graduate outcomes for basic medical education” to provide medical colleges a guidance which can implement OBME in Korea.

Development of graduate outcomes for basic medical education

1. Process and principles for graduate outcomes development

The Education Committee of the KAMC formed a working group to develop graduate outcomes of basic medical education based on “The role of Korean doctor, 2014.” Members of the working group for the project on graduate outcomes development (WGGOD) were selected based on their majors. Professors who took part in developing “The role of Korean doctor, 2014” and experts who were experienced in developing OBME curricula participated in the group. We did not include other stakeholders in the working group because a lot of stakeholders, such as students, citizens, general practitioners, and other organizations, were included in the process of developing “The role of Korean doctor, 2014.” The WGGOD was active between April 2016 and November 2017 in developing graduate outcomes of basic medical education. To establish a common foundation for members in development of the outcomes, the WGGOD studied changes in the healthcare environment, issues related to OBME development, and cases of the ACGME competency framework of the United States, CanMEDS 2015 physician competency framework of Canada, and the learning outcomes framework for Scottish doctors. As Harden [18] mentioned, we found that the level of outcome emphasize a broad overview with a design-down approach to a more detailed competency. Also in terms of classification and interrelationship of outcomes, it is structured in the form of learning outcomes with nesting of outcomes, knowledge embedded and meta-competences recognized. The WGGOD also reviewed the appropriateness of the 65 competencies stipulated in “The role of Korean doctor, 2014” as graduate outcomes of basic medical education. The review process involved group discussions and consensus. The definition, level and classification of defined in “The role of Korean doctor, 2014” were not much different from foreign outcome framework. Fig. 1 describes the WGGOD’s process for developing the outcomes. The following are principles applied during the review process. First, the graduate outcomes of basic medical education were based on the 65 competencies stipulated in “The role of Korean doctor, 2014.” Second, graduate outcomes for basic medical education should be demonstrated upon entering a graduate medical education program. Third, graduate outcomes of basic medical education are common and core outcomes that can be used by all medical schools. Medical schools should be able to expand graduate outcomes according to their own educational goals and objectives. Fourth, graduate outcomes of basic medical education should have an association with the learning outcomes developed by the KAMC (e.g., the KAMC published in several books, guides to clinical performance, guides to basic clinical skills, learning outcomes of basic medical education: scientific concept- and principle-centered outcomes, learning outcomes of basic medical education: clinical competency-centered outcomes, and learning outcomes of basic medical education: human- and society-centered outcomes). Fifth, graduate outcomes of basic medical education include competencies required by doctors in the future. Sixth, they reflect the feasibility of education and assessment of competencies in medical schools.

Fig. 1.

Working Flows for Graduate Outcomes Development

2. Development of preliminary graduate outcomes

Each of the 65 competencies proposed in “The role of Korean doctor, 2014” was analyzed according to the developmental principles for the graduate outcomes of basic medical education mentioned previously. We classified the competencies as follows: (1) competencies that should be completed during the basic medical education phase (category I); (2) competencies that begin to develop during the basic medical education phase and should be attained during the graduate medical education phase (category II); and (3) competencies that begin to develop and which should be completed during the graduate medical education phase (category III). We excluded category III from the graduate outcomes for basic medical education, according to the principle that such outcomes should be achieved upon entering a graduate medical education program. Categories I and II were adjusted and integrated to fit the basic medical education phase. We identified 34 competencies as preliminary graduate outcomes (PGOs). Table 1 shows the division by domain for the PGOs.

Table 1.

Results of Preliminary Graduate Outcomes Development

Domain	No. of competency		Deleted items[a)] in the role of Korean doctor, 2014
	The role of Korean doctor, 2014	Preliminary graduate outcomes
Patient care	15	9	6, 7, 8, 9, 10, 11, 12, 14
Communication and collaboration	17	5	18, 19, 20, 21, 23, 26, 27, 28, 29, 30, 31, 32
Social accountability	11	5	35, 42, 43
Professionalism	15	8	44, 45, 47, 53, 54, 56, 57, 58
Education and research	7	-	63
Self-improvement	-	5
Knowledge application	-	2
Sum	65	34

Of the 65 competencies in “The role of Korean doctor, 2014,” category III items that begin at the postgraduate training stage and competencies that could be integrated with others were deleted. The competency corresponding to the number of deleted items can be found in Appendix 1.

3. Validation of preliminary graduate outcomes by professionals

The advisory committee for the WGGOD included 11 professors from medical colleges who reviewed the validity of the PGOs. This committee consisted of medical educators from the fields of internal medicine, surgery, obstetrics and gynecology, pediatrics, psychiatry, and family medicine, and they have been in charge of medical education and residency training in their colleges. They were selected based on their expertise in the field of medical education and activities related to OBME. Regarding the 34 PGOs, we asked for the advisory committee’s written opinion on the following: (1) the appropriateness of PGOs, (2) the validity of achievement levels, (3) exclusiveness in terms of other competencies, (4) clarity of the descriptions, and (5) feasibility of education and assessment. An offline discussion with the advisory committee was held on April 10, 2017, and the following conclusions were formulated. First, there were some overlaps among the 34 PGOs developed by the WGGOD, indicating that they had to be reviewed and configured as mutually exclusive items. Second, to increase the feasibility of education and assessment at medical schools, the number of PGOs should be reduced. Third, because achievement of some PGOs could not be realized at the point when a student completes basic medical education, the achievement level of the PGOs should be adjusted for students entering a graduate medical education institution. The WGGOD revised the graduate outcomes of basic medical education based on the consensus of the advisory committee and developed 19 outcomes, as shown in Table 2.

Table 2.

The Number of Revised Graduate Outcomes Developed by the Advisory Committee

Domain	The role of Korean doctor, 2014	Preliminary graduate outcomes	Revised graduate outcomes
Patient care	15	9	8
Communication and collaboration	17	5	3
Social accountability	11	5	2
Professionalism	15	8	3
Education and research	7	-	-
Self-improvement	-	5	-
Knowledge application	-	2	3
Sum	65	34	19

4. Hearing and finalization for revised graduate outcomes

The WGGOD requested a review of the appropriateness of the revised graduate outcomes (RGOs) from 40 medical schools to verify their acceptability for the basic medical education phase. In May 2017, we held a public hearing for faculty members who were recommended by medical schools. The following is a summary of the opinions of medical schools and the results of the public hearing. First, the RGOs of basic medical education that have been developed were appropriate and are very likely to be accepted by medical schools. Second, application of the RGOs should not be imposed on all medical schools. Additionally, medical schools should be able to expand or modify the RGOS according to their own educational goals and objectives. Third, it is important to develop assessment tools for evaluating the RGOs to increase the feasibility of their implementation in medical schools. Fourth, the context in which the RGOs are situated needs to be addressed. The WGGOD developed the RGOs based on reviews by medical schools and opinions expressed during the public hearing. In November 2017, the 19 graduate outcomes of basic medical education were approved by the KAMC board of directors and officially announced. Table 3 shows the final graduate outcomes of basic medical education according to the five domains of patient care, application of knowledge, communication and collaboration, social accountability, and professionalism.

Table 3.

Final Graduate Outcomes of Basic Medical Education Developed by the Korean Association of Medical Colleges

Domain	Graduate outcomes
Patient care	1.	The graduate is able to obtain the patient's medical history accurately in an effort to solve his/her problems.
	2.	The graduate is able to perform a physical examination competently in a given medical interview situation.
	3.	The graduate is able to perform basic clinical skills and procedures necessary for patient care.
	4.	The graduate is able to choose diagnostic and screening tests required to solve the patient's problems and interpret the results.
	5.	The graduate is able to carry out reasonable diagnostic inferences based on patient information and scientific evidence.
	6.	The graduate is able to plan and manage patient-centered care based on the proper medical evidence.
	7.	The graduate is able to maintain accurate medical records and make the best use of them.
	8.	The graduate should recognize prevention and handling strategies for patient safety incidents and conduct timely intervention.
Knowledge application	9.	The graduate is expected to acquire scientific knowledge and apply it to solve medical problems.
	10.	The graduate is able to utilize the latest medical research and scientific methods needed to solve patients' problems.
	11.	The graduate should learn various research methodologies pertaining to medicine and design medical research studies according to ethics and guidelines research.
Communication and collaboration	12.	The graduate is able to empathize with the patient's perspective and communicate effectively.
	13.	The graduate can identify challenging situations, such as those requiring the delivery of bad news or difficulty with communication, and is able to adopt effective strategies in such situations.
	14.	The graduate is able to communicate and collaborate with colleagues participating in medical practices.
Social accountability	15.	The graduate can identify the health issues of the community or population and make plans for improvement.
	16.	The graduate should understand the healthcare policy and should be able to suggest response strategies according to the healthcare paradigm for the future.
Professionalism	17.	The graduate should understand the principles of bioethics and medical ethics and be able to apply them in medical practice.
	18.	The graduate should internalize honesty, sincerity, altruism, humility, respect for others, and empathy as core values of his/her profession and should be able to demonstrate them in daily practices.
	19.	The graduate should reflect his/her own competencies and develop the ability to be a lifelong learner.

The final graduate outcomes for basic medical education were proposed by outcome name, context of performance (context in which the outcome is performed), achievement level (required achievement level at the basic medical education phase), and relevance (how it relates to “The role of Korean doctor, 2014”). Table 4 shows the final graduate outcomes of basic medical education and a sample format.

Table 4.

Sample Format for Presenting Graduate Outcomes

Domain: patient care	Description
Graduate outcome 1	1.	The graduate is able to obtain a patient's medical history accurately to solve his/her problems.
Context	A process for collecting information and interacting with patients is the foundation for understanding the patient and managing his/her care. The physician must be able to prioritize problems and obtain an accurate medical history using a systematic approach. When obtaining patients' medical histories, doctors should consider certain clinical situations and the individuation of patients to ensure a patient-centered approach. Doctors should obtain appropriate medical histories according to general practice situations, emergency situations, patient symptoms, acute or chronic disease, and so on.
Achievement criterion	Medical graduates are able to identify the priorities of the problem and obtain medical history accurately and systematically according to a patient-centered approach.
Relation to future role of the Korean doctor	1.	Doctors shall possess professional medical knowledge and appropriate clinical skills.
	2.	Doctors shall maintain a patient-centered approach when assessing, diagnosing,and treating patients and be capable of delivering accurate medical judgments and appropriate clinical decisions.
	3.	Patient care shall be based on scientific evidence and the individual nature of each patient.

Discussion

The development of graduate outcomes of basic medical education started with the need to establish core competencies to be cultivated by medical schools based on “The role of Korean doctor, 2014.” The KAMC developed the “Graduate outcomes for basic medical education” as a guide for basic medical education for the 40 medical schools throughout Korea. Notwithstanding, there are still several issues that we should consider in relation to developing and applying these outcomes.

First, there are differences in the number of outcomes that are included in five domains. There is the possibility of misunderstanding the importance of each outcome because of these differences. The time durations required for teaching and learning each outcome are also different. Additionally, some outcomes can be acquired within a certain period, but others require continuous attention throughout the basic medical education curriculum. It is, therefore, not appropriate to consider all outcomes as equal. The second issue relates to the achievement levels of graduate outcomes of basic medical education. To elaborate, there are debates on whether the specified outcome to be achieved should describe the ultimate achievement level expected of doctors, or if it should describe the achievement level that medical students are expected to demonstrate at the point of entry into the graduate medical education program. It is best to describe the achievement level of medical students after specifying the ultimate achievement level expected of doctors upon completing the medical education continuum. Regarding the achievement of graduate outcomes, it is important that stakeholders agree on the transition between basic medical education and graduate medical education. In this project, such agreements were not made; therefore, we have defined the achievement level expected of medical students at the point of entry into the graduate medical education program. The purpose of developing graduate outcomes of basic medical education is to reinforce OBME in medical schools. We hope that follow-up research will address these issues.

The third issue is how to measure and assess whether medical students are achieving the graduate outcomes of basic medical education. The key to the success of OBME is developing valid and reliable assessment tools to verify the achievements of students. The assessment will not only be a criterion for determining whether individual students have achieved the outcomes, but will also drive learning and provide information on the effectiveness of medical education [19-21]. Many medical schools have petitioned that it is difficult to develop valid assessment tools [17]. In addition, medical schools must determine how to assess not only visible outcomes but also invisible outcomes when designing OBME curricula [19]. For further study, we are left with the task of developing assessment tools and studying examples to measure students’ achievements of the graduate outcomes of basic medical education.

The “Graduate outcomes for basic medical education” is a set of guidelines for curriculum development in medical schools. However, the intention is not to imply that the outcomes must be adopted and implemented as graduate outcomes for all medical schools. Medical schools can expand the graduate outcomes according to their educational goals and modify them according to their own context. The competencies presented in “The role of Korean doctor, 2014” are expectations regarding the roles of doctors in our society. The graduate outcomes developed by the KAMC are core outcomes that medical students should achieve at the point of entry into the graduate medical education. We know that it is important to have a continuum from graduate outcomes of basic medical education to outcomes in residency training phase. Now the Korean Academy of Medical Sciences, which is in charge of the residency training, is working on the transformation of residency training into competency-based medical education based on “The role of Korean doctor, 2014.” However, we were not able to carry out these tasks at the same time because it needed more time for consensus and coordination. We believe that graduate outcomes of basic medical education can be a starting point for linking outcomes of residency training program. In this context, the KAMC is working on a project to develop EPAs for entering intern and residency program to ensure a successful transition from the basic medical education phase to the graduate medical education phase in Korea.