Comparison of competency level of medical, non-medical students and its relevance for admission policy.

PURPOSE: This study aims to analyze the competency of medical students and its relevance for admission policy in medical schools.
METHODS: This study examined the competency of 63 medical students from the 6-year program (group A) and 41 medical students from the 4-year program (group B) at Yonsei University using the Korea Collegiate Essential Skills Assessment (KCESA). The competency of groups A and B were compared to the corresponding competency levels of non-medical students (groups C and D). Group C is freshmen and D is senior students in universities. The KCESA is computer-based ability test composed of 228 items. The competency of participants were calculated on a T-scores (mean=50, standard deviation=10) based on KCESA norm-references. We conducted independent t-test for group comparisons of competency levels.
RESULTS: There are no differences in competency levels between groups A and B. Compared with the non-medical students (group B), the medical students showed a significantly stronger ability to use resources, information-technology and higher-order thinking. In the comparison between groups B and D, medical students showed lower levels of self-management, interpersonal, and cooperative skills.
CONCLUSION: The cognitive ability serves as an important indicator for the decision on admission to a basic medical education program. The efforts should be made to foster the competency that medical students have been found to lack, such as self-management, interpersonal, and cooperative skills. The admission committee should assess the cognitive and non-cognitive competency of applicants in a balanced manner.

Introduction

Medical schools are increasingly giving attention to competency-based education (CBE) [1]. According to literature on curriculum development, CBE is a preferred educational model in medical education [2]. As regards the concept of competency, several researchers have presented different definitions [3-7]. A shared sense is that competency relates to the underlying characteristics of an individual that is causally linked to superior performance in a job [6]. In the medical sphere, CBE has gained increasing attention as emphasis is being placed on graduates’ practical competencies, and because the social responsibility of educational institutions has emerged as an important issue [8]. In the United States, the Accreditation Council for Graduate Medical Education initiated its core competency project in 1999 to identify competencies that residents should be equipped with during the course of graduate medical education. Similar efforts have been implemented in Canada through the CanMEDS 2005 Project, which aimed to establish a CBE model. Other examples of efforts to identify the required components of CBE include the Tuning Project 2000, funded by the European Commission, and Tomorrow’s Doctors, which was published in 2009 by Britain’s General Medical Council. These projects have repeatedly indicated that doctors’ competencies include communication skills, interpersonal skills, ability to collaborate or work in teams, critical thinking, information management, clinical skills, patient care, and self-directed learning.

One central issue in competency-based medical education is the selection of candidates most likely to possess these competencies. Typically, admission criteria serve to select applicants most likely to succeed in the respective undergraduate medical program and become upstanding members of the medical profession [9]. Based on several years of study, the Association for American Medical Colleges defined the following nine competencies that should be considered in admission policy: ethical responsibility to self and others, reliability and dependability, service orientation, social skills, capacity for improvement, resilience and adaptability, cultural competence, oral communication, and teamwork [10]. Thus far, the primary selection criteria used in most of these models are cognitive in nature, and include traditional academic measures, such as undergraduate grade point average (GPA) and medical college admission test scores [11]. Although past academic achievement can be effective variables for predicting academic progress through medical school [12], students’ academic excellence in cognitive domains does not necessarily translate to clinical excellence in the field [13,14]. Therefore, determining admission based on cognitive competencies does students a disservice, as potentially excellent candidates might be eliminated based on the assessment of arguably irrelevant competencies.

It takes more than academic ability to become a good doctor, and thus, selection criteria that assess important generic attributes need to be established [15]. Koenig et al. [16] noted that of the competencies necessary to be a doctor, prospective students’ internal traits should be screened prior to acceptance into medical school. In explaining the structure of doctors’ core competencies, Niven [17] pointed out that an individual’s underlying characteristics, such as beliefs, motivation, values, and personality, are difficult to foster through education. Kulatunga-Moruzi and Norman [11] also argued that medical students’ integrity, leadership ability, communication skills, work ethic, and orientation towards service should be seriously considered during the admission process because it is difficult to teach these skills and qualities. Although there are tools to assess these competencies, certain traits, such as integrity, belief, and values, are not easy to assess during the admission process. Continuing with this line of thought, Harden [18] asserted that institutions should not only assess if applicants possess the necessary competencies at the admission process but also try to assist students in cultivating these competencies during their study through curriculum design.

In South Korea, there has been significant debate regarding undergraduate medical education since 2004. The focus has been on changing the medical education system (from a 6-year medical education program after high school to a 4-year program after undergraduate studies) that cultivates medical professionals. As of 2016, the country had 41 medical education institutions, of which 16 offer the 4-year program type, 14 retain the 6-year program, and 11 provide both. In 2012, the accreditation standard required the identification of competencies medical students must have and the admission policy of the medical school that takes such competencies into account [19]. This change in accreditation standard has driven the development of CBE curriculum and influenced dramatically the admission policy of medical schools. Attention is now given to competency and subjective factors, such as personality traits and behaviours needed be a ‘good doctor’ in the admission policy. Although there are considerable efforts in South Korea to define the required competencies for medical students, no national consensus has been reached on the specific competencies that applicants should have and that should be evaluated in the admission process. Moreover, no objective assessment method has been developed to screen for and evaluate the said competencies. Consequently, the students selected by medical schools tend to be those who obtain high scores on the competencies that can be measured objectively. Powis [20] mentioned that an expert’s subjective judgment on non-cognitive qualities does not necessarily imply a decline in objectivity. Nevertheless, the Ministry of Education of Korea [21] declared that all medical schools should evaluate applicants’ competency based on quantitative data to guarantee objectivity in admission. This rule implies that medical schools have limited autonomy in evaluating applicants in terms of whether they have the personality traits or characteristics required to be a good doctor.

The 6- and 4-year medical education programs follow different admission policies. In the former, new students are selected for admission based on their high school transcript and Korea Scholastic Aptitude Test score (KSAT, similar to the scholastic aptitude test used in the United States), which are purely objective materials. Students are required to submit recommendation letters and take an interview, but these are deemed less important variables than academic performance indicators. Meanwhile, students in the 4-year program are selected based on their university GPA, medical education eligibility test score (similar to the medical college admission test used in the United States), recommendation letters, personal statement, and interview. In this admission process, evaluators consider objective as well as subjective materials, such as letters and interview, to predict the likelihood of a student to succeed in basic medical education and become a good doctor.

The present study aims to identify the characteristics of competency of medical students in Korea and investigate its relevance for admission policy. Thus, this work compares the competency of first-year medical students in both the 6- and 4-year programs, as students in these two tracks are accepted using different admissions processes and materials. The research also analyses the differences in the competencies possessed by medical and non-medical students. Based the results, the nature and types of competencies possessed by medical students and its relevance for admission policy are discussed.

Methods

Study participants included 63 students (95.5% of 66) from the 6-year medical education program (group A) and 41 students (74.5% of 55) from the 4-year medical education program (group B) accepted at Yonsei University, which provides both program types. There are 83 male students (group A, n=50; group B, n=33) and 21 female students (group A, n=13; group B, n=8). Group A is composed of first-year medical students who began their program after high school; the program consists of 2 years of premedical education and 4 years of basic medical education. They are equivalent to university freshmen. Group B is composed of first-year medical students who had completed a bachelor’s degree in university; the program consists of 4 years of basic medical education. The competency of the two groups were also compared to the corresponding competency levels of non-medical students in universities nationwide (group C, 1,000; group D, 1,000), majoring in liberal arts and social science, natural science, engineering, and others. Group C is composed of university freshmen comparable to group A. Group D members are senior students in a university comparable to group B. The admission policy and process of groups C and D are similar to those of the 6-year medical education program.

Participants’ competencies of groups A and B were evaluated using the Korea Collegiate Essential Skills Assessment (KCESA, equivalent to the Graduate Skills Assessment in Australia and Collegiate Learning Assessment or Measure of Academic Proficiency and Progress in the United States). To compare the competency between medical and non-medical students, which may inform the relationship between competency and admission policy, it is desirable to use the KCESA inventory developed in the context of Korea. KCESA was jointly developed by the Korea Ministry of Education and the Korea Research Institute for Vocational Education and Training (KRIVET). KCESA is a standardized inventory that assesses college students’ core competencies. This inventory was developed by the Delphi process to measure the level of knowledge, skills, and attitudes commonly required in various job markets in fiscal years 2006–2009. KCESA’s criterion validity was established through consistency with other assessment tasks, whereas construct validity was determined through confirmatory factor analysis of 813 college students. After then, a total of 48,000 college students participated in the KCESA test by 2011, and over 20,000 students participate in the test every year. This inventory is a standardized tool developed by assuming 4-year college students in Korea [22].

As presented in Table 1, the KCESA is designed to assess six core competencies. The KCESA is a computerbased ability test composed of 228 items including of 110 multiple-choice questions, eight essay items, and 110 items rated on a 5-point Likert scale. All participants (groups A and B) completed the test in a computer lab in Yonsei University, and T-scores (mean scores=50, standard deviation=10) of individuals were calculated based on the mean and standard deviation of the each group. The data for groups C and D is norm-referenced score given by the KRIVET. The score is calculated by sampling 1,000 students in each grade of university considering gender and university location. Therefore, it can be said that the higher the average score of the individual or group is, the higher the competency level is. We conducted independent t-test for group comparison of competency levels. The p-value for the statistical test set to <0.008 (=0.05/6, Bonferroni correction) considering six components of the KCESA. The assumption of homogeneity of variances among groups was met. Data were statistically analyzed using PASW SPSS ver. 18.0 (SPSS Inc., Chicago, USA). Ethical approval was obtained from the Severance Hospital Institutional Review Board (4-2011-0006). Informed consent was obtained from the participants according to the guideline and form of the Institutional Review Board.

Table 1.

Composition of the Korea Collegiate Essential Skills Assessment Inventory

Competency	Definition	No. of items	Time (min)	Type
Resources-information-technology	Competency involving collecting, analyzing, and using resources, information, and technology	30	45	Multiple-choice question
Global skills	International sensibility and an international worldview	48	30	Multiple-choice question
Communication skills	Competency involving understanding, creating, and transmitting a message appropriate for the situation, using communication skills and knowledge	32	80	Multiple-choice question
Higher-order thinking	Competency involving solving a problem by diagnosing the situation and using logical reasoning and problem solving to analyze clearly the problem situation through higher-order thinking skills, and offering a valid solution from among various ideas	8	90	Essay
Self-management	Self-managing competency involving motivation, cognitive strategies, stress management, planning ability, and profession’s ethic, etc.	60	Unlimited	Five-point Likert scale
Interpersonal and cooperative	Competency involving cooperating with others and acting appropriately to achieve goals in social situations and maintaining positive relations with a variety of people	50	Unlimited	Five-point Likert scale

Results

1. Competency levels between groups A and B

Table 2 presents the core competency levels of the medical students. No significant differences in core competency levels were observed between groups A and B. As regards the six core competencies, both groups obtained above-average scores of t-distribution on items pertaining to the ability to use resources, information, and technology; communication skills; and higher-order thinking. In contrast, group B obtained below-average t-scores on items pertaining to self-management and interpersonal and cooperative skills. In terms of global competency, group A obtained above-average scores of t-distribution, whereas group B obtained below-average scores. Nonetheless, both groups are regarded as being within the normal range on that dimension because t-scores are within one standard deviation above or below the mean reference value.

Table 2.

Comparison of Competency T-Scores between Groups A and B

Competency	Group	No. of students	Mean±SD	t-value[*]	p-value
Resources-information-technology	A	62	55.51±5.51	0.23	0.8185
	B	40	56.11±6.11
Global skills	A	63	50.92±9.04	-0.62	0.5358
	B	40	49.74±9.74
Communication skills	A	34	50.55±0.55	1.11	0.2737
	B	16	54.41±4.41
Higher-order thinking	A	50	53.93±3.93	0.66	0.5140
	B	31	55.39±5.39
Self-management	A	63	47.51±11.02	-0.02	0.9804
	B	40	47.46±7.46
Interpersonal and cooperative	A	62	48.04±8.04	-0.29	0.7712
	B	40	47.43±7.43

Group A: 6-year basic medical education program after high school. Group B: 4-year basic medical education program after a bachelor’s degree.

SD: Standard deviation.

p<0.008 (=0.05/6, Bonferroni correction: significance level 0.05 is divided by six competencies of the Korea Collegiate Essential Skills Assessment).

2. Comparison of specific competencies between medical and non-medical students

Table 3 presents comparisons of the competency levels of medical and non-medical students (groups A and C, B and D). Compared with the non-medical students (groups A and B), the medical students showed a significantly stronger ability to use resources (e.g., information and technology) and higher-order thinking ability. Meanwhile, no significant between-group differences in global skills and communication were observed. In the comparison between groups B and D, medical students showed lower levels of self-management, interpersonal, and cooperative skills compared with the fourth-year students of group D.

Table 3.

Comparison of Competencies between Medical and Non-Medical Students

Competency	Group (A)	Group (C)	t-value	p-value	Group (B)	Group (D)	t-value	p-value
Resources, information-technology	55.50	48.40	5.595[*]	0.000	56.10	51.40	2.023	0.050
Global skills	51.03	48.50	1.864	0.063	49.85	51.50	-1.118	0.264
Communication skills	50.62	48.20	1.569	0.121	54.44	51.60	1.564	0.118
Higher-order thinking	54.04	47.50	5.186[*]	0.000	55.55	51.90	2.154	0.032
Self-management	47.46	47.00	0.372	0.710	47.15	52.20	-3.24[*]	0.001
Interpersonal and cooperative	48.06	47.60	0.272	0.786	47.37	51.90	-2.592[*]	0.003

Group A: 6-year basic medical education program after high school. Group B: 4-year basic medical education program after a bachelor degree. Group C: first-year non-medical students from other majors. Group D: fourth-year non-medical students from other majors.

p<0.008 (=0.05/6, Bonferroni correction: significance level 0.05 is divided by six competencies of the Korea Collegiate Essential Skills Assessment).

Discussion

This study conducted an analysis of the competency levels of first-year medical students enrolled in the two types of medical school program. No significant difference in general competency level was observed between the two groups (groups A and B); differences were observed in some competencies between first-year medical and non-medical students. It is worth noting that the first-year medical students (group A) accepted into the basic medical education program in Korea are likely to possess such competencies as higher-order thinking and ability to use resources, information, and technology compared with non-medical students (group C) in university. The medical students of group B, however, showed lower self-management, interpersonal, and cooperative skills compared with fourth-year non-medical students (group D). The results of analyzing the data of 28,399 students who participated in KCESA in 2010 are the same as those of this study [22].

A few factors underlie these tendencies. First, students’ cognitive ability is given more importance in the selection of applicants for 6-year basic medical education program after high school. A study on KCESA pointed out that students’ competence are related to the entrance screening system [23]. There is a possibility that the difference in competence among groups may be caused by the selection effect [22]. In admission processes in Korea, many assessors consider objective variables, such as test scores (e.g., KSAT), high school transcript, and GPA, as the best measurement of students’ ability to use resources (including information and technology) and higher-order thinking. As mentioned, past academic achievement can be effective as a variable to predict academic progress through medical school [12]. The results of this study confirm that cognitive ability serves as an important indicator for the decision on admission to basic medical education program. Second, in this study, the students who had been accepted into 4-year basic medical education program after a bachelor degree had lower levels of selfmanagement, interpersonal, and cooperative skills compared with non-medical students. This may be attributed to Korea’s educational environment. Kim [24] reported that Korean students’ learning experiences take place under the supervision of their parents or in private educational institutions (hakwon) rather than on their own. Park [25] also noted that Korean medical students tend to view learning in terms of competing with others and believe that studying is an activity that should be done alone. In such a competitive learning environment where academic rank (established based on a student’s performance compared with others) is perceived as important, there is a perception that interpersonal and cooperative skills are unimportant.

Analysis of the competency of students admitted to medical schools provides meaningful implications for developing admission policy and improving competency through medical curricula. First, the specific competencies medical students should possess at the time of admission should be clarified. Subsequently, admission policies based on these competency requirements should be established. Second, efforts should be made to foster the competencies that medical students have been found to lack, such as self-management, interpersonal, and cooperative skills. Therefore, fostering these competencies, including the ability to connect emotionally with others, work with diverse individuals, work effectively in a team, and act as a mediator, as well as developing leadership skills and systematic thinking, should be regarded as essential goals in basic medical education. Third, the non-cognitive competencies of applicants should be recognized as important admission criteria because they are difficult to cultivate during basic medical education, which means that the admission committee is given no room to assess non-cognitive competencies because of the regulation of the Ministry of Education. A potential approach is to allow admission committees autonomy to evaluate non-cognitive competency.

A limitation of this study is that it could not explore the correlation between first-year medical students’ competency and assessment scores in the admission process directly owing to lack of access to admission data. Such data would be helpful to assess whether differences in admission scores are statistically significant. The admission process of medical schools and measurement of medical students’ competency have largely remained unchanged since 2011, and critics have noted that current research data may be outdated. In a follow-up study, it is necessary to conduct a follow-up study to compare the scores of the KCESA according to students majors, and to track the cause of the differences among the groups. The limitations of the KCESA inventory are as follows. The KCESA does not help to identify the strengths and weaknesses of core competencies based on student background variables. In addition, the KCESA itself does not provide the information necessary for the improvement of education [26]. The KCESA inventory is based on quantitative assessment and tends to simplify and reduce the complexity of college students’ core competencies [27]. Also, there is a limit to diagnose competencies with non-intellectual characteristics because of focusing on specific areas and contents of cognitive domains [28]. This study contributes insights and data for the design of basic medical education curriculum aimed at addressing the deficiencies of first-year medical students in competencies needed not only to survive medical school but also succeed in professional practice, which may be explored in future studies.