Competencies required for graduated physicians: the integration of Englander's common taxonomy in a validated scale for the assessment of competency acquiring in undergraduate medical education.

BACKGROUND: Competency-based medical education has been emerging as a new educational model for learning. The development of medical education in Middle Eastern countries lacks basic literature in the area of medical education, particularly competency-based medical education (CBME). AIM: Our objective is to validate a brief/rapid scale that informs us on the current perception of the acquiring of competencies of medical students and graduates from the Lebanese University, Lebanon.
METHODS: A survey containing the Englander's common taxonomy for medical competencies-a set of 57 items-was distributed to medical students and graduates from the Lebanese University. Three parameters were used for the validation of this scale: construct validity using the principal component analysis technique, internal reliability using Cronbach alpha, and convergent validity using Pearson correlation.
RESULTS: A total of 377 participants responded to the survey. Factor analysis, using Varimax rotation, showed appropriate loading of domains; loadings varied between 0.648 and 0.899. Construct validity showed for all competencies loaded appropriately in their respective domains; loadings ranged between 0.656 and 0.913. Cronbach alpha values were very good to excellent (0.868 to 0.932). Pearson correlation coefficients showed a good correlation with the total score and varied between 0.739 and 0.893 (p < 0.0001).
CONCLUSION: The Englander's common taxonomy scale is valid and reliable among Lebanese medical students and could be used for concise assessment of current and past medical education in faculties among Lebanese medical students. Using such scales can be used in assessing the improvement of medical education towards CBME.

Introduction

Medical education worldwide is witnessing new reforms in its curricula and syllabi. New paradigms are shifting education from a “classical” structure and process-based to competency-based medical education (CBME) [1]. The major distinction between these two learning models is that the first focuses on the process and the period of time while the latter focuses on the desired learning outcome, which is competency. In fact, this revolution began in the early twenty-first century, where the American Board of Medical Specialties (ABMS) and Accreditation Council for Graduate Medical Education (ACGME) aspired to initiate standardized, well-reported acquiring of competencies [2]. Consequently, several developed countries other than the USA initiated competency learning in medical settings such as the Royal College of Physicians and Surgeons in Canada [3], The Netherlands [4], and Scotland [5].

Developing countries have lagged behind in the implementation of such educational reforms. The majority of Arab countries have also failed to document reforms in medical education with the exception of the United Arab Emirates, in which the government implemented CBME in 2009 [6], and Saudi Arabia [7]. In other Arab and Middle Eastern countries, namely Lebanon, CBME literature is scarce. New efforts are being put by Lebanon’s universities such as the Lebanese-American University (LAU) [8] and the American University of Beirut (AUB) [9] to implement CBME in residency programs and post-graduate medical education (GME). The Lebanese University (LU) has not yet implemented CBME in its medical curriculum and is on the pathway of the World Federation of Medical Education accreditation process. Self-assessment of medical competencies is useful, as it may allow administrators to trace the improvement in competency acquiring through education till graduation, the instructors to locate the perceived weaknesses and correct them, and the students to become self-aware and make supplementary efforts in specific domains, if necessary.

Based on (1) the need to document undergraduate medical education and (2) the lack of a standard scale that measures competency acquiring, the authors aimed to develop and validate a new Likert scale that assesses competency acquiring in LU medical students, providing a concise and rapid assessment of undergraduate medical education, using Englander’s common taxonomy for competency-based medical education [10]. These efforts come in parallel with the faculty’s willingness to obtain the World Federation of Medical Education (WFME) accreditation. The accreditation requires several assessments in the faculty, including students’ and professors’ opinions, as well as the evaluation of the curriculum and extracurricular activities.

Objective

The objective of this study is to perform a principal component analysis of the Englander’s common taxonomy for competencies, in effort to validate a new scale for perceived competency acquiring of medical students/graduates at the Lebanese University.

Methods

Design, population, and procedure

This is a cross-sectional study, conducted between 21 April 2020 and 19 May 2020. Participants consisted of LU medical students and graduates, including both residents and practicing physicians. Applicants responded to a 45-min questionnaire, disseminated through the Google Forms platform and on social media platforms such as WhatsApp and Facebook. They were asked to rate their current perception of their acquiring of professional medical competencies; students were asked to answer about their current status, while graduates were to answer about their status upon graduation.

Study population

Number of participants

The minimal sample size was calculated using Slovin’s formula. The formula is as follows: N/(1 + Ne2), where N is the total number of the population and e is the margin of error (in %) [11].

Students and medical staff at the Lebanese University, Faculty of Medical Science, were 1509 individuals, according to the faculty’s website [12]. This number was used as the total population in Slovin’s formula. The margin of error was set at 5%. Hence, the minimum required sample size was 316 individuals. In our study, we collected a total of 385 responses, exceeding the minimal size required and with a confidence interval of 95%.

Ethical aspect

Participants’ confidentiality was respected during this study. The Institutional Review Board of the LU approved this study protocol.

Questionnaire and variables

The questionnaire consisted of two parts. The first part contained questions about gender, age, and year of education for medical students (2nd to 7th year) and residents (PGY1, PGY2) as well as the year since obtaining a general medicine degree for doctors. The second part consisted of questions, on the different competencies of medical students, extracted from two references: Englander’s common taxonomy of competency domains and the Association of American Medical College’s (AAMC) Undergraduate Medical Education (UME) competencies [13, 14]. Participants were asked to assess the acquiring of these competencies during their pre-graduation years using a 5-point Likert scale, as follows: 0 for “too new to rate,” 1 for “below expectations,” 2 for “needs improvement,” 3 for “exactly meets expectations,” 4 for “periodically exceeds expectations,” and 5 for “regularly exceeds expectations.” Englander’s domains of competencies consisted of 57 questions that were used as a common taxonomy for competencies and were grouped into 8 main categories: patient care, knowledge for practice, practice-based learning and improvement, interpersonal and communication skills, professionalism, system-based practice, inter professional collaboration, and personal and professional development. However, the AAMC’s UME competencies consisted of an exhaustive listing of all competencies, grouped into 12 categories, and composed of 168 questions; thus, its results were not used. In this study, we validated only the Englander’s scale.

Statistical analysis

Data was converted to the Statistical Package for Social Sciences (SPSS), version 23.0. Assumptions of sample adequacy (anti-image, KMO, and Bartlett’s test of sphericity) were checked prior to conducting factor analyses; communalities were higher than 0.3. An explorative factor analysis using the Varimax rotation was conducted to assess the adequacy of the distribution of competency domains. Loadings of competencies (used as items) on the domain were recorded. In addition, the Cronbach alpha was calculated for every competency and overall domain, to assess the internal consistency (reliability) of the domain-related competencies. Furthermore, Pearson correlation coefficients were calculated to assess the convergent validity of the domains within the framework, and the association of the domains with the overall framework.

Results

Description of sample

Overall, the sample contains 377 participants. We note that in the population that we are most interested in, which is between the 2nd and the 7th academic year, the minimal number of responses per year was 34 responses, seen in the 2nd and 3rd years, with a response rate close to 30%, and increasing to 76, 64, and 61 responses in the 5th, 6th, and 7th year with response rates as high as 63%, 56%, and 53%, respectively.

The mean age of the sample was 23.53 years (S.D. 2.93). Males accounted for 43.2% of the sample (163 responses), and females accounted for 55.7% (210 responses), along with 1.1% (4 responses) with unknown gender. Responses came from both pre-graduates and graduates. Practicing physicians were the smallest category with only 3.2% (12 responses) of the sample and 5th-year medical students had the highest participation with a frequency of 20.2% (76 responses), followed by 6th-year medical students with a frequency of 17.0% (64 responses), 7th-year students with 16.2% (61 responses), and residents with 14.3% (54 responses). Missing values accounted for 1.1% (4 responses) of the total responses.

Scale structure

Factor analysis, using Varimax rotation, revealed the same distribution of domains as in the Englander’s original domains: patient care, knowledge for practice, practice-based learning and improvement, interpersonal and communication skills, professionalism, system-based practice, interprofessional collaboration, and personal and professional development. Loadings ranged from 0.648 for the interpersonal and communication skills domain to 0.899 for the professionalism domain.

Construct validity of the framework

Based on all factor analyses (Tables 1, 2, 3, 4, 5, 6, 7 and 8; Fig. 1), all competencies loaded appropriately on their respective domain and varied from a minimum of 0.656 for the “use information technology to optimize learning” competency in the practice-based learning and improvement domain and a maximum of 0.913 for the “use the knowledge of one’s own role and the roles of other health professionals to appropriately assess and address the health care needs of the patients and populations served” competency in the interprofessional collaboration domain (Table 9).

Table 1

Patient care competency domain and items

Competency domain	Behavior/outcome	Loading/competency
Patient care Number of items = 11 Cronbach alpha = 0.932 Total variance explained = 59.62%	PC1. Perform all medical, diagnostic, and surgical procedures considered essential for the area of practice	0.739
	PC2. Gather essential and accurate information about patients and their conditions through history-taking, physical examination, and the use of laboratory data, imaging, and other tests	0.779
	PC3.Organize and prioritize responsibilities to provide care that is safe, effective, and efficient	0.788
	PC4. Interpret laboratory data, imaging studies, and other tests required for the area of practice	0.747
	PC5. Make informed decisions about diagnostic and therapeutic interventions based on patient information and preferences, up-to-date scientific evidence, and clinical judgment	0.805
	PC6. Develop and carry out patient management plans	0.822
	PC7. Counsel and educate patients and their families to empower them to participate in their care and enable shared decision making	0.746
	PC8. Provide appropriate referral of patients including ensuring continuity of care throughout transitions between providers and settings and following up on patient progress and outcomes	0.789
	PC9. Provide health care services to patients, families, and communities aimed at preventing health problems or maintaining health	0.753
	PC10. Provide appropriate role modeling	0.739
	PC11. Perform supervisory responsibilities commensurate with one’s roles, abilities, and qualifications	0.781

Table 2

Knowledge for practice competency domain and items

Competency domain	Behavior/outcome	Loading/competency
Knowledge for practice Number of items = 6 Cronbach alpha = 0.868 Total variance explained = 60.53%	KFP1. Demonstrate an investigatory and analytic approach to clinical situations	0.754
	KFP2. Apply established and emerging bio-physical scientific principles fundamental to health care for patients and populations	0.808
	KFP3. Apply established and emerging principles of clinical sciences to diagnostic and therapeutic decision-making, clinical problem-solving, and other aspects of evidence-based health care	0.835
	KFP4. Apply principles of epidemiological sciences to the identification of health problems, risk factors, treatment strategies, resources, and disease prevention/health promotion efforts for patients and populations	0.775
	KFP5. Apply principles of social-behavioral sciences to the provision of patient care, including assessment of the impact of psychosocial and cultural influences on health, disease, care-seeking, care compliance, and barriers to and attitudes toward care	0.765
	KFP6. Contribute to the creation, dissemination, application, and translation of new health care knowledge and practices	0.728

Table 3

Practice-based learning and improvement competency domain and items

Competency domain	Behavior/outcome	Loading/competency
Practice-based learning and improvement Number of items = 10 Cronbach alpha = 0.897 Total variance explained = 52.05%	PBLI1. Set learning and improvement goals	0.708
	PBLI2. Identify strengths, deficiencies, and limits in one’s knowledge and expertise	0.759
	PBLI3. Identify and perform learning activities that address one’s gaps in knowledge, skills, and/or attitudes	0.776
	PBLI4. Systematically analyze practice using quality improvement methods, and implement changes with the goal of practice improvement	0.742
	PBLI5. Incorporate feedback into daily practice	0.763
	PBLI6. Locate, appraise, and assimilate evidence from scientific studies related to patients’ health problems	0.665
	PBLI7. Use information technology to optimize learning	0.656
	PBLI8. Participate in the education of patients, families, students, trainees, peers, and other health professionals	0.73
	PBLI9. Obtain and utilize information about individual patients, populations of patients, or communities from which patients are drawn to improve care	0.724
	PBLI10. Continually identify, analyze, and implement new knowledge, guidelines, standards, technologies, products, or services that have been demonstrated to improve outcomes	0.68

Table 4

Interpersonal and communication skills competency domain and items

Competency domain	Behavior/outcome	Loading/competency
Interpersonal and communication skills Number of items = 7 Cronbach alpha = 0.902 Total variance explained = 63.14%	ICS1. Communicate effectively with patients, families, and the public, as appropriate, across a broad range of socioeconomic and cultural backgrounds	0.787
	ICS2. Communicate effectively with colleagues within one’s profession or specialty, other health professionals, and health-related agencies	0.838
	ICS3. Work effectively with others as a member or leader of a health care team or other professional groups	0.777
	ICS4. Act in a consultative role to other health professionals	0.779
	ICS5. Maintain comprehensive, timely, and legible medical records	0.794
	ICS6. Demonstrate sensitivity, honesty, and compassion in difficult conversations, including those about death, end of life, adverse events, bad news, disclosure of errors, and other sensitive topics	0.798
	ICS7. Demonstrate insight and understanding about emotions and human responses to emotions that allow one to develop and manage interpersonal interactions	0.788

Table 5

Professionalism competency domain and items

Competency domain	Behavior/outcome	Loading/competency
Professionalism Number of items = 6 Cronbach alpha = 0.93 Total variance explained = 74.22%	Pr1. Demonstrate compassion, integrity, and respect for others	0.821
	Pr2. Demonstrate responsiveness to patient needs that supersede self-interest	0.859
	Pr3. Demonstrate respect for patient privacy and autonomy	0.883
	Pr4. Demonstrate accountability to patients, society, and the profession	0.884
	Pr5. Demonstrate sensitivity and responsiveness to a diverse patient population, including but not limited to diversity in gender, age, culture, race, religion, disabilities, and sexual orientation	0.884
	Pr6. Demonstrate a commitment to ethical principles pertaining to provision or withholding of care, confidentiality, informed consent, and business practices, including compliance with relevant laws, policies, and regulations	0.837

Table 6

System-based practice competency domain and items

Competency domain	Behavior/outcome	Loading/competency
System-based practice Number of items = 5 Cronbach alpha = 0.896 Total variance explained = 70.89%	SBP1. Work effectively in various health care delivery settings and systems relevant to one’s clinical specialty	0.818
	SBP2. Incorporate considerations of cost awareness and risk–benefit analysis in patient- and/or population-based care	0.847
	SBP3. Advocate for quality patient care and optimal patient care systems	0.858
	SBP4. Participate in identifying system errors and implementing potential systems solutions	0.862
	SBP5. Perform administrative and practice management responsibilities commensurate with one’s role, abilities, and qualifications	0.825

Table 7

Interprofessional collaboration competency domain and items

Competency domain	Behavior/outcome	Loading/competency
Interprofessional collaboration Number of items = 4 Cronbach alpha = 0.906 Total variance explained = 78.09%	IPC1. Work with other health professionals to establish and maintain a climate of mutual respect, dignity, diversity, ethical integrity, and trust	0.863
	IPC2. Use the knowledge of one’s own role and the roles of other health professionals to appropriately assess and address the health care needs of the patients and populations served	0.913
	IPC3. Communicate with other health professionals in a responsive and responsible manner that supports the maintenance of health and the treatment of disease in individual patients and populations	0.907
	IPC4. Participate in different team roles to establish, develop, and continuously enhance interprofessional teams to provide patient- and population-centered care that is safe, timely, efficient, effective, and equitable	0.85

Table 8

Personal and professional development collaboration competency domain and items

Competency domain	Behavior/outcome	Loading/competency
Personal and professional development Number of items = 8 Cronbach alpha = 0.912 Total variance explained = 61.94%	PPD1. Develop the ability to use self-awareness of knowledge, skills, and emotional limitations to engage in appropriate help-seeking behaviors	0.742
	PPD2. Demonstrate healthy coping mechanisms to respond to stress	0.736
	PPD3. Manage conflict between personal and professional responsibilities	0.822
	PPD4. Practice flexibility and maturity in adjusting to change with the capacity to alter one’s behavior	0.808
	PPD5. Demonstrate trustworthiness that makes colleagues feel secure when one is responsible for the care of patients	0.799
	PPD6. Provide leadership skills that enhance team functioning, the learning environment, and/or the health care delivery system	0.808
	PPD7. Demonstrate self-confidence that puts patients, families, and members of the health care team at ease	0.803
	PPD8. Recognize that ambiguity is part of clinical health care and respond by utilizing appropriate resources in dealing with uncertainty	0.772

Fig. 1

Component plot in rotated space using Varimax rotation

Table 9

Factor analysis of the Englander’s competency domains using Varimax rotation

Component	1	2	3	4	5	6	7	8
Patient care		0.842
Knowledge for practice					0.821
Practice-based learning and improvement						0.808
Interpersonal and communication skills								0.648
Professionalism	0.899
System-based practice							0.782
Interprofessional collaboration				0.796
Personal and professional collaboration			0.831

Reliability (internal consistency) of the framework

Cronbach alpha values were very good to excellent: a minimum of 0.868 was recorded for the knowledge for practice and a maximum of 0.932 for the patient care domain (Tables 1, 2, 3, 4, 5, 6, 7 and 8).

Convergent validity of domains

In Table 10 and Fig. 2, all domains were found to be correlated. Pearson correlation coefficients varied between 0.510 (patient care with professionalism) and 0.735 (interpersonal and communication skills with system-based practice). More importantly, all domains correlated well with the total score; coefficients varied from 0.739 in the professionalism domain and 0.893 in the interpersonal and communication skills domain. All correlations were statistically significant (p < 0.0001).

Table 10

Domain inter-correlation and association with total competence

Correlations	Patient care	Knowledge for practice	Practice-based learning and development	Interpersonal and communication skills	Professionalism	System-based practice	Interprofessional collaboration	Personal and professional collaboration
Patient care	1
Knowledge for practice	.691**	1
Practice-based learning and improvement	.611**	.698**	1
Interpersonal and communication skills	.659**	.657**	.710**	1
Professionalism	.510**	.538**	.547**	.684**	1
System based practice	.648**	.676**	.651**	.735**	.528**	1
Interprofessional collaboration	.622**	.566**	.609**	.794**	.569**	.754**	1
Personal and professional collaboration	.594**	.567**	.685**	.706**	.548**	.637**	.682**	1
Totals core	.830**	.812**	.842**	.893**	.739**	.835**	.827**	.824**

**p-value < 0.0001

Fig. 2

Pearson correlation coefficients between domains and total score of the scale

Discussion

This is the first study in Lebanon to validate a detailed and comprehensive set of competencies required for graduate physicians. In fact, several challenges are imposed when implementing CBME in undergraduate medical education; data in the Arab world is scarce in this area [13]. Therefore, there is a tremendous need for a valid scale to be implemented and used in areas where CBME in UME needs to be implemented. This scale is made out of 57 core and common items/competencies and therefore could be used as a concise assessment of competencies. The results we obtained showed the overall scale to be valid and reliable; moreover, each domain had appropriate structural validity and internal consistency. Based on the excellent results obtained, we suggest this scale as a tool for competency self-assessment among university students and graduates. As explained, this study was done in the LU and showed a valid scale that assesses competency acquiring in the pre-graduation years of medical education.

On the global level, Englander’s taxonomy was not yet validated previously in any country. This taxonomy gathered common competencies in the different countries that implemented CBME. Its high internal reliability, adequate loading of factors and items, and well-suited convergent validity show that this taxonomy was well constructed and that it can be used in several countries as a common scale that assesses UME competencies acquiring. This is of tremendous importance for eventual comparison between different medical curricula; thus, this scale provides standardized, common, and original items that can be used in all medical faculties. Combining international forces of different medical faculties is highly recommended to evaluate and refine medical education worldwide since this would lead to a better physician workforce and adequate clinical and professional care. This is especially needed in Arab countries and universities seeking CBME implementation [6-9].

This scale could also be eventually used in other universities in Lebanon, such as the American University of Beirut, the Lebanese American University, and those covering a medical curriculum. This would lead to an eventual assessment of all Lebanese medical curricula, in a CBME-centered manner. Flaws and gaps in the medical curricula in the country would be revealed, and new guidelines and policies would categorically revolutionize medical education in the country. Such strategies are highly needed in Lebanon, especially since there is no unified medical curriculum in the country, where French and English curricula are present with no clear policies on Lebanon’s special population and needs [14]. The Ministry of Public Health and the Ministry of Education and Higher Education, in cooperation with professional associations of medicine (Lebanese Orders of Physicians), should invest more in this field; in addition, research implicating medical education should be promoted.

Moreover, this study was done on 2nd- to 7th-year medical students. Therefore, this scale can help in tracking the mean of competencies and domains in each year and plot the different mean scores across the years to assess the continuous acquiring of competencies and pinpoint gaps in the years of medical education. A global strategy on the timing of implementation of competencies and the level required, known as milestones [15] (from novice to expert), can be based on this type of analysis. In fact, this scale can be used several years later in the same faculty, as a cohort, to assess whether the application of CBME increased competency acquiring or not. Adequate follow-up of medical curricula is essential and can be achieved using this scale.

Among the several uses of this scale is the assessment of the past undergraduate medical education by residents and physicians, who rated their acquiring of competencies during undergraduate years. All things considered, different factors could affect scores in the resident and physician sample and should be taken into consideration when interpreting results. From these factors, we mention the possibility that some residents and physicians could have rated their present medical education instead of their past medical education and the potential memory bias due to the recall of past medical education. In addition, information bias could have taken place. Graduates could have better valued their education with time, leading to higher scores among older graduates.

The need for student feedback and formative assessment is crucial in the modern era of medical education and has been adequately elucidated by Tormey [16]. The review focuses on the importance of objective structured clinical examinations and the necessity to remove bias in the examination by using tested and validated scales. The World Federation for Medical Education (WFME) developed nine standards “to guide the development and evaluation of medical programs.” Each standard has two categories: basic and “quality” standard [17]. Tormey discusses several assessment methods, including Angoff, Bookmark Ebel, Hofstee, borderline group, and contrast by group, to be used as a guide by medical schools/programs.

Despite the limitations, this type of assessment could lead to a retrospective root analysis of medical education by responsible authorities being eager to improve medical education, including educators, medical education researchers, administrators, and policymakers in the university. This should ultimately lead to a retrospective root cause analysis of all events and decisions made in the last 20 years at the faculty to identify the possible underlying problems in the educational system. Therefore, constructive criticism, in the faculty’s best interest, should be a priority to be able to pinpoint potential outfalls in the past. In adjunct to the root cause analysis, a systematic review could be conducted with available published and non-published data, in addition to previous theses of graduates describing different aspects of education in the faculty (education strategies, environment, satisfaction of students, mental health, etc.), supplying us with more evidence on the topic of past medical education in the faculty. Finally, after gathering all of the sufficient data, a forward mode and effect analysis should be carried out before the implementation of a new educational strategy. In this systematic approach, coupled with scientific reasoning, new solutions could be afforded while also avoiding drawbacks and pitfalls.

Limitations

This article did not include participants outside the LU, thus making a generalization on all Lebanese medical students invalid. This scale has also to be tested in future studies in other universities and countries allowing standardized comparison. Information bias could have occurred in the residents’ and physicians’ population due to the difficulty of recalling past perceived acquiring of competencies and/or conveying another picture of reality. Selection bias could have occurred since the survey was conducted in the COVID-19 pandemic, which hindered participation among busy medical staff and physicians.

Conclusion

This is the first valid scale among Lebanese medical students that tackles competency acquiring in undergraduate medical education, using a 5-point Likert scale. This concise scale is essential for medical faculties that seek to measure perceived competency acquiring and to participate in the reform towards CBME.