Perspectives of the Key Stakeholders of the Alignment and Integration of the SaudiMEDs Framework into the Saudi Medical Licensure Examination: A Qualitative Study.

PURPOSE: The purpose of the Saudi Medical Education Directives Framework (SaudiMEDs) is to assure the essential level of competencies for medical graduates, which should be reflected in the Saudi Medical Licensure Examination (SMLE). This study explored the opinions of key stakeholders of the alignment and integration of the SMLE's blueprint and contents with the SaudiMEDs competency framework's themes and domains. PARTICIPANTS AND METHODS: This was a qualitative case study, using a purposive sampling technique. Ten participants participated in the study representing the main stakeholders. The participants were four deans, an assistant dean, two residents, and three interns of various colleges of medicine (COM). In-depth interviews were conducted through a semi-structured format, either online or in-person. The interviews were recorded, transcribed verbatim and analyzed according to the general guidelines of qualitative content analysis.
RESULTS: Four major themes emerged from the data, including the current alignment practices of the COM, competencies enhanced through preparing according to the SMLE, the SaudiMEDs representation in the SMLE, and finally a roadmap to achieve optimum alignment between the SaudiMEDs and the SMLE.
CONCLUSION: The participants were knowledgeable about the SaudiMEDs and perceived the SMLE blueprint and contents to be partially aligned with the themes and domains of the SaudiMEDs competency framework. The responses suggested that additional effort is required to improve the current alignment to assess the competencies of COM graduates appropriately.

Introduction

The progression in the daily practice of medicine initiated a medical education revolution early in this century, which resulted in the development and improvement of many educational models.1 One of these popular models is the competency/outcome based learning.2,3 Competence-based medical education contribute to improve the students’ and graduates’ competencies and performance, and it was adopted by most of the developed countries.4,5 There are many competency frameworks, for example the USA uses the competency framework of the Accreditation Council for Graduate Medical Education (ACGME),6 Canada embraces the implementation of the CanMEDs framework,7 and in the UK, the General Medical Council (GMC) guideline of “Tomorrow’s Doctors.”8 The equivalent framework embraced in KSA is the “Saudi Medical Education Directives Framework” commonly called the SaudiMEDs.9 The framework was developed by the Deans Committee of the Saudi Colleges of Medicine (COM). It is a Competence-based framework reflecting the principles of professional practice, learning outcomes and the national specifications for Saudi doctors. The framework was approved by the National Center for Academic Accreditation and Evaluation (NCAAA) as the minimal learning outcomes for all programs of theCOMs in Saudi Arabia.10

The SaudiMEDs outlines six major themes, namely the scientific approach to practice, patient care, community-oriented practice, communication and collaboration, professionalism, as well as research and scholarship. It also entails 17 key competencies (Program Learning Outcomes) and 80 enabling competencies (course-level learning outcomes).10 It was recommended that all the national medical school programs develop and review their curriculum according to the SaudiMEDs in a spiral approach, and to integrate the competencies in their curricula. Focusing on the major competency domains which will have a positive effect on the performance at the SMLE as well as others assessments, such as the objective structured clinical examination (OSCE).10 To increase the awareness of medical schools regarding the SaudiMEDs framework, the Deans Committee conducted a culture awareness and training course for the medical schools (a series of workshops).

The SMLE is a summative assessment tool established by the Saudi Commission for Health Specialties (SCFHS) to license national or international physicians to practice medicine in the Kingdom or to authorize postgraduate medical trainees to apply for local residency programs. The SMLE is obligatory for all potentially graduated doctors during the internship of their bachelor’s degree. The examination blueprint was developed by the SMLE steering committee, which included representatives from the Saudi Medical College Deans. The examination was prepared to assess a particular learning domain, including knowledge and cognitive skills (interpretation, analysis, decision-making, reasoning and problem solving) and attitude.11

Based on the literature, there is no study exploring the alignment between the SMLE test blueprint and the SaudiMEDs framework. Since there is no evidence regarding the awareness of the stakeholders about the alignment, this study is expected to raise awareness regarding the incorporation of the SMLE test contents and blueprint in the SaudiMEDs and to generate recommendations to improve the current alignment.

The key stakeholder population included medical school faculties and medical interns to explore their points of view regarding to the alignment of the categorical concurrence (general match) between the SMLE blueprint and contents with the competencies and enabling competencies of the SaudiMEDs.11 The outcome of this study should highlight the awareness of the stakeholders about both the SMLE and SaudiMEDs.

Materials and Methods

Study Design and Sampling

The research was designed as a qualitative case study, with participants from 10 governmental and private medical schools from the different regions in Saudi Arabia. The sample size was five from each stakeholder group.12 A purposive sampling technique was used. Stakeholders with opinions relevant to the purpose of this study were identified and recruited, including:

1) COM faculty involved in curriculum/assessment (including deans, the associate, and assistant deans)

2) The medical interns and junior residents who graduated from the national medical schools.

This study included informed participants and the inclusion criteria were:

The faculty involved in the curriculum and assessment of the COM.

Employment level of dean, associate dean, or assistant dean at the COM.

Medical interns and residents who took the SMLE examination recently (within the last year).

It was challenging to engage individuals involved in developing the SMLE and none was included in the study.

Data Collection

The method of data collection was semi-structured, in-depth, individual exploratory interviews. The researcher conducted the interviews mostly through electronic virtual means, with two face-to-face interviews at the workplace of the participant. The interviews were scheduled as appropriate for the researcher and interviewee. The interviewees were allowed to speak freely about their opinion regarding the topic. The length of each interview was between 20–30 minutes.

The interview questions were created by the researcher, then reviewed and consensus was reached by four experts in the field of Medical Education. The interview guide included eight questions. As some of the questions were not relevant to specific groups, they were skipped by the researcher. Additional probing questions were asked when explanation or elaboration was required. The questions were open ended to obtain the maximum information and input concerning the alignment between the SMLE test blueprint and contents with the SaudiMEDs. The interview question focused two categories: Firstly, the perception of the alignment between the SMLE blueprint and SaudiMEDs. The second category was the realization of the SMLE examination items and its alignment with SaudiMEDs.

Data Analysis

After the interviews were conducted, the main researcher used the audio recording and created verbatim transcripts. Two verbatim transcripts were randomly selected and two different raters from the research team (i.e., the main researcher and another member) individually conducted the open coding. The two raters met and reviewed their open coding to create a comprehensive open coding structure. The main researcher used the open coding structure and coded the other eight transcripts. The same two researchers proceeded with axial and selective coding and met several times to discuss and finalize the sub-themes and main themes, based on the study objectives. The axial and selective coding and developed themes and subthemes were shared with the whole research team for final revisions and improvement.13 Though the analysis focused on exploring the participants’ opinion regarding the alignment between the SMLE blueprint and contents with the SaudiMEDs, it included the challenges and solutions reported by the participants.

Ethical Approval

Institutional Review Board (IRB) approval was obtained and the participants signed a written informed consent document. The aim and objectives was emailed to the participants a few weeks before the date of the interview. The participants were informed that the interview would be recorded and transcribed verbatim by the researcher. They were also informed that the transcripts of the interview could be sent back upon request for review before the analysis. No emotional distress or risks were imposed on the participants due to participating in the study. The data were secured by the researcher on a protected external hard disk. The privacy and confidentiality of the participants were maintained, and no names or any identification was collected.

Results

In total, 10 participants were interviewed (n=10). Six were male (60%). The faculty members and students were evenly represented in the sample. The level of knowledge of the sample with the SaudiMEDs and SMLE was substantial and they were well informed. The sources of information for the SaudiMEDs varied between the faculty and students.

For example, a faculty member said:

[In reference to SaudiMEDs], I was part of the foundation committee of these competencies for the physician as a member of College of Medicine. (#7)

The students had their own sources of information. For example, in reference to SaudiMEDs a student stated:

I google[d] it [out] for curiosity [to find out] if it is similar to the CanMEDS or not and [I] found them alike. However, I never heard about it [indicating to the SaudiMEDs] in my undergraduate studies. (#9)

An overview of the participants familiarity with the SMLE considered two aspects, the blueprint and contents. The following response is an example from one of the participants regarding their knowledge of the blueprint:

It was very easy accessibility to blueprint you can find it in the SCFHS website regarding the contents we got some courses before the exam. (#1)

Although the blueprint was perceived to be informative, fair and helpful, they did not have the final version. For instance, the participants stated:

I think [the] preparation of the blueprint [was done] not only [by] one body. Put this blueprint [and] you can see there is no homogeneity in preparation and final outcome between the branch [i.e. Obstetrics and Gynecology and Pediatrics etc.] of the examination which is different in the methods of setting up the different branches of the blueprint. (#7)

As for the second aspect (SMLE content), it was not officially communicated to the any of the participants.

In regarding the content [stating to SMLE] we do not have an official way in looking for contents. The SCFHS did not consult us at all in terms of the contents yet we have unstructured feedback from our candidates who [took] the exam. (#6)

Considering the unofficial version of the SMLE examination contents, several problems were perceived by the participants. The problems included an inconsistency between what was presented in the SMLE blueprint and what was actually in the examination, imbalanced subject distribution, and a lack of involvement in setting the SMLE examination.

I wished that the development of the blueprint and content was done based on the opinion with the [faculties in the] Colleges of Medicine and [in alignment with] SaudiMEDs. (#8)

Using qualitative content analysis, the data has been coded and analyzed, which resulted in four themes and 13 subthemes (Table 1).

Table 1

Themes/Subthemes That Emerged from Participants’ Answers Concerning How They Perceive the Alignment and Integration of SMLE and SaudiMEDs

● 1.1. COM’s* Current Alignment Practices

● 1.1. No Alignment

● 2. Indirect Alignment

● 1.3. Stakeholders’ Satisfaction With Current Alignment

● 1.2. Competencies Enhanced Through Preparing for SMLE**

● 2.1. No improvement:

● 2.2. Partial improvement (Mainly Knowledge Domain)

● 1.3. SaudiMEDs*** representation in SMLE

● 3.1. SaudiMEDs Not Represented in SMLE

● 3.2. SMLE Partially Promoted SaudiMEDS’ domains/themes.

● 3.3. Not sure about alignment status

● 1.4. The Roadmap Toward Optimum Alignment Between SaudiMEDS and SMLE

● 4.1. Involvement of Stakeholders in aligning SMLE with SaudiMEDs

● 4.2. Assess current alignment between SaudiMEDs and SMLE

● 4.3. Improve Governance of SMLE

● 4.4. Plan for smooth transition from pre-grad to post-grad assessment

● 4.5. Expand and advance current SMLE format

Notes: *=Colleges of medicine; **=Saudi Medical Licensure Examination; ***=Saudi Medical Education Directives.

In addition, the analysis also generated recommendations to improve the alignment and integration between the SMLE and SaudiMEDs, which is a secondary objective of the search.

Theme One: The COM’s Current Alignment Practices

The response of the COM participants (n=5) generated three sub-categories explaining the current alignment.

No Alignment

Some participants expressed that there is no need for an alignment due to several reasons, including they were not involved in the process of setting the SMLE examination and the SMLE mapping is based on the CanMEDS, not the SaudiMEDs. Some perceived that the SMLE does not fit the Saudi Medical curricula, and others pointed out that they do not understand the rationale behind establishing an alignment between the SMLE and the SaudiMEDs.

In our college nothing has been done yet to make any changes or in depth reading in this matter [referring to the alignment between the SMLE and the SaudiMEDs] because it [the request to align] was not sent to us in an official way. (#8)

We do not have to align with a specific examination [indicating to the SMLE]. (#6)

Indirect Alignment

According to the participants (n=4), some of the colleges had started to take measures to improve the alignment between the SaudiMEDs and the SMLE directly and indirectly. One of the most direct actions is targeting the medical school curricula development towards adopting the SaudiMEDs. Also, direct communication with the SCFHS, encouraging the adoption of the SaudiMEDs as a guide to set the SMLE examination. The following statements are examples of how the participants referred to the direct alignment activities in their responses:

We adopted the strategy of aligning our curriculum with the SaudiMEDs. (#7)

Participants described the different indirect alignment approaches currently done by the colleges.

An example of an indirect attempt to align the SMLE and the SaudiMEDs is aligning the students’ progress or mock tests with both the SMLE and SaudiMEDs.

We [modified] the progress test to be in alignment with the SaudiMEDs and with the SMLE. (#5)

Another indirect alignment approach was using the feedback provided from the data published by the SCFHS (e.g., student performance reports) as well as the feedback from the students who completed the SMLE. For example, the participants stated:

We are doing an analysis of the result of the [SMLE] examination data, as well as checking the statistics [provided by SCFHS]. (#6)

The Stakeholders’ Satisfaction with the Current Alignment

The participants’ level of contentment was explored to describe the current alignment status. The majority (n=7) are partially satisfied with the alignment between the SMLE and the SaudiMEDs,

It is not so satisfying when I see the alignment of the SMLE with the SaudiMEDs. It is fine with the SaudiMEDs in a few components, but another component is not covered. (#4)

However, a few participants expressed their total dissatisfaction

I’m not satisfied with the alignment because none of the competencies have been mention by our faculty or the SCFHS, it was a personal effort to know it and the other reason” (#9)

The participants mentioned many reasons for the level of dissatisfaction, such as the discrepancy between the level of questions and the expected cognitive level of the medical graduates, the examination items assessed memorization and passive learning, the SMLE overlooked many important clinical examination competencies, there is a clear discrepancy between the undergraduate curricula and examinations, and finally, the lack of clear communication and available data about the examination.

I did not show that I am [a] graduate of a university that applied the SaudiMEDs in comparison of other colleagues. I was not very happy about the exam [relating to SMLE] because [it did not match what] we learned. (#2)

Theme Two: Competencies Enhanced Through Preparing in Terms of the SMLE

When asked about the competencies that they (as residents) were able to acquire because of the SMLE, one resident responded that her/his competencies did not improve while preparing for SMLE. S/he stated:

They [indicating to who set the SMLE examination] did not focus much on a theme. It [the SMLE examination] mainly was a recall exam so you have information and recall it, so it’s mainly like a mental question like a diagnosis of disease. (#2)

However, all other participants expressed perceiving a partial improvement in their competencies. The participants pointed out that the SMLE mainly endorsed the knowledge domain, while the other important domains, such as the clinical competencies of the SaudiMEDs, which are not included in current version of the SMLE. The following statements are an example of what was shared by the participants:

Other competencies, like professionalism and research, are not been represented well by SMLE as well for example, it is difficult to assess professionalism by theoretical exam like SMLE because you have to have a real encounter with patients and colleagues for proper assessment. (#1)

Theme Three: The SaudiMEDs Representation in the SMLE

The responses regarding whether the SaudiMEDs themes and domains are represented in the SMLE examination, resulted in three sub-themes.

The SaudiMEDs’ Domains and Themes are Partially Represented in SMLE

The majority of the students (n=4) were remarking that the SMLE partially promote the SaudiMEDs’ domains and themes. An example from the participant’s statements is:

I think there [is] big overlap [between SaudiMEDs and SMLE. Most of the competencies are represented in SMLE. (#1)

In rationalizing their opinion, the participants explained that they perceive the SMLE to fairly assess the minimum abilities required to be able to practice as a competent physician. These competencies are derived from the SaudiMEDs. However, the participants acknowledged that many important SaudiMEDs competencies were not represented in the SMLE (e.g., physical examination or communication). They related the absence of these competencies to the nature of the SMLE format, which is Multiple Choice Questions.

How the SMLE scenario [could help] in applying competency when you read a question [in a comparison to dealing with a] scenario with a difficult patient or a patient who just received some bad diagnosis or news. (#1)

I think [there is] partial [alignment] between SMLE and SaudiMEDs. Many competencies were never mentioned. (#10)

SaudiMEDs Not Represented in the SMLE

The general responses that resulted in this sub-category highlighted that the SMLE was not representative or below standard in measuring the SaudiMEDs’ domains and themes. The participants (n=3) reached this conclusion based on their perception of the current quality of the SMLE. The SMLE format (MCQ) is knowledge-based recall questions, which does not need any integration skills, explaining why the SaudiMEDs domains and themes are not represented in the SMLE. A participant stated:

I do not think it is [in reference to SaudiMEDs] integrated or align with the exam

[with regard to SMLE questions] mostly testing clinical knowledge. (#9)

In explaining why the SaudiMEDs domains and themes were not represented in the SMLE, a participant mentioned the quality of the questions posed in the SMLE. According to her/him, the level of the SMLE is much lower than the level of COM graduates. According to the participant:

[SMLE is] asking about things that [a] physician needs to know [when s/he is in] second year of medical school not [at] graduate level. (#5)

Not Sure if the SaudiMEDs is Represented in the SMLE

The last group of the participants (n=3) declared that they were not sure about the alignment status and that they are taking a neutral position about assessing the current alignment. The inadequacy of the official SMLE content information was considered a requirement to judge the alignment. An example from the participants statements:

I’m not sure because I did not look at a full exam paper [referring to SMLE] never ever so I cannot judge that [pointing to alignment]. (#6)

The SCFHS are detached from the Colleges of Medicine [so] I can’t give you [an] answer. (#8)

Theme Four: The Roadmap to an Optimum Alignment Between the SaudiMEDs and the SMLE

There was a unanimous agreement that the current alignment requires improvement. The participants recommended several measures to achieve a better alignment. These recommendations are reported here as it serves the secondary objective of this study. The recommendations were categorized in five sub-themes.

Involvement of Stakeholders in Aligning the SMLE to the SaudiMEDs

The participants indicated that the current communication channel is not ideal to ensure proper alignment of the SMLE to the SaudiMEDs.

If there is an annual and biannual meeting between all medical school and SCFHS they will really know what is going on. (#4)

The participants suggested the following measures to improve involvement:

Inviting SCFHS representatives to the medical schools’ committees.

The director general of the SCFHS is a member in our social media channel and sometimes he or a representative can be invited to the committee meeting. (#7)

Scheduling meetings between COM and SMLE bodies to review the student performance reports issued by the SCFHS and get feedback for future plans.

The only communication with them [SCFHS] is the performance of our students. We need an open channel between the medical schools and the SCFHS. We wanted to discuss the students’ curriculum and performance, as well the alignment with the blueprint of the SMLE. (#4)

Students as main stakeholders should be involved through being educated about the SMLE, SaudiMEDs and the alignment between the two.

I think they [pointing to members from SCFHS] should find a better way to educate undergraduate students. (#3)

The SCFHSs should adopt the SaudiMEDs and integrate the framework in the blueprint instructions. So candidates will be aware of the alignment. (#10)

Assess the Current Alignment Between the SaudiMEDs and the SMLE

The participants recommended mapping the current version of the SMLE to the SaudiMEDs to explore the extent and percentage of the alignment and identifying areas that needs improvement.

As a first recommendation, we need to do [an] alignment study because I am not sure [if SMLE is aligned with SaudiMEDs]. Even if we assumed it is aligned, the second questions will be to what extent or what percentage. (#6)

Improve the Governance of the SMLE

Clarification of the SMLE rationale and details, such as its reasons, necessity, mission, scope and magnitude has to be communicated to the stakeholders.

I am [satisfied] and fine but [SMLE] needs more work in term of governance as far as I know. (#5)

The use of an independent quality assurance agency or external members to ensure the alignment of the SMLE and the SaudiMEDs.

It [in reference to alignment between SMLE and SaudiMEDs] can be done also by asking an independent member from outside the educational institution for the evaluation [of alignment]. (#7)

The Plan for a Smooth Transition from the Undergraduate to Postgraduate Assessment

Developing a plan to bridge the gap between the undergraduate and postgraduate assessment as a continuum practice would be a great step for better alignment between the SMLE and the SaudiMEDs.

I always ask for the link between the undergraduate and postgraduate. [To have] this link [stakeholders] has to cooperate to have this harmony. (#5)

The participants highlighted using curriculum development efforts for paving the way toward an optimum alignment between the SMLE and the SaudiMEDs.

Also incorporating SaudiMEDs in undergraduate curriculum is a must and [it] has to be the case. (#4)

Expand and Advance the Current SMLE Format

In addition, some important skills, such as the clinical examination (which are vital for patient care competency of the SaudiMEDs) are not currently considered in the SMLE. To consider such important skills and other missed competencies, the participants suggested adding other assessment aspects to the SMLE, such as an oral examination or OSCE, which can be considered as SMLE part II.

Instead of SMLE being written exam, I think that some part of it should be an oral examination or OSCE station. (#1)

The SCFHS [should] add OSCE to test for these two competencies physical examination and procedures. (#3)

Discussion

An exploration of the opinions of the key stakeholders, the COMs and the graduated students, about the integration and alignment between the SaudiMEDs and the SMLE, was the main aim of this study. In general, most of the medical students, residents and faculty were knowledgeable about both the SaudiMEDs and the SMLE. This familiarity is logical because the establishment of the SaudiMEDs framework has been initiated and advocated for by most public and private schools of medicine curricula in Saudi Arabia.14 Regardless the familiarity with the SMLE, the findings indicated that the participants perceived themselves to have a good level of knowledge of the SMLE blueprint, as it is officially posted in the SCFHS website. The student group was more knowledgeable about the content of the SMLE, compared with the COM faculty group. This could be explained by the students’ acquiring the content in multiple examination attempts. In addition, as the assessment drives learning, the students are more motivated to know about the SMLE content to prepare for the examination. They access sampling questions through subscribing to the SCFHS, which they use as a form of mock examination. The students also participate in the courses organized by the SCFHS regarding the SMLE.

The first theme revealed that currently nothing is officially done to directly align the SMLE to the SaudiMEDs. The absence of alignment practices resulted in a level of stakeholder resistance. Available literature suggest that stakeholder resistance may occur due to several reasons, such as lack of information, disagreement with available data, and confusion of what the meaning or reason is behind the intervention or project.15 In their responses, the participants indicated two of the reasons, including the lack of information required and the purpose of the alignment process. This might indicate that other stakeholders were not involved in the process of constructing the SMLE. The data showed that all the current practices are indirect, which may imply that they are limited and not expected to ensure a proper level of alignment. It should be noted that the majority of the participants reported being partially satisfied with the current practices, as they perceive the knowledge domain as the most important domain in the SaudiMEDs.16

In the second theme, (Competencies enhanced through preparing for the SMLE), more than two thirds of student group indicated that the SMLE enhanced some competencies in the SaudiMEDs. They mainly referred to the Scientific Approach to Practice and the Patient care competency. This enhancement occurred mainly through augmenting their basic knowledge. This indication is understandable because the format of the existing SMLE is MCQs. It is not uncommon for a high-stake examination to improve or boost the knowledge construction of the students by serving as an extrinsic motivation, as explained by the Self-Determination Theory (SDT),17 however, being knowledgeable is not enough to be competent. The current examination is limited by focusing only on one segment of a competency. Through SDT, literature suggested that high stake tests could serve as motivation to enhance competencies of interest.18 The SMLE has to be expanded to achieve its potential and consider the skills that are important to the Scientific Approach to Practice and Patient care competencies or other competencies, such as communication and professionalism. This consideration may require including other assessment formats, for example a live encounter such as an OSCE.

The limited enhancement of some the SaudiMEDs competencies in the SMLE, as perceived by the participants, indicates that the SMLE and the SaudiMEDs alignment is modest and require significant improvement. Such an alignment could have formative and educational value. For instance, the alignment would provide insight for medical schools facilitating an evaluation of the need for enhancing resources, improvement of theoretical and practical teaching and subsequently, the learning outcomes of their current curricula. The alignment would enable students to detect deficiencies not only in their knowledge, but also in their competencies in the context of the SaudiMEDs.18

The third theme was the SaudiMEDs representation in the SMLE. Most of sample declared that there was a partial advocacy of the SaudiMEDs domains in the SMLE. The bulk of the composition of the SMLE was knowledge assessment of some competencies, namely Patient care and Approach to Practice, and to a minimal degree other competencies such as Research and Ethics. This minimal representation can be acceptable for safe practice as mentioned in the literature.18 Regardless of an acceptable representation, all competencies should be considered when constructing the SMLE.

A third of the sample expressed the absence of any evidence of endorsement of the SaudiMEDs in the SMLE, which could be due to their personal judgment of the quality of the SMLE. They believed that the SMLE items only focused on assessing knowledge, and no psychomotor skills. The SMLE overlooked the assessment of important competencies, such as communication and clinical skills. This judgment of the SMLE appears as the main reason for indicating no alignment, rather than a partial alignment.

The final subtheme resulted from a small proportion who declared not having an opinion or not being sure of the alignment status, due to the lack of the required information to make a judgment. The lack of resources and information to raise awareness of the alignment presents a serious deficit in the communication with the key stakeholders. As mentioned in theme one, such a deficit can cause resistance, which may contribute to the failure of the efforts to align the SMLE and the SaudiMEDs.

In an attempt to gain a deeper understanding of the perceptions of the participants regarding partial or no alignment, generated the following reasons:

The current SMLE format is not corresponding to the SaudiMEDs Framework, which contains the competency framework that underpins most of the COM competency-based curricula (CBC).

Though knowledge acquisition is the main predictor of educational fulfilment,19 the SMLE is expected to be a comprehensive assessment approach that should include assessing the most important competencies. Stated differently, the SMLE is a licensing examination that needs to ensure that only competent future physicians are able to practice.20

In the final theme, the Roadmap for Optimum Alignment between the SaudiMEDs and the SMLE, recommendations were suggested to improve the current situation between all key stakeholders, to achieve an ideal alignment of the SMLE with the SaudiMEDs competency framework.

The first recommendation was ensuring the involvement of the stakeholders in the development of the SMLE and the alignment process of the SMLE with the SaudiMEDs. This cooperation, however, cannot exist without having clear dialogue between all concerned parties to ensure the easy and successful execution of the plans and achieving mutual targets.21 The participants suggested setting regular meetings between the key stakeholders to ensure constant communication throughout the alignment process.22

The second recommendation related to the fact that there is no official notification regarding the current situation of the alignment between the SMLE and the SaudiMEDs. There was a general agreement about the need to assess the current alignment status between the SaudiMEDs and the SMLE through appropriate mapping to identify deficits and planning for improvement.

The third recommendation was uncertainty regarding the goals, mission, laws and policies of the SMLE examination. This lack of information emphasizes the need to clearly set and re-evaluate the scope of the SMLE and its boundaries.23 There is an online instruction reference for the SMLE.11 However, it is brief, limited and lack important information of interest to stakeholders. It should be updated with detailed instructions regarding the blueprint, to support the expectations of the stakeholders.

The fourth recommendation highlights the need for coherence between the undergraduate and postgraduate assessment systems, through improved preparation of undergraduate students for the SMLE. Evidence in current literature supported the benefit of such preparation, which should be considered while planning for both the undergraduate assessment as well as the SMLE.24,25 Achieving a smooth transition and coherence between the under- and postgraduate assessment require a high level of cooperation between the COMs and the SCFHS to ensure that the student will pass the SMLE.

In the fifth recommendation, the participants called for a reform of the current SMLE format. The existing SMLE disregards the assessment of important SaudiMEDs competencies, which cannot be tested with the current MCQs style. In fact, some of these competencies, such as the physical examination and professionalism require evaluating skills through a different assessment format, for example, an OSCE, and literature highlights the addition of these tools to the SMLE.26 The examination should be comprehensive enough to ensure that whoever is practicing in Saudi Arabia has an acceptable level of competence. Expanding the SMLE format would enhance its alignment with the SaudiMEDs. The students would invest more time and effort to master most of the SaudiMEDs competencies, in addition to the cognitive aspects, which is currently assessed by the MCQs format.27,28

In conclusion, a concern is the inexplicit relationship between the SMLE and the COMs curricula, considering that the assessment derives from learning. It is logical to request the unification of the efforts of the COMs to ensure an improved alignment of the curricula with the SaudiMEDs. Increased effort is also required to ensure that the SMLE is aligned with the SaudiMEDs. Collectively, these efforts can result in an appropriate alignment between the SMLE with the SaudiMEDs (Figure 1).

Figure 1

Indicates That Aligning Both SMLEa and COMsb Curricula with SaudiMEDsc Can Ultimately Improve Alignment Between the Two.

The research team supports the views of the participants in emphasizing the importance of extending the SMLE to assess as much competencies as possible from the SaudiMEDs framework. This recommendation will directly support the relationship between the curricula and the SMLE. The annual SMLE result ranking report of the national medical schools issued by the SCFHS should be used by the medical schools as a reference benchmark to improve their alignment process to reflect positively on the performance of their students when assessed through the SMLE.29

Limitations of the Study

Two limitations were identified; firstly, the study did not include the perceptions of members from the SCFHS. The Commission’s members are vital stakeholders as they are the SMLE committee participants. Their absence reduces the result’s indisputability. It is worth noting that the research team experienced challenges in inviting the SCFHS members to participate in the study. Secondly, the official implementation of the SaudiMEDs in the country started in 2015, and due to the short duration, the implementation of the framework is limited by some of the stakeholders, which may have affected their perception.

Recommendation for Future Research

Alignment and integration studies are important references for good quality and guidance for improvement of any educational project.30 In the context of no previous study regarding the mapping between the SMLE blueprint and contents with the SaudiMEDs, it is recommended that future research should focus on how to bridge this gap.

Conclusion

The outcome of the study indicated clearly that the integration and alignment between the SaudiMEDs and the SMLE were suboptimal. The current situation is below expectation, causing a lack of satisfaction in the stakeholders. There are valuable recommendations by experts in the educational process of the country to improve the current situation between all stakeholders for an ideal SMLE alignment with the SaudiMEDs competency framework. This study emphasizes the alignment between the SMLE and the SaudiMEDs as a high-stake endeavor that requires project management strategies to ensure a successful planning, development, and implementation of a wide variety of operations and tasks.