Evaluation of a faculty development workshop aimed at development and implementation of a competency-based curriculum for medical undergraduates.

INTRODUCTION: Faculty development to implement competency-based medical education (CBME) is urgently needed as the Medical Council of India has implemented the competency-based curriculum this year onwards.
OBJECTIVES: To evaluate a 2-day faculty development workshop in terms of: (a) increase in knowledge about CBME terminology and concepts, (b) self-reported capacity to develop and implement a competency-based module in their respective disciplines, and (c) satisfaction of the participants.
METHODOLOGY: A single arm interventional study using mixed methods was carried out in which faculty members were purposively identified and requested to volunteer for a two-day faculty development workshop on the development and implementation of CBME. The workshop was evaluated (open-ended and Likert scored items) by the participants for self-reported gain in knowledge, gain in their confidence to develop and implement CBME, and level of satisfaction with respect to the components of the workshop. Quantitative data was analyzed by Wilcoxon signed rank test and Kruskal-Wallis test. Qualitative data was analyzed by doing content analysis and emerging themes have been presented.
RESULTS: Eleven faculty members attended a capacity building workshop for developing and implementing a competency-based curriculum for medical undergraduates. There was a significant improvement in their self-reported knowledge and attitude regarding the competency-based curriculum. New concepts learned fell into four domains: modification of the existing curriculum toward competency-based curriculum, knowledge of CBME, teaching-learning and assessment methods, and beneficial to the students. The participants were highly satisfied with the workshop in its current form.
CONCLUSION: A two-day faculty development workshop can increase the knowledge and understanding of competency-based medical education and can be an important first step in the journey to more specialized training. Copyright:

Introduction

The need for competency-based medical education and training (CBME) is being felt at an international level with the goal being to produce physicians who are competent to practice. In other words, not only should they possess the required knowledge and skills but they should also demonstrate the ability to apply them in independent practice. Replacement of traditional medical education curricula with CBME has been shown to be a viable and effective change.[12] The Medical Council of India, too, has initiated change by implementing a CBME curriculum across India from this year. The Indian Medical Graduate is expected to undertake the responsibilities of a primary care physicians/doctors of the first contact who is able to provide preventive, promotive, curative, palliative, and holistic care with compassion.[3] The prerequisite is to design and implement a curriculum that focuses on the acquisition of desired and observable abilities in real-life situations and assesses learners for the same.[4]

CBME is vastly different from the traditional curriculum as it focuses on learner centeredness, is not time-dependent, and its nature is multi-dimensional, dynamic, developmental, and contextual; thus, it has a significant learning curve and training is required for its successful implementation.[56] It is recognized that current faculty is insufficiently equipped and a global initiative of faculty development is needed around CBME.[78]

We report the results of a workshop that was designed around the hypothesis that a two-day workshop could lead to capacity building of the faculty for the development and implementation of competency-based modules for undergraduate medical students. The primary objectives were to evaluate the effect of the workshop on the participants in terms of: (a) increase in knowledge about CBME terminology and concepts and (b) self-reported capacity to develop and implement a competency-based module in their respective disciplines. The secondary objective was to determine the satisfaction levels of participants with the content and the methods employed in the workshop and their suggestions for improvement.

Methods

This single arm interventional study using mixed methods (quantitative and qualitative) was conducted at the University College of Medical Sciences, Delhi, in July 2015. After approval from the Institutional Ethics Committee–Human Research, email invites were sent to faculty members of the institution. Approval of the ethics committee was obtained. It was in 2015 itself. Since the workshop depended on participants having a basic knowledge of educational principles, the selection of participants was purposive (non-random). A list of faculty who had already attended the basic medical education workshop recognized by the Medical Council of India was acquired from the Medical Education Unit of the Institution and email invites were sent to them (n = 25). The invitation to volunteer for the workshop included the background and aim, the proposed program, and the expected outcomes. It was mentioned that there was no registration fee and that the Delhi Medical Council had accredited the two-day workshop with 7.5 credit hours.

The workshop module had been developed in-house by the authors. Employing the concept of moving the participants from the known to the unknown when teaching a new concept, we built the knowledge and understanding of competencies on the bedrock of learning objectives. The teaching-learning methods adopted were primarily interactive PowerPoint-based presentations, group work, and group discussions.

Volunteering faculty were provided resource materials (published articles on competency-based medical education) beforehand via email. These and some other articles were also given to the participants as hard copy handouts when they came to attend the workshop.

On day one of the workshop, the first four sessions included: an icebreaking session, an introduction to competencies and competency-based curriculum, a recap of Bloom's domain of learning and Kirkpatrick's evaluation model, and a session on methods of teaching and assessment. Thereafter, the participants were divided into groups of two to three faculty members who worked together with a facilitator (one of the authors facilitated each group) to discuss the framework of developing a competency-based curriculum and what strategies could be used for its implementation. At the end of the first day, the groups were asked to frame a competency about their specialty.

The second day was dedicated to group work wherein participants developed the entire competency-based curriculum module for undergraduate medical students. Each group's module was discussed within the groups and then was presented to all groups who then critiqued it to give it a final shape.

Signed informed consent was obtained from the participants for the use of all generated data for the purpose of reporting the results of the study. Privacy and confidentiality of the data was maintained. The workshop was evaluated as follows:

At the end of each session, the participants rated their satisfaction with various components of the session on a five-point Likert scale (1 = highly unsatisfied to 5 = highly satisfied).

A retrospective pre-post test questionnaire was used to assess their self-perceived change in knowledge and understanding of competency-based medical concepts. This questionnaire was made available to them at the end of the two-day workshop.

On the last day of the workshop, an open-ended questionnaire sought participants’ self-reported ability to develop and implement a competency-based medical undergraduate module (CBMUM) in their respective disciplines. There was an open-ended section for them to draw a timeline for implementation in their departments and to give feedback on what went well and what could be changed to make the workshop more effective in bringing about learning.

The questionnaires were designed through discussions between the authors and pretested and validated for content and face validity.

The outcome measures were:

Pre and post knowledge and confidence scores of the participants about competency-based medical education.

Themes emerging from their responses to the new concepts learnt by them during the workshop.

Satisfaction levels of the participants with the sessions of the workshop.

Analysis of data

Data was entered in an MS Excel spreadsheet. SPSS 20.0 and R-software version 3.5.3 for Windows was used for quantitative data analysis. Descriptive analysis was done, and the results are presented in the form of medians with interquartile ranges and proportions. Non-parametric tests of significance were used to compare the quantitative data. For paired data, Wilcoxon signed rank test was applied to compare two groups. Kruskal–Wallis test was used for comparing means of more than two groups. P values less than 0.05 were considered significant. Reliability analysis of the questionnaire on the satisfaction of the participants with respect to different sessions was done using Cronbach's alpha and is presented along with its 95% confidence interval. Qualitative analysis was done by doing content analysis of the responses obtained from open-ended questions to find out the emerging themes. Open coding was done in an iterative manner, and similar or overlapping codes were merged or modified through discussion between the authors and then classified into overarching themes. The themes with some quotations have been presented in the results section.

Results

Twenty-five eligible faculty members were identified and were invited to participate in the workshop. Fourteen of them registered; however, three could not attend the workshop: one due to illness, and two others due to some other pressing assignment. Thus, eleven faculty members finally volunteered; seven were Assistant Professors, two Associate Professors, and two Professors. The departments they represented included Physiology, Pharmacology, Pathology, Microbiology, Community Medicine, Otorhinolaryngology, and General Medicine.

When the retrospective pre and post workshop scores were compared, there was a significant self-reported improvement in the knowledge and understanding of a competency-based curriculum [Table 1].

Table 1

Comparison of scores of the retrospective pre-post test questionnaire on self-reported knowledge and understanding of competency-based curriculum (CBC)

Items	Median (IQR) scores before the workshop	Median (IQR) scores after the workshop	Median (IQR) scores of Pre-Post score differences	P (Wilcoxon signed rank test)
My knowledge about the term “competency” in the context of medical educational	2 (1.75, 2)	4 (4, 4.5)	3 (1.5, 3)	0.003*
My understanding of the difference between “competencies” and “objectives”	2 (1.5, 2.5)	4 (4, 5)	3 (1.5, 3)	0.007*
My knowledge of appropriate teaching-learning methods for a CBC	3 (3, 4)	4 (4, 4.5)	1 (0, 1)	0.03*
My knowledge of appropriate assessment methods for a CBC	3 (3, 4)	4 (4, 4.5)	1 (0, 1)	0.01*
My ability to design a competency-based curriculum	1 (1, 2)	3 (3, 4)	2 (3, 4)	0.002*
My level of Knowledge regarding curriculum	3 (2, 3)	4 (3, 4)	1 (1, 1.5)	0.002*
My understanding that competency-based medical education has the potential to make medical education more meaningful to the student	3 (1.5, 3)	4 (4, 5)	2 (1, 3)	0.005*
My understanding that competency-based medical education has the potential to make medical education more meaningful to society	3 (1.5, 3)	4 (4, 5)	2 (1, 3)	0.005*

IQR=Interquartile Range with 25th and 75th percentile values; *Statistically significant

The median (IQR) of self-reported confidence scores of the participants in developing a competency-based module for medical undergraduates was 4 (4, 4) while in implementing such a curriculum was 4 (1.5, 4). Wilcoxon signed rank test showed that there was no significant difference between these two scores (P = 0.06).

Satisfaction level, across all sessions, was high; the median (IQR) satisfaction scores ranged from 4 (4, 5) to 5 (4, 5) [Table 2]. The internal consistency of the satisfaction level measurement instrument was high as the Cronbach's alpha was 0.8 (95% CI: 0.7–0.9).

Table 2

Mean satisfaction scores of the participants with respect to certain session characteristics for each of the teaching-learning sessions of the workshop

Name of the session	Characteristics of the sessions
	Time allotted to the session	Content covered	Facilitator’s engagement with the participants	Resource material provided	Overall satisfaction
	Median (IQR)	Median (IQR)	Median (IQR)	Median (IQR)	Median (IQR)
	Competencies and competency-based curriculum	4 (4, 5)	4 (4, 5)	5 (5, 5)	4 (4, 5)	5 (4, 5)
Domains of learning and assessment	4 (4, 5)	4 (4, 5)	5 (5, 5)	4 (4, 5)	4 (4, 5)
Teaching-learning methods and assessment methods	5 (4, 5)	4 (4, 5)	5 (4, 5)	4 (4, 5)	4 (4, 5)
Framework and implementation strategies of competency-based curriculum	4 (4, 5)	4 (4, 5)	5 (4.5, 5)	4 (4, 4.5)	4 (4, 5)
P (Kruskal-Wallis test)	0.86	0.98	0.47	0.48	0.87

IQR=Interquartile Range with 25th and 75th percentile values

New concepts learned by the participants because of the workshop fell into four domains (illustrated with exemplary quotes):

1. How the existing curriculum could be modified toward a competency-based curriculum:

“Preparing SLO (specific learning objectives) and linking it to (current) teaching-learning methods and assessment methods.”

“One can link the present curriculum with competencies.”

“Good way to structure the undergraduate module.”

2. Knowledge and understanding of competency-based medical education improved:

“Learned how to make a competency statement”

“Competency-based and outcome-based curriculum designs”

“Understood what competencies are”

“Demonstration of ability is the goal”

“Doable rather than wishful”

3. Knowledge of teaching-learning and assessment methods for a competency-based curriculum improved

“Learned of the assessment and teaching-learning methods”

“Miller's pyramid and Bloom's domains”

“Feedback is important”

4. Belief in the benefits of a competency-based curriculum

“It will help a student in the long run as what is expected from a student is known”

The themes which emerged from the responses to the strengths of the workshop were:

Learning by doing technique:

“Do it yourself format.”

“Learnt how to make a competency.”

“Group tasks.”

Discussion within and between groups.

“Good interaction with peers. Learnt how to make a competency.”

“Healthy discussion.”

“Intergroup discussion during the session of developing the competency-based curriculum module and its presentation.”

About scope for improvement to the workshop, the themes that emerged were:

1. Need for more resource materials

“Some more resource materials and some more group activities”

“More planning by the participants before the workshop.”

2. Need for more group tasks

“Some more resource materials and some more group activities”

3. Need for having an integrated curriculum for competency-based medical education

“More emphasis on competencies derived from the integration of departments.”

“Include all faculty members from different disciplines.”

4. Need for follow-up workshops

“Have another follow-up workshop.”

“Follow-up of the implementation of the plan derived in the workshop.”

Discussion

Faculty has a vital role in facilitating the learning of medical graduates toward becoming the primary health care physicians of the future. Being able to implement CBME might help in achieving this goal. This study attempts to evaluate a two-day faculty development workshop for implementing a competency-based curriculum. The literature is lacking on this front both in India and globally, and we could not find any study which reports the structure and the effect of a faculty development workshop around the topic of competency-based medical curriculum. In India, a three-day faculty development workshop (Basic Course Workshop) was earlier being conducted for the medical faculty through the nodal and regional centers of the Medical Council of India. Since 2014, however, these workshops have been changed to a four-day version (revised Basic Course Workshops: rBCW) wherein two days are dedicated to competency-based medical education (CBME) and the teaching of the attitude, ethics, and communication (AETCOM) module. Thus far, no center has evaluated the effect of the rBCW training in competency-based medical education, although a baseline evaluation of a rBCW did discover that only 3.8% of the faculty participants were aware of CBME.[9] That bit of data, considering that the study was conducted about the same time as ours, is surprising given that the medical education community has been abuzz with the requirement to move to competency-based education for quite some years now. Clearly, there is a need to promote research in this exciting, though challenging, field and this paper sets out to detail how a basic training workshop can help in this regard.

The two-day workshop for faculty led to a significant self-reported improvement in knowledge of and an enhancement in the attitude toward competency-based curriculum for medical undergraduates. Acquiring knowledge and developing a positive attitude regarding the competency-based curriculum is an important first step toward implementing the concept of CBME.

Self-reported confidence to develop and to implement the competency-based module was just above average. The qualitative responses also highlight that the participants are keen to have greater resources and more future workshops on CBME. Their lack of confidence is understandable since this was the first faculty development workshop on competency-based medical education attended by the faculty participants. It seems that for those who are uninitiated, a two-day workshop is not comprehensive enough to make them confident to develop and implement a CBM. We understand that self-reports on improved teaching skills do not necessarily reflect a change.[10] However, self-reports do show the degree of ownership and self-assessment by the respondents which is important in medical education. More longitudinal studies are needed to assess long-term impacts of such workshops. With the recent introduction of a competency-based curriculum for medical undergraduates all over the country, it is possible and necessary to design such studies. This report is an important precursor and may encourage greater endeavor in this nascent field in our country.

The faculty members gave high satisfaction ratings for all the sessions. This is particularly encouraging since there were no workshop modules available in the literature, and we had to develop our own model as well as design the strategies to train faculty in the concepts and the implementation of a competency-based curriculum in our medical college. There were no statistically significant differences between the different sessions assessed, suggesting that the quality of the teaching-learning sessions was maintained uniformly throughout the workshop. We had kept the feedback anonymous to eliminate response bias and increase its reliability. The interactive nature and activity-based sessions may be the reason for high satisfaction among the participants. The vast medical education experience of the medical educators who were all active members of the Medical Education Unit of the college might also have contributed to enhancing the quality of the sessions.

This interaction motivated the faculty members toward developing competency-based medical undergraduate module. The faculty members at the end of the workshop opinionated that the modification of the existing curriculum toward competency-based curriculum is possible. They also mentioned that it will benefit the students.

Learning by doing and interactive sessions were the key strengths which emerged from the qualitative data analysis. Adult learning principles, such as these were utilized while developing and implementing this faculty development workshop.[11] The participants’ responses that these were the strengths of the workshop show that these principles were executed effectively throughout the implementation of the workshop.

The participants’ feedback regarding the improvement of the workshop revealed that they expected more resource materials and follow-up workshops. This indicates that they are eager to learn more about CBME. It seems that even though the participants were satisfied with the workshop, they were not fully confident whether they would be able to implement it and so expected to have more resource materials and capacity building training to implement CBME. The competency-based curriculum being an entirely new concept needs to be reinforced with multiple workshops and built upon their experiences as it is implemented in the country. Right now, the Medical Council of India has mandated that all the faculty members of all the medical colleges of India should undergo the revised Basic Course workshop which includes one day of orientation toward Competency-based curriculum and the Curriculum Implementation Support Program aimed at empowering medical faculty to implement the competency-based module in their respective departments.[12] Our research suggests that a one-time training will not be sufficient to fully enable medical faculty in implementing CBME and multiple workshops may be required.

The increase in knowledge, high satisfaction levels, and the emerging themes from the participants’ responses regarding the newer concepts learned, strengths, and limitations of the workshop hints toward acceptance and build-up of the concept of competency-based curriculum among the participants.

It is to be noted that generalizability of this study is limited, as it is a single-center study, with purposively selected faculty members as participants and in an institution which has a functional Medical Education Unit; nevertheless, the results are important in that they validate a preliminary workshop geared toward introducing novice faculty to CBME.

Conclusions

Based on the self-reported gains of a small group of participants, it seems that a two-day workshop can increase the knowledge and understanding of competency-based undergraduate curriculum among the faculty members. More research is needed which assess the effect of faculty development workshops regarding CBME.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.