Qualitative study to identify the perception and challenges faced by the faculty of community medicine in the implementation of competency-based medical education for postgraduate students.

OBJECTIVES: To identify perception about the key aspects of competency-based medical education (CBME) among community medicine faculty members and to ascertain the various challenges faced by them during its implementation.
METHODS: A descriptive qualitative study of 2 months' duration was conducted among the faculty members of the community medicine department. Non-probability purposive sampling was employed in the study. Free listing was done initially to elicit the views of faculty members to meet the intended objectives. Visual Anthropac software was used to identify the salient variables using Smith's Salience Score, and then pile sorting was done to identify the association between the salient variables.
RESULTS: Three faculty members participated in the free listing and pile sorting. A total of 20 responses were obtained pertaining to the key aspects of CBME, of which 12 were identified as the salient variables depending on the cut-off value of 0.125 (Smith's Salience Score) and subjected to pile sorting. Similarly, eight challenges were identified in the implementation of the programme during the free listing, and all were included in the second stage of pile sorting. Cognitive maps were drawn to understand the relationship between the key aspects of CBME and involved challenges separately.
CONCLUSION: On employing the free listing and pile sorting methods, formulation of entrustable professional activities and their assessment using appropriate tools were the identified crucial areas in CBME, while the lack of sensitisation of stakeholders and inadequate planning were identified as the predominant challenges in the implementation of CBME. © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Introduction

Competency-based medical education (CBME) is an approach to ensure that the medical student develop those competencies which are desired to meet the needs of patients in a community and at the same time meet international standards.1 2 A total of five core competencies have been identified in CBME in the Indian set-up, namely clinician, communicator, leader and member of the healthcare team, lifelong learner, and professionalism.1 2 CBME has attracted immense attention as it negates the multiple limitations attributed to the traditional mode of education delivery.2 CBME enables the alignment of teaching-learning with assessment in authentic workplace settings.3 4 However, regardless of the multiple associated advantages and acknowledging the need for the same in the field of medical education, not many universities or institutes across India have yet adopted the same.1 3

The delay in the adoption of the CBME within their set-up could be due to the various challenges which have been encountered earlier.1–6 These include a supportive administration, absence of a vision and a plan to bring about the reforms in curricular delivery, infrastructure and learning resources, reluctant teaching staff, resistance to change, no guidelines from the regulatory body for the mandatory implementation, financial support, poor coordination between the undergraduate and postgraduate curriculum, untrained teaching staff, no comprehensive plan to streamline student assessment, and lack of support from the medical education unit of the medical colleges.3–9 CBME has been implemented in the field of public health, epidemiology and as part of postgraduate residency programmes across different institutes.10 11

Since 2016–2017, CBME for postgraduate students has been implemented in our institute, including in the Department of Community Medicine. In fact, the faculty members of the department has formulated 50 entrustable professional activities (EPAs). However, in the initial year itself, problems in the field of assessment have surfaced. During this time of change management, the current study has been planned with the objectives to identify perception about the key aspects of CBME among faculty members and to ascertain the various challenges faced by them during its implementation. The findings of the study will aid in the better implementation of the programme in the department, and will be of great help to other postgraduate (PG) departments of the institute and other teaching institutions which are planning to implement CBME in the future.

Study design

This is a descriptive qualitative study.

Study duration

The study was conducted for 2 months (June–July 2018).

Study area

The study was conducted in the Department of Community Medicine.

Study population

The study population included faculty members of the Department of Community Medicine.

Inclusion and exclusion criteria

The study included faculty members who are trained in the basics of CBME and are involved in the implementation of the programme for at least 6 months. Out of the six faculty members in the department, three were untrained (as they were recruited in the last 1 month in the institute) and were thus excluded from the study.

Sampling method

The study used non-probability purposive sampling.

Free listing: To identify perception about the key aspects of CBME and the challenges involved in its implementation.

Pile sorting: To establish a relationship between the identified key aspects and the salient challenges.

Approval from the Medical Research Unit and Institutional Ethics Committee.

Free listing.

The study involved two stimulus research questions: (1) In your opinion, what are the key aspects of CBME? (2) In your opinion, what are the challenges which are preventing the successful implementation of the CBME in the department?

The access protocol was developed and information was collected by the principal investigator in a face-to-face interaction. At the start of the interaction, a brief and easy instruction was given to the participants about what is expected of them. The research questions were read to the participant one after the other, and it was ensured that they understand the question. Further, it was emphasised that the exercise is not a test of their knowledge. Participants were given the option to record their responses on their own in writing (in short sentences). All the three probes, namely silent probe, echo-probe and Uh-huh probe, were used to elicit maximum responses from the participants. Once the responses were recorded, the investigator reread the same to the participants and ensured that they themselves are clarified about what the participants meant about them.

The responses obtained from the participants were analysed using the Visual Anthropac software to identify those items which are prominent and representative of the cognitive domain. The Smith’s Salience Score was calculated, and depending on the elbow (cut-off) observed, salient variables were selected for the next step of pile sorting.

Pile sorting.

This step was done to identify the similarities and differences among the recorded items as perceived by the participants. The idea was to convert a large number of items into specific themes (groups). The free pile sorting method was adopted for the current study, in which the pile sorting was done on a one-to-one basis. The identified salient items were written on the cards (front side), while on the back side numbers were mentioned to enable data entry. Initially, the participants were made to relax by telling that it is a game and not a test of their abilities. All the cards with the item name were placed in front of the participant and they were asked to group them together using their own criteria.

It was ensured that the participants understand what is written on the card and they were even asked to read the same for confirmation. Once the groups were formed, they were asked to explain why they grouped them in that particular way and their responses were recorded in the recording format. Also, each time, the pack of cards was shuffled before giving to the next participant. In this step, no probing was done and participants were given the chance to change the pile, if they wish to. The participants were allowed to rearrange the piles. The obtained results or categories were again subjected to analysis using the Visual Anthropac software, and cognitive maps were drawn to identify a meaningful relationship between the salient variables.

Methodology

Statistical analysis

Qualitative analysis was done, and Visual Anthropac software was used to interpret the results of the free listing and pile sorting.

Ethical considerations

Consent was obtained from the faculty members before enrolling them into the study.

Results

The current study was carried out involving three faculty members who were already trained in CBME as part of the Revised Basic Medical Education Workshop. All the included faculty members have been part of the implementation process for the last 2 years. Free listing 1 with a stimulus research question of eliciting the opinion of faculty members about the key aspects of CBME was done. A total of 20 responses were obtained from three respondents (table 1), which were then fed into the Visual Anthropac software. A Smith Salience Score of 0.125 was taken as the cut-off and 12 salient variables (out of the 20 listed) were selected and eventually subjected to the second stage of pile sorting.

Table 1

Free listing: key aspects of CBME

Responses obtained
Respondent 1	Respondent 2	Respondent 3
Involved assessment methods. Acts as a self-assessment tool. Constructive feedback. Reflection. Identification of must-know EPAs. Focus on acquisition of skills.	Measurable. Year-wise grading of student possible. Emphasis on formative assessment. Self-assessment tool. Specific importance to ATCOM module. Computer skills required. Not friendly for senior faculty.	Student-oriented. Skills-focused. Skills from all 3 domains can be assessed. Constructive feedback. Syllabus can be broken down to individual EPAs. Formative assessment. Focus on the final outcome.

ATCOM, Attitude and Communication Module; CBME, competency-based medical education; EPAs, entrustable professional activities

On a similar note, free listing 2 was done with an aim to elicit the opinion of three respondents regarding the challenges faced by them in the implementation of CBME in the department. Eight challenges were identified by the three respondents (table 2), which were subsequently fed into the software. All challenges were selected for the second stage of the study to identify the association between the enlisted challenges.

Table 2

Free listing: challenges faced by faculty members in the implementation of CBME in the department

Responses obtained
Respondent 1	Respondent 2	Respondent 3
Formation of rubrics of milestones. No periodicity in assessment. Lack of clarity about CBME among faculty. Lack of clarity about CBME among Postgraduates (no sensitisation).	Each month specific time to be allotted by faculty for assessment.	No involvement in formation of EPAs. No guidelines about assessment of EPAs. Dilemma about the scope of CBME in final exams.

CBME, competency-based medical education; PGs, postgraduates.

The salient items identified during free listing 1 were used for pile sorting 1. Overall, the items were grouped in three to four categories by three respondents (table 3). The reasons cited for categorisation were assessment-related, EPAs-related, workplace-based assessment (WPBA)-related, e-portfolio-related, skills required for EPAs and strengths of CBME. These observations were subjected to pile sorting analysis through the Visual Anthropac software, and a cognitive map was drawn (figure 1). The cognitive map revealed the distribution of the key aspects of CBME in three categories, namely assessment, EPAs and advantages of CBME.

Table 3

Pile sorting to ascertain the relation between key aspects of CBME

Respondent number	Pile as formed by the respondent	Reasons for the same
1	1, 2, 7, 8, 10, 12	Assessment-related.
	5, 6, 11	Related with EPAs.
	3, 4, 9	Strengths of CBME.
2	1, 6, 10, 12	Related with WPBA.
	2, 5, 7, 8, 11	Related with EPAs.
	3, 4	Related to e-portfolio.
	9	Separate.
3	1, 2, 7, 8, 10, 12	Related with assessment of EPAs.
	5, 6, 11	Skills required for EPAs.
	3, 4, 9	Related to feedback.

CBME, competency-based medical education; EPAs, entrustable professional activities; WPBA, workplace-based assessment.

Figure 1

Key aspects of CBME. CBME, competency-based medical education; EPAs, entrustable professional activities.

Similarly, the salient items identified during free listing 2 were used for pile sorting 2. In general, all three respondents grouped the challenges in three categories (table 4) due to reasons like assessment, formation of EPAs, faculty-related, student-related, lack of clarity among stakeholders and so on. Once again, these observations were subjected to pile sorting through the software, and a cognitive map was drawn (figure 2). The cognitive map depicted the challenges faced under three broad categories, namely the involvement of faculty, poor planning and assessment-related (figure 2).

Table 4

Pile sorting to assess the relationship between the identified salient challenges

Respondent number	Pile as formed by the respondent	Reasons for the same
1	2, 5, 7	Assessment-related.
	1, 6	Formation of EPAs.
	3, 4, 8	Negative aspects.
2	3, 5, 6	Faculty-related.
	1, 2, 7, 8	CBME-related.
	4	Student-related.
3	3, 4, 6, 8	Lack of clarity among faculty and postgraduates.
	1, 2, 5, 7	Assessment.

CBME, competency-based medical education; PGs, postgraduates.

Figure 2

Challenges in competency-based medical education (CBME).

Discussion

The present qualitative study was conducted among the faculty members of the Department of Community Medicine who were already sensitised in CBME and have been part of the implementation of the programme for at least 6 months. Free listing and pile sorting methods were adopted to obtain the detailed perspectives of the programme. The free listing and pile sorting methods have been adopted in different settings in the field of medicine.12–14 In short, these qualitative methods have been adopted to explore the cultural or the cognitive domains about any issue.

These selected faculty members enlisted factors pertaining to the assessment in terms of the WPBA tools which can be employed or that skill from all three domains of learning can be assessed, or that more emphasis is given on formative assessment and that the performance in the assessment can be used as a self-assessment tool. The findings of a study done in a postgraduate residency programme indicated that assessment in the workplace, followed by a feedback on the performance, was the crucial area of the CBME implementation programme.11 Significant emphasis has been given to assessment and its principles in order to ensure that competent healthcare professionals are produced.15 16

In the current study, respondents gave due emphasis to the issues that CBME is student-friendly, allows gradation of students on a yearly basis and that it helps in the identification of must-know EPAs. Similar findings were reported in studies done in medical schools from Sub-Saharan Africa and in an institute with public health and epidemiology courses, as part of the implementation of CBME.5 10 In fact, the component of flexibility has been highlighted in CBME even in higher education centres.17 In another study, a remarkable improvement in all the learning domains was observed among the undergraduate medical students who were exposed to CBME.18 Furthermore, participants from our study gave emphasis to the component of constructive feedback and reflections in the CBME programme, which actually differentiates it from the traditional mode of curricular delivery. On a similar note, most of the earlier conducted studies have identified feedback from the teachers and reflections from the students as the two pillars of CBME.1 5 6 11 18

The second free listing was done to identify the challenges which the faculty members encountered during the process of implementation of the programme in the last 2 years. It was reported in our study that the formation of the rubric of milestones for individual EPA is the most difficult task. These rubrics of milestones are the key elements which have to be framed precisely (and further have to be validated as well), so that the issue of subjectivity can be neutralised. In fact, milestones have been used in the assessment process across different settings.19 20 In the current study, lack of clarity among faculty about CBME or with regard to limited involvement in the formulation of subject-specific EPAs was identified as the challenges. In order to successfully implement the programme and sustain as well, it becomes really important to train the faculty and sensitise them about their roles and responsibilities.7 8

In our study, the faculty members even opined that the postgraduate students have also been not adequately sensitised and thus we are unable to achieve the desired outcomes. Once again, CBME is an approach that is learner-driven, and acknowledging this fact it is not right to initiate the programme without their sensitisation. Regardless of the target population (viz, postgraduate or undergraduate students), they have to be primed about CBME and adequately prepared so that the students can acquire the desired skills and become competent.9–11

The department faculty members reported that they have not received any guidelines from the university about the assessment of EPAs and thus they are finding it difficult to streamline the entire process. As we are shifting from the traditional curriculum to competency-driven curriculum, it is extremely important to ensure that the instructions pertaining to the assessment or their periodicity are clear, so that both the department faculty and postgraduate students can plan their schedule.21–23 The findings of another study emphasised on the need to have extensive curricular reforms and systematic planning before CBME is launched in the institute.24 As the regulatory body has still not made it mandatory for the medical colleges to implement CBME within their set-up, despite the implementation of CBME in the institute, a dilemma still prevails pertaining to the scope of CBME in final exams. Thus, it is high time that the Medical Council of India should soon come out with specific guidelines regarding the status of CBME in medical colleges.

The strength of the current study is that very few medical colleges in India have taken an initiative to launch CBME, and henceforth the results of the study will prove to be of great help for other medical institutions which are planning to implement CBME in their settings. Further, this is the first study of its kind in which qualitative research methodology has been adopted to explore the perspectives of faculty members who already have the experience of implementing CBME. However, the limitation of the study is that very few faculty members were involved, and it is quite obvious that we have not attained saturation. Nevertheless, many important key aspects of CBME and challenges have been identified, and it is a lesson for the administrators to soon rectify the existing shortcomings.

Conclusion

CBME for postgraduate residents has been adopted in the community medicine department of the institute. On employing the free listing and pile sorting methods, formulation of EPAs and their assessment using appropriate tools were identified as crucial areas in CBME, while the lack of sensitisation of stakeholders and inadequate planning were identified as the predominant challenges in the implementation of CBME.

What is the key aspects of Competency Based Medical Education (CBME) among Community Medicine faculty members and the possible challenges during its implementation in Tamil Nadu, India?

Formulation of Entrustable Professional Activities (EPAs) and their assessment using appropriate tools were the identified crucial areas in CBME on employing the free listing and pile sorting methods. The lack of sensitization of stake holders & inadequate planning were the main challenges in the implementation of CBME.

This study provides reference for the quality improvement of Competency Based Medical Education for postgraduate residents in Tamil Nadu. It is beneficial to the career development of local medical students.