The Acceptability and Feasibility of Mini-clinical Evaluation Exercise as a Learning Tool for Pediatric Postgraduate Students.

BACKGROUND: The mini-clinical evaluation exercise (Mini-CEX) is a valid and reliable tool that facilitates the assessment of skills essential for a physician and provision of immediate feedback. AIMS: This study aimed to assess the acceptability and feasibility of Mini-CEX as a learning tool for pediatric residents.
MATERIALS AND METHODS: Following the sensitization with the concept of Mini-CEX, the actual process of assessment of residents was done using the "standardized American Board of Internal Medicine Mini-CEX evaluation form." Feedback about the Mini-CEX was taken from the residents and faculty on separate questionnaires consisting of close- and open-ended questions. A total of 87 Mini-CEX encounters were done with 13 faculty and 29 residents over 6-month study period.
RESULTS: Residents perceived that it is a method that does the assessment of skills, prerequisite for good clinical performance with provision of immediate feedback. Most of the residents felt that it improved their clinical skills, uplifted the personal development, and impart a better one to one student-teacher interaction. Almost all the faculty were satisfied with this method of assessment. They found it useful for improved learning of themselves also. Both residents and faculty suggested to incorporate Mini-CEX in curriculum.
CONCLUSIONS: Mini-CEX is an acceptable learning tool as reflected by the residents and faculty. It is feasible to use mini-CEX for assessment of residents.

Introduction

The definitive goal of medical education is to impart the best health care to community.[1] At present, there is a paradigm shift in medical education, which focuses on competency-based medical education, empowering the learner to work with appropriate responsiveness in real-life patient scenario.[2] It requires an assessment method that assesses the trainee in real clinical settings. Workplace-based assessment (WPBA) is such an approach with its fundamental elements of direct observation, conducted in workplace and contextual feedback.[3] It conforms to the “Does” level, that is, the highest level of Miller's pyramid. WPBA is concerned with clinical skills, which are essential to diagnosis and treatment including history taking, physical examination, and certain soft skills, for example, communication skills, professionalism, or empathy.[4] The key component of immediate and contextual feedback following performance enables the trainee to direct their learning toward the desired outcome. A meta-analysis documents that feedback about the performance is a most important factor to influence achievement, with an effect size of 0.79.[5] However, the studies document that a minimal proportion of postgraduate students are observed during their actual clinical work.[6]

The mini-clinical evaluation exercise (Mini-CEX) is a valid and reliable WPBA tool that facilitates the assessment of skills that are essential for a physician and provision of immediate feedback.

This study was conducted to explore the use of Mini-CEX as a learning tool for improvement in the clinical competence of pediatric postgraduate residents. The primary objective was to assess the acceptability and feasibility of Mini-CEX as a learning tool for pediatric postgraduate students.

Materials and Methods

The study was conducted from May 2016 to October 2016 in the department of pediatrics at a tertiary level medical university in Northern India. Foremost the approval from the institutional ethics committee and written informed consent from the residents and faculty were obtained. The study was carried out with the 29 junior residents and 13 evaluators including 3 senior residents and 10 faculty. With the help of an audiovisual presentation, prior sensitization of residents and faculty about the actual process of Mini-CEX was done. A blueprint of mini-CEX sessions depicting the name of residents and faculty spread over a period of 6 months was made. It was planned in a way to ensure that a resident rotates through different evaluators.

This was followed by the conduct of actual sessions of Mini-CEX encounter using the “standardized American Board of Internal Medicine Mini-CEX evaluation form.”[7] The record of all such encounter was maintained. At the end, faculty and residents were asked to give feedback about their experiences with the Mini-CEX. The questionnaire consisted of closed-end questions where the rating was done using a 5-point Likert scale. It also includes open-ended questions including positive and negative points about Mini-CEX and suggestions to improve it further. The peer validation of questionnaire was done before utilizing it. Separate feedback forms were used for faculty and residents. The data were entered in excel sheet and analysis was done with SPSS. For the closed-ended questions, median score was calculated, whereas for the open-ended questions, themes were drawn.

Results

A total of 87 mini-CEX encounters were done with 13 faculty (including 3 SR) and 29 residents. It was planned to do 6 encounters per resident though only 3 encounters per resident (50.0%) were completed. Out of 29 residents, 8 were 3rd-year resident and 21 were 2nd-year resident. The basic details about such encounters are shown in Figure 1.

Figure 1

Basic details about the mini-clinical evaluation exercise encounters. *(1) Medical interviewing skills; (2) Physical examination skills; (3) Professionalism; (4) Clinical judgment; (5) Counseling skills; (6) Organization; (7) Overall clinical competence

Table 1 depicts the median scores of the Mini-CEX encounters. Median score on the first encounter was highest for medical interviewing skills and counseling skills. The score was lowest for physical examination skills, clinical judgment, and organization/efficiency. Maximum improvement between the first encounter and last encounter was seen in physical examination skills, organization and efficiency, and clinical judgment, followed by counseling skills and humanistic qualities/professionalism. None of the skill showed any statistically significant improvement between the first and last encounter. Tables 2 and 3 depict the residents and faculty feedback on Mini-CEX.

Table 1

Scores of mini-clinical evaluation exercise encounters

Table 2

Student feedback questionnaire on mini-clinical evaluation exercise

Table 3

Faculty feedback questionnaire on mini-clinical evaluation exercise

In response to open-ended questions, residents mentioned that it improved their clinical skills (72.4%), uplifted the personal development (62.0%), and imparted a better one to one student–teacher interaction (62.0%). They desired for more of such encounters (48.2%). However, some of the residents felt that they need to balance it with their other clinical activities (34.4%) and busy faculty (37.9%). Faculty perceived it as a better method of assessment (76.9%). An important inference was that they found it useful for improved learning for themselves also (61.5%). However, the faculty felt that it requires more time to assess the students so shall not be possible for a larger batch. Some faculty raised the concern that direct observation can create bias as “It does not mimic real-life situation; the students put his/her best show”. However, both residents and faculty suggested to incorporate Mini-CEX in the curriculum.

Discussion

The project was done to assess the acceptability and feasibility of Mini-CEX as a learning tool for pediatric postgraduate residents. This study exposed the pediatric residents and faculty to the Mini-CEX for the first time as a method of WPBA, which aims for improved learning. It is a method where the trainee is directly observed by an assessor while performing a focused clinical task during a specific patient encounter. The assessor rates and provides structured feedback on the trainee's performance in this specific instance.[8] A meta-analysis documents that it is an important tool for the demonstration of trainee's performance.[910] The biggest advantage of Mini-CEX is the provision of immediate contextual feedback, which helps the trainee to improve their clinical judgment, decision making, and professional skills.[1112] It encourages a reflective approach to learning among the residents.

Out of the planned 6 encounters per resident over a period of 6 months in the study, only 50% of the Mini-CEX encounters were completed. The possible reasons are the busy schedule of the faculty, examinations, vacation of the faculty, and the residents. Constant reinforcement and demonstration of its utility can improve the completion rate.[13] None of the clinical skill showed statistically significant improvement. It may be due to the lesser number of encounters. Norcini et al. found the maximum improvement in clinical judgment and organization and efficiency while the lowest improvement in professionalism.[8] Other studies reported maximum improvement in humanistic qualities/professionalism.[1415]

Since it was the first exposure of residents and faculty to the Mini-CEX, the ambulatory patients were chosen for the convenience. It was similar to other studies.[1316] To increase the reliability of the tool, cases from the emergency area, in patient facility and other areas, should be included.

The residents and faculty accepted the Mini-CEX very well. It is reflected in their feedback. Most of the residents felt comfortable with it and its duration. Although some of the residents were frightened by the feedback, almost all of them agreed that feedback helped them to improve their clinical skills. A similar conclusion was reported in a systematic review which states that multisource feedback can lead to performance improvement although individual factors, the context of the feedback, and the presence of facilitation have a profound effect on the response.[9] The acceptability by the students is also reflected in the narrations in which they desire for more of such exercises with inclusion of this tool in curriculum.

Faculty found it as a better method of formative assessment since it has the potential for immediate and individual feedback. An important inference was that they found it useful for improved learning of themselves also. Both the faculty and residents felt that it improves the relation between the teacher and students. A systematic review document that Mini-CEX showed largely positive results in terms of learner satisfaction.[9]

Both the faculty and residents perceived that paucity of time was one of the factors that get in the way of completing all the planned encounters. Similar concerns were reported in other studies also.[1317] This obstacle can be overcome by the constant reinforcement.

Some of the residents and faculty were concerned that chances of bias are high as residents know that they are assessed, so they try to put their best show. This may be possible, but the scores obtained by the residents reveal that it was in “satisfactory” zone, leaving the space for improvement up to the zone of superior. It can be made possible with diversity of cases and examiners and frequent encounters.[1718]

Strength of the study is that for the first time in this part of India, the faculty and residents were exposed to method of WPBA. Limitation was small number of encounters and lack of diversity of type of patients.

Conclusions

Mini-CEX is an acceptable learning tool as reflected by the residents and faculty. It is feasible to use mini-CEX for assessment of residents.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.