Acceptability and Feasibility of Mini Clinical Evaluation Exercise (Mini-CEX) in the Busy Emergency Department.

BACKGROUND: Mini Clinical Evaluation Exercise (Mini-CEX) has been adapted to different specialties in clinical practice but with very little evidence documented about its use for residency training in the emergency department (ED). This study aims to assess its acceptability and feasibility as a formative tool in the busy emergency department.
MATERIALS AND METHODS: Both the faculty members and the emergency medicine residents were sent a validated questionnaire using Google forms, and the results were analyzed using simple statistical tools.
RESULTS: Forty-nine residents and 58 faculty participated in the survey. The study was carried out over a period of 4 months. The resident's completion rate was 96% (49 out of 51), while faculty completion rate was 96% (58 out of 60). The time for Mini-CEX completion ranged from 10 to 20 minutes. Most of the residents were satisfied with Mini-CEX as an assessment tool. Twelve residents expressed their concern regarding available time during busy clinical shifts. Most of the faculty agreed with the benefits of using Mini-CEX as a formative assessment tool. Several of them commented that they need "protected time" and "more training" to use this tool to provide maximum benefit to the residents.
CONCLUSION: Despite busy nature of ED, Mini-CEX has been identified as an acceptable learning tool for residents in emergency medicine. Based on the faculty's feedback and comments, several faculty development workshops were conducted to improve faculty skills in carrying assessments by using Mini-CEX, and protected time is provided to some faculty members to carry out these formative assessments for the benefit of the residents.

Introduction

The use of Mini Clinical Evaluation Exercise (Mini-CEX) in the Emergency department is not widely documented, even though Mini-CEX has been applied over a wide range of clinical situations.1–8 Mini-CEX is a method of assessing clinical skills developed by the American Board of Internal Medicine.9–13

Directly observing the medical students and residents while they are with the patient is crucial to their education.14 This is to ensure the proper application of skills in the areas of medical interviewing, physical examination, and counselling. This direct observation is compulsory for the reliability and validity of the assessment of interviewing and counselling skills.15 The Mini-CEX has been used as a feedback tool for assessing and educating residents in training.16 It is used for assessing the clinical skills of medical students and residents while observing them with actual patients on a variety of skills. These include interviewing skills, physical examination skills, professionalism, clinical judgment, counselling skills, organization, and overall clinical competence.

The Mini-CEX is a valuable assessment tool because of the number and breadth of feedback comments by both the residents and their evaluators (rater). It involves a single rater, usually a faculty member, who observes a resident while that resident performs a clinical assessment of the patient, like a focused history taking and physical examination within a stipulated time interval of 10–20 minutes. The rater then asks the resident for a diagnosis and treatment plan, then rates the resident and provides educational feedback.

The Mini-CEX assesses residents in a broader range of clinical situations than the traditional Clinical Evaluation Exercise because it has better reproducibility.15 It is a validated tool8,18 and it offers residents the opportunity to be assessed by more than one faculty member and with multiple patients.

The Emergency Department (ED) is a hectic place due to patients’ emergent nature being managed there. They need apt attention; hence, assessment of resident trainees is always a challenge. Lack of time has been cited commonly by the ED educators to teach medical students and residents during busy clinical shifts. Due to increased number and complexity of the patients, emergency physicians are usually under pressure not only to manage undifferentiated patients competently but also support the needs of the department such as improving the communication with the patients, their families, and the other departments. ED unlike other departments is unable to set a structured teaching program during clinical work, such as lunchtime conferences, or morning reports. ED teaching occurs spontaneously in between managing severely injured or ill patients, answering questions to the nurses and staff members. The learners range from medical students to residents and fellows with the different learning needs.19

The Mini-CEX was introduced in our residency training program in 2013. Before the introduction of Mini-CEX there were informal feedback sessions provided by the supervising faculty. In 2013, the residency program was in the process of being accredited by the international arm of Accreditation of Graduate Medical Education (ACGEM-I), USA; hence, it was decided to introduce Mini-CEX, which is a well-recognized formative assessment tool. This study aims to assess its acceptability and feasibility as a formative assessment tool in the busy emergency department. Many other studies have investigated other parameters such as validity, reliability, implementation, etc., none has investigated its applicability and feasibility in a large middle eastern Emergency department.

Methodology

The questionnaire used in our study2 has been previously studied among the postgraduate trainees. We consulted our ED educators and questionnaire was changed to suit the needs of the ED through Delphi technique. Delphi technique has been used in the past when the available data instrument or other methods are inadequate. The aim of Delphi technique was to prepare a questionnaire based on expert opinions.

The study was conducted in the ED from April to July 2020. The study approval letter was circulated to ED physician through an e-mail from the program director’s office and explained that the study authors will approach ED physicians for completion of the survey. The ED is the major teaching institute hospital in Doha, the capital of Qatar. The study center hosts a four-year emergency medicine training program, which is accredited by the international branch of the USA-based Accreditation Council for Graduate Medical Education. Both the Faculty members and the Emergency Medicine residents sent a validated questionnaire using google forms. The total time taken for the whole process to be completed is then recorded. The results obtained were analyzed using simple statistical tools. The study was approved by the Institutional Review Board (IRB) of Medical Research Centre (MRC), Hamad Medical Corporation (HMC) as a quality improvement project (MRC-01-19-463). The study has been marked as exempt by IRB committee and the questionnaire link was sent to Faculty and Residents through Google forms along with the benefits, confidentiality, anonymity and the right to withdraw at any time through an information sheet. No personal details were collected. The participants were not required to provide an informed consent to complete the survey.

Results

Forty-nine residents and 58 faculty participated in the survey. The study was carried out over a period of 4 months. The resident’s completion rate was 96% (49 out of 51) while faculty completion rate was 96% (58 out of 60). The time for Mini-CEX completion ranged from 10 to 20 minutes. Among the residents (Table 1), 75.5% are aware of the competencies being assessed. 51% agree that the duration of the feedback was appropriate. 73.5% agree that the feedback made them aware of their strong points, while 77.5% agree that feedback made them aware of their weak points. 63.2% agree that they had the opportunity to express my view during feedback. 61.2% were satisfied with Mini-CEX as a method of assessment. 57.2% agree that Mini-CEX enhanced their medical interviewing skills. 51% agree that Mini-CEX enhanced their skills in physical examination. 59.2% agree that Mini-CEX enhanced their communication skills. 61.3% agree that Mini-CEX enhanced organization skills. 53% agree that assessors were able to provide appropriate time during their clinical working hours. 79.6% agreed it would be helpful if they were assessed by the same shift consultant multiple times. 73.4% agreed that there must be a Mini-CEX performed during the clinical supervision shift.

Table 1

Descriptive Analysis of Mini-CEX Questionnaire Feedback from Emergency Medicine Residents

Questionnaire Item	Strongly Agree (%)	Agree (%)	Cannot Say (%)	Disagree (%)	Strongly Disagree (%)
Aware of the competencies being assessed	5 (10.2)	32 (65.3)	2 (4.1)	10 (20.4)	0
Comfortable while being examined	13(26.5)	19 (38.8)	11 (22.4)	6 (12.2)	0
Directly observed for a specific patient	5 (10.2)	26 (53.1)	7 (14.3)	8 (16.3)	3 (6.1)
Duration of examination exercise was adequate.	3 (6.1)	22 (44.9)	11 (22.4)	13 (26.5)	0
Duration of feedback was appropriate	5(10.2)	27 (55.1)	8 (16.3)	9 (18.4)	0
Feedback made me aware of my strong points	5 (10.2)	31 (63.3)	9 (18.4)	1 (2.0)	3 (6.1)
Feedback made me aware of my weak points	6 (12.2)	32 (65.3)	4 (8.2)	7 (14.3)	0
Opportunity to express my views during feedback	3(6.1)	28 (57.1)	13 (26.5)	5 (10.2)	0
Satisfied with Mini-CEX as method of assessment	5 (10.2)	25 (51.0)	9 (18.4)	9 (18.4)	1 (2.0)
Mini-CEX enhanced my skills in medical interviewing	4 (8.2)	24 (49.0)	11 (22.4)	8 (16.3)	2 (4.1)
Mini-CEX enhanced my skills in physical examination	3 (6.1)	22 (44.9)	14 (28.6)	10 (20.4)	3 (6.1)
Mini-CEX enhanced my communication skills	4 (8.2)	25 (51.0)	12 (24.5)	7 (14.3)	0
Mini-CEX enhanced my professionalism skills	4 (8.2)	26 (53.1)	11 (22.4)	8 (16.3)	0
Mini-CEX enhanced my organization skills	4 (8.2)	26 (53.1)	11 (22.4)	8 (16.3)	0
Assessors are able to provide appropriate time during their clinical working hours.	3 (6.1)	23 (46.9)	5 (10.2)	12 (24.5)	4 (8.2)
Would it be helpful if assessed by same shift consultant multiple times	12 (24.5)	27 (55.1)	4 (8.2)	6 (12.2%)	0
Most Mini-CEX were performed during the clinical supervision shift.	6 (12.2)	30 (61.2)	8 (16.3)	4 (8.2%)	1 (2.0)

Among the faculty (Table 2), 72.4% agree that they got prior training on how to assess a trainee using workplace-based assessment tools (WBA). 75.8% agree that prior training about WBA has improved their own understanding of the mini-CEX. 75.6% of the raters were able to provide appropriate time for the mini-CEX during the clinical shift, while 70.7% carried out most of the mini-CEX during the clinical supervision shift. 80.9% provided feedback to the residents discussing both their strengths and weaknesses.7 5.8% agree that Mini-CEX improved their own attitude towards residents’ training. 74.1% of the assessors directly observed the trainee for a specific patient encounter rather than using Mini-CEX for general assessment. Step to improve the Mini-CEX tool as outlined by assessors varied and includes training for assessors and more protected time.

Table 2

Descriptive Analysis of Mini-CEX Questionnaire Feedback from Emergency Medicine Faculty

Questions	Strongly (%)	Agree (%)	Neither Agree Nor Disagree (%)	Disagree (%)		Strongly Disagree (%)
I got prior training on how to assess a trainee using workplace-based assessment tools.	7 (12.1)	35 (60.3)	9 (15.5)	7 (12.0)		0
I feel that this prior training about WBA has improved my own understanding of the Mini-CEX.	10 (17.2)	34 (58.6)	12 (20.6)	2 (3.5)		0
I was able to provide an appropriate time for the Mini-CEX during my clinical shift.	6 (10.3)	35 (60.3)	10 (17.2)	7 (12.1)		0
Most of the Mini-CEX were completed during the clinical supervision shift.	2 (3.5)	39 (67.2)	11 (18.9)	6 (10.3)		0
I provided appropriate feedback to the residents discussing both their weaknesses and strengths.	6 (10.3)	41 (70.6)	7 (12.1)	4 (6.9)		0
Mini-CEX has improved my own attitude towards resident training.	9 (15.5)	35 (60.3)	11 (18.9)	3 (5.2)		0
For most Mini-CEX, I directly observed the trainee for a specific patient encounter.	8 (13.8)	35 (60.3)	12 (20.7)	3 (5.2)		0
From an assessor’s perspective, briefly explain what steps can be taken to improve the Mini-CEX tool.	● There should be a specific and protected time for mini-CEX and not during a clinical shift
	● There should be a specific faculty member who are properly trained in doing mini-CEX for quality assurance.
	● Physicians who are involved in teaching and assessment should be given proper protected time than they will be able to do appropriate assessment and feedback
Time taken to complete Mini-CEX (Minutes)	0–10	10–20	20–30		>30
	17 (29.3)	31 (53.5)	8 (16.3)		2(3.5)

Discussion

The Mini-CEX is a method of clinical assessment developed by the American Board of internal medicine for the evaluation of competence of undergraduate and postgraduate trainees. Studies have shown that it has been applied to specialty training like dermatology, neurology, Anaesthesia, Internal Medicine, Psychiatry and Cardiology.17,19–23

The ED Residents reported (Table 1) overall satisfaction with Mini-CEX as an assessment tool. A total of 61.2% agree that they were satisfied with Mini-CEX as an assessment tool, and it helped them uplift their personal development and focus on their weak points. 73.5% agree that the feedback made them aware of their strong points, while 77.5% agree that feedback made them aware of their weak points. 63.2% agree that they had the opportunity to express my view during feedback. This confirms this study; according to Schopper Rosenbaum & Axel,17 students value observation and feedback.

The Mini-CEX format used in the study is computer-based as both assessors and the residents’ utilized online platform (Med Hub) to perform the evaluation. This agrees with the study by Chang et al,24 which shows that the computer-based Mini-CEX tool was highly rated by both evaluators and residents. The authors also noted that modifying the Mini-CEX format may produce more recorded observations and improve overall accuracy. This was demonstrated in this study because the response rate was 99%. The high completion rate confirmed the feasibility of using the Mini-CEX in ED. 99% was obtained in this study despite the very busy and crowded nature of the ED. This is in line with the study by Chang et al24 where there was a high completion rate of weekly Mini-CEX study in an emergency department.

The Mini-CEX tool is a widely accepted tool for clinical assessment and evaluation of residents in training. Extensive studies have been carried out in different clinical settings as evidence of its usage acceptability in medical training worldwide. According to Sidhu et al,25 a post-assessment survey of reliability and acceptability of Mini-CEX showed that rates (94%) and residents assessed (75%) both felt that Mini-CEX is an acceptable assessment of practicing physician.

Twelve residents expressed their concern regarding available time during busy clinical shifts. Residents strongly recommended being evaluated by the same shift clinical supervisor. Several of them commented that they need “protected time”, “more training”, feedback and “quality assurance” on their formative assessments to use this tool to provide maximum benefit to the residents. Similar results were found in another study by Amouzeshi et al, which26 recognised lack of time as a limiting factor and recommended faculty development as a key to improving the education in ED.

The survey results from the ED faculty (Table 2) found that 72.4% agreed that they had prior training on how to access a trainee using a workplace-based assessment tool, while 75.8% said this prior training about the work-based assessment improved their understanding of the Mini-CEX tool. Liao et al1 found that Faculty development is a prerequisite to train evaluators to implement a successful Mini-CEX assessment program. This is also in line with some of the assessors’ recommendations in this study that need more training on the use of Mini-CEX.

The first limitation of the study is using a questionnaire for gauging opinion. A better approach would have been focus group discussion to obtain detailed assessment and evaluation of their opinion about feasibility and acceptability of Mini-CEX. This approach would have helped in thematic analysis. Due to the busy nature of emergency medicine, questionnaire was the pragmatic approach. The second limitation was the single centre study. The results may not be applicable to other centres. We recommend further studies to endorse the results.

Conclusion

Mini-CEX is feasible and acceptable learning tool for Residents in ED. Due to the nature of ED work and lack of protected teaching time, several modifications have been suggested to optimise educational experience for the learners. Based on the feedback and comments from the faculty, several faculty development workshops were conducted to improve the skills of faculty in carrying out formative assessments by using Mini-CEX, and protected time is provided to some faculty member to carry out these formative assessments for the benefit of the trainee. We have developed a core educational faculty to provide training and feedback to residents.