Review of mini-clinical evaluation exercise (mini-CEX) in a psychiatry clerkship.

BACKGROUND: Direct observation of medical students with actual patients is important for the assessment of clinical skills including interviewing and counseling skills. This article describes medical students' experience of mini-clinical evaluation exercise (mini-CEX) during their clerkship in consultation psychiatry.
MATERIALS AND METHODS: In our center during inpatient consultation psychiatry clerkship, all rotating students are expected to complete one mini-CEX assessment as part of their clinical training. We conducted retrospective analysis of mini-CEX ratings completed from 2013 to 2016. All evaluations took place at inpatient medical setting in patients admitted with medical conditions and psychiatric comorbidities.
RESULTS: A total of 113 evaluations were reviewed. The time examiner observed the interaction of a student with the patient was 14.24 minutes (mean), and the time spent in providing feedback to the student was 9.71 minutes. Complexity of problem was rated as low in 0.88% (n=1), moderate in 50.44% (n=57), and high in 48.67% (n=55). Highest ratings were for professionalism, similar to previous reports. Total score calculated by examiner showed no difference by the complexity of the patient; however, we observed a trend in higher counseling score for the high complexity group.
CONCLUSION: Mini-CEX assessment during busy clerkship is feasible with good outcomes. Direct observation of medical trainees with actual patients is important for the assessment of performance-based clinical skills. Hospital psychiatry rotation covering various medical and surgical units offers a great opportunity for exposure in patient communication.

Background

Greater faculty involvement in teaching and direct observation of medical trainees with actual patients are important for the assessment of clinical skills.1 Application of skills in the areas of medical interviewing, physical examination, and counseling is required for the successful practice of medicine.2 However, medical students report that they are rarely being observed during patient encounters; one main reason is a lack of faculty time.3 The direct observation is mandatory for the reliable and valid assessment of interviewing and counseling skills.4

The mini-clinical evaluation exercise (mini-CEX) is a method of the assessment of clinical skills including counseling skills and professionalism, developed by the American Board of Internal Medicine,5 and has been used in clerkships as a feedback tool.6 After an evaluator observes a trainee’s performance in a normal clinical encounter, trainee receives immediate feedback.7 For each encounter, an examiner (attending) recorded the date, the complexity of the patient’s problem on 3-point scale (low, moderate, and high),8 the sex of the patient, the number of minutes spent in observing the encounter, and the number of minutes spent in giving feedback.9 Using 9-point scale (in which 1–3 were “unsatisfactory,” 4–6 were “satisfactory,” and 7–9 were “above expected”), the examiner (attending) rated the student on interviewing, physical examination (mental status examination), professionalism, clinical judgment, counseling, organization and efficiency, and overall competence.8 The emphasis on counseling included student’s interaction with the patient professionally (introduced himself/herself, maintained good eye contact) and education (explained the rationale for test/treatment). After the interview, the examiner (attending) completed the rating form and provided feedback.8

In 2013, we in our center decided to use mini-CEX tool as part of consultation liaison (CL) psychiatry clerkship, Stritch School of Medicine, Loyola University Medical Center, May-wood, IL, USA, and this assessment is ongoing. The Loyola consultation psychiatry service focuses on collaboration across medical settings and disciplines providing consultation and covers the whole hospital every day including various divisions of medicine and surgery. The service provides 1800 consultations a year in the areas including mania, psychosis, depression and anxiety in medically ill patients, delirium, crisis, psycho-oncology, end-of-life issues, alcohol withdrawal, intensive care unit and burn unit psychiatry, postpartum depression/psychosis, and transplant evaluations. Promoting good patient interviewing skills during medical student clerkship is an important goal. Psychological aspects of medical care give an additional opportunity for hospital psychiatry, and the clerkship offers interpersonal experiences of the clinical encounter and medical care.10 Given this position, clerkship provides a great opportunity for exposure in patient communication.

Hypothesis

Assessing the feasibility of mini-CEX in hospital consultation psychiatry setting, ie, mini-CEX as an educational curriculum, is feasible and has an impact on patient interviewing experience irrespective of exposure to the complexity of problems.

Materials and methods

About 4 years after implementation, we reviewed the completed mini-CEX forms as a process for feedback for faculty and to broaden this teaching experience. The Loyola University’s institutional review board reviewed this research proposal and granted exemption as an evaluation of medical student curriculum according to 45 Codes of Federal Regulation 46.101 b(1). In our center, all rotating third-year medical students during inpatient psychosomatic medicine clerkship (total duration is 6 weeks) are expected to complete one mini-CEX assessment for experience and feedback. Students are informed about this as part of clinical orientation. The mini-CEX was conducted during the fifth week of the rotation. Students selected the patients from their own case load. A total of 117 students completed clerkship during the time period, and mini-CEX scores of 113 students were available.

Results

Examiner observed time of student interview was 15 (mean±SD: 14.24±1.79) minutes and the time spent in providing feedback to the student was 9.71±1.13 minutes. Table 1 shows that the rating score of overall clinical competence is 7.26 (n=113). Table 2 shows that the rating score of overall clinical competence among complexity of problems is 8 (n=1) in low complexity problems, 7.25 (n=57) in moderate complexity problems, and 7.25 (n=55) in high complexity problems. The rating score of medical interviewing skills is 7.40 (n=57) in moderate complexity problems versus 7.27 (n=55) in high complexity problems. The score of counseling skills is 7.21 (n=57) in moderate complexity problems versus 7.38 (n=55) in high complexity problems. The rating score of mental status examination skills is 7.09 (n=57) in moderate complexity problems versus 7.29 (n=55) in high complexity problems. List of diagnoses for moderate complexity (n=57) are as follows: major depressive disorders (21; 36%), delirium (8; 14%), bipolar disorders (3; 5%), substance abuse/dependence (6; 10%), psychotic disorder (2; 3%), anxiety disorder (5; 8%), mood disorder, not otherwise specified (7; 12%), acute stress disorder (1), impulse control disorder (1), bereavement (1), insomnia (1), posttraumatic stress disorder (1); high complexity (n=55): major depressive disorders (4; 7%), delirium (12; 21%), bipolar disorders (13; 23%), substance abuse/dependence (7; 12%), psychotic disorders (7; 12%), mood disorder, not otherwise specified (2), developmental delay (2), acute stress disorder (2), posttraumatic stress disorder (1), obsessive-compulsive disorder (1), hoarding (1), catatonia (1), eating disorder (1), suicide attempt (1); low complexity: adjustment disorder (1).

Table 1

Overall rating of objectives of mini-CEX scale

Variables of mini-CEX	Encounters=n	Rating	Mean	SD	Min	Max
Medical interviewing skills	113	7.35±0.58	7.35	0.58	6	9
Mental status examination skills	113	7.19±0.56	7.19	0.56	6	8
Humanistic qualities/professionalism	113	7.42±0.53	7.42	0.53	7	9
Clinical judgment	113	7.20±0.54	7.20	0.54	6	9
Counseling skills	113	7.29±0.61	7.29	0.61	6	9
Organization/efficiency	113	7.26±0.48	7.26	0.48	6	8
Overall clinical competence	113	7.26±0.46	7.26	0.46	6	8
Total score	113	43.70±2.28	43.70	2.28	39	50

Abbreviations: CEX, clinical evaluation exercise; Max, maximum; Min, minimum.

Table 2

Mean score rating of all mini-CEX components among complexity of patient’s problems

Variables of mini-CEX	Total encounters (n=113)	Complexity
		Low (n=1), mean	Moderate (n=57)	High (n=55)
Medical interviewing skills	113	8	7.40±0.08 (7.24, 7.57)	7.27±0.07 (7.13, 7.41)
Mental status examination skills	113	7	7.09±0.07 (6.95, 7.23)	7.29±0.08 (7.14, 7.44)
Humanistic qualities/professionalism	113	8	7.37±0.07 (7.23, 7.51)	7.45±0.07 (7.31, 7.60)
Clinical judgment	113	7	7.16±0.07 (7.02, 7.30)	7.24±0.07 (7.09, 7.38)
Counseling skills	113	8	7.21±0.07 (7.06, 7.36)	7.38±0.09 (7.21, 7.56)
Organization/efficiency	113	8	7.25±0.06 (7.13, 7.36)	7.25±0.07 (7.12, 7.39)
Overall clinical competence	113	8	7.25±0.06 (7.13, 7.36)	7.25±0.06 (7.13, 7.38)
Total score	113	54	50.72±0.35 (50.02, 51.42)	51.15±0.36 (50.44, 51.85)

Notes: Values with the ± sign represent the mean±SD. Values in parenthesis represent range.

Abbreviation: CEX, clinical evaluation exercise.

Statistical analyses

One-way analysis of variance (ANOVA; parametric test) was performed to assess seven objectives of mini-CEX scale and the complexity of patient’s problem. The analysis was conducted by using tata software (Stata Corp LP, College Station, TX, USA). The time spent in observing (p=0.1947) and giving feedback (p=0.1998) showed no difference compared to the complexity of patient’s problems. We did not observe the statistical difference in the mean score of overall clinical competence (p=0.36) and mean score of seven objectives of mini-CEX scale compared among high versus moderate complexity problems.

Student feedback

We have quoted few feedbacks that have been received as part of overall clerkship rating. Feedbacks are mentioned as follows:

“Met with us individually and watched us with patients, giving feedback and questioning us on our thought process. Great experience”

“My only comment would be to give us some earlier heads up about evaluating us with patients so that we could have more time to work for a good experience”

“I liked that we were directly observed doing follow-up interview with a patient and got feedback”

“Before giving feedback on an observed interview, he could ask ‘How do you think you did?’ to see what the student is already aware of ”

“He provides direct feedback after watching an interview”

“Good feedback during the rotation (observed interviewing a patient) and at the end of the rotation.”

“I learned a lot of valuable information and clinical skills during my rotation”

“His feedback after patient interviews was helpful and constructive”

“We also had great interactions with our attending”

Examiner feedback

Examiner’s experience has been very good during this educational curriculum in interviewing patients during a busy clerkship rotation. This is a learning experience and gives the examiner an opportunity to appreciate the value of teaching patient-centered care during medical student clerkship. The mini-CEX has given the examiner the opportunity to guide the students in maintaining the balance of structured clinical interview, empathetic engagement, and bedside manners. Opportunity for brief patient counseling and education is one important area of focus that the examiner has found helpful from this exercise. After covering key areas including mental status examination, students get the opportunity to counsel patients regarding adherence to antidepressants and other medications, eg, antihypertensives. Students’ satisfaction of the overall rotation is very good with various factors contributing including mini-CEXs. Student survey of consultation psychiatry clerkship before mini-CEX module implementation (on 5-point scale with 5 being the best) was 4.5, and has increased to 4.9 after implementation of the mini-CEX module. Rating for the examiner before mini-CEX module implementation (on 5-point scale with 5 being the best) was 4.78 and has increased to 4.88 (after mini-CEX implementation).

Discussion

Previous reports on mini-CEX have shown the highest ratings for professionalism, similar to our findings, and lowest ratings for the physical examination (analogous to mental status examination).9,11 The mean score of overall clinical competence (7.26) is also similar to other study reports.11 In the comparison of students’ mini-CEX scores between differing complexities of patients interviewed, we have observed slightly higher mean score of counseling skills among the high complexity patient group (7.38; higher delirium, bipolar disorder, psychotic disorder patients) compared to the moderate complexity patient group (7.21; higher depression patients). This could be related to the patient’s alertness, cognition, and relatedness contributing to the flow of the interview. Counseling in mini-CEX also includes explaining rationale for test/treatment, educating/counseling regarding disease management, assisting the patient with implementing a plan, and using summarizing statements. It is likely that student while interviewing attempted to make sure that patient with high complex diagnosis (delirium) understood the treatment plan well. As this mini-CEX was performed at consultation psychiatry inpatient clerkship, prospective studies are needed to see whether the complexity of patient group is based on the complexity of diagnosis (depression versus delirium) or other factors (medical, psychosocial factors, different settings) play a role as well.

Based on the experience of faculty who completed the evaluation, students’ performance in the follow-up of the cues could be enhanced. As an observed interview, students want to make sure that they are not missing information. While information gathering is a very integral part of interviewing a patient, at times, patient could bring up details emotionally challenging. In these scenarios, students maintaining good eye contact and follow-up of the cues are important. Immediately after the interview, before the examiner gives the feedback, students should reflect and convey what they think about their interview. This self-evaluation by students should be followed by examiners’ feedback. One evaluation needs to be performed in the first week. Our goal is to continually work on this model, develop a framework, and demonstrate that training in patient-centered care is possible during routine rigorous clinical training.

There is a need for an educational module and feasible instrument where patient interviewing skills can be taught as part of actual patient care and clinical training. Given time constraints, any module must be feasible and practical in the context of the busy inpatient clinical challenges. With an average time of 15 minutes for each interview and covering the key areas as shown by the quantitative data, mini-CEX is feasible. We were able to identify only two articles on mini-CEX in psychiatry clerkship.12,13 Mini-CEX evaluation is feasible during hospital psychiatry rotation. The future goal is to generalize the use of mini-CEX module with feedback rating comparison among the complexity of problems at various clerkship centers.

There are limitations to this retrospective analysis. Our main purpose of this preliminary exercise was to assess feasibility. Being conducted at one clerkship site limits the generalizability. Challenges such as patient’s schedule and privacy were manageable. We believe that there is a need for multiple evaluations in the beginning, middle, and the end of the rotation by multiple evaluators to establish validity and reliability of mini-CEX module. As psychiatry examination predominantly consists of interviewing skills, validity and reliability of mini-CEX in psychiatry clerkship need to be established for mini-CEX to be recognized as an important tool for the assessment of student’s clinical skills.

Conclusion

Students value observation and feedback.14 Direct observation of medical trainees with actual patients and mini-CEX during busy clerkship is feasible with good outcomes. Students are pleased with the observation and feedback process. To our knowledge, this is the first report of mini-CEX examination in hospital consultation psychiatry clerkship. Prospective studies are needed in this important area of training in patient-centered care. Hospital psychiatry rotation covering various medical and surgical units offers a great opportunity for exposure in patient-centered care and assessment of clinical and communicative skills.