A Pilot Study of Modified Mini-Clinical Evaluation Exercises (Mini-CEX) in Rotation Students in the Department of Endocrinology.

Background: The mini-clinical evaluation exercise (mini-CEX) is an excellent tool for assessing the clinical abilities of medical students in intense clinical practice. In this study, the Mini-CEX was adapted to professional questionnaires for Diabetes Mellitus (DM), and examined in medical students completing their clerkship rotation in the department of endocrinology.
Methods: From January 2021 to January 2022, all rotating medical students at Shanghai Pudong Hospital completed two mini-CEX exams before and following their rotation under the supervision and guidance of six tutors. The mini-CEX form was modified in this study primarily for inpatient management based on our clinical experience and updated DM guidelines of the American Diabetes Association (ADA), the European Association for the Study of Diabetes (EASD), and the Chinese Diabetes Society (CDS). Each component of the mini-CEX assessment, including medical interviews, physical examination, clinical judgment, clinical management, and overall clinical competence was evaluated using a nine-item questionnaire.
Results: Our findings revealed that the second-round performance on the assessments significantly improved, as indicated by higher scores on each component. The Pearson association analysis revealed that the feedback time of the first examination was markedly associated with improved overall scores (r= 0.391, p<0.001). However, no correlations were discovered between patient age, gender, disease severity disparity, or the interval between examinations (p>0.05). Additional regression analysis revealed that the feedback time during the initial examination was the most significant contributor to the increased overall scores (β=0.391, p<0.001).
Conclusion: This newly designed mini-CEX form based on current ADA and EASD guidelines may assist trainees in more effectively diagnosing and managing DM in inpatients, particularly those with macrovascular, microvascular, or peripheral nerve neuropathy. This study aims to assess the efficacy of administering a modified mini-CEX form to rotating trainees participating in an endocrine clerkship.

Background

The mini-Clinical Evaluation Exercise (mini-CEX) is a clinical evaluation instrument used to assess trainee-patient interactions in hospitals and other clinical settings.1 It is a brief assessment that takes approximately 15 minutes for each domain and is incredibly valuable during a hectic medical rotation. The original mini-CEX evaluation form requires that trainees focus on at least five categories, including medical interviewing ability, physical examination ability, clinical judgment ability, therapeutic ability, and overall clinical competence.2 Trainers grade these five domains on a nine-point Likert scale ranging from “inadequate” to “outstanding.” A feedback section is mentioned at the end of the assessment.

Mini-CEX has demonstrated its clinical utility and universality in medical student and resident training.3–5 However, due to a lack of comprehensive and specialized knowledge on a single specific disease, its operationalization is currently limited.6 Therefore, the original mini-CEX is frequently modified for use in specialized training fields.7–9 Type 2 diabetes (T2D) is a prevalent metabolic disease that causes many complications in multiple organs and systems,10,11 necessitating the completion of numerous questionnaires for medical history and physical examination.12 It is also a disease that is commonly used in the Department of Endocrinology to assess the efficacy of training in medical students and clinicians. As a consequence, a special mini-CEX form is required for the assessment of residents or medical students participating in an endocrine rotation clerkship. In this study, the original form of mini-CEX was modified, and its educational impact on trainee performance was investigated.

Methods

Instrument Development

The nine-point Likert scale in the original form of mini-CEX, ranging from “unsatisfactory” to “outstanding”, was designated as the 9-item questionnaire on diabetes diagnosis and treatment in this study, and was categorized into the 5 specific domains described above (Table 1). These nine items were referred to the American Diabetes Association (ADA) 2022 guideline,13 the European Association for the Study of Diabetes (EASD) guideline,14 and China’s 2020 standard of medical care for type 2 diabetes (Chinese Diabetes Society).15 Because diabetes is a chronic, complex condition, and outpatient management would take longer than a month to master, as would the application of diabetes pharmacology alone, we required that each student master inpatient diabetes clinical practice for the duration of a one-month rotation.

Table 1

The Modified Mini-CEX Form for Assessment of Trainees Who Interviewed Diabetes Mellitus (DM) Patients

Domains of Mini-CEX	Contents	Score
Medical interview skills	1. Introduce yourself and other staff to the patient in a way that makes them feel comfortable	1
	2. Respect the patient, ascertain the patient’s primary health problems, communicate in plain language, and encourage the patient to ask questions	1
	3. Appropriate questions and counseling to elicit adequate information, summarize the major complaint and the history of the current disease	1
	4. Have an appropriate response to the patient’s body language, (For example, if the patient complains about an unpleasant or painful spot in his body, the trainee should immediately do a physical examination (PE) and then proceed with the medical interview)	1
	5. Create a family tree for the patient, including each relative (the age starts to suffer from diabetes, health, major illness, and cause of death)	1
	6. In patients suspected of having type 1 diabetes or latent autoimmune diabetes in adults (LADA), obtain a medical history to rule out thyroid disease or other autoimmune diseases and perform a PE for the thyroid. Inquire about any autoimmune disease symptoms, such as joint pain or stiffness	1
	7. Macrovascular problems such as myocardial infarction, chest discomfort, orthopnea; stroke, and Transient Ischemic Arrack (TIA) are also possible	1
	8. Microvascular complications: blurred vision, foamy urine, etc.	1
	9. Diabetic neuropathy’s history includes changes in superficial and deep feeling, including numbness and discomfort in the lower limbs, as well as skin color	1
Physical examination skills	1. Protect the patient’s privacy	1
	2. Inform the patient about the aim of the PE	1
	3. Humanistic care, such as pre-warming the stethoscope head with the palm, etc.	1
	4. Handle the patient’s discomfort with care during palpation	1
	5. Utilize the GCS (Glasgow coma scale) to assess the patient’s mental state	1
	6. The PE performed for the cardiovascular system	1
	7. The PE performed for the neurological system	1
	8. The PE performed for the respiration system	1
	9. The PE was performed for diabetic neuropathy, 10g nylon wire inspection required	1
Clinical judgment	1. Could outline the primary complaint, the current illness’s history, previous medical history, and family history	1
	2. Could make a concise description of the physical examination	1
	3. Could describe the differential diagnostic possibilities	1
	4. Possess the ability to examine the complications of type 2 diabetes	1
	5. The ability to interpret laboratory data and special PE for macrovascular problems (eg, cardiovascular and cerebrovascular disorders)	1
	6. The ability to interpret laboratory data and specialized examinations for microvascular disorders (eg, renal and retinal disorders)	1
	7. The ability to interpret laboratory data and specialized examinations for diabetic peripheral neuropathy	1
	8. Preliminary diagnosis of latent autoimmune diabetes in adults (LADA), type 1 and type 2 diabetes.	1
	9. Clinical judgment regarding the outcome	1
Medical management skills	1. Adequate medical orders for nursing care, diet modification, and blood glucose monitoring.	1
	2. Appropriate medical orders for vital sign monitoring, oxygenation/respiratory care as determined by the GCS	1
	3. Appropriate oral medications for the treatment of type 2 diabetes	1
	4. Appropriate insulin dosage for treatment of hyperglycemia in type 2 diabetes, if necessary	1
	5. If necessary, appropriate medical orders for a rehydration regiment	1
	6. If necessary, the appropriate physician orders antibiotics	1
	7. If necessary, appropriate medical orders for laboratory tests	1
	8. If necessary, appropriate medical orders for the special assessment. CTA, MRA, and MRI are a few examples	1
	9. Determine whether additional consultations with other departments are necessary for the patient	1
Overall clinical competence	1. Above all, the mention is unsatisfying.	1–3
	2. Above all, the mention is satisfactory.	4–6
	3. Above all, excellent in the mention	7–9

Participants and Setting

From January 2021 to January 2022, all 79 rotating students at the hospital participated in two mini-CEX sessions before and after clerkship training, under the monitoring and guidance of 10 professional tutors in the department of Endocrinology. The trainees were notified of these assessments as part of a rotation during their first mini-CEX assessment. The modified mini-CEX form was used as a self-evaluation tool throughout the approximately one-month rotation period preceding the second assessment. The initial evaluation was conducted by each rotation student’s tutor (6 tutors) under the supervision of a senior physician of the treatment group, who were experienced attending and chief physicians in our department, proficient in the diagnosis and treatment of both diabetes and its complications. In addition, this evaluation was undertaken without any guidance from the tutor, and the student was blinded to the patient’s diagnosis and treatment information unless each part of the assessment was completed. The final assessment was performed by the same student’s rotation tutor under the same conditions, with the exception that the patient was reselected and the student was blinded to the patient’s information. Both initial and final assessment durations were specified to be within 40 minutes.

Statistical Analyses

All statistical analyses were performed using SPSS software (Version 25, USA). The Kolmogorov–Smirnov test was used to assess the normal distribution of the data. The 5 domains of the mini-CEX scale were compared using a paired t-test. Pearson correlation was used to determine the primary contributor to the overall increase in scale. Multilinear regression analyses were performed to identify the critical traits assessed by mini-CEX that contributed to the overall score. For each analysis, p<0.05 was considered statistically significant.

Result

Comparisons of the Progress of Scores Assessing Five Mini-CEX Domains

The mini-CEX forms were distributed to all 79 rotation students before and after the approximately one-month training, and response rates were 100%. As expected, the scores for each domain on the mini-CEX form significantly increased when medical interviewing skills, physical examination skills, clinical judgment, medical management skills, overall clinical competencies, and total scores were evaluated (p<0.0001) (Figure 1).

Figure 1

Increased scores on the mini-CEX in 5 domains including medical interview skills (A), physical examination skills (B), clinical judgment (C), clinical management skills (D), and overall clinical competence (E) as well as an increase in total scores (F) among rotating medical students before and after type 2 diabetes-specific training. N=79, ****P<0.0001.

The Factors Associated with the Increase in Total Score Before and After Training

Correlational analyses of the increasing total score revealed that, in addition to the five components of primary assessment throughout training, the feedback time of the first examination is critical (p<0.001). However, the association between patient age, gender, disease severity disparity, examination interval, consultation type, observation time disparity, and second examination feedback time were relatively weak. The precise r and p-values are shown in Table 2.

Table 2

The Correlation Between Changes in Age, Patients’ Gender, the Interval Between Examinations, Clinical Skills Scores, Observation Time, Feedback Time, and Change in Disease Severity in Relation to Increases in Total Scores

Variables	Increase in Total Score
	r	p
Age disparity among patients (years)	−0.025	0.826
Gender disparity among patients	0.206	0.069
Disease severity disparity	0.071	0.536
The interval between examinations (days)	0.035	0.761
Consultation type disparity (first/ repeated)	0.068	0.554
Medical interview scores change	0.840***	<0.001
Physical examinations score change	0.688***	<0.001
Clinical judgment score change	0.789***	<0.001
Medical management score change	0.660***	<0.001
Overall clinical competence score change	0.355***	0.001
Observation time disparity (min)	0.009	0.940
Feedback time (first exam) (min)	0.391***	<0.001
Feedback time (second exam) (min)	0.160	0.159

Note: ***: P<0.001.

Multilinear Regression Analyses of the Major Components Contributed to the Progress of the Increased Overall Score

Based on a stepwise regression analysis, the feedback time of the first examination was determined to be the only significant factor in this study (β=0.391, p<0.001) (Table 3). Patient-related variables such as age, gender, severity, consultation type, and exam interval period had no effect on the increase in the total score.

Table 3

Multiple Linear Regression Analysis of Factors Influencing the Increase in the Total Score Before and After Training

Variables	Increment of Total Score R2=0.153 F=14.400 P<0.001
	B	SE	β	t	p
Constants	−1.745	0.471		−3.708	<0.001
Feedback time (first)	0.080	0.021	0.391	3.795	<0.001

Abbreviations: B, unstandardized coefficient; SE, standard error; β, standardized coefficient.

Discussion

Previous studies have demonstrated that using mini-CEX during a demanding clerkship rotation is not only convenient but also professional. This customized mini-CEX assessment form benefits our clinical training plan in the department of endocrinology. Trainees in the Department of endocrinology who are interested in advancing their knowledge and skills can benefit from this assessment. In addition, as with previous studies, this form should assist trainers in guiding trainees through more professional structured interviews, in-depth sympathetic engagement, excellent communication skills, and bedside manner.9,16

Within the Endocrinology Department, the trainee can use this form as an extended instructional resource for learning DM. The five domains in the form have been modified in accordance with the DM diagnostic and therapy approach. We later discovered that the progression of each student’s total score was independent of changes in the patients’ properties and the background environment, indicating that the exam is objective enough and has a high degree of reliability to accurately reflect each resident’s virtual ability and progress during the rotation, with the findings of previous research.17 Our examination includes five domains, which encompass nearly all aspects and fundamental skills necessary for students’ development and routine work in our department. There are 9 items in the medical interview that are primarily concerned with establishing compatible relationships with patients and completing the initial impression on diagnoses and the patient’s condition. This section primarily requires students to have concise and clear logical communication abilities in counseling, as well as the ability to understand the primary paradox at the moment patients require treatment in a hospital. The 9 major items in the physical examinations (PE) section encompass the numerous aspects that we should conduct in diabetic patients because the PE is critical not only for correctly diagnosing patients but also for further communication with patients. In addition, there are nine items in the clinical judgment section that assess a prospective trainee’s analytical ability, experience, and knowledge as demonstrated by his or her ability to collect illness histories, perform physical examinations, and interpret laboratory tests, as well as make numerous differential diagnoses and develop an appropriate prognosis for each patient. The medical management skills major assess a prospective physician’s practical competence to integrate newly acquired data with previous experiences and knowledge to solve medical problems for patients. The overall clinical competence component summarizes and assigns a score based on the previous performance in the other four sections. We design these examination frameworks to evaluate participants’ comprehensive qualities, provide them with specific constructive suggestions, and guide them in overcoming their deficiencies in their future medical careers.18

In addition, immediately following testing, participants receive feedback from tutors confirming the correct clinical process and indicating their errors, which help them improve their skills and attain a higher standard in their future routine work. In reality, the tutors identified the exact issues that the student encountered in his or her evaluation and provided suitable solutions that the student must immediately investigate on his or her own based on the daily assessment questionnaire. In accordance with the current diabetic guidelines, the supported reference tool could be discovered in official internal medicine or diagnosis training materials. A student may also obtain the answers by searching clinical resources like PubMed. Throughout each rotation, the tutors will provide the students with substantial clinical errors relevant to the examination, for which they will be required to discover remedies. In addition, the department regularly held training rounds and mini-lectures provided by senior physicians. We discovered that each participant’s performance improved significantly one month later on the second examination and that this effect was strongly related to the feedback time of the first examination, but had little correlation with the interval period between the two exams, implying that the feedback time is the most critical factor in improving trainee’s clinical abilities. During their one-month rotation, participants were asked to complete additional self-assessments following the first mini-CEX evaluation. Although the interval period varied by the student, our study found no correlation between the duration of the interval period and performance on the second test. One possibility is that one month is sufficient time for a trainee to develop clinical skills in DM clinical practice for general inpatient management, given that diabetes is a chronic, complex condition, and outpatient management would take longer than a month to master, as would the application of diabetes pharmacology alone. Furthermore, the study found that the majority of trainees prefer this type of training because it enables them to concentrate on a specific area of weakness.19 Consequently, as demonstrated by this study, they can devote more time and achieve a higher score on the second min-CEX exam. Several studies have demonstrated that feedback time and the structuring of this evaluation process are critical components of mini-CEX.20 In this study, however, feedback time was found to have only a limited causal effect (R2=0.153) on trainees’ total score improvement. We did not know how long each participant spent following the initial examination, nor did we know about the participant’s attributes, such as intellect and personal characteristics,21,22 which may have played a significant effect in this improvement. In a future study, a more customized mini-CEX will be developed to account for each participant’s distinguishing characteristics.

Conclusion

In the current study, we modified a mini-CEX form often used for in-patient treatment based on the most recent guideline for diabetes mellitus diagnosis and treatment. We found that regardless of the patient’s characteristics or clinical circumstances, the feedback time during the initial examination is essential for attaining improved overall scores. This mini-CEX form could be expanded to include additional variables, such as trainees’ personality qualities from other standard scales or methods, to support trainers in assisting trainees both personally and in achieving greater success in their future inpatient clinical practices.