The Mental Status Exam: An Online Teaching Exercise Using Video-Based Depictions by Simulated Patients.

Introduction: The mental status exam (MSE) is a structured approach to gathering a patient's behavioral and cognitive information. Analogous to the physical exam, it provides a template to collect clinical data in a systematic fashion. The MSE is a core competency of undergraduate medical education (UME) and an entrustable professional activity in clinical psychiatry.
Methods: We developed video clips of simulated patients depicting three adults respectively diagnosed with schizophrenia, obsessive-compulsive disorder, and bipolar disorder. We used three short video clips per condition to demonstrate an incremental number of psychiatric signs and symptoms. We used the nine video clips as calibrated stimuli for learners to identify components of the MSE using an online tool.
Results: We piloted this online exercise among 37 volunteer students. Experienced learners performed better than novice ones on overall identification of MSE components (p <.001). Specifically, they were able to identify elements of the MSE following an ABC-STAMPS (appearance, behavior, cooperation; and speech, thought process and content, affect, mood, perceptions, suicidality) rubric. Discussion: This video-based scoring tool was easy to implement in a UME setting and well received by students as a formative didactic exercise and educational complement.

Educational Objectives

By the end of this activity, students will be able to:

List the components of the mental status exam (MSE) following an ABC-STAMPS rubric (Appearance, Behavior, Cooperation; and Speech, Thought process and content, Affect, Mood, Perceptions, Suicidality).

Use an online application to identify ABC-STAMPS components depicted in video-based portrayals by simulated patients of common forms of serious psychopathology.

Obtain immediate feedback to compare their personal performance to a gold standard in identifying MSE components.

Use the exercise as a springboard for formative discussion and feedback.

Introduction

The mental status exam (MSE) is a structured approach to gathering a patient's behavioral and cognitive information. Analogous to the physical exam, it provides a template to collect clinical information in a systematic fashion. The MSE is a routine component of a patient's examination, and although most pertinent to behavioral presentations and psychiatric patients, is applicable and relevant across medical practice. The MSE is routinely collected during the course of even a short interview, and provides important and shifting data points over the course of a patient's care.

The MSE is a core competency of undergraduate medical education and a key component of coursework in clinical psychiatry, as articulated in the Association of Directors of Medical Student Education in Psychiatry Milestones Project.[1] The MSE is also a psychiatry-specific entrustable professional activity (EPA),[2] for students to identify, describe, and document abnormal findings as part of a first-level EPA (i.e., gathering a history and performing a physical examination).[3] In psychiatry, this EPA may refer to the MSE, the primary (and often only) portion of the psychiatric examination at each patient encounter. There are numerous approaches to teaching the MSE, including traditional lectures, applying a rubric to a video of a psychiatric interview,[4] role-play exercises,[5] or the use of standardized patients in either small groups using video,[6] or large classroom settings.[7] As the use of standardized patient simulation (SPS) and online resources become more commonplace, there are innovative opportunities to teach and assess the MSE in psychiatric education.

A recent review[8] identified 63 studies that used some form of simulation in undergraduate medical education in psychiatry, 48 of which included SPS. The authors applied Kolb's Learning Cycle[9] to the retrieved studies, and found that even though all 63 studies provided a concrete learning experience (stage 1), only 19 included opportunities for reflective observation (stage 2). A study that used a six-question survey to evaluate medical students’ (n = 170) feelings regarding the use of SPS in teaching the MSE found that the majority of students felt it was a useful teaching tool.[4] Nevertheless, relying on SPS live interactions can be a logistically complicated and expensive task. A structured curriculum including video-based clinical depictions and an online tool could help enhance educational consistency and practice for learners.

In this study we described an online MSE tool that used video-based depictions by simulated patients (SPs) as calibrated clinical stimuli. The tool provided a structured exercise for practice and reflective observation. The online tool specifically sought to quantify students’ ability to objectively identify psychiatric symptoms seen in routine medical practice. It aimed to measure objectively quantified, performance-based psychiatric symptom recognition, as opposed to only capturing subjective measures of confidence, or knowledge-based tests, as has been done in prior studies.[7]

The online exercise was intended to advance learners’ knowledge and skills relevant to the MSE. It was developed for use among medical and nursing students in their preclinical years of training. Learners should have received at least 1 hour of didactic content on the MSE in order to benefit from the exercise and put their knowledge to practice. The online exercise was designed as an aid to classroom instruction and small-group discussion.

Methods

Video-Based Depictions of Psychopathology by SPs

We followed best practices in SP case development[10,11] to construct scripts based on patient composites from our combined clinical experience in inpatient and outpatient psychiatric settings. Cases depicted three adults respectively diagnosed with schizophrenia, obsessive-compulsive disorder, and bipolar disorder (Appendices A–C). Using professional actors as SPs, we recorded the scripts to yield a 27-minute video sequence that was divided into three segments for each condition (Appendices D–L). Each of the component video segments were under 3 minutes in duration, and highlighted an incremental number of psychiatric signs and symptoms, following a scoring rubric anchored in ABC-STAMPS (Appearance, Behavior, Cooperation; and Speech, Thought process and content, Affect, Mood, Perceptions, Suicidality). The core MSE constructs, and their underlying items, as exemplified in the instrument, were summarized in Appendix M, which served as a blank sheet for manual scoring of the instrument outlined next.

Web-Based Scoring Instrument

After viewing each 3-minute segment, learners identified MSE components by entering responses through their preferred WiFi-enabled devices into the electronic interface Qualtrics. Data were collected securely following a standard MSE template, and scoring took place automatically, with coding of specific signs and symptoms yielding ordinal scores at each time interval. The complete exercise, including viewing and scoring, was typically completed in under 30 minutes. Learners could manually compare their responses to a gold standard answer key (Appendix N) or receive quantitative feedback immediately after completing the last online data entry field (Appendix O).

We derived gold standard ratings and assessed the internal validity of the instrument through independent scoring of the videos by the four authors, who identified discrepant ratings, and, after an iterative process, arrived at final consensual scoring. Final interrater agreement was 89%, and kappa coefficient 0.66 (t = 14.29, p < .001).

Pilot Testing of the Online Instrument

The initial application of the video-based tool was conducted in the winter of 2019 among volunteer undergraduate students in the Yale Schools of Medicine and Nursing. We obtained institutional review approvals from the Yale Human Investigations Committee (#2000024005) before starting data collection. We collected no personally identifying information, and data collection and analysis were deemed exempt from requiring informed consent. The pilot application of the teaching tool was included as part of regular didactic activities, with learners invited to participate as volunteers, but notified that their participation was not mandatory and would have no impact on their evaluations.

We compared scores on the instrument and its subscales across levels of clinical expertise. For the purpose of this study, we defined expertise as having received training on the MSE or spent at least 1 month on a psychiatry clinical placement. We hypothesized that the online instrument would discriminate across levels of experience, resulting in higher overall and subscale scores among experienced, as compared to novice, learners.

MSE Instrument in Practice: Paper or (Optional) Online Scoring

Learners could apply and consolidate their knowledge by identifying elements of the MSE in the videos included in this report. Video clips could be embedded into the slide deck provided (Appendix P) and scoring could be completed either manually or through the online instrument described above. These teaching materials could be easily included into a 1-hour didactic, as described in the facilitator guide (Appendix Q).

Results

Thirty-seven undergraduate students (76% women; 65% medicine; 35% nursing or physician assistant) completed the scoring tool across four separate 100-minute sessions (online component mean duration, 34 minutes; range, 31–36). Experienced learners (N = 20) had received previous formal instruction on the MSE or spent at least 1 month in a psychiatric setting, as compared to novice learners (N = 17, p < .001). As summarized in the Table, experienced learners performed better than novices on overall score (p < .001), and on seven of 10 subscale scores: (1) speech; (2) thought content (TC): delusions; (3) TC: compulsions; (4) affect and mood (p < .001 for all); (5) thought process (TP): speed (p = .011); (6) TP: coherence (p = .012); and (7) appearance, behavior, and cooperation (p = .04). The scores for TC: obsessions, TC: perceptions, and TC: suicidality did not differ across groups. Taking into consideration adjustments for multiple comparisons (11), the threshold for significance increased from .05 to .005 after Bonferroni correction, leaving as significant the differences in overall score, speech, delusions, compulsions, and affect and mood.

Table.

Performance on the Video-Based Tool Across Experience Levels

Completion of the online instrument proved to be a valuable educational complement and a meaningful springboard for clinical discussion. Specifically, after ratings were completed, learners were eager to review the material in order to clarify and share their observations. Learners considered the exercise a synthetic and useful way of making the MSE content educationally sticky and memorable. Learners’ free-text comments included how “videos made the material come alive,” “were organized and helpful for learning and recognizing symptoms,” made them feel “prepared to evaluate some of the patients we are likely to meet, as I had never seen a psychiatric patient and was apprehensive based on depictions in the media.” Several learners commented on how the “structure of the exercise led to clinically relevant opportunities for group participation and analysis,” in which “even though I did not identify several symptoms, the scoring helped me see what I was missing, rather than frustrate me. I appreciate that the exercise was not formally graded; that would have been a killjoy for an otherwise engaging and instructive activity.” Finally, and speaking to the verisimilitude of the videos, several learners wondered if the patients were real, and if so, whether their informed consents were valid or ethical, given how ill some of them appeared to be. We had no intention of deceiving the learners, and before finishing the exercise always clarified that patients were professional actors following a script.

Discussion

In this report we described a novel, video- and online-based exercise that complemented MSE instruction by quantifying learners’ ability to objectively identify psychiatric symptoms seen in routine medical practice.

The main utility of the video-based depictions and electronic scoring exercise was as an educational complement. The online instrument was specifically designed as a formative exercise, rather than as a summative or evaluative measure. Its educational use may be most relevant as a way to solidify didactic content on the components of the MSE. For example, facilitators in medical or nursing school preclinical courses, in psychiatry clerkships, or residency directors in psychiatry may consider a 2-hour session divided into a 50-minute initial presentation structured around the MSE rubric—ideally including embedded video examples—followed by the 30-minute video/electronic scoring exercise and the clinically rich discussion it is certain to foster.

Our tool was not intended as an evaluative instrument, as it was not designed for repeated use over time. Such use could not only introduce a practice bias but render it repetitive and burdensome to students as well. Moreover, several of the items did not clearly differentiate, even among groups with such different backgrounds, making it an unlikely measure of overall competence. Our tool may instead be construed as mapping onto the “knows how” level of Miller's pyramid,[12,13] allowing for the demonstration of clinical skills in a supervised setting.

We note several limitations to our report. First, the video depictions and rubric used in this instrument did not formally incorporate elements of the cognitive exam, such as those covered by the mini mental status exam[14] or the Montreal Cognitive Assessment.[15] Second, generalizability, particularly to graduate medical learners, was limited given our small undergraduate pilot sample. Third, we did not collect long-term follow-up of outcomes, so cannot comment on the stability of results over time, or how responsive they may be to greater clinical experience or to educational intervention, something we have started to explore elsewhere.[16] Finally, three instrument subscales appeared particularly limited in their ability to distinguish experienced from novice learners, and other components of the MSE, such as insight and judgment, were not included at all.

In summary, this MSE educational exercise was easy to implement, and its online tool effectively differentiated experienced from novice learners on global and most subscale measures. The module in general, and the online instrument in particular, were well received by medical and nursing students as a valuable didactic exercise and an educational complement. It is our hope that educators may find these video-based depictions and related online scoring instruments an engaging and easy to adapt complement to their teaching activities.

Case Script Ben OCD.docx

Case Script Robbin BD.docx

Case Script Karthik Schizophrenia.docx

Ben Video 1.mov

Ben Video 2.mov

Ben Video 3.mov

Robbin Video 1.mov

Robbin Video 2.mov

Robbin Video 3.mov

Karthik Video 1.mov

Karthik Video 2.mov

Karthik Video 3.mov

Scoring Sheet.docx

Answer Key.docx

Electronic Interface MSE instruments.docx

MSE Instrument Slide Deck.pptx

Facilitator Guide.docx

All appendices are peer reviewed as integral parts of the Original Publication.