SVI Self-Study Guide: A Study Tool for Independent Preparation for the AAMC Standardized Video Interview.

Introduction: Since 2017, students applying to emergency medicine residencies must take the AAMC Standardized Video Interview (SVI) to assess their knowledge of professional behaviors and interpersonal and communication skills. Due to the SVI's novelty, there are not many study tools available to prepare for it, outside of the resources provided by the AAMC.
Methods: The SVI Self-Study Guide is a PowerPoint document that learners can use to prepare for the SVI independently. It is intended for fourth-year medical students who are applying to emergency medicine residencies and therefore planning to take the SVI. The guide was distributed via email and assessed with a pre- and postquiz measuring subjective feelings of preparedness as well as testing knowledge of professionalism and interpersonal and communication skills.
Results: Eleven students were invited to use the SVI Self-Study Guide, of whom 10 and eight took the pre- and postquiz, respectively. There was a statistically significant increase in learners' self-rated feelings of preparedness to take the SVI (p < .05). Although there was no significant change in the average score on four knowledge-based questions (p = .29), the average score increased from 72.50% to 93.25%. Discussion: Overall, there was a higher improvement in feelings of preparedness to take the SVI than in performance on knowledge-based questions. This suggests that learners benefited the most from practicing using the SVI testing format. The SVI Self-Study Guide can be distributed via email to supplement existing resources in preparing for the SVI.

Educational Objectives

During the months leading up to the deadline to complete the Standardized Video Interview (SVI), students preparing to take the SVI will:

Review guidelines for answering questions effectively in the provided 3 minutes.

Experience an increase in feelings of preparedness to take the SVI.

Gain knowledge on specific professionalism topics, including delivering bad news, disclosing medical errors, time management, addressing patient nonadherence, and patient autonomy and informed consent.

Practice explaining professionalism topics verbally while using examples from their own life and clinical experiences.

Gain knowledge on specific interpersonal and communication skills, including conflict resolution, SBAR (situation-background-assessment-recommendation) and CUS communication tools, and providing effective feedback.

Practice explaining specific interpersonal and communication skills verbally while using examples from their own life and clinical experiences.

Introduction

In 2017, for the first time, students applying to emergency medicine residencies participated in the AAMC Standardized Video Interview (SVI) as part of their residency application process. Students who chose not to participate in the SVI during the 2017 application cycle were still allowed to apply to emergency medicine residencies, but they were warned that their applications might be considered incomplete. During the previous application cycle in 2016, approximately 600 students participated voluntarily in the original pilot of the SVI in return for financial compensation.[1] During the most recent application cycle, in the summer of 2018, students applying to emergency medicine residencies were again required to participate in the SVI.[2]

The SVI uses a combination of behavioral and situational questions to measure two Accreditation Council for Graduate Medical Education competencies: (1) knowledge of professional behaviors (professionalism) and (2) interpersonal and communication skills.[3] The AAMC provides some material to help students prepare for this new assessment tool.[4] However, due to the novelty of the SVI, there are not many study tools published by parties other than the AAMC. A literature review did not yield any comprehensive study guides to prepare for the SVI other than the resources provided by the AAMC; therefore, the SVI Self-Study Guide aims to fill this gap in availability of resources.

The SVI's format consists of six questions that must be answered by students orally in front of a camera. Students have 30 seconds to prepare each response, and 3 minutes to deliver a recorded answer.[3] The SVI Self-Study Guide aims to increase students' feelings of preparedness in approaching this new evaluation tool by providing practice questions in a similar format.

The AAMC offers limited information concerning the specific content tested in the SVI. The AAMC provides three example questions regarding both professional behaviors and interpersonal and communication skills, but it does not publish a curriculum. The AAMC's example questions cover topics that include working with challenging patients, communicating difficult messages, leading multidisciplinary teams, patient autonomy, addressing mistakes, and communicating bad news.[2] The questions in the SVI Self-Study Guide were written in a style similar to the AAMC's example questions. In addition, we looked at example questions in the Residency Interview Guide published by the Emergency Medicine Residents' Association to further inform the SVI Self-Study Guide's practice questions.[5]

The SVI Self-Study Guide reviews important professionalism and communication skills topics, including the SPIKES protocol for delivering bad news,[6] disclosing medical errors, time management, addressing patient nonadherence, patient autonomy and informed consent, conflict resolution, SBAR (situation-background-assessment-recommendation) and CUS communication tools,[7,8] and giving effective feedback.

The SVI Self-Study Guide is a self-guided resource that learners can utilize on their own. We chose a self-guided rather than an in-class methodology due to the benefits of easy accessibility and flexibility. This tool can be distributed via email, it only requires access to Microsoft PowerPoint, and learners have the choice to utilize it at their leisure. There is no requirement to gather students in one geographic location for didactic teaching.[9]

Methods

The University of Miami Miller School of Medicine Institutional Review Board reviewed and approved the research protocol for this study.

The SVI Self-Study Guide (Appendix A) was developed by an emergency medicine resident and two faculty members utilizing resources published by the AAMC[2-4] and validated curricula on relevant topics.[5-8] In addition, we consulted the expertise of a faculty member involved in the University of Miami Miller School of Medicine's emergency medicine residency program selection process and the Council of Emergency Medicine Residency Directors.

The guide consisted of a 50-slide PowerPoint presentation that learners could use independently. The PowerPoint included an introduction to the study guide, a brief introduction to the SVI with links to additional resources, eight SVI practice questions followed by background information that could be used to inform answers, and a description of the guide's overall objectives. The study module required approximately 1.5 hours to complete. Links to external resources were included within the SVI Self-Study Guide. These were suggested but not required to complete the activity.

Via email, we invited 11 rising fourth-year medical students applying to emergency medicine residency training at the University of Miami Miller School of Medicine to use the SVI Self-Study Guide and complete a pre- and postquiz (Appendices B & C) on a voluntary basis. The email was sent 6 weeks prior to the national deadline for completing the SVI. Students had to have access to a computer with PowerPoint software in order to use the study tool.

The prequiz contained a consent form that learners had to complete in order to participate in this study. The prequiz was distributed as a link at the beginning of the PowerPoint presentation. The postquiz was distributed as a link at the end of the PowerPoint presentation. Students were instructed to first take the prequiz, then complete the study guide, and finally take the postquiz. We did not track students' adherence to these instructions or the time frame in which the quizzes and study guide were completed.

Both quizzes consisted of five questions evaluating students' perceived preparedness to take the SVI, followed by four questions evaluating knowledge based on the content in the SVI study tool. The questions on perceived readiness were designed to measure confidence in regard to the tested content (professionalism and interpersonal and communication skills) and testing format, based on the AAMC's published information.[2-4] The knowledge-based questions were designed to test retention of information covered in the study guide. To ensure anonymity, as only 11 students were invited to participate, we did not collect participants' demographic data.

Results

Eleven students who were planning to take the SVI were invited to use the SVI Self-Study Guide. Ten learners completed the prequiz, and eight learners completed the postquiz. Using Excel software, we analyzed and compared the pre- and postquiz results. We compared students' self-rated perceived preparedness on each of the five questions and on an overall average using a two-tailed t test (Table 1). Due to the small sample size, we here provide the raw data on students' self-rated perceived preparedness (Table 2). We also compared students' overall percentage scores on knowledge-based questions on the pre- and postquiz using a two-tailed t test (Table 3).

Table 1.

Learners' Perceived Preparedness to Take the Standardized Video Interview

Test-Taking Componenta	Prequiz (N = 10)	Postquiz (N = 8)	p
Knowledge of professional behaviors	6.800	7.625	.693
Knowledge of interpersonal and communication skills	6.800	7.875	.173
Ability to deliver a complete answer within 3 minutes	4.500	7.250	<.001b
Ability to provide an organized response	4.300	6.875	.002b
Overall confidence and readiness	4.500	6.875	.004b
Average	5.380	7.300	.014b

Rated from 1 to 10.

p < .05.

Table 2.

Learners' Perceived Preparedness to Take the Standardized Video Interview: Raw Data

Knowledge of Professional Behaviors (Professionalism)		Knowledge of Interpersonal and Communication Skills		Ability to Deliver a Complete Answer Within 3 Minutes		Ability to Provide an Organized Response		Overall Confidence and Readiness
Pre	Post	Pre	Post	Pre	Post	Pre	Post	Pre	Post
2	6	4	7	2	5	1	5	2	5
5	7	5	7	4	7	3	6	3	6
6	7	5	7	4	7	3	7	3	7
7	7	6	7	4	7	4	7	3	7
7	8	7	8	4	7	4	7	4	7
7	8	7	8	4	8	5	7	5	7
7	9	7	9	5	8	5	7	5	8
8	9	8	10	5	9	5	9	6	8
9		9		6		6		7
10		10		7		7		7

Learners (N =10 pre, N = 8 post) were asked to rate from 1 to 10 how prepared they felt to take the Standardized Video Interview with regard to the test-taking components listed as column headings here. We include raw data on  individuals' ratings due to the small sample sizes.

Table 3.

Learners' Performance on Knowledge-Based Questions

Question	% Correct Answer
	Prequiz (N = 10)	Postquiz (N = 8)
1	100	100
2	20.00	87.50
3	90.00	100
4	80.00	87.50
Averagea	72.50	93.75

p = .29.

There was a significant difference (p < .05) in learners' perceived preparedness to take the SVI in three out of five questions assessing perceived preparedness, as well as on the overall average (Table 1).

There was no significant difference between the pre- and postquiz average score on the four knowledge-based questions (p = .29). However, the average score increased from 72.50% to 93.25% (Table 3).

Discussion

Overall, there was a higher improvement in feelings of preparedness to take the SVI (Tables 1–2) than in performance on knowledge-based questions (Table 3). There was an especially high increase in perceived preparedness in ability to provide a complete answer within 3 minutes, ability to provide an organized response, and overall confidence and readiness. This suggests that students benefited the most from the SVI Self-Study Guide's opportunity to practice answering questions in the SVI test format. This is consistent with the fact that the SVI has a novel testing format.

There was no significant difference between the pre- and postquiz average score on the four knowledge-based questions (p = .29), but the average score increased from 72.50% to 93.25% (Table 3). This is consistent with the fact that the students were taught professionalism and interpersonal and communication skills in medical school; therefore, their scores on the knowledge-based questions were already 72.50% in the prequiz. In fact, the Liaison Committee on Medical Education includes communication skills and interprofessional collaborative skills as two of its nine types of mandated curricular content, so all medical schools are required to teach these knowledge areas.[10] These findings suggest that in creating resources to prepare for the SVI, it is most useful to provide tools to practice using the SVI testing format.

Limitations

Students were instructed to take the prequiz before completing the study guide and the postquiz afterwards, but the timing for taking the tests was not enforced or standardized. Therefore, the results may be limited by recall bias. However, this bias is reduced by the fact that students were not provided with the right answers to the questions after taking the prequiz. In the future, we will track completion of the prequiz, study guide, and postquiz in order to ensure completion in the correct order and monitor the time frame. In addition, the sample size was only 10 for the prequiz and eight for the postquiz, so our findings may not be generalizable to all students. In the future, we could implement the SVI Self-Study Guide at several medical schools in order to increase the sample size and ensure generalizability. The pre- and postquiz can be used to assess the SVI Self-Study Guide's utility when implemented in new student populations. In this study, the SVI Self-Study Guide was assessed via pre- and postquizzes, but we did not investigate the impact of the SVI Self-Study Guide on students' actual SVI scores. In the future, a randomized controlled trial study design could be used to investigate the SVI Self-Study Guide's impact on students' SVI scores.

Conclusion

To implement the SVI Self-Study Guide, we recommend emailing it to all fourth-year medical students planning to apply to emergency medicine residencies 6–10 weeks before the deadline to complete the SVI. We found that implementing the SVI Self-Study Guide needed minimal resources as it only required distribution via email. While the SVI Self-Study Guide provides the opportunity to practice using the SVI testing format through practice questions, it includes only a limited number of practice questions and does not give individualized feedback for students' responses. Students would benefit from individual counseling and feedback on their responses as well as from additional practice questions in order to further prepare for the SVI. We did not collect individual feedback on students' experience using the SVI Self-Study Guide. In the future, we will organize focus groups after the SVI completion deadline in order to expand and improve the SVI Self-Study Guide, in addition to supplementing it with additional resources, based on student feedback.

A recent critique of the SVI by Buckley, Hoch, and Huang suggests that “until the validity of the SVI's ability to predict future performance is determined, mandatory participation and reporting to residency programs should not be implemented.”[1] Buckley and colleagues warn that the SVI may unintentionally diminish diversity in the field of emergency medicine and state that other markers in an applicant's portfolio may already fill the needs that the SVI aims to fill. Even though the utility of the SVI continues to be debated, we recommend that while it remains in use, students can benefit from resources to prepare, including the SVI Self-Study Guide.

The SVI Self-Study Guide is a teaching tool that learners can use independently. The only requirement for use is access to a computer with PowerPoint software. The guide can easily be distributed via email in order to provide students planning to take the SVI with an additional resource. The SVI Self-Study Guide is not meant to replace existing resources, such as the AAMC's Preparing for the AAMC Standardized Video Interview: Tips for Applicants,[4] but rather to supplement them.

A. Standardized Video Interview Self-Study Guide.pptx

B. Prequiz With Consent Document.docx

C. Postquiz.docx

All appendices are peer reviewed as integral parts of the Original Publication.