Standardized Patients to Teach Medical Students about Intimate Partner Violence.

OBJECTIVE: To use 360-degree evaluations within an Observed Structured Clinical Examination (OSCE) to assess medical student comfort level and communication skills with intimate partner violence (IPV) patients.
METHODS: We assessed a cohort of fourth year medical students' performance using an IPV standardized patient (SP) encounter in an OSCE. Blinded pre- and post-tests determined the students' knowledge and comfort level with core IPV assessment. Students, SPs and investigators completed a 360-degree evaluation that focused on each student's communication and competency skills. We computed frequencies, means and correlations.
RESULTS: Forty-one students participated in the SP exercise during three separate evaluation periods. Results noted insignificant increase in students' comfort level pre-test (2.7) and post-test (2.9). Although 88% of students screened for IPV and 98% asked about the injury, only 39% asked about verbal abuse, 17% asked if the patient had a safety plan, and 13% communicated to the patient that IPV is illegal. Using Likert scoring on the competency and overall evaluation (1, very poor and 5, very good), the mean score for each evaluator was 4.1 (competency) and 3.7 (overall). The correlations between trainee comfort level and the specific competencies of patient care, communication skill and professionalism were positive and significant (p<0.05).
CONCLUSION: Students felt somewhat comfortable caring for patients with IPV. OSCEs with SPs can be used to assess student competencies in caring for patients with IPV.

INTRODUCTION

Coker et al.1 reported 29% of women have experienced physical, sexual or psychological intimate partner violence (IPV) in their lifetime. Additionally, according to the Bureau of Justice Statistics 42% of women victimized by an intimate partner sustained injuries, with females comprising 58% of murder victims.2 Emergency departments (ED) and other primary care providers are often the entry point for individuals experiencing family violence.3–11 Therefore, healthcare providers should play a vital role in ensuring that their patients receive the assistance they need to address violence in the home.

Recently the Institute of Medicine (IOM) compiled a comprehensive review of the current state of family violence education in the IOM’s 2001 report, “Confronting Chronic Neglect: The Education and Training of Health Care Professionals on Family Violence.” In addition, the IOM made specific recommendations for family violence curricular development.4 As noted in the IOM report, basic competency skills for health providers, as recommended by the American Association of Colleges of Nursing, can serve as template for describing competency skills for all healthcare professionals.4 In our educational setting, there had been sporadic teaching of family violence in the M1 and M2 years. An informal survey of our course directors revealed three hours over four years dedicated to teaching family violence.

There is increasing commitment by the academic community to improve IPV education in the United States (U.S.). The Liaison Committee on Medical Education accreditation standards state “the curriculum must prepare students for their role in addressing the medical consequences of common societal problems, for example, [by] providing instruction in the diagnosis, prevention, appropriate reporting and treatment of violence and abuse.”12 Despite the importance, the literature is sparse with examples on how to teach this topic. Typically, the lectures are sporadic and interspersed between courses that may serve as the appropriate vector for teaching, i.e. teaching child maltreatment in pediatrics or IPV in the women’s health module.4 One university instituted formal lectures on IPV, including a mock patient evaluation, during the first year medical school curriculum.13

Observed Structured Clinical Examinations (OSCE) that use standardized patients (SP) have been used for over 30 years to assess clinical skills in increasingly high stakes settings.14 Both OSCE and SPs have been shown to be effective in training and assessing core components of clinical skills for topics typically considered difficult communications. With respect to IPV, SPs have been used in the OSCE format to teach IPV skills to fourth year students rotating in the primary care setting. This technique has been shown to be an effective modality to teach them core concepts in IPV and improve clinical skills with lasting effects.20 While competency-based evaluation is the goal on an emergency medicine (EM)clerkship, OSCE models in the emergency setting have not yet been widely developed.21 Our review of the literature indicates that there is only one research study regarding the reliability of OSCE when applied to EM resident training in the U.S.22 In addition, Johnson and Reynard23, assessed OSCE as an evaluative method in Great Britain in 1994 with undergraduate students in EM. Another study in Israel developed a national domestic violence (DV) experiential training program for physicians and noted that they improved their capabilities to manage DV cases.24

The purpose of this study was to implement and assess an IPV OSCE module using SPs for fourth year medical students during their EM rotation. Secondary objectives were to measure competency in IPV, comfort (self-efficacy), communication skills and professionalism.

METHODS

Study Design

This was a University Institutional Review Board (IRB) approved prospective, observational study. All students provided informed verbal consent in compliance with our university IRB.

Study Methodology

We assessed IPV knowledge and skills as part of an OSCE at the end of a four-week EM fourth year rotation. All students performed one IPV SP encounter during three separate OSCE evaluations.

Study Protocol

Students received interactive didactic training during a one-hour small group session on family violence and IPV with trained faculty. Prior to the didactic training, each student’s knowledge, attitudes and belief about IPV was assessed from a pre-test. The pre-test also assessed the student’s comfort level in caring for IPV patients. Within several weeks of this educational session students participated in an ED OSCE piloted as a summative evaluation at the end of a mandatory EM rotation. As an adjunct to the OSCE we conducted our study of IPV skills.

The SPs received two hours of IPV training by the grant principal investigator (PI) on the SP encounter, which was adapted from Reteguiz and Cornel-Avendano’s25 workbook on SP cases. Specific items on the SP checklist included: screening for IPV, assessing safety of the victim, addressing injuries, and providing resources and counseling.

Students, SPs, and grant personnel completed the same single evaluation of the student during the SP encounter. The evaluation included communication and competency skills.

EM grant personnel served as evaluators for the SP encounter. The grant PI and Co-PIs had an average of eight years of EM training and an average minimum of 40 hours of formal IPV training. Grant personnel created the SP case and reviewed the checklist reviewed during a grant personnel meeting. The role of the evaluator included observing the history and physical (H&P) and completing the check-off list relating to the H&P. In addition, individual feedback by grant personnel was given to the student immediately after the SP exercise. Feedback was based upon the checklist, grant personnel observation of the student’s communication skills, the student’s physical examination technique, and the student’s discussion of diagnosis and plan with the SP. The post-test was administered to assess any improvement in the student’s knowledge, recognition and awareness of family violence.

Measurements

Competency

The 360-degree evaluation form, which was developed by grant personnel to assess IPV skills of fourth year medical students in the IPV OSCE station, measured student competency. The evaluation instrument included a checklist of responses that would assist faculty trained in family violence education in making a final determination of the competency level of the student to appropriately manage IPV patients. There were 25 items specific to the SP encounter, three related to competency skills (patient care, communication and professionalism) and three global items (student’s overall comfort, confidence and informativeness). A trained faculty, the SP and the student also completed the same evaluation form providing a 360-degree assessment of the student’s performance.

Comfort

Pre- and post-test and the 360-degree evaluation measured student comfort. Grant personnel with IPV expertise developed and reviewed pre- and post-test content to ensure their face validity. The pre- and post-test were the same instrument. The 25-item checklist assessed the trainee’s communication skills and comfort in working with IPV patients. The pre- and post-test were not graded, but rather compared to see a difference in response by the trainee after the SP encounter.

Professionalism and Patient Car

Professionalism and patient care were measured by the 360-degree, 25-item checklist. Using a five point Likert scale, the SP and faculty rated the students on patient care, communication skills and professionalism.

Data Analysis

We imported the checklist information and assessment of competency from the SP, attending and student into an Excel database, and then computed frequencies, means and correlations using SAS version 8.0

RESULTS

Forty-one students participated in the SP exercise. The students’ pre-test was noted at mildly uncomfortable to comfortable (2.7), with minimal change on the post-test (2.9) after the SP encounter (p=NS).

In this cohort, which includes students from 29 states and four countries, approximately 13 % are underrepresented minorities and 49% are women. Table 1 exhibits H&P findings. Although 88% of the students screened for IPV and 98% asked about the patient’s injury, only 39% asked about verbal abuse and only 11% documented their findings in the chart.

Table 1

Domestic violence standardized patient encounter percent totals from all fields (n=118)

Standardized Patient Checklist	No (%)	Yes (%)
Patient Care
Screened for DV*	11.86	88.14
Assessment (Asked about…)
Injuries*	1.71	98.29
Medical problems*	11.97	88.03
Medication use*	29.31	70.69
Whether husband was physically abuse*	11.21	88.79
Whether husband was verbally abuse*	60.87	39.13
Patient was afraid at home*	30.97	69.03
Physical Examination
Checked arm for trauma*	2.56	97.44
Monitored fetal heart tones*	77.19	22.81
Checked body for other areas of bruising*	18.10	81.90
Document findings in chart*	89.38	10.62
Assesses for safety (Asks if …)
Patient feels safe at home*	17.24	82.76
Firearms or other weapons in home*	84.96	15.04
Children safe in home*	43.10	56.90
Patient has emergency plan*	82.76	17.24
Communication skills
Patient does not deserve abuse*	48.28	51.72
Acknowledged leaving is difficult*	70.69	29.31
Offer DV support and counseling*	19.13	80.87
DV in home affects future behavior of children*	87.93	12.07
DV is illegal*	87.07	12.93
Made Appropriate Referral
Social work*	49.14	50.86
Law enforcement*	82.46	17.54
DV advocates/support groups*	38.05	61.95
Shelter/Safe house*	50.43	49.57
Hotline Number*	45.22	54.78

Indicate missing values

DV, domestic violence

SP, attending and student assessment for competency, professionalism and patient care show average to very good composite scores (Table 2 and 3). The majority of students ranged in their communication skills from good to very good (Table 2). We analyzed the correlations analyzed using three evaluators. Correlation between overall trainee comfort level and each of the competencies (patient care, empathetic communication skill and professionalism) were positive (Table 4). Correlations when stratified by evaluators were not universally statistically significant.

Table 2

Assessment by standardized patient, grant personnel and student for student’s competency.

Standardized Patient – Domestic Violence Standardized Patient Encounter

Competency – Totals (%)
	Very Poor	Poor	Acceptable	Good	Very Good
Patient Care*	1.83	0.92	28.44	45.87	22.94
Empathetic Communication Skills	1.72		17.24	47.41	33.62
Professionalism	0.87		13.04	46.09	40.00

Four missing values

Average means:

(1=Very Poor, 2=Poor, 3=Acceptable, 4=Good, 5= Very Good)

Patient Care: 3.87 (n=109)

Empathetic communication skills: 4.11 (n=116)

Professionalism: 4.24 (n=115)

Table 3

Assessment by standardized patient, attending physician and student for student’s comfort level and informativeness overall evaluation – totals (%)

	Very Poor	Poor	Acceptable	Good	Very Good
Trainee comfort level	1.72	2.56	27.59	41.38	26.72
Trainee’s confidence level	1.72	2.56	22.41	47.41	25.86
Informativeness of trainee	5.17	10.34	40.52	31.03	12.93

Average Means:

(1=Very Poor, 2=Poor, 3=Acceptable, 4=Good, 5= Very Good)

Trainee comfort level: 3.89 (n=116)

Trainee’s confidence level: 3.93 (n=116)

Informativeness of trainee: 3.36 (n=116)

Table 4

Correlation with competency and trainee comfort level standardized patient – domestic violence standardized patient

Correlation:	Overall Trainee Comfort Level
Competency – patient care	0.6382
(n=109)	<0.0001
Competency – empathetic communication skills	0.6548
(n=116)	<0.0001
Competency – professionalism	0.6850
(n=115)	<0.0001

The correlation between overall trainee comfort level and each of the competencies (patient care, empathetic communication skill and professionalism) are significant. All correlations are positive.

Note: The above correlation was run using all three evaluators. When the correlation was stratified by evaluator, the correlation was not significant between comfort and professionalism for the principal investigator (p=0.0859).

DISCUSSION

With the recent emphasis on clinical competency, the use of SPs is being recognized as a useful evaluation tool to assess competency in medical schools. Currently, almost half of medical schools use SPs in clerkships and in fourth-year examinations.26 Recently, specific classes have been developed to teach medical students these communication skills. One example is the Uniformed Services University of Health Sciences, which developed a pilot course as part of a psychiatry clerkship to improve medical student communication skills.27

Goals of an undergraduate EM rotation include the assessment of acutely sick and injured patients as well as experience in evaluating the undifferentiated patient.21, 28 Many IPV victims in the ED setting will not present with IPV as their chief complaint and therefore may go unrecognized, unless students are taught to inquire specifically about IPV. Further, many patients may not present to the ED with IPV as their primary complaint but would be willing to disclose IPV if asked directly by a physician.6 One way to assist students in considering the diagnosis of IPV is to introduce the subject early in medical student education. Using the SP encounter in IPV can be one such tool. This study sought to develop an IPV SP encounter and an evaluation tool for this educational goal.

SP encounters have several advantages. It allows for direct observation of the student, an opportunity for immediate feedback to the student by faculty, and a specific focus on the student’s ability to manage IPV. In addition, students have the opportunity to ask specific questions related to IPV and are encouraged to include it as part of their differential when caring for patients. Our use of the pre- and post-test after using the SP IPV scenario did not significantly improve student comfort; however, Boulet et al29 caution that relatively little research has been done to assess how to best set a passing standard on SP encounters and recommend that future research using SPs should include standard-setting methods. To assess or replicate research findings of published reports, journal editors, reviewers and authors should provide adequate detail when describing SP methodology.30

LIMITATIONS

Limitation of this study include not assessing students previous experience with IPV, either through clinical experience, didactics or personal experience prior to training. Second, due to the small sample size of this pilot study, we cannot extrapolate these findings to the entire medical school class and we don’t know how these findings would persist long term. Third, the students did not have demographic markers on the OSCE pre-test and post-test that determined ethnicity and gender; thus we were unable to infer any biases based on ethnicity and gender, although a larger study should look at those demographics and how they affect the results. Fourth, we did not achieve a 100% completion rate for all the forms, which resulted in missing data. Lastly, the SP case and form has not been validated and would warrant use and repetition in other student ED rotations to test its validity and reliability.

CONCLUSION

Effective methods for training medical students in IPV are needed. Using SPs in an OSCE setting could help trainees focus on the affective elements of training (such as attitudes) in caring for patients who experience IPV. This methodology should be further developed to use as a tool to measure students’ competencies in caring for patients with IPV. We found correlation among the trainee comfort level, patient care, communication skills and professionalism. In our investigation, the OSCE did not show a clinically significant improvement in students’ comfort level in caring for IPV victims. The SP OSCE did help us identify likely gaps in our educational curriculum. Further studies using SP with IPV should be conducted to determine effectiveness of this educational technique within the medical student curriculum.