Case-Based Curriculum With Integrated Smartphone Applications Improves Internal Medicine Resident Knowledge Of Contraceptive Care.

BACKGROUND: Contraception is an essential preventive service for all women. However, the literature suggests that internal medicine residents have low levels of confidence and knowledge of contraceptive care.
OBJECTIVE: We designed and implemented a novel contraception curriculum promoting active, collaborative learning. We sought to evaluate whether this curriculum improved internal medicine resident knowledge of and comfort with contraceptive care through the administration of pre-/post-surveys.
METHODS: Our curriculum was delivered in a two-hour session as part of the mandatory ambulatory curriculum for internal medicine interns at our institution in the academic year 2017-2018. Interns were provided with select online resources and two smartphone applications at the beginning of the session, which they then used in case-based small group work. Small group work was followed by a large group case review, co-facilitated by OB/GYN and internal medicine faculty.
RESULTS: Thirty-eight participants completed surveys assessing knowledge of and comfort with contraceptive care immediately before and after the curriculum; 20 participants completed surveys assessing the same domains 4-6 months after the curriculum. Data from surveys administered immediately post-curriculum demonstrated significant improvements in knowledge about and comfort with counseling about, assessing medical eligibility for, and initiating multiple forms of contraception. Many of these improvements in knowledge and comfort were maintained on follow-up surveys 4-6 months following the curriculum.
CONCLUSION: Our case-based curriculum with integrated smartphone applications resulted in significant improvements in internal medicine resident knowledge of and comfort with the key skills of contraceptive care. In contrast to active, collaborative learning methodologies such as the flipped classroom, our methodology supports active, collaborative learning without requiring advance learner preparation, and is thus well suited to the time constraints of the graduate medical education setting. Our methodology is readily translatable to other clinical topics and residency curricula.

Introduction

Almost half of United States pregnancies are unintended.1 Unintended pregnancy is associated with adverse health, economic, and social consequences, particularly for women with chronic diseases who are at higher risk for maternal and fetal complications during pregnancy.2,3 As women with chronic diseases are more likely to receive primary care from internists compared to other types of providers, contraceptive care is an essential component of the care general internists provide to these patients, and to all patients seeking to avoid pregnancy.4

Despite the importance of contraceptive care in primary care and internal medicine, internal medicine residents report low levels of self-confidence and perceived competence surrounding contraceptive care.5–7 In a 2004 study, greater than 80 percent of primary care internal medicine residency program directors believed that their residents should master the prescription of oral contraceptive pills by the end of residency; however, only 10 percent believed that their residents did master this skill by the end of the residency.7 In a 2012 study at the University of Colorado, over 80 percent of residents reported perceived inadequate knowledge as a barrier to performing contraceptive counseling; greater than 20 percent of residents in this same study reported “low self-efficacy” around contraceptive counseling as a barrier to providing contraceptive counseling to patients.6

This lack of knowledge and comfort around contraceptive care perceived by both residents and by their supervisors likely results from limited curricula and training opportunities in women’s health in internal medicine residency programs.8,9 A national 2015 survey of internal medicine residency program directors demonstrated that inclusion of women’s health core competencies defined by the American Board of Internal Medicine was variable between programs nationally; contraception specifically was included in only approximately 70 percent of internal medicine residency curricula nationally.9

To address the need for improved contraceptive education among internal medicine residents, we designed, implemented and evaluated a novel contraception curriculum. In recognition of the increasing movement away from lectures in medical education and toward active, collaborative learning activities such as the flipped classroom, as well as growing use of smartphones as a portable educational and clinical resource, we employed a case-based learning design in which residents were provided with select online resources and smartphone applications for use in collaborative, case-based problem-solving.10–15

Methods

Settings And Participants

Our contraception curriculum was integrated into the ambulatory curriculum in the academic year 2017–2018 for first-year internal medicine residents (interns) at an urban tertiary academic medical center in the northeastern United States. The Partners HealthCare Institutional Review Board deemed this research exempt from further review.

Curriculum Description

Our curriculum consisted of a two-hour session facilitated by one internal medicine faculty member (L.P., H.G. and C.A.) or senior resident (A.B.) knowledgeable in reproductive health and one Obstetrics/Gynecology faculty member (K.B., D.B. and W.J.W.). We held four sessions, each with 10 participating interns. At the opening of each session, participants were provided a “toolbox” of references about contraception to guide their engagement with clinical cases, including the websites for the Center for Disease Prevention and Control (CDC) Medical Eligibility Criteria and the Reproductive Health Access Project, as well as two free smartphone applications (the CDC Contraception application and Contraception Point-of-Care application), which the participants were encouraged to download onto their phones.16–19

Participants were then split into groups of 2–3 interns and assigned one of four clinical cases to work through using the provided references, as well as any literature they located using their laptops. Each case led the participants through a clinical vignette with a series of discussion questions that covered pre-identified learning objectives for the case (see ). Facilitators circulated during this time to assist with questions. After 20 mins of small group work, the Medicine and Obstetrics/Gynecology facilitators led an 80-min review of the cases where each group presented its case and responses to the discussion questions to the other groups, all of whom had worked on different clinical cases. During this review, participants were encouraged to ask clarifying questions about the cases, both of each other and the session facilitators.

Program Evaluation

We developed and administered anonymous written surveys to participants immediately before and after the session, as well as 4–6 months following the session (“delayed post-survey”), with the intent to: (1) describe baseline knowledge about and comfort with providing contraceptive care, and (2) assess the effectiveness of our curriculum in improving knowledge about and comfort with providing contraceptive care, both immediately post-session and over time (See –). This pre-/post-survey approach to curriculum evaluation is standard in education literature as a means of assessing “changes in proficiency [that] have occurred in learners during the course of the curriculum.”20 The inclusion of a delayed post-survey is frequently used as a means of assessing whether changes in proficiency persist over time.20

To assess participant knowledge of contraceptive care, our surveys included 10 board exam-style questions with clinical vignettes, which assessed participants’ ability to match patient preferences to contraceptive method, facility with initiating a variety of contraceptive methods and knowledge of medical eligibility criteria for contraception. These questions were designed in accordance with the National Board of Medical Examiners’ guidelines for test question construction.21

Based on the literature cited above indicating low “self-efficacy” among residents as a barrier to provision of contraceptive counseling, our survey also incorporated questions about participants’ comfort with or confidence in various aspects of contraceptive care.6 Surveys included questions about participant comfort with (1) counseling about different forms of contraception, (2) initiation of contraception, and (3) assessing medical safety of contraception for patients with a range of medical conditions. Participants used a 5-point Likert scale to rate their level of comfort or confidence, ranging from “very comfortable/confident” (1) to “not at all comfortable/confident” (5). Five-point Likert scales for evaluation of comfort or confidence around clinical skills can be found elsewhere in the medical education literature, such as for the evaluation of trainee confidence or comfort with end-of-life care.22

The survey administered immediately after the session also asked participants to rate the value of the session, using another 5-point Likert scale. Similar 5-point Likert scales are used at our institution for medical education evaluation, and can be found elsewhere in the medical education literature for assessment of trainee satisfaction with educational programs.23,24

We used parametric tests for data analysis, which are appropriate for numerical data, including Likert-type data.20 We used paired t-tests to compare mean knowledge scores, as well as comfort and confidence scores, before and immediately after the curriculum. We used independent t-tests to compare the same domains between the pre- and delayed post-surveys, as the delayed post-surveys were anonymous. Participants used self-selected study identification codes for the pre- and immediate-post-surveys; most were unable to remember these identification codes for the delayed post-surveys and thus these surveys were analyzed independently. In addition to assessing and comparing global knowledge scores before and after the curriculum, we assessed and compared performance specifically in each knowledge domain (patient preference, contraceptive initiation and medical eligibility).

Results

In total, 40 interns participated in the curriculum. Thirty-nine interns completed the pre-curriculum survey, 39 completed the post-curriculum survey (for a total of 38 paired pre- and immediate post-surveys, or 95% of possible survey participants) and 20 (or 50% of possible survey participants) completed the delayed post-survey 4–6 months after the curriculum.

Baseline Knowledge Of And Comfort With Contraceptive Care

Prior to the session, the mean knowledge score (ie percent of 10 questions answered correctly) was less than 50% among all participants (Figure 1). Mean ratings of comfort and confidence around contraceptive care ranged from 3.0 for comfort with counseling about the intrauterine device (IUD) or implant to 3.6 for confidence in initiating birth control pills, patch, ring or injection (where 1= very comfortable or confident and 5= very uncomfortable) (Table 1).

Figure 1

Resident performance on knowledge assessment before and after curriculum participation.

Notes: Study participants completed a knowledge assessment, composed of 10 boards-style questions with clinical vignettes, immediately before, immediately after and 4–6 months after participation in the curriculum. P values to assess the statistical significance of changes between pre- and immediate post-survey data were calculated using a paired t-test; p values to assess the statistical significance of changes between pre- and delayed post-survey were calculated using an independent t-test.

Table 1

Resident Comfort And Confidence With Contraceptive Counseling And Prescription Before And After Curriculum Participation (1= Very Comfortable/Confident, 5= Not At All Comfortable/Confident)

	Pre-TestMean Score, 1–5 (SD), n=38	Immediate Post TestMean Score, 1–5 (SD), n = 38	P-Valuea	Delayed Post TestMean Score, 1–5 (SD), n= 20	P-Valueb
Comfort
With counseling, in general	3.2 (0.94)	2.2 (0.59)	< 0.001	2.9 (0.85)	0.246
With counseling about pills, patch, ring or injection	3.4 (0.93)	2.2 (0.56)	<0.001	2.8 (0.89)	0.044
With counseling about IUD, implant	3.0 (1.003)	1.9 (0.57)	<0.001	2.5 (1.1)	0.073
Confidence
Evaluating medical eligibility	3.2 (0.89)	2.1 (0.76)	< 0.001	2.7 (0.81)	0.004
Initiating pill, patch, ring, injection	3.6 (0.84)	2.3 (0.58)	< 0.001	3.0 (0.75)	0.002
Preparation for placement IUD, implant	3.4 (1.07)	2.1 (0.72)	<0.001	2.7 (0.99)	0.04

Notes: aPaired t-test comparing immediate post-curriculum scores to pre-curriculum scores. bIndependent t-test comparing delayed post-curriculum scores to pre-curriculum scores.

Post-Curriculum Knowledge Of And Comfort With Contraceptive Care

Immediately following the curriculum, the mean knowledge score improved significantly, nearly doubling from 49.2% correct answers on the pre-survey to 84.5% correct on the post-survey (Figure 1). Comfort and confidence scores also improved significantly across all domains (Table 1).

Many of these gains were maintained in delayed post-surveys at 4–6 months. Analysis of knowledge scores between pre- and delayed post-surveys demonstrated persistent statistically significant improvements in all knowledge domains. Compared with pre-curriculum scores there remained statistically significant gains in comfort with counseling about shorter-acting methods, confidence in assessing medically eligibility, and confidence in initiating both shorter and long-acting reversible forms of contraception, when compared to the pre-surveys (Table 1). However, improvements were no longer statistically significant around comfort with counseling in general, and comfort with counseling about the IUD or implant in particular.

Resident Feedback

Interns rated the session at a mean of 1.44 (1= very valuable, 5= not valuable at all). One participant commented, “Loved having time to download and use apps! Loved case based approach. Loved having [internal medicine] doc and GYN doc. Really excellent.”

Feasibility

This curriculum was integrated into the intern ambulatory curriculum and delivered as part of a morning of classroom-based didactics, thus ensuring access to the curriculum. Resources required for delivery of the curriculum included faculty teaching and curriculum materials. We had access to faculty from Obstetrics/Gynecology and Internal Medicine with interests in medical education and knowledge of contraceptive care. As a result of the difficulty coordinating different faculty schedules, there was no in-person faculty development session prior to curriculum facilitation; rather, faculty were provided with the curriculum materials and instructions about curriculum delivery electronically.

Discussion

Internal medicine residencies must equip their trainees to provide comprehensive women’s health care through providing training in contraception – a recommendation supported by recent policy statements by the American College of Physicians, which call for more robust training in women’s health, including contraception, at all levels of medical education.25 In this article we describe an innovative pedagogical approach, pairing active, collaborative learning activities with use of smartphone applications and online resources in “real-time,” to engage internal medicine interns and successfully improve resident knowledge and comfort around this critical clinical skillset.

The term active learning activities refers to instructional methods that promote learner engagement, including individual journaling, individual quizzes, group or paired discussion, solicitation of audience response through verbal questions or interactive audience response systems, and peer to peer teaching, among others.26–29 In general, these activities are thought to promote higher-order cognitive processes such as interpretation, application and analysis when compared to passive learning activities such as traditional lecture—and evidence from general and medical education literature indicates that active learning activities promote deeper understanding and superior recall relative to the traditional lecture.27,29

Collaborative learning activities represent a subset of active learning activities in which individual learners “work together toward a common goal and [the emphasis is on] group interactions” rather than working in isolation.29 In addition to conferring the benefits of active learning, collaborative learning activities are associated with improved quality of interpersonal interaction relative to individualistic learning.27 This is a benefit of significant value for medical trainees in an era where effective communication and functioning in interdisciplinary teams are key competencies for practicing physicians.30

In recognition of these advantages of active, collaborative learning activities, instructional methods incorporating these kinds of activities have increasingly replaced traditional lectures in medical education over the last three decades.11,12 The flipped classroom approach, in particular, has gained popularity in higher education, including medical education.11 The flipped classroom describes an educational strategy in which learners read or view content individually or in small groups prior to classroom time, and then use classroom time to engage in a guided application of concepts through a variety of active learning activities, such as problem sets.11 Though the flipped classroom model is defined by the use of classroom time for active learning activities and not necessarily collaborative ones, in practice its use in higher education and medical education has favored collaborative learning activities during classroom time, in which small groups work through clinical cases or other problem-solving activities together.11,29,31

Despite its growing popularity in medical education, implementation of the traditional flipped classroom poses unique challenges in graduate medical education settings. Specifically, resident time and motivation to research learning objectives, review content or complete assessments prior to a didactic session may be scarce or non-existent in the setting of demanding residency work hours, even on less hectic outpatient rotations. In Graham et al’s recent report on the flipped classroom, the authors note that time limitations are a major “logistical challenge” in utilizing the flipped classroom in graduate medical education.10 Emergency Medicine residents participating in a different study of the flipped classroom noted that “poor compliance” with content review and “finding time” to complete content review were weaknesses of the flipped classroom.10,32

In our model, we incorporate active, collaborative learning activities through case-based problem-solving in small and large groups without requiring advance preparation. Instead of requiring learners to review content ahead of classroom time, we provided them with curated online resources and smartphone applications to support problem-solving. This innovative approach not only addresses issues of feasibility with the flipped classroom, but also respects a tenet of adult learning by increasing the learning activity’s relevance to the learners’ professional, clinical role.33 Multiple studies demonstrate that over 50 percent of residents use smartphone applications in their clinical practice; residents also use their smartphones to access general web-based resources.13,15 Thus, by incorporating online resources and smartphone applications as problem-solving resources for our residents, we are mirroring their clinical practice and increasing the relevance of our learning activity to their daily professional tasks.

In summary, we created a curriculum that incorporates active and collaborative learning activities in a practical and highly clinically relevant manner. We received positive feedback from residents about their experience with our curriculum, and our data demonstrate its effectiveness in improving knowledge of and comfort with contraceptive care immediately following participation in the curriculum. Though our assessment of knowledge and comfort/confidence retention was limited by the anonymity and sample size of the delayed post-survey data, independent t-test analysis of these data suggests persistent knowledge and comfort gains in many domains 4–6 months following participation in the curriculum.

While there was a signal toward retention of knowledge and comfort, there was also some attrition of both between the immediate post-survey and delayed post-survey. This may indicate the need for implementation of a spaced learning strategy, either through clinical exposure as suggested above or through regular, interval review of material in a more didactic style.34

The integration of smartphone applications in our curriculum may also have a unique benefit of scaffolding this spaced learning. As we mention above, residents regularly use smartphone applications in their clinical practice, and our provision of applications relating to contraception may facilitate ongoing content review and clinical application as residents encounter opportunities for contraceptive care in their clinical practice.13,35

Our study has several limitations. Though our pre- and immediate post-survey data sample sizes represent 95 percent participation by eligible individuals (ie interns), our sample size is small. We could not conduct a paired analysis of delayed post-survey results due to participants’ inability to remember their unique identifiers. Further, since only 50 percent of curriculum participants completed delayed post-surveys, there may have been some degree of participant bias. Additionally, we did not include a control group in our study, as our pool of eligible individuals was already quite small. As a consequence, we cannot compare the results achieved with our curriculum and methods to results achieved by alternative approaches. Despite these limitations, we are able to pick up on statistically significant changes in knowledge and comfort in all domains on pre- and immediate post-surveys, and in many domains in delayed post-survey data. Finally, our survey tools assessed only self-reported comfort, without objective clinical assessment of residents’ practices around contraceptive counseling and prescribing.

Conclusion

We present a practical and translatable model for incorporating active, collaborative learning activities in the graduate medical education context without the need for advance preparation. We demonstrate our model’s success in improving knowledge of and comfort with contraceptive care. Our curriculum approach is readily translatable to other clinical topics. In ongoing research, we will develop clinically relevant strategies for evaluating trainees’ skills and patient outcomes.