Using Interactive Learning Modules to Teach Value-Based Health Care to Health Professions Trainees Across the United States.

PROBLEM: Despite prominent calls to incorporate value-based health care (VBHC) into medical education, there is still a global need for robust programs to teach VBHC concepts throughout health professions training. APPROACH: In June 2017, Dell Medical School released the first collection (three modules) of a set of free interactive online learning modules, which aim to teach the basic foundations of VBHC to health professions learners at any stage of training and can be incorporated across diverse educational settings. These modules were designed by an interprofessional team based on principles of cognitive engagement for active learning. OUTCOMES: From June 2017 to September 2018, the website received 130,098 pageviews from 8,546 unique users (2,072 registered users), representing 45 states in the United States and 10 foreign countries. As of October 15, 2018, 568 (27%) of registered users completed modules 1-3. Five-hundred thirty-five of these users completed a survey (94% response rate). Nearly all (484/535; 90%) reported overall satisfaction with the curriculum, 522/535 (98%) agreed "after completing the modules, I can define value in health care," and 520/535 (97%) agreed "after completing the modules, I can provide examples of low- and high-value care." Second-year Dell Medical School students reported that they have incorporated value into their clinical clerkships (e.g., by discussing VBHC with peers [43/45; 96%]) as a result of completing the modules. NEXT STEPS: Future plans for the curriculum include the release of additional modules, more robust knowledge assessment, and an expanded learning platform that allows for further community engagement.

Problem

To improve health care delivery, clinicians need to have a firm grasp on the concepts of high-value care.[1] Thus, medical education leaders and policy experts have called for the incorporation of value-based health care (VBHC) into medical training.[2,3] Although there are a few nationally available high-value care curricula,[4] and medical schools have begun to integrate programs dedicated to value,[5] there is still a global need for robust programs to teach concepts related to VBHC throughout health professions training.[6,7] In a recent national survey, less than one-third of internal medicine clerkship directors reported having high-value care curricula, despite 91% of them agreeing that “medical schools have a responsibility to teach about [high-value care] across all phases of the curriculum.”[6] They cited a lack of generalizable curricular materials and local faculty expertise as main barriers to implementing high-value care education.

One strategy for addressing these challenges on a national scale is to introduce online educational tools that can be accessed independently, asynchronously, and incorporated across diverse educational settings. In June 2017, Dell Medical School at the University of Texas at Austin released the first set of interactive online learning modules in a new curriculum, titled Discovering Value-Based Health Care (Discovering VBHC),[8] which aims to teach the foundations of VBHC to health professions learners at any stage of training. This approach to content delivery resembles many of the components of massive open online courses (MOOCs), which have been successfully employed in continuing medical education.[9] However, unlike many current MOOC offerings, these modules were designed based on cognitive engagement theories, such as the ICAP (Interactive, Constructive, Active, and Passive) framework, for achieving active learning outcomes.[10] In this Innovation Report, we describe our experiences and early outcomes from developing and releasing the Discovering VBHC interactive learning modules, which may inform the work of others interested in similar MOOC-like models for health professions education.

Approach

Discovering VBHC is a set of free interactive online learning modules that teach the basic foundations of VBHC in a self-paced format.

Module development

Module development was supported by a grant from the Episcopal Health Foundation to Dell Medical School to create VBHC curricula that could be used by health professions learners. Because VBHC delivery requires the understanding and contributions of all members of the health care team, the modules were specifically designed to be applicable to different types of health professionals, including physicians, pharmacists, nurses, therapists, and others.

The development team (which included content experts, medical students, clinicians, web developers, video producers, and instructional designers) first agreed on a number of design principles (from cognitive engagement theories for active learning) for the modules:

Each module would actively engage learners and tie to daily work in health care, using different modes of engagement to achieve active learning[10];

The content would be accessible and applicable to health care learners at any stage of training;

The content could be flexibly implemented in independent study, classroom, workshop, or other educational settings;

Each module would include a patient narrative and a system redesign case (see below);

The delivery of the modules should include community-building activities and features to create interactive dialoguing.[10]

Module development was a multistep process. Over the course of approximately six months, the development team wrote the first three modules and reviewed the specific learning objectives and content with a multidisciplinary expert review panel, which included medical students. The team then worked with educational design experts and professional film and web development companies to create interactive content that is grounded in best practices in online learning. The team performed cognitive testing of the modules with two to five medical students per module to make improvements before release. The Undergraduate Medical Education Committee reviewed and approved the learning objectives and overall structure of the curriculum.

Module description

Each module opens with a short video vignette about an individual patient interacting with the health care system (patient narrative). The didactic content is presented through interactive multimedia formats, including videos, text, pictures, animations, quiz questions, and activities. The next part of each module is a care redesign case study about an institution or tool that has successfully been implemented (system redesign case), such as the value-driven outcome (VDO) tool at the University of Utah. Finally, users participate in an interactive activity to highlight the learning objectives of the case study, such as analyzing a mock VDO tool. The modules also provide supplementary materials to allow students to dive deeper on specific topics and to connect with national organizations. We have plans to annually review and refresh the content in each module to ensure their longevity.

Each module takes approximately 45 minutes to complete. The modules are organized into collections, with the first three modules, released in June 2017, composing the Introduction to VBHC collection. Continuing medical education credit is available on completion of each collection. We released all 10 modules (organized into four collections, including the Introduction to VBHC collection) in the Discovering VBHC curriculum by the end of 2018 (Table 1).

Table 1

Description of the Discovering Value-Based Health Care Curriculum, Dell Medical School, Released in 2017–2018

Setting and participants

Dell Medical School.

Dell Medical School at the University of Texas at Austin opened on July 1, 2016, with its first class of 50 medical students. In June 2017, we integrated the available Discovering VBHC modules into the Intersessions curriculum, a daylong, classroom-based session that takes place six times per year between clinical clerkships, which occur during the second academic year. All 50 second-year Dell Medical School students completed the Introduction to VBHC collection (modules 1–3) during the first half of the 2017–2018 academic year.

National dissemination.

In addition to incorporation into the Dell Medical School curriculum, the modules were released online on June 8, 2017,[8] in conjunction with a keynote presentation at the annual Association of American Medical Colleges (AAMC) Integrating Quality meeting in Chicago, Illinois. We also announced the release of the modules via social media platforms and through medical education–related channels (e.g., AAMCNews). In addition, members of the development team shared the modules through a number of invited webinar, poster, and oral presentations at regional and national meetings throughout the 2017–2018 academic year.

Implementation models.

There are multiple models for introducing these modules into curricula. Users can complete the modules asynchronously and independently, without the need for dedicated classroom time or faculty mentorship, and many initial users accessed the content in an independent study format. Alternatively, at Dell Medical School, we created six 90-minute sessions dedicated to VBHC that take place during Intersessions days between clinical clerkships. During these sessions, a faculty leader uses the module content to guide the students through an interactive session and discussion. The modules can also be used as preparatory assignments for in-class discussion. For example, Dell Medical School internal medicine residents completed a module on their own during their ambulatory medicine rotation and then met as a small group before afternoon clinic to discuss the content with a faculty facilitator. As another example, orthopedic residents at Dell Medical School participated in a three-hour workshop during an academic half-day, in which they completed modules on their laptops. Each module was immediately followed by a moderated 15-minute large-group discussion. We created facilitator guides to assist local faculty with both of these formats.[8]

Outcomes

National/online evaluation

During the first 15 months following the release of the first collection (June 2017–September 2018), the curriculum website received 130,098 pageviews from 8,546 unique users (2,072 registered users), representing 45 states in the United States and 10 foreign countries.

Although the open dissemination of the modules makes certain aspects of evaluation challenging, we have thus far been able to evaluate this educational program across a number of levels, including engagement, satisfaction, and self-reported knowledge, attitudes, and behaviors. As of October 15, 2018, of the 2,072 registered users, 568 completed modules 1–3 (27%), 976 completed module 1 (47%), 656 completed module 2 (32%), and 655 completed module 3 (32%). Similar to other online education programs,[9] a relatively small proportion of individuals who began the modules finished them. It is likely that a mixed online and in-person class arrangement would be most effective at maximizing engagement.

Additionally, 535 of the 568 users who completed all three modules in collection 1 took our assessment survey (94% response rate). Nearly all of these users (484/535; 90%) reported overall satisfaction with the Discovering VBHC curriculum, and 434/535 (81%) said that they were “likely to recommend [the modules] to a friend.” In addition, 489/535 (91%) agreed that “the content of the modules was aligned with the module outcomes,” 522/535 (98%) agreed that “after completing the modules, I can define value in health care,” and 520/535 (97%) agreed that “after completing the modules, I can provide examples of low- and high-value care” (Figure 1A). For 355/535 (66%), the set had been assigned as part of a required course. Approximately two-thirds of respondents said they were “moderately familiar with the concept and application of VBHC,” and one-third were “new to the concept of VBHC” before taking the course. Of the 535 users that completed the course, 429 (80%) identified their role, with the majority of these users being medical students (281/429; 66%) and residents (85/429; 20%). Despite the inherent differences between undergraduate and graduate medical education learners, there was no significant difference between their satisfaction with the modules.

Figure 1

Learner assessments for the first three modules (which make up the Introduction to Value-Based Health Care collection) of the Discovering Value-Based Health Care curriculum, released June 2017. Distribution of (Panel A) assessment survey responses from users who completed the collection online (n = 535) and (Panels B and C) impact survey responses from Dell Medical School (DMS) second-year students (n = 45).

Dell Medical School evaluation

We also separately surveyed Dell Medical School second-year students immediately after they completed the third module. Forty-five of the 50 students in the class completed our survey (90% response rate). The majority of students agreed that as a result of completing the Introduction to VBHC modules, they have incorporated value into their clerkship work in proposing treatment plans, thought about value during patient care, and been “more cognizant of patient needs” (Figure 1B). In addition, 43/45 (96%) said that as a result of the curriculum, they have discussed VBHC with their peers, and 44/45 (98%) have thought about how to make value-based changes to health care systems during their clerkships (Figure 1C).

Limitations

The open dissemination of the modules makes robust evaluation challenging. We do not yet have detailed information about how users outside of Dell Medical School accessed and used these modules. In having evaluations only from users who completed the first three modules, we may have overrepresented the degree of satisfaction. However, the majority of these users (66%) completed the set as an assigned part of a required course and, therefore, are less likely to have a positive bias. Another limitation is that the Dell Medical School residents who completed our curriculum were asked to complete the standard postmodule survey for all online users and were not separately surveyed.

Development of the interactive modules required the collaboration of an interprofessional team including content experts, medical students, clinicians, web developers, video producers, and instructional designers to create a state-of-the-art product for health professions learners. This requires a significant up-front resource investment, which, in our case, was made possible by a sizable grant.

We also did not capture in these early evaluations the nature of the clinical learning environment in which the users were working or learning. Learning about a changed model of health care delivery while users are working in a clinical environment that does not support that change is unlikely to produce the desired educational objectives given the power of the hidden curriculum. Local role models and health system practices serve to codify behaviors.[1] This curriculum merely provides foundational VBHC knowledge and should not be expected to create meaningful and long-lasting behavioral change unless paired with efforts to also change the clinical learning environment.

Next Steps

Future plans for the Discovering VBHC curriculum include development of robust knowledge assessment tools and an expanded learning platform that allows for more community engagement among users. For example, this platform could be used to host national (or local) challenges, such as a challenge for the best resident-led value improvement interventions, to ensure that these concepts are integrated into the clinical learning environment. The initial full release now includes four collections (10 modules) of content (Table 1). We also plan to expand these four collections by developing additional mini-modules that are adjunct to the modules and focus on specific health care delivery settings.

The development process and early experiences of the Discovering VBHC curriculum, an interactive educational tool, may help inform other programs seeking to disseminate content on health systems science, including VBHC, and other topics on a large scale to a diverse group of health professions learners.