Development of a health care systems curriculum.

BACKGROUND: There is currently no gold standard for delivery of systems-based practice in medical education, and it is challenging to incorporate into medical education. Health systems competence requires physicians to understand patient care within the broader health care system and is vital to improving the quality of care clinicians provide. We describe a health systems curriculum that utilizes problem-based learning across 4 years of systems-based practice medical education at a single institution.
METHODS: This case study describes the application of a problem-based learning approach to system-based practice medical education. A series of behavioral statements, called entrustable professional activities, was created to assess student health system competence. Student evaluation of course curriculum design, delivery, and assessment was provided through web-based surveys.
RESULTS: To meet competency standards for system-based practice, a health systems curriculum was developed and delivered across 4 years of medical school training. Each of the health system lectures and problem-based learning activities are described herein. The majority of first and second year medical students stated they gained working knowledge of health systems by engaging in these sessions. The majority of the 2016 graduating students (88.24%) felt that the course content, overall, prepared them for their career.
CONCLUSION: A health systems curriculum in undergraduate medical education using a problem-based learning approach is feasible. The majority of students learning health systems curriculum through this format reported being prepared to improve individual patient care and optimize the health system's value (better care and health for lower cost).

Background

Knowledge of basic science is crucial for understanding how the human body works and experience in clinical practice is essential for developing prevention or treatment strategies.1 Along with these basic and clinical sciences, health systems science represents the “third pillar” of medical practice. Considered indispensable to improving the quality and safety of patient care,2 the health systems competence requires physicians to understand patient care within the broader health care system.1 Health systems training in undergraduate medical education, also known as systems-based practice, constitutes one of the six core competencies defined by the Accreditation Council on Graduate Medical Education.3 This competency may be the most challenging to incorporate into medical education since it is hard for medical students to recognize the contributions of the systems when the current cornerstone of their education focuses on mastering knowledge of disease, diagnostic skills, and treatment at the level of physician-patient interactions.2

While there is no gold standard for delivering systems-based content, well over 50 years ago medical educators recognized that the traditional lecture approach ineffectively taught clinical skills.4 In response, a wide implementation of a problem-based learning approach improved medical students’ assimilation of basic science into clinical decision-making.5 This approach gradually expanded to other professional schools such as business, law, nursing, and engineering.6 Despite the success of a problem-based learning approach in clinical medical education and its expansion to other fields, we found no existing literature describing this method for the graduate medical education council’s system-based practice competency.

To date, the majority of published work on system-based practice is at the graduate medical education level rather than undergraduate level.7 At the undergraduate level, an audit of educational curricular content found that only 8% of content was related to systems-based practice.5 A survey of medical student knowledge of systems-based practice found no improvement from first to fourth years, causing those surveyed to express a desire for greater undergraduate systems-based practice content.8 Another study showed that an 8-hour (total) undergraduate medical education curriculum significantly increased systems based practice (SBP) knowledge.9 While institutions have formalized curriculums, broadly there appears to be a limited amount ascribed in most undergraduate programs to this topic, albeit several institutions provide notable systems-based practice experience through student-run clinics.10 Most clinic experience is voluntary,11 so utilizing that experience as a generalized driver of this type of education may be ineffective. This case describes the curriculum development of a problem-based learning approach of a single institution’s undergraduate medical education program to design, deliver, and assess a health systems curriculum to address systems-based practice. Among other educational objectives, the problem-based learning approach has been found to enable students to develop recall and application of information in the context of medical practice, improve clinical reasoning, self-directed learning skills, and increase motivation for learning.12

Methods

Case study research enables description of characteristics, patterns, structures, or processes of single representative instances.13 This explanatory case study describes the undergraduate medical education health systems curriculum design, content delivery, assessment approach, and course evaluation results. Our analytic strategy was to develop a descriptive framework to gain insight into the proposition that application of problem-based learning approach to systems-based practice curriculum is appropriate for undergraduate medical education. The problem-based learning approach is characterized as a student-centered approach in which small groups work with a tutor to solve a professionally relevant problem.4 Among other educational objectives, this approach enables students to develop recall and application of information in the context of medical practice, improve clinical reasoning, self-directed learning skills, and increase motivation for learning.12

We organized our case description using a three part descriptive framework – curriculum design, content delivery, and assessment approach – to explain how a problem-based learning approach can be applied to systems-based practice curriculum. While less rigorous than theoretically-driven analysis, a case description can serve as an alternative to more quantitatively-supported case study designs.13 The unit of analysis is the undergraduate medical education program and our examination occurred at an institution in which training is delivered in two separate geographic regions (Florida and Pennsylvania). Using an informal nominal group technique,14 the team of educators and experts gathered information and iteratively identified and reviewed important themes that led to the development of objectives, entrustable behaviors, and module content. The lead faculty, experienced in health systems education, training, and research, presented curricular ideas at faculty retreats and worked with course directors to create course content and assessment approaches.

The assessment of a student’s health system curriculum learning consists of a series of competencies that a first year resident would be entrusted to demonstrate; called entrustable professional activities.15 The “pre-entrustable” assessment statements describe behaviors that exemplify a student who did not attain competency in systems-based practice. The “post-entrustable” assessment statements describe behaviors related to health systems that residents should be able to demonstrate on their first day of graduate medical education. For example, “a first year resident would not be able to recognize opportunities for health system improvement”. In contrast, a first year resident learning through this health systems curriculum “could identify a need for health system improvement and implement a best practice to patient care”. Entrustable behaviors are described in context of the program’s five critical health systems competencies (terminal program objectives): 1) demonstrate advanced knowledge of US and international health systems, policy, and finance; 2) demonstrate the ability to strategize, practice, and advocate for quality improvement in patient care and health care systems; 3) integrate knowledge of health care systems into individual patient care; 4) demonstrate the ability to analyze a health care environment/system and recommend changes to improve patient outcomes; and 5) integrate knowledge of health care systems into individual patient care. Additionally, end-of-year student course evaluations are conducted through surveys and data are reported in simple proportions.

Results

Curriculum design

The program originators developed a comprehensive set of competencies (objectives), of which, systems-based practice was one component. After establishing the desired student outcomes, the health systems curriculum design modeled the three components of the problem-based learning approach: problem selection, coach facilitation, and thorough debriefing. Effective use of this approach required the selection of an authentic and professionally relevant problem that embraced complexity, mirrored medical practice,6 emphasized autonomous exploration,12 and engaged collaborative discussion.16 Each activity necessitated coach facilitation. Most faculty training to facilitate the coaches in their effectiveness to support the learning was done in the 30 minutes before the class session and was also supplemented by quarterly faculty development meetings.

In our curriculum design, the physician coaches supported questioning, critical thinking, and problem solving, but did not provide information related to the problem. These coaches – not experts in health systems problems, but rather clinical professionals with real-world experience with systems-based practice – actively engaged the student teams in the evaluation, analysis, and resolution of the problem.

Following problem resolutions, student teams delivered a thorough debriefing to the entire class. Each team defended their actions as appropriate to the problem discussed in their small groups. According to Barrows,4 “the purpose of the post-experience debriefing process was to consolidate the learning and ensure that the experience has been reflected upon”.

Content delivery

The systems-based practice content, following a problem-based learning approach, was delivered to fourth year medical students in 22 sessions that usually lasted from 3 to 4 hours each (Table 1). The first and second year modules were delivered weekly in five session blocks, whereas the third and fourth year sessions were integrated longitudinally throughout the 2 years of training (Supplementary material).

Table 1

Course session and hours

Year	Topic	Hours
1	Access, quality, cost, and health care policy	3
	Health care costs	3
	Access to health care	3
	Health care quality	3
	Best practices of health systems Organizations	3
2	Current case in health care systems (Zika activity)	3
	Role of informatics health care	3
	Health systems performance	3
	Current case in health care access (pediatric trauma center activity)	3
	Reducing physician burnout	3
3	Risk management and disclosure	4
	Health care workforce	3
	Health systems engineering	3
	Health insurance	3
	LEAN quality improvement methodology	4
	Quality improvement	6
	Safety at the bedside	4
4	Health care advocacy	7
	Public health	3
	Health systems finance	3
	Quality improvement	6
	Teamwork at the bedside	8

Assessment approach

To assess student learning of the health systems curriculum, a series of pre-entrustable and post-entrustable behaviors was developed (Table 2). In the third year, students undergo two assessments and at the end of the fourth year, students undergo a final assessment. Both assessments use multiple choice questions, essay responses, and simulation exercises to demonstrate students’ absorption of the materials presented.

Table 2

Assessment through health systems entrustable professional activities (EPA)

Description of the activity	Day 1: residents should be able to integrate knowledge of health care systems into individual patient care, including identifying health care system performance issues and demonstrating the ability to strategize, practice, and advocate for value (quality and cost of health care) in patient care.
	Functions
	1) Identify health care system quality issues in the US at an individual patient level.
	2) Apply a health care cost control mechanism related to a specific health care management decision.
	3) Demonstrate effective oral and written communications to advocate for health system improvement.
	4) Apply the decision making (e.g., health care provider orders) in line with value-based (quality and cost) health care provider compensation.
	5) Identify barriers to health care access and suggest possible solutions.
	6) Distinguish value added from non-value added health care activities using tools such as registries, dashboards, and other metric tracking.
	7) Identify and describe hazards and inefficiencies processes at point of care and utilize tools like root cause analysis to identify areas of improvement.
	8) Complete an adverse advent report and effectively complete a medical error disclosure to patients and their family.
	9) Effectively advocate for improved patient outcomes considering multiple stakeholder positions (such as departmental, local, state, and federal government) in health care.
Most relevant domains of competence - SELECT terminal program objectives (TPO) that relate to this EPA	Relevant domains of competence include:
	S-MK1
	S-SBP2
	S SBP3
	S-PBL2

Note: EPA 1: the resident should be able to integrate understanding of health systems into individual patient care.

Abbreviation: SELECT, Scholarly Excellence, Leadership Experiences, and Collaborative Training; S-MK1, SELECT Medical Knowledge Competency #1; S-SBP2, SELECT Systems-Based Practice Competency #2; S-SBP3, SELECT Systems-Based Practice Competency #3; S-PBL2, SELECT Practice-based Learning and Improvement.

Course evaluation results

At end of the 2016 academic year, the majority of first and second year students stated they gained working knowledge of health systems and analytical skills by engaging in the sessions on their evaluations. At the end of the 2017 academic year, most students agreed that the course was well-organized and administered (89.5%), that the workload allowed an appropriate balance between academic and personal life (87.7%), and that the assigned readings and online materials effectively enhanced their learning (82%). Overall, the students felt prepared to improve the value of patient care. The majority of the 45 2016 graduating students (88.24%) and the majority of the 56 2017 graduating students (96%) agreed that the content of the course prepared them to embark upon their careers with enhanced health system knowledge and awareness to enable successful medical practice.

Discussion

The landmark publications from the Institute of Medicine17 focused national attention on health system performance and underscored the need for health system improvement. In particular, the US health care system is the most costly in the world, accounting for 17% of the gross domestic product with estimates that percentage will grow to nearly 20% by 2020.18 Despite this economical dedication to health care, the system remains uneven and fragmented, patient harm is quite common, care is often uncoordinated, and many mishaps occur.19 Physicians are essential contributors to this effort to improve quality and decrease cost.20 Research on educating physicians to deliver high-value, cost-conscious care suggests that learning by students is promoted by combining specific knowledge transmission, reflective practice, and a supportive environment.21 We present a 4-year undergraduate curriculum in health systems to be delivered in problem-based learning approach modules. Other health systems curriculum literature have been described. For instance, Halbach and Sullivan22 describe teaching medical students about medical errors and patient safety; Boonyasai et al23 performed a systematic review of the effectiveness of published quality improvement curriculum. However, we believe this is the first description of a problem-based learning approach application to health system curriculum in undergraduate medical education.

Our health systems curriculum was delivered as part of an overarching medical school program, entitled, SELECT (Scholarly Excellence, Leadership Experiences, and Collaborative Training) with information initiated in 2011 between the Morsani College of Medicine, University of South Florida (Tampa, FL) and the Lehigh Valley Health Network (Allentown, PA). This program was designed to equip medical students with knowledge, skills, and behaviors to transform the nature of health care.24 Part of this program includes fostering a 4-year coaching relationship between preceptors from each campus and students. There is little doubt that the investment in developing the relationship between the preceptors and students in this fashion, contributed to the success of the training. The program places emphasis on leadership, health systems, and values-based patient-centered care. The health systems training is approximately one third of the content delivered in the enhanced undergraduate medical education curriculum, with health systems content reinforced in other program modules. The students receive a graduate school certificate for the content in addition to their MD degree.

It is unclear whether the health systems curriculum would have the same effectiveness if delivered outside of this model. This case report serves primarily to describe the development of a curriculum with implementation using a unique problem-based learning format for this content; it is limited by the lack of scientific rigor in methodology. While pre- and post-entrustable professional activity assessments have not been conducted on a student cohort at this time, future opportunities exist for evaluating this program both in the short term, through pre-graduation activity assessment, and long term, through continuing demonstration of these professional activities.

Conclusion

A health systems curriculum in undergraduate medical education using a problem-based learning approach is feasible. The majority of students learning health systems curriculum through this format reported being prepared to improve individual patient care and optimize the health system’s value (better care and health for lower cost).

Supplementary material

Detailed description of the problem-based learning approach to systems-based practice

MS1 sessions

Health care cost, quality, access, and health care policy

In this introductory session, students examine how policy influences health care access, quality, and costs. MS groups analyze evidence to identify successes and failures of an assigned health policy (e.g., Affordable Care Act). This problem requires that the team formulate health policy alternatives and predict positive and negative impacts of their solutions on the value of the US health care system.

Health care costs

Following a brief introduction on the trends, drivers, and issues related to health care costs in the US, student teams apply cost control concepts to medical care problems. Teams are given one of many possible scenarios, including using comparative effectiveness research evidence to clinical care, engaging in pay-for-performance schemes, or advocating for (or against) increased patient responsibility and accountability for health care costs.

Access to health care

Students begin by recognizing common health care access barriers, such as financial, geographical, clinical, and cultural. To develop critical thinking and political advocacy skills, students are presented controversial (and hypothetical) health care access policies, such as creating single payer health insurance, mandating minimum Medicaid participation for physicians, and diverting heavy users of emergency department care. To provoke discussion, student teams defend opposing policy interventions as they relate to patients, physicians, organizations, and the health care system.

Health care quality

Students are introduced to relevant health care quality concepts, including quality measures and improvement approaches. Within teams, students address problems such as pediatric asthma exacerbation, patient satisfaction, inpatient readmission, health care-associated infections, and patient falls. Teams are encouraged to recommend SBP-related solutions, such as continuous quality improvement, technology enhancements, health care team redesign, provider incentive programs, and health system reorganization.

Best practices in health care delivery

As a review of the first year, MS2s examine best practices in health care delivery organizations. Specifically, MS are presented with various access-, quality-, and cost-related problems for which highly-regarded health systems have developed solutions. Using examples of successful health system innovations, student groups create solutions to health system problems, including fragmentation of care, unnecessary care, avoidable harm to patients, chronic disease, and emergency department crowding.

MS2 sessions

Current case in health care

To start year two, MS are presented with contemporary public health emergency problems, such as Zika, Ebola, hurricanes, or terrorism. The objectives of the PBL activity are to analyze the roles of government in prevention and control of public health emergencies and to construct evidence-based health system solutions to public health emergencies. Specific health systems problems include: develop and justify funding priorities, institute travel ban for affected area, evaluate patient confidentiality laws, and construct physician staff surge capacity plan.

Role of informatics in health care

The objectives of this PBL activity are to develop potential solutions to health care informatics challenges, particularly the varying types and classes of data found in databases. Along with their coaches, student groups develop solutions to health informatics challenges using PBL scenarios, such as electronic health records, consumer health informatics, information exchange, decision support systems, physician’s involvement, and public health informatics.

Health systems performance

In a deeper examination of health care quality, the objectives of this PBL activity are to apply quality measurement concepts to the development of a health care system performance score card. Student teams discuss the relative importance of performance concepts such as equity, efficiency, quality, and access. Based on each group’s choices of quality measures, the final performance rankings are calculated using comparative data from World Health Organization Countries Health Profile, OECD Health Care Quality Indicators, and the Commonwealth Fund.

Cases in access to health care

Student groups develop access solutions to controversial problems developed after recent events in the news media. Examples of news events related to health care access include trauma medical system policies, restricting access to opioid prescriptions, and end-of-life medical decision-making. Each student group represents different stakeholders, such as paramedics, physicians, hospital administrators, local politicians, directors of the public health departments, and community members (patients, families, advocates, or legal representatives). The objective of the session is to encourage the problem-solving from ethical, financial, clinical, political, and regulatory perspectives.

Impact of health system’s improvement on physician

In the final activity session of year two, students learn to evaluate the impact of external pressures of health system improvements on physicians. Example problems include physician burnout, decreased patient interaction time, scope of practice boundary issues, and litigation associated with error reporting. Student teams develop responses to the negative consequences of health system changes on their professional and personal lives.

MS3 sessions

Risk management and disclosure

Facilitated by network risk management experts, such as the hospital attorney, risk manager, and patient safety director, MS learn how to review and report safety events and issues that must be corrected or reported to authorities. Consent for treatment and authorized representatives are studied. The PBL activity involves examples of risk management cases that require patient disclosures. MS also practice a patient disclosure in peer pair exercises.

Health care workforce

A panel of multidisciplinary experts, such as the president of a large multispecialty physician group, a hospital vice-president of patient care services, a leader from the network advanced practice clinicians, convene to describe the supply and demand for physicians, physician assistants, nurse practitioners, nurses, and other health professionals. Experts and MS discuss the ramifications of health care policy on health care/medical education, patient access, cost, and demographic trends of the health care workforce. MS solve health care workforce problems, such as the impact of a new electronic medical record system and the effectiveness of workforce diversity interventions.

Health systems engineering

Health systems engineers describe the core systems engineering applications used in health care. Student groups demonstrate the application of qualitative and quantitative engineering tools to solve quality and safety problems, including the assessment of process-related failures and the development of potential system fixes.

Health insurance

A diverse panel of experts, including patients with chronic diseases, a hospital case manager, and a medical director of a large insurance company represent the different perspectives in health insurance. MS discuss the impact of cost sharing and the incentives structure of the shared risk model on patient decisions. The PBL table exercises include assessing a family’s premiums for a variety of plan enrollment options.

LEAN

Facilitated by several LEAN coaches, students apply the problem-solving methodology to a team-based pharmacy simulation. After an initial round of filling a prescription, students debrief and strategize ways to improve the process. A second round allows them to successfully improve the prescription filling process with markedly less waste (more completed prescriptions, less time per prescription, and higher quality outcomes). This activity illustrates that eliminating waste in a health service process can increase patient value.

Quality improvement

Building on the health care quality introduction from their first year, students are exposed to both inpatient and outpatient quality improvement initiatives. These sessions are facilitated by quality and patient safety experts such as the senior vice-president of quality and patient safety, the department of medicine vice-chair of quality improvement, and the network medical director for quality and patient safety. Several PBL cases illustrate quality improvement strategies that can improve performance metrics, assess population health, align behavior of physicians, and link these factors to patient outcomes and safety.

Safety at the bedside

Facilitated by the patient safety officer and pharmacy clinical specialists (pharmacists), MS learn about managing the hazardous environment at the bedside. They are exposed to the basic elements of medication safety and the vulnerabilities that affect safety in health care. Using a tool developed to assess the contributing factors that lead to latent errors, the students rotate through three simulated bedside experiences that illustrate deficiencies in a variety of areas (impaired physician, poor team environment, organizational management issues, staffing problems, task-specific errors, etc.).

MS4 sessions

Health care advocacy

Facilitated by the network’s vice president of government and legislative affairs, MS learn about the roles of local state and federal governments in health care, non-governmental advocacy organizations such as professional organizations, and how to appropriately engage with legislators. The PBL activity includes an exercise centered around proposed marijuana legislation and ends with students each participating in PBL activities, such as debating their positions to a mock panel of legislators.

Public health

Facilitated by the director of the local health department, students learn about implementing and evaluating public health initiatives and how to persuasively communicate public health ideas to target populations. Small group problem exercises teach MS how to access and retrieve Centers for Disease Control information germane to patient care. The PBL exercise requires student groups to solve current local public health problems, such as the decreasing vaccination rates in children, the increasing frequency of falls in the elderly, rising narcotic addiction and overdose rates, and the increasing smoking rates in pregnant teens.

Health systems finance

Facilitated by network financial leaders (Vice President of Revenue Management, Vice President of Finance and Controller, Director of Revenue Integrity), students are exposed to health care financial metrics and how health systems make decisions based on budgets and revenue. The PBL experience uses the small group to work through issues of liquidity, debt ratio, leverage, impact of managed care contract rates, and optimal revenue management practices.

Quality improvement approaches

MS complete a longitudinal series of Institute for Health care Improvement modules on patient safety, root cause analysis, and the culture of safety.

Teamwork at the bedside

Student groups review the Team Strategies and Tools to Enhance Performance and Patient Safety (TeamSTEPPS) approach to teamwork at the bedside. They practice teamwork in the simulation lab in three different cases that illustrate complex approaches, the seizing patient, respiratory distress, and four patient multiple casualty scenarios. MS rotate opportunities as participants and as observers in a simulation that assesses the team structure, leadership, situation monitoring, and communication functions.

Other MS3 and MS4 curriculum-enhancing opportunities

Skills developed from these sessions were used by MS3 and 4 as a basis for applied learning in quality improvement, and by many as the foundation for an individual capstone experience required for the graduate level certificate of the program.