Preparing Faculty to Incorporate Health Systems Science into the Clinical Learning Environment: Factors Associated with Sustained Outcomes.

This study assesses participants' perceptions of long-term impacts of the Teachers of Quality Academy, a medical school faculty development program designed to prepare faculty to both practice and teach health system science. A previously published 1-year evaluation of the first cohort of 27 participants showed improved perceived skills, with positive career and health system impacts. In this 5-year evaluation, a mixed-methods design included a questionnaire followed by semistructured interviews to assess perceived long-term impacts on participants. Quantitative and qualitative analyses were completed. Questionnaire response rate was 88% (N = 22), and 14 interviews were analyzed. Results demonstrated that participants had incorporated quality improvement concepts into their clinical work and teaching, better understood interprofessionalism, and observed continued improvements in care delivery. They felt the longitudinal training, delivered in a shared setting, created a learning community with lasting positive effects in institutional culture, supported long-term professional development, and had broader institutional impact. Advancements in clinical care, medical education, and professional and academic advancements were noted.

Introduction

Despite myriad efforts to create systematic change in health care delivery, the United States continues to have poorer health outcomes, greater inequities, and higher costs than other developed countries.[1] Reasons for this are multifactorial, and one important contributor to slow progress on this front is the lack of physician knowledge and skills needed to meaningfully improve care.[2,3] Until recently, medical education has lagged in developing and integrating these skills into the curriculum, with a major limitation being a lack of faculty who were trained in patient safety, quality improvement (QI), interprofessional practice, and population health.[4-6] Furthermore, the competing demands for increasing clinical productivity, teaching and evaluating students and residents, and generating scholarly productivity have left most faculty with insufficient time for learning how to develop effective systems that improve care.[7,8]

Few options have existed for preparing clinical faculty to practice and teach these new competencies. Recently, these previously scattered systems-related competencies were unified in a cohesive Health Systems Science (HSS) framework with the aim to develop a unified approach to its practice and education. This unified HSS framework was developed to serve as the third pillar of medical education, alongside the competencies required to master the basic and clinical sciences. Ultimately, the goal of this 3-pillar framework is to train learners in the principles and methods of providing safer, higher quality, and comprehensive systems-based care.[2]

Most available programs that provide HSS training for faculty are offered as time-limited courses outside of one’s own institutional setting. Many programs focus on either patient safety or quality improvement alone, without the broader context of HSS competencies, and few include preparation of faculty to develop curriculum or teach learners.[6,9] Faculty who participate in external courses may return to their home institution to find a lack of shared language and understanding among colleagues, as well as deference to established culture and priorities, thus limiting their ability for change agency.[3] While these programs demonstrate increases in self-perceived knowledge and skills, none have reported long-term impacts on the individual faculty participants or their institutions, thus limiting understanding of the enduring value of these types of programs.[10-12]

In response to the need to adequately prepare and support faculty to achieve the necessary knowledge and skills in HSS, the Brody School of Medicine at East Carolina University (ECU) established the Teachers of Quality Academy (TQA) to prepare faculty for the dual purpose of leading clinical transformation and engaging students to learn and practice HSS competencies.[13] While anecdotal signs of desired change were apparent even during their participation in TQA, and post-training results after 1 year have been published previously,[14] meaningful individual and perceived outcomes of TQA could only be assessed by demonstrating sustained effects of this professional development program on faculty themselves and the clinical learning environment in years to come. In this study, the authors describe the perceived impacts of TQA among the initial faculty cohort 5 years after completing the program.

Methods

Context and Intervention

The first TQA cohort of 27 faculty were enrolled in a 12-month series of six 2-day learning sessions beginning in January 2014. These sessions were coupled with experiential exercises and practical application of new knowledge and skills between sessions. Content of sessions is shown in Table 1. The design was explicitly intended to foster a learning community who were embedded in the clinical and educational environments in which they worked. Initial outcomes of the TQA have been described elsewhere.[13,14] Since 2014, 3 additional cohorts of faculty and clinical care partners (N = 116) have participated in TQA, with a long-term goal of establishing a critical mass of faculty and frontline health care providers prepared to positively impact clinical and educational outcomes.

Table 1.

Overview of Content Provided Over Six 2-day Learning Sessions of the Teachers of Quality Academy Professional Development Program and Graduate Education Course Completed by Participants.[13]

Session number and theme	Overview of content
1. Focus on improvement	Health care systems
	Forming a QI project team
	QI project management
	Clinical microsystems
	Measuring quality in clinical education
	Performance improvement and maintenance of certification
	The model for Improvement and tools for improvement
2. Measuring for quality and introduction to adult education	Measurement for quality
	Big data in health care and informatics
	Data security and IRB issues for QI work
	Small group work for project planning
3. Leading and managing change	Leadership for improvement
	Changing the conversation to influence change
	The influencer matrix
	Small group work
4. Leadership for patient safety	National movement for patient safety
	Human factors and diagnostic failures
	Just culture and second victims
	System factors
	Leadership in crisis
	Identifying gaps in QI and PS in the medical school curriculum
5. Linking interprofessional education into practice	Focus on interprofessional education and practice
	National and local perspectives on IPE
	Interprofessional education and lessons learned
	TeamSTEPPS® training
	Importance of teams in QI
	Local interprofessional practice initiatives and Examples
	Learner-centered education
6. Population health	What is population health?
	Perspectives from public health, clinically integrated networks, and the community
	Population health curriculum framework and the curriculum at the medical school
	Medical education instructional strategies for QI and patient safety
Graduate courses
• Introduction to medical education
• Instructional strategies
• Program evaluation

Abbreviations: IPE, interprofessional education; PS, patient safety; QI, quality improvement.

Design

The authors employed a mixed-methods, sequential design beginning with a participant questionnaire followed by semistructured interviews of a subset of respondents.[15] The quantitative questionnaire was conducted to reassess outcomes previously measured in the 1-year evaluation on perceived knowledge, skills, and incorporation of HSS concepts into faculty responsibilities, and to inquire about additional, long-term effects of TQA participation to inform the subsequent interviews. The qualitative portion of the study was utilized to describe and characterize participants’ experiences and long-term perceptions of the program’s impact on their professional development and the work they have done since TQA. Human subjects study approval was obtained by the East Carolina University Institutional Review Board (UMCIRB 14-000005).

Participants

Of the 27 faculty members in the first cohort of TQA graduates from 2015, 25 (92.6%) are included in this follow-up study; losses included retired and moved faculty without contact information available.

Procedure and Instruments

The quantitative questionnaire included items about incorporation of learning into daily work and perception of self-confidence in HSS topics and perceived broader program impact, career changes, continued work in clinical improvement efforts, and professional collaborations resulting from the training. Six questions utilized 4-point Likert scales for participants to rate their perceptions, confidence, and degree to which any increase in confidence was attributable to TQA. The remaining questions solicited information with “yes/no” or short answer responses. All participants were invited by email to complete the questionnaire (Qualtrics, Provo, UT) in October 2018, with 3 weekly reminders, and a final reminder after 6 weeks.

The interview guide explored in depth about postprogram perceptions of the TQA experience, whether participants implemented what they learned into practice, and the overall impact of the program on professional growth. Final selection of the topics in the questionnaire and interview was informed by pilot phone interviews discussing the proposed question topics with 3 participants, whose input influenced the final design. One of the authors (M.D.), who was not from ECU or involved with the TQA program, conducted the phone interviews. Participants were recruited sequentially in batches of 4–6, with order of invitation determined by applying a combination of purposeful and convenience sampling.[16] The first group was selected deliberately to get a balanced and varied view of those known to have positive or negative views based on previous evaluations at 1 year; thereafter for convenience the authors invited respondents based on the order of questionnaire completion. Up to 3 invitations to participate were issued in the interview. (Questionnaire and interview guide are presented in Supplemental Digital Content, Appendix A, http://links.lww.com/AJMQ/A55.)

Interviews were conducted between November 2018 and June 2019, lasted 20 to 60 minutes, and recorded for preparation of verbatim transcripts. The first three transcripts were reviewed by two authors (A.T. and S.L.) and discussed with the interviewer (M.D.) to finalize the interview guide.

Analysis

Questionnaire data were analyzed using descriptive statistics and binomial tests (R version 4.0.2, 2020-06-22) for comparing related results to 1-year postprogram measures. The authors used conventional qualitative content analysis[17] to analyze the interview data. Qualitative analyses of the interview transcripts began simultaneous to data collection. Two individuals (A.T., research assistant) analyzed 3 interviews to create the initial codebook which was reviewed and modified with input from the research team. The included codes were identified in the analysis (ie, no pre-existing frameworks were used as codes). Additional refinement to the codebook was completed after a fourth interview and the final codebook was then used to code the remaining transcripts. The authors noted that no new patterns were emerging during the analysis of the 11th interview. The authors conducted 4 subsequent interviews to ensure data saturation was reached.[18] Each individual transcript was coded by a research assistant and one investigator (A.T.), discussed, and reconciled into a final coded transcript. Data were analyzed using Dedoose (version 8.3.17).

Results

Questionnaire

Participant characteristics are shown in Table 2. Response rate for the questionnaire was 88% (n = 22 of 25). Descriptive results from the responses to the questionnaire are shown in Table 3. Overall, most respondents continued to perceive TQA participation as valuable. More than 75% reported that they had completed their TQA QI project and one-third had gone on to engage in additional QI activities. Additional impacts attributed to TQA included additional or new job opportunities, perceived positive culture changes within the institution and perceived value-added to the health care system when applying new knowledge and skills in practice. Half or more of respondents (50%–78%) reported incorporation of HSS topics into clinical responsibilities and scholarly work to at least a moderate degree. Nearly all (86%–96%) incorporated HSS topics into teaching, with one exception in the area of population health (59%). All participants formally presented their work at a university sponsored symposium and over half went on to present or publish external scholarly products from their work in TQA.

Table 2.

Comparison of Baseline Characteristics of the Participants in the Teachers of Quality Academy Program that Completed the Questionnaire and that Participated in the Interviews.

Characteristic	Survey respondents	Interview subjects
	N = 22	N = 15
Age, y, mean (SD)	45.5 (9.4)	46.6 (8.6)
Years since terminal degree, mean (SD)	14.4 (8.7)	13.7 (7.8)
Years in current department, mean (SD)	4.9 (4.4)	5.0 (4.3)
	n (%)	n (%)
Sex, female	11 (50.0)	8 (53.3)
Race
Asian	3 (13.64	3 (20.0)
Black/African American	7 (31.8)	5 (33.3)
White	13 (59.1)	7 (46.7)
Ethnicity, non-Hispanic	21 (95.5)	14 (93.3)
Rank/position type
Tenured	6 (27.2)	4 (26.7)
Tenure-track	4 18.2)	4 (26.7)
Other	12 (54.4)	7 (46.7)
College or department
Medicine	15 (68.2)	10 (66.7)
Nursing	3 (13.6)	2 (13.3)
Public health	1 (4.5)	1 (6.7)
Bioethics	1 (4.5)	1 (6.7)
Allied health informatics	2 (9.1)	1 (6.7)

Abbreviation: SD, standard deviation.

Table 3.

Summary of Participant Responses From the Questionnaire About Perceptions and Impacts 5 Years After Completion of the Teachers of Quality Academy Program.

Topic	Responses, % Yes
(unless noted, all response options Yes/No, or not applicable)	(unless otherwise noted)
Perceived value of the participating in the Teachers of Quality Academy (TQA) (rating, 4-point Likert scale)	86% Valuable or very valuable
Status of original TQA Quality Improvement project	Complete, no more projects: 41%
	Complete, on to other projects: 36%
	Ongoing: 13%
	Abandoned: 5%
New administrative or job responsibilities, and/or professional opportunities as a result of training in TQA	50% (11/22)
Other faculty in unit/division participated in TQA; If yes, has that contributed to advancing progress or accelerating changes	59% (13/22);
	77% (10/13)
For clinical unit, since participation in TQA, rating of perceived culture change (eg, awareness; prioritization) related to these topics: (rating: 4-point Likert scale)	To moderate or substantial degree
	Quality Improvement: 61%
	Patient Safety: 66%
	Population Health: 11%
	Teamwork 31%
Work completed in TQA, or subsequent work stemming from TQA project or training, added value to the larger system of care	68%
Impact on participant in any of the following:	New career path: 27%
	Relationships/collaboration: 82%
	Resilience/vitality: 45%
	Skills to do different things: 82%
	Change in approach to role: 55%
	Change, perceived meaning of work: 41%
To what degree participant incorporated things learned about each of the following topics from TQA into clinical responsibilities and/or scholarly work: (rating: 4-point Likert scale)	To moderate or substantial degree
	Clinical Scholarship
	Quality Improvement: 72%, 56%
	Patient Safety: 72%, 22%
	Population Health: 33%, 17%
	Teamwork 78%, 50%
	Leadership: 72%, 50%
Incorporation of things learned about each of the following topics from TQA into teaching setting(s)	Quality Improvement: 86%
	Patient Safety: 91%
	Population Health: 59%
	Educational Strategies: 86%
	Teamwork: 96%
	Leadership: 86%
Since TQA, submitted or published QI project or other work related to topics covered in TQA	54% (11 abstracts; 8 manuscripts)

Abbreviations: TQA, Teachers of Quality Academy.

Interviews

Fifteen participants (68%) who completed the questionnaire subsequently participated in interviews. Findings were grouped into themes of long-term perceptions of the program, implementation of learnings into practice, additional impacts, and barriers to impact.

Overall Long-Term Perceptions

Participants voiced that TQA was an invaluable experience. While noting that the program was time-intensive, participants noted it was “time well spent” (TQA10) and useful to many aspects of their career. One participant called the program a “transformational educational experience” (TQA4).

Specifically, participants cited that training in how to communicate, teach, and develop curriculum in HSS competencies was critical to their learning and leadership development. Participants reported gaining a fundamental understanding of how to leverage the system to facilitate change. As one participant noted, systems issues are present in the day-to-day work health care providers experience and TQA provided a “lens” by which to view these issues and feel “empowered” to address them. (TQA11) A couple of participants commented on how TQA has helped increase their understanding of the system and others’ roles in it. As TQA11 explains:

“previously you didn’t realize that there are system issues at play that drive outcomes and you chalk everything up to…your abilities or inabilities …All of a sudden, you view how you can be impactful [and] feel more empowered having the awareness...” (TQA11).

Participants reported that the education courses they completed provided new skills and a network of colleagues from different educational backgrounds. Interprofessional collaboration was highlighted as a particularly powerful learning experience, which broadening the participants’ professional worldview.

The experiential components of TQA required that participants develop and implement team-based improvement projects that they would not otherwise have undertaken, with a focus on the institutional impact. The professional relationships developed through this process appeared to have lasting impact. As one participant noted:

“[TQA] has helped establish relationships with some other folks that really have helped improve care outside of TQA. [Working with others who were in TQA] just helped facilitate our discussion, our relationship, and so that was really, really good. … it’s just created an environment and a culture and a sort of cohort for us to share stories, share resources, and how we can improve and just help support each other…[creating a] community for improving quality.” (TQA1)

Implementation into Practice

Health care Systems and Patient Care

All participants commented on how they have implemented what they learned from TQA into their current practice through efforts aimed at improving systems of care delivery and outcomes. Notable examples are shown in Table 4. For others, the impact was broader. One participant reported that TQA “affected…the way I look at the health systems… you can’t look at one piece in isolation from the rest, because you change one piece, everything else is affected” (TQA10). This participant now asks “what are the problems [and] how can we solve [them]? If it’s working, how can we make it work even better?” (TQA10). TQA has enabled participants to help others in this quest: “now knowing …more about quality improvement” has meant being “able to help out … with those QI projects that [the fellows] are required to do” (TQA2). Similarly, participants noted that the most useful aspect of training was being able to demonstrate a “technique for how to know that the project or improvement they’re trying to do is actually having the desired outcome” (TQA15).

Table 4.

Examples Provided by TQA Participants of Ways They Have Applied Things They Learned From TQA into Their Clinical Practice.

Streamlining the system to “substantially reduce…the number of inappropriate [procedures] being done” (TQA13).

Incorporating evidence-based practice into standardized perioperative care pathways, which resulted in decreasing length of stay and complication rates (TQA7). This new procedure was then adopted by other departments across the hospital with the goal to adopt them across the broader health system.

Using a plan-do-study-act (PDSA) approach to improve care delivery relating to “high flow nasal cannula administration for children with respiratory distress” (TQA12)

Becoming certified in TeamStepps®, and now doing “teamwork lectures at other hospitals about quality improvement and the need for teamwork” (TQA13).

Working within existing structures “to make…subtle changes to get at the goal of preventative ethics” (TQA9).

Assuming medical directorship of quality soon after completing the program, now using “quality improvement principles as a part of my formal role in quality improvement” (TQA11).

Abbreviations: TQA, Teachers of Quality Academy.

Teaching

Participants applied what they learned about HSS across the continuum of medical education and increased their impact through greater awareness of “learning styles and approaches” and the importance of learners “actively participating in the learning process.” (TQA11) TQA participants created a number of new, formal courses in HSS. For example, one participant organized an integrated interprofessional approach to teaching HSS topics by placing nursing students with medical students allowing students to know “what each other does as a profession” (TQA8). This same participant subsequently became a faculty advisor for a HSS education group. Another participant designed a course in which medical, nursing, and dental students work together so that they “can learn to see the value of each profession and learn to build teams” (TQA10).

Furthermore, most now strive to incorporate HSS topics into their teaching because they “see the value in it” (TQA2). As such, courses taught by TQA participants are “more system-based, …more focused on...the collaborative care model...making sure that our students are able to see a whole system” (TQA10). Some participants influenced the institution to “include HSS in basic training for incoming residents” (TQA2) and to “incorporate not only the didactics but some expectations of what [the residents] can actually do for…quality improvement or patient safety projects” (TQA6).

Program Impact

Professional Impact

Participation in TQA impacted the professional growth of participants, both educationally and in practice. Learning HSS competencies empowered participants to incorporate systems approaches in clinical practice and had a lasting effect on their professional thinking:

“I think they’re just so weaved into my life that it is who I have become, or I hope to aspire to be…” (TQA11)

“So I think everybody who’s gone through TQA, they naturally look at the world in those systems and we work in a different way. And you, you can’t undo that. You just think differently.” (TQA1)

Several participants pursued additional training in HSS, including courses and/or certifications in health economics, value-based care, and informatics. They also engaged in broader professional development in areas of leadership, teamwork, or communication. A number of participants have continued work in HSS in clinical leadership. For example, one had “a position created…[as] Associate Medical Director of the practice, through [which] I was apt to focus exclusively on quality improvement and population health initiatives for the entire medical group practice. And that work then spun into my promotion as…Medical Director of the practice….” (TQA3).

Perceived Impacts to the Clinical Environment

Participants cited that implementation of what they learned into their clinical teaching and practices improved care systems, workflows and outcomes. One participant described how teaching HSS competencies learned during TQA had changed the culture of their department to one focused on improvement (TQA6). Seeing these successes, they reported spread of successful changes to other departments and units, who subsequently adopted what they learned. One participant described how they became the “go-to” person in their department for HSS, with their faculty role increasing in responsibility and effort in this area. This same department enrolled additional faculty to subsequent TQA cohorts to build a critical mass of faculty with this training. Other departments followed, resulting in the sense of culture change within the institution and tangible examples of moving from previously ineffective efforts to existing problems to now having the resources to be able to effect meaningful change. In some situations, this added participation created a shared experience and stronger voice for what was learned across the department.

“I think that is the way you turn the tide is by having a critical mass at every department, at every division level that have been through this... to strengthen the voice overall, the collective voice, to be able to drive change” (TQA11).

Because of the successful interprofessional nature of the program, participants also noted that they began to observe a change within the institution to become more “interprofessional in the approach to problem solving and quality improvement” (TQA11).

Barriers to Impact

Some participants expressed frustration with their inability to implement what they learned into their professional settings. A few departments were less open to new ideas, leading participants to conclude that the department did not make use of their new skills to bring about change. As one participant noted:

“I wish that they would make better use of me and my skills, because every time I come up with something to do, it’s because I am pushing. It’s not because they are asking me. I am pushing, and sometimes I feel like I am being a bother to them” (TQA10)

Other barriers to impact were institution size and opportunity. A few participants noted that the clinical area in which they worked was small and there was insufficient infrastructure to implement what they learned. Similarly, for some, the day-to-day work demands did not allow the time or opportunity to implement what they learned.

Discussion

An important barrier to integration of HSS in medical education has been the lack of a critical mass of faculty who are prepared to teach and model HSS competencies. The TQA program was designed to address this deficiency through a comprehensive, longitudinal model of professional development that provided faculty with experiential training and empowered them to develop and implement novel curricula in HSS immediately after experiencing it themselves.[19,20] This investigation of impacts on participants after 5 years demonstrates substantial continued faculty participation in QI efforts, improved care in many clinical areas, increased faculty confidence to implement and teach HSS concepts and perceived culture change related to systems and interprofessional approaches to problems. Overall, the long-term impacts on faculty have met, and in some aspects exceeded, the original program objectives.

While several of the outcomes had been noted in the previously published 1-year evaluation,[14] these findings reveal increased breadth and degree of impacts on faculty over time and include some unanticipated, powerful effects such as an enduring cohort effect, further career development, leadership roles, scholarship, and strengthening of interprofessional relationships. While self-reported incorporation of most HSS concepts into clinical work was largely sustained over time, the questionnaire responses suggested population health principles were incorporated to a lesser degree. Thus, the program effects for all components of HSS may not have been consistent and future efforts can focus on strengthening the integration of population health components to build confidence in this area.

Core to the TQA approach was the creation of a learning community with sufficient number and diversity of participants, grounded in the context of the institution, as a necessary component to developing a common language, set of skills, and the environment needed to support sustained culture change. Additionally, the supportive nature of the program bolstered participants as subsequent challenges were encountered, which was important to sustaining efforts and empowering faculty during a period of rapid and consequential changes in health care. Programs that provide faculty professional development in patient safety and quality improvement have been increasingly offered in recent years, but most often occur in locations external to participants’ home institutions, are offered for individuals rather than for groups of faculty who work together, and may lack a significant experiential component. Separating medical education from immediate application in a clinical setting that acknowledges the need for effective modeling of what is taught fails to recognize the impact of the clinical learning environment on learners’ future behavior.[21] The AAMC recommended that academic and clinical leadership share a common commitment to quality improvement and patient safety.[22] Alignment of local faculty development programs with the efforts in the clinical enterprise to improve health outcomes provides an infrastructure for sustainability and communicates the values of the academic health system.[23] Understanding both the clinical and educational contexts of HSS is necessary for faculty to both teach these competencies and to serve as change agents that can help transform their local clinical learning environment to model and reinforce them.[21]

While addressing traditional faculty development outcomes, this work begins to incorporate and highlight the importance of professional development that occurs in the workplace, in connection with others embedded in that setting, and includes time for application of learning while in the work setting.[24,25] Higher education embraces the concept of “communities of practice” as a powerful nidus for enhancing professional learning that is integral and central to the workplace and change processes that allow for collaboration with diverse participants and other local nonparticipants.[26] Specifically in the context of HSS, this type of environment allows learners to expand their professional identity and perspective on the health care system.[27]

The limitations of this study include that it was based on a program that was offered at a single site with a limited number of participants. Three participants were lost to follow up or did not respond to invitations for interview.

Implications and Conclusion

Faculty development in HSS is essential to align education with health care transformation, to establish a focus on patient-centered outcomes, and to impact integration of these competencies in the ethos of the institution for sustained culture change. The faculty who participated in the first offering of TQA have a common set of knowledge and skills, are able to support each other in continued growth, have created meaningful change in their own clinical microsystems and, along with others who have followed in subsequent TQA cohorts, have formed a critical mass able to ignite broad change in the educational and clinical learning environment. The TQA successfully created a learning community with potential to impact the experience of learners, colleagues, team members, patients, and ultimately the health care system.

Acknowledgments

The authors wish to thank Sally Collins, Research Analyst, University of California, San Francisco for her help with analyzing transcripts.

Conflicts of Interest

Dr Lawson is a co-editor of the textbook Health Systems Science, 2nd edition (Elsevier, 2020). Drs Baxley and Higginson are contributors to the Health Systems Science textbook. All the other authors have no conflicts of interest to disclose.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article. This work was prepared with financial support from the American Medical Association as part of the Accelerating Change in Medical Education Initiative. The contents reflect the views of the authors and do not necessarily represent the views of the American Medical Association or other participants in this initiative.

Ethical approval for this study has been granted by the University and Medical Center Institutional Review Board (UMCIRB 14-000005).