A Systematic Review of Approaches for Continuous Quality Improvement Capacity-Building.

CONTEXT: Continuous quality improvement (CQI) has become prominent in public health settings; yet, little consolidated guidance exists for building CQI capacity of community-based organizations.
OBJECTIVE: To synthesize relevant literature to identify guiding principles and core components critical to building the capacity of organizations to adopt and use CQI.
DESIGN: We employed a systematic review approach to assess guiding principles and core components for CQI capacity-building as outlined in the literature. ELIGIBILITY CRITERIA: Studies meeting the following criteria were eligible for review: (1) empirical, peer-reviewed journal article, evaluation study, review, or systematic review; (2) published in 2010 or later; and (3) capacity-building activities were described in enough detail to be replicable. Studies not including human subjects, published in a language other than English, or for which full text was not available were excluded. STUDY SELECTION: The initial return of records included 6557 articles, of which 1455 were duplicates. The research team single-screened titles and abstracts of 5102 studies, resulting in the exclusion of 4842 studies. Two hundred sixty-two studies were double-screened during full-text review, yielding a final sample of 61 studies from which data were extracted. MAIN OUTCOME MEASURES: Outcome measures of interest were operationalized descriptions of guiding principles and core components of the CQI capacity-building approach.
RESULTS: Results yielded articles from medical education, health care, and public health settings. Findings included guiding principles and core components of CQI capacity-building identified in current practice, as well as infrastructural and contextual elements needed to build CQI capacity.
CONCLUSIONS: This consolidation of guiding principles and core components for CQI capacity-building is valuable for public health and related workforces. Despite the uneven distribution of articles from health care, medical education, and public health settings, our findings can be used to guide public health organizations in building CQI capacity in a well-informed, systematic manner.

Continuous quality improvement (CQI) is defined as “a culture of sustained improvement targeting the elimination of waste in all systems and processes of an organization.”1(p761) CQI has its roots in manufacturing: Mass production of automobiles was systematized by Henry Ford in 1901 and improved upon by Japan's Toyota Production System, or Lean manufacturing, in the 1930s. Other CQI methodologies, such as Total Quality Management, Six Sigma, and the Model for Improvement (MFI), have since emerged, as well as hybrid models such as Lean Six Sigma. The development of more sophisticated methodologies has resulted in a more comprehensive CQI toolbox that can be used across a wide variety of systems beyond manufacturing.1 Uptake of CQI in health care has been considerable with the seminal publication of the Institute of Medicine's Crossing the Quality Chasm. Medical errors and the resulting quality gap became a call to action for medical professionals to implement change, and CQI was a mechanism for doing so.2 CQI continues to be a useful tool for bridging research evidence, direct service with patients, and population health outcomes.3 This consideration of population health outcomes has extended CQI's practicality beyond clinical encounters and into the realm of public health.

In the public health context, Riley et al4 define quality improvement (QI) as “the use of a deliberate and defined improvement process ... which is focused on activities that are responsive to community needs and improving population health.”(p6) This definition underscores both the importance of a coordinated and methodologically sound approach and consideration of the community context in which QI efforts take place.4 CQI involves an organizational commitment to systems change using QI as previously defined. This organizational commitment is critical to ensuring the spread and sustainability of QI.4,5

Commitment to CQI's potential for improving both clinical and population outcomes has led to the institutionalization of QI in medical education.6,7 In many medical disciplines, QI is required during residency for accreditation. For example, the Accreditation Council for Graduate Medical Education (ACGME) and the American Board of Neurological Surgeons (ABNS) require QI in neurosurgery residency.7 Integrating QI training in undergraduate medical education is also gaining traction.8

The success of CQI in health care has been one of the key drivers of its adoption in public health. With a focus on CQI in its standards, the Public Health Accreditation Board's (PHAB's) voluntary accreditation program has been a positive force in spurring the adoption of CQI in public health departments as the field endeavors to do more with less, given tightened budgets. Unlike medical education programs, public health programs have not yet included QI in their undergraduate and graduate education programs. One of the key challenges for the future of public health is the need to build QI capacity within the public health workforce so that these principles can be applied in community-based programs.5

Another potential advantage of applying CQI in public health contexts relates to community efforts to implement evidence-based practices (EBPs). Replication and scaling of EBPs can be a challenge for many communities due to a lack of resources, infrastructure (eg, staffing) necessary for effective implementation, and/or contextual fit of the EBP with the population's needs and assets.9–12 In such cases, improving and strengthening the community's capacity to deliver existing, culturally relevant public health programs may be a more pragmatic strategy to improve equitable outcomes for the population than attempting to invest heavily in EBPs.4,13 One mechanism for making such improvements in existing public health programming is through building the capacity of local organizations to apply CQI principles to their existing programs.4,14 This hyper-local strategy not only builds the capacity of leaders, managers, and practitioners to use CQI but also provides a lever for improving the quality of programs for which the community has already established funding, infrastructure, and buy-in.

Although CQI has become increasingly popular across disciplines, there remain valuable opportunities to compile what we know about how to effectively build an agency's or community's capacity to use CQI and what is needed to position an agency or community for the successful implementation and sustained use of CQI methods to improve its public health programming. To our knowledge, there are no available compilations or systematic reviews of best practices associated with CQI capacity-building; yet, published work in this area appears ripe for synthesis and aggregation.

We applied an implementation science lens, specifically the Active Implementation Formula for Success, to this review.15 The formula attends to 3 domains that enable intended outcomes: a well-defined and well-operationalized intervention (“what works”); necessary infrastructure to support implementation of the intervention across its stages of development (“how it works”); and an enabling context of data use and communication among implementation teams (“where and with whom it works”). Within each included study, we focused on implicitly or explicitly stated elements of a well-defined intervention: (a) guiding principles, (b) core components, and (c) associated activities related to CQI capacity-building efforts (elements of “what works” in the formula).16 We also attended to aspects of implementation infrastructure (elements of “how it works” in the formula)17 and enabling contexts (elements of “where and with whom it works” in the formula)18,19 for CQI capacity-building efforts. We also noted whether CQI capacity-building efforts in the literature were subjected to formal evaluation, in the event readers would like to review any particular study's evaluation approach and findings.

Methods

Literature search

Our systematic review procedures adhered to best practices as outlined by Cooper20 and Littell and colleagues.21 We also incorporated A Measurement Tool to Assess Systematic Reviews (AMSTAR) and Preferred Reporting Items for Systematic Reviews and Meta Analyses (PRISMA) guidelines—2 tools designed to optimize the conduct and reporting of systematic reviews.22–24 Our search employed the following 7 electronic databases: Cumulative Index of Nursing and Allied Health Literature (CINAHL), PsycINFO, PubMed, Social Services Abstracts, Social Work Abstracts, Sociological Abstracts, and Web of Science. Because of its “changing content, unknown updating practices and poor reliability,”(p8) we elected not to conduct a search on Google Scholar.25 The final search was conducted in June 2019.

Search terms

To select our search terms, we consulted a university social science reference librarian with expertise in conducting systematic reviews. Our final string of search terms was (“quality improvement”) AND (capacity OR train* OR technical assistance OR coach*). Note that an asterisk indicates the search string captures words with alternative endings or forms (eg, train* could flag terms such as train, training, trainer, and trained).

Inclusion and exclusion criteria

Studies were considered eligible for review if they met the following a priori inclusion criteria: (1) empirical, peer-reviewed journal article, evaluation study, review, or systematic review; (2) published in 2010 or later; and (3) capacity-building activities were described in enough detail to be replicable. Studies that did not include human subjects, were published in a language other than English, or for which full text was not available were excluded from review.

Study identification, screening, and selection

Figure 1 displays a PRISMA diagram for the systematic review process. The initial return of records included 6557 articles, of which 1455 were duplicates. The research team single-screened the titles and abstracts of 5102 studies, resulting in the exclusion of 4842 studies and retention of 262 studies for full-text review, which were double-screened. Full-text review resulted in a final sample of 61 studies meeting inclusion criteria from which data were extracted.

FIGURE 1

PRISMA Diagram of Systematic Review Process: Study Identification, Screening, and Selection

Abbreviation: QI, quality improvement.

Data extraction

Coding sheets were developed to capture information pertaining to guiding principles, core components, and other characteristics (ie, study authors, study year, field, CQI framework, whether capacity-building efforts were evaluated) across the included studies. Consistent with a hybrid approach to inductive and deductive coding and theme development,26,27 the review team used emerging guiding principles and core components from community-based CQI work as an initial deductive coding frame for data extraction, while also inductively refining or adding emergent guiding principles, core components, and/or activities (refer to the Table for brief operational definitions of the final iteration of guiding principles and core components). Data related to infrastructure and enabling context were included to inform implementation of the guiding principles and core components and were inductively coded. All coding was conducted by the research team, with 2 coders assigned to each study. During the coding process, the research team met weekly with the first author to review and discuss coding procedures. Because 2 coders were assigned to each article for data extraction, we estimated rater agreement percentages with respect to the coding of guiding principles and core components. Rater agreement was 80% for guiding principles and 84% for core components, resulting in 82% rater agreement overall. To account for agreement by chance, we also estimated Cohen's kappa (). Our estimated was 0.62, which indicates moderate-to-substantial agreement.29 Any discrepancies between coders were discussed by the research team until consensus was reached.

Results

Study characteristics

We summarized several key study characteristics, including the last name of first authors, study year, the general field that describes the context of the study, CQI frameworks selected for capacity-building efforts, and whether CQI capacity-building efforts were evaluated (see Table, Supplemental Digital Content 1, available at: http://links.lww.com/JPHMP/A839, which summarizes key characteristics from extracted studies). The 61 studies from which data were extracted fell into one of 3 core fields: health care (n = 26), medical education (n = 26), and public health (n = 9).

Nineteen studies (31%) explicitly listed the MFI as the selected CQI framework. Fifteen studies (25%) referenced Plan, Do, Study, Act (PDSA) cycles, a component of MFI (Institute for Healthcare Improvement [IHI] Open School), as the guiding framework. Lean (n = 11) and Six Sigma (n = 7) were the next frequently referenced established CQI frameworks; followed by Define, Measure, Analyze, Improve, and Control (DMAIC) (n = 4), a component of Six Sigma; and Improvement Collaboratives (n = 4). The remaining studies either used another CQI framework (n = 6), a site developed CQI framework (n = 9), or did not specify a selected framework (n = 9). Taken together, MFI and associated PDSA cycles were the predominant CQI frameworks across studies.

Although it is beyond the scope of this article to detail findings related to the evaluation of CQI capacity-building efforts across studies, we note that 55 (90%) studies evaluated the efficacy of CQI capacity-building efforts. We now turn to specific findings related to guiding principles and core components, as well as themes related to specific activities, implementation infrastructure, and enabling contexts.

Guiding principles and core components

With respect to guiding principles of CQI capacity-building, rigorous, data- and improvement-driven, collaborative, impact-focused, and responsive were widely identified across studies. Specifically, 56 studies (92%) showcased rigorous, 56 studies (92%) showcased data- and improvement-driven, 54 studies (89%) showcased collaborative, 52 studies (85%) showcased impact-focused, and 41 studies (67%) showcased responsive. These guiding principles were well represented across all 3 core fields (ie, health care, medical education, and public health). See Figure 2 for more details related to the frequency of guiding principles identified by field.

FIGURE 2

Frequency of Guiding Principles and Core Components Identified Across Studies by Field Health care (n = 26); Medical education (n = 26); Public health (n = 9). This figure is available in color online (www.JPHMP.com).

In terms of core components of CQI capacity-building, each core component was identified in at least half of the reviewed studies. Specifically, 60 studies (98%) showcased facilitate shared learning; 55 studies (91%) showcased coach for data use and improvement; 45 studies (74%) showcased use data for assessment, improvement, and evaluation; 42 studies (69%) showcased communicate and support feedback loops; and 34 studies (56%) showcased cultivate a culture of CQI. Figure 2 also displays frequencies of core components identified across studies by field.

Specific activities, implementation infrastructure, and enabling contexts

The following themes emerged throughout the systematic review related to the implementation infrastructure and enabling contexts that could support CQI capacity-building efforts:

Use of didactic instruction and experiential learning on CQI: Nearly all (n = 61) studies referenced the use of didactic training, either in person or via distance learning technology, to teach the fundamentals of CQI. In nearly all cases, this didactic instruction was coupled with experiential learning, with students learning CQI terminology, concepts, frameworks, and tools “just-in-time” to apply these learnings to a CQI project. “Just-in-time” generally refers to participants receiving instruction at the very moment it is needed in the context of the CQI project. Having the necessary infrastructure (eg, training materials, expert trainers) in place to support training is critical for delivering this component of CQI capacity-building.

Use of expert faculty and coaches. Among many of the studies reviewed, faculty and coaches who were building the capacity of others were described as experts in CQI, as well as in the content area for the team's CQI project. Criteria for CQI experts included formal training and certification in a CQI method.30,31 In some cases, this expertise was divided among 2 or more coaches.32,33 In other cases, where strong expertise was not available, training and professional development opportunities were leveraged to build the capacity of faculty and coaches to train on CQI topics and tools and support teams in the development and implementation of CQI projects.34,35 Hiring, onboarding, and deploying expert faculty serve as another key infrastructure component that must be in place before delivering CQI capacity-building.

Use of an established QI methodology. As noted earlier, the majority of studies emphasized the rigorous application of an established and tested CQI framework to guide didactic instruction and experiential learning through CQI projects. Using an established QI framework was also demonstrated as critical to creating a shared language for communication with partners. This theme was observed across all 3 key fields. As trainers and coaches build the capacity of others to do CQI, the framework of an established methodology is a critical jumping-off point for both didactic training on CQI and also how coaches guide teams through their CQI projects.

Use of teams for experiential learning and CQI work. The use of teams to engage in CQI work was noted throughout the studies. For instance, Bonnes et al36 refer to the inherently “team-based nature” of QI.(p101) The importance of teaming extends to didactic instruction and coaching. Groups were often provided information on team formation and best practices and then were provided technical assistance on selecting interdisciplinary team members, forming team charters, and using effective meeting practices. Also critical was the role of the team in implementing a CQI project and then spreading and sustaining CQI within their organization. Teams are a cornerstone of creating an enabling context for implementation, providing an interdisciplinary and diverse mechanism for moving the work through all stages of CQI implementation (developing a problem statement, scoping the work, carrying out tests of change, etc).

Discussion

Our review of the literature elucidated guiding principles of CQI capacity-building: Rigorous, data- and improvement-driven, collaborative, impact-focused, and responsive were consistently identified across articles. Those implementing CQI capacity-building efforts may find it useful to reflect on which guiding principles are most relevant, given their context.

For the core components, the literature noted the critical importance of communicating to support feedback loops; facilitating shared learning; coaching for data use and improvement; and using data to assess, improve, and evaluate the capacity-building approach. While cultivating a culture of CQI appeared less consistently, this may be largely due to the context in which CQI capacity was built. Public health systems have not institutionalized CQI to the extent that health care and medical education have, so it is unsurprising that building a culture of CQI was more prominent in public health settings. As noted by Randolph and Lea,5 there is a distinction between QI and CQI in which the latter attends to institutional uptake of QI. In some articles, capacity-building was specific only to learning relevant tools and implementing a project, which would be defined per Randolph and Lea5 as QI. In other articles, the capacity-building approach extended beyond didactic learning and project implementation to include engagement of senior leaders and other agency staff to spread and sustain the benefits of the project and the team's learnings, which would be CQI.

Considerations for infrastructure and enabling context provide key insights into CQI capacity-building. For example, any entity seeking to deliver a CQI capacity-building approach should consider its ability to provide didactic instruction and experiential learning on an established, evidence-based CQI methodology by expert faculty and coaches and to support the formation of CQI teams to sustain the work. Such reflections on infrastructure and enabling context could serve as a diagnostic for those exploring CQI capacity-building: Without these elements in place or a clear plan to develop them, the CQI capacity-building approach will likely struggle to achieve its intended outcomes.

One significant limitation to this review is the uneven distribution of articles from health care, medical education, and public health. In reviewing the findings, we compared the guiding principles and core components identified in the public health literature with those from the health care and medical education fields. Among the core components, the most significant difference was public heath's greater inclusion of communicate and support feedback loops and cultivate a culture of CQI. This is likely due to the contextual nature of where the capacity-building was being done. For residents, the fact that CQI is part of their curriculum speaks to the program's uptake of CQI. In these settings, there may be less need for residents to spread or sustain CQI or to communicate about their work outside of their cohort. This may also be the case for health care settings, whose leadership have already invested in building a culture of CQI within their clinics or hospitals. Future reviews could incorporate materials from the gray literature (eg, unpublished theses, dissertations, book chapters, white papers), such as from the IHI Breakthrough Series, which might provide additional information on CQI capacity-building.

Our review of the literature indicates a need for further investigation of best practices for supporting CQI capacity-building. The findings presented here provide a framework of guiding principles and core components while providing meaningful data to improve upon and enhance the operationalization of these elements. The setting in which CQI capacity is built is critical, and we note that some of the guiding principles and core components may be more or less relevant, given the implementation context. Those wishing to build the CQI capacity of their organization may benefit from these findings as they look to develop an approach that is appropriate for their context.

Entities seeking to build the CQI capacity of others should have the infrastructure in place to provide didactic instruction and experiential learning on established, evidence-based CQI methodologies.

Building CQI capacity should be grounded in a team-based approach, as CQI teams are more likely than individuals to sustain CQI efforts.

Public health may benefit from institutionalizing CQI in undergraduate and graduate education programs as has been demonstrated in medical education.

TABLE

Brief Descriptions of Guiding Principles and Core Componentsa

	Description
Guiding principles
Collaborative	CQI staff form mutually beneficial and supportive partnerships to achieve the goals of CQI capacity-building. CQI staff learn and work together with capacity-building recipients and the community and participate equally in codeveloping strategies to improve the quality of services. There is a commitment to acknowledging and valuing feedback from diverse stakeholders, and value is placed on a team-based approach to CQI.
Data- and improvement-driven	CQI staff and capacity-building recipients consistently use data to drive decision-making and approach improvement efforts with curiosity and an inquiry-driven approach. This work requires a safe space to experiment and work through challenges, vulnerability to tolerate failure, and a commitment to innovation and change that includes minimizing internal barriers.
Impact-focused	CQI staff and capacity-building recipients create a space for, and bringing a sense of urgency to, improving the lives of service recipients. Strategies to achieve this outcome include a focus on systems thinking and change and coordinating and aligning efforts in a unified approach by which service recipients are seen as experts and define what quality looks like.
Responsive	CQI staff provided tailored, just-in-time support, training, and technical assistance. CQI staff respond appropriately and positively to the needs of capacity-building recipients and customize their response to the context. CQI staff are flexible and practical when challenges arise and agile and adaptive in their technical assistance.
Rigorous	CQI staff use a systematic and established evidence-informed QI methodology for building capacity, including formal communication processes to ensure transparency. CQI staff are experts in the methodology, and their roles are structured and standardized.
Core components
Communicate and support feedback loops	Share or exchange information, expectations, and vision for CQI capacity-building and its related activities, both verbally and in writing, with capacity-building recipients and other key stakeholders. Listen to, acknowledge, and respond to feedback from capacity-building recipients and other key stakeholders.
Facilitate shared learning	Enable a process of participatory learning, problem-solving, and support with capacity-building recipients in a context of the recognized need for improvement and supportive interpersonal relationships. Successful facilitation promotes cycles of mutual consultations between CQI staff and capacity-building recipients to ensure that different forms of knowledge and ways of knowing are integrated into CQI capacity-building.28
Coach for data use and improvement	Promote positive, self-supporting teams rooted in the local context. Develop the capacity of capacity-building recipients to collect and use data toward quality improvement. CQI staff should also support the capacity-building recipients' learning of CQI frameworks, principles, and practices in order to effectively build capacity.
Cultivate a CQI culture	Support capacity-building recipients and community members in shifting to a CQI mindset to spread and sustain CQI in their organizations and throughout the community.
Use data for assessment, improvement, and evaluation	Intentional collection, management, and analysis of qualitative and quantitative data to inform decision-making and CQI feedback loops to determine the impact of the CQI capacity-building approach.

Abbreviation: CQI, continuous quality improvement.

aCQI staff refer to those who are delivering the capacity-building approach. Capacity-building recipients refer to those who are receiving the capacity-building approach.