Quality improvement in pediatric intensive care: A systematic review of the literature.

IMPORTANCE: Measuring and improving performance is an essential component of any high-risk industry, including intensive care medicine. We undertook this systematic review to describe the current state of quality improvement efforts in pediatric intensive care medicine.
OBJECTIVE: To evaluate the quality and rigor of all published literature on quality improvement efforts in the pediatric intensive care unit in the current era.
METHODS: We conducted a literature search on MEDLINE, Embase, and Cochrane for studies that met two broad inclusion criteria: 1) the terms "pediatric critical care" and "quality improvement" and 2) they were completed in the past ten years. In the initial search, we also included academic and professional societies or organizations devoted to providing resources on quality improvement in intensive care medicine. We excluded studies that examined quality improvement processes exclusively for neonatal or adult patients receiving intensive care.
RESULTS: Forty-nine of 332 identified articles were selected for final review by two reviewers who independently rated the quality of the methodology and rigor of the evidence reported for each study. Of these, 23 studies targeted structural issues, 14 studies targeted process issues, and 12 targeted an outcome as the focus of the intensive care quality improvement effort.
INTERPRETATION: Our review of the published literature on quality improvement efforts in the pediatric intensive care unit in the current era found that 85% of studies were limited in methodology or analysis. Fifteen high-quality studies are reported here and serve as helpful examples of rigorous research methodology in this domain going forward.

INTRODUCTION

Measuring and improving performance has likely been a high priority for physicians for many centuries, as revealed in the accompanying quote from Confucius. The modern era of quality improvement in health care can be legitimately traced to many individuals and places across the globe, but many frequently cite the foundational work of Avedis Donabedian, a professor of medical care organization at the University of Michigan School of Public Health. In his 1966 landmark article, “Evaluating the quality of medical care”, cited over 6,000 times to date, Donabedian proposed using the triad of structure, process, and outcome to evaluate the quality of health care. As Donabedian defined them, “structure” refers to the settings, qualifications of providers, and administrative systems through which care takes place; “process” refers to the components of care delivered; and “outcome” refers to recovery, restoration of function, and survival.1 This triad framework for measuring and improving quality of care remains the conceptual foundation for quality assessment across all areas of health care to the present day.2

In the intervening 50 years, quality improvement has emerged as a discrete and vital domain across all aspects of health care, based on rigorous quantitative and qualitative methodology, leading to improvements in the provision of safe, effective, and efficient medical care. The impact of quality improvement efforts has been especially notable in intensive care medicine, as critically ill patients have both the smallest margin of reserve to tolerate lapses in optimal care, and yet the greatest ability to benefit in any marginal improvement in the quality of the care they receive. Critical care medicine is also fundamentally multi‐professional and multi‐dimensional, where structure and process variables affect patient outcomes.3

The objective of this study is to review the current state of quality improvement efforts in pediatric intensive care medicine systematically by evaluating the quality and rigor of the published literature on quality improvement efforts for critically ill pediatric patients in the current era and to assess resources available on the topic of quality improvement in this domain.

METHODS

With the assistance of a trained medical librarian, we conducted a literature search on MEDLINE, as well as Embase and the Cochrane Database of Systematic Reviews. Studies were eligible for inclusion if they satisfied the following two criteria: 1) they contained the terms “pediatric critical care” and “quality improvement” and 2) they were completed in the past ten years. We excluded studies that examined quality improvement processes exclusively for neonatal or adult patients receiving intensive care. According to these search criteria, 332 articles were included in our initial review.

For the first round of assessment, two independent reviewers rated the quality of the methodology and rigor of the evidence reported for each of the 332 studies in the initial review based on a modified adaptation of the Grades of Recommendation, Assessment, Development, and Evaluation (GRADE) for assigning the grade of evidence.4 We also categorized them into one of the following categories of types of health care quality measures: structural, process, or outcome measures. We used the definitions of these categories as outlined by the Agency for Healthcare Research and Quality (AHRQ) listed in Table 1. Subsequently, using the Delphi method, the two independent reviewers systematically and interactively came to a consensus of articles to be included in the final review. From 332 articles, as a result of the Delphi method, the reviewers included 49 articles in the final review. Additionally, as part of this Delphi process and additional discussion, the pair of reviewers categorized these articles into structural, process, or outcome measures according to the AHRQ definitions (Figure 1).

Table 1

Three‐part classic framework for quality improvement5

Categories	Definitions
Structural measures	Structural measures give consumers a sense of a health care provider's capacity, systems, and processes to provide high‐quality care.
Process measures	Process measures indicate what a provider does to maintain or improve health, either for healthy people or for those who are diagnosed with a health care condition. These measures typically reflect generally accepted recommendations for clinical practice
Outcome measures	Outcome measures reflect the impact of the health care service or intervention on the health status of patients.

Figure 1

Flow diagram for included quality improvement studies

RESULTS

The initial search strategy generated 332 citations; of these, 283 (85%) were excluded due to lapses in study methodology or failure to report meaningful variables of interest as based on an adaptation of the GRADE criteria for assessing the quality of a study. Of the 49 (15%) studies included in this analysis, 23 studies targeted structural issues, 14 studies targeted process issues, and 12 targeted an outcome as the focus of the intensive care quality improvement effort. Fifteen studies (5 for each domain) were independently selected in the final round based on the rigor of their quality improvement methodology and the potential of the findings to change practice in the context of pediatric critical care.

The structure of an intensive care unit (ICU) can greatly affect safe, effective, and efficient patient care. As seen in Table 2, the study by Cifra and colleagues highlights the importance of introducing a multidisciplinary and standardized framework at Morbidity and Mortality Conferences. So too the work by Kresch and colleagues demonstrates the importance of improved patient outcomes by creating and implementing a structured handover process, while Landrigan and colleagues demonstrate an association between reducing the work hours of junior physicians in the ICU and the reduction of serious medical errors in this setting. Pasek and colleagues importantly sought to improve the experience of parents with a critically ill child and found improved outcomes after implementing a program that gives parents the option to be present during a resuscitation attempt. Finally, the seminal work by Hickey and colleagues demonstrates the association of improved patient outcomes at pediatric ICUs staffed with more experienced nurses.

Table 2

Structural measures: five notable citations

Citation	Key findings
Cifra et al10	The use of a systems‐based Morbidity and Mortality Conference brings to light diagnostic errors, communication problems, and workflow issues and provides the space for stakeholders to create multidisciplinary solutions for improved follow‐through.
Hickey et al3	Results of this study demonstrate that higher levels of experience and education for critical care nurses improve patient outcomes and decrease the number of patient deaths. These results support the implementation of structural improvements in nursing education including developing and improving nursing residency experiences and mentorship programs.
Kresch et al11	Investigators analyzed the use of patient handovers in transfers from the critical care transport team to the neonatal intensive care unit and highlighted the success of implementing consistent structural improvements in the handover process.
Landrigan et al12	Investigators compared the use of a traditional schedule and an intervention schedule and found that interns made 36% more serious medical errors on a traditional work schedule than an intervention schedule. The study notes that schedule design is a critical but not exclusive factor in reducing serious medical errors in ICU settings.
Pasek et al13	This study discusses the ethical, cultural, and medical implications of providing parents with the option to be present during patient resuscitation and demonstrates the improved care for families when hospitals implement programs such as PAGER (the Parent Advocacy Group for Events of Resuscitation).

Emerging evidence also reveals that good outcomes are driven by evidence‐based processes in care. As seen in Table 3, Connelly and colleagues demonstrate that the implementation of an evidence‐based program for C‐Spine clearance reduced exposure to radiation without any adverse impact on timely diagnoses of serious neurologic injury. Khan and colleagues demonstrate the reduction in nosocomial infections at a congenital heart surgical program in Pakistan following the implementation of a nurse‐driven intervention, while Lovett and colleagues report an intervention to reduce secondary brain injury following traumatic brain injury through systematic implementation of a cooling blanket on patient arrival to prevent hyperthermia. Wieczorek and colleagues report improved outcomes for select critically ill children who received an early mobilization intervention, while the seminal study by Mehta and colleagues reveals the association between improved patient outcomes and the implementation of an individualized nutritional assessment and nutrient delivery plan in the pediatric ICU (PICU).

Table 3

Process measures: five notable citations

Citation	Key findings
Connelly et al14	The implementation of a performance improvement and patient safety (PIPS) program initiative for C‐spine clearance decreased the number of C‐spine CT scans and subsequently decreased lifetime attributable risk for thyroid cancer. This quality improvement investigation supports the implementation of simple but high compliance institutional protocols to improve patient safety.
Khan et al15	Investigators compared postoperative outcomes of congenital heart disease (CHD) surgeries before and after the implementation of the International Quality Improvement Collaborative (IQIC). Results demonstrate decreases in surgical site infections, incidents of bacterial sepsis, duration of ventilation and duration of hospital and ICU stay after the implementation of the IQIC, through a nurse‐empowered team‐based approach in which the nurse acts as a liaison between hospital management and bedside clinical teams.
Lovett et al16	In order to prevent secondary brain injury following a primary insult, investigators implemented a process to reduce both incidence and duration of hyperthermia by placing a cooling blanket on the patient bed prior to arrival.
Mehta et al17	This quality improvement study demonstrates the need for more specific nutrition support therapy guidelines in the pediatric critically ill patient. Investigators emphasize the importance for an individualized approach to nutritional assessment, nutritional status, and nutrient delivery as part of the processes that aim to improve clinical outcomes.
Wieczorek et al18	Assessing each child's activity level and classifying it on a level range of 1‐3 allowed the interdisciplinary team on morning rounds to implement individualized activities tailored to each child's specific skills and needs. The increased formal involvement of both occupational and physical therapists significantly increased because of this process, thus supporting the shift to a culture of PICU mobility.

Among the m ost notable improvements in patient outcomes in the PICU over the past 25 years are those achieved because of rigorous quality improvement efforts designed to reduce hospital acquired conditions. As seen in Table 4, the landmark work of the Children's Hospitals’ solutions for patient safety collaborative (as seen in the publication by Lyren and colleagues, the work by Miller and colleagues, and the work by Frank and colleagues) demonstrates the remarkable effectiveness of a national collaborative created and dedicated to collaborative efforts across dozens of PICUs across an entire country. New national collaborations on improving the quality and outcome of care in the PICU are revealed by the work of Nishisaki and colleagues (tracheal intubation) and Betters and colleagues (early mobilization).

Table 4

Outcome measures: five notable citations

Citation	Key findings
Betters et al19	The development of an Early Mobility (EM) protocol made EM more feasible for pediatric patients receiving mechanical ventilation, therefore improving early mobility for critically ill pediatric patients.
Frank et al20	A 5‐part pressure injury prevention bundle was associated with lower pressure injury rates in critically ill pediatric patients, thus supporting the use of bundles to prevent hospital‐acquired pressure injuries.
Lyren et al21	Implementing a structured patient safety collaborative reduced the rates of hospital‐acquired conditions and serious safety events.
Miller et al22	The sustained use of central line insertion and maintenance bundles resulted in reduced PICU central line‐associated bloodstream infections. These bundles focused on both improving daily maintenance care for central lines and reliably performing tasks for each PICU patient multiple times per day.
Nishisaki et al23	In order to identify areas for process improvement and ultimately improve outcomes of emergent tracheal intubations, investigators created the National Emergency Airway Registry for Children (NEAR4KIDS), which compiled process of care and safety outcomes for tracheal intubations across fifteen academic PICUs in North America.

These organizations have developed online resources of potential interest to clinicians across the world. As seen in Table 5, these organizations offer a range of valuable tools and methodologies to measure, improve, and benchmark important quality metrics. These websites can serve as an important training resource for those seeking to learn and master the science of quality improvement. They can also be valuable resources for physicians, nurses, and other quality improvement program leaders seeking resources to better measure and improve their performance.

Table 5

Helpful online resources for quality improvement

Organization	Name of resource	Exclusive to pediatrics?	Description
Agency for Healthcare Research and Quality (AHRQ)	“Types of Health Care Quality Measures”	No	The Donabedian model is a classification system that categorizes health care quality measures as related to structure, process, or outcome. The AHRQ uses this model to assess and compare the quality of health care organizations.5
Agency for Healthcare Research and Quality (AHRQ)	“Six Domains of Health Care Quality”	No	The AHRQ defines six domains of health care quality to help consumers understand the meaning and relevance of quality measures. The six aims of health care quality as outlined by the AHRQ are to make health care: safe, effective, patient‐centered, timely, efficient, and equitable.24
American Society for Quality	“Failure Mode & Effects Analysis (FMEA)”	No	FMEA is a widely‐used analysis tool that helps to identify failures in a process with the goal of eliminating factors in a process that could cause failures.25
Canadian Patient Safety Institute (CPSI)	“Tools and Resources”	No	The CPSI has developed evidence‐based tools and resources for use by both individuals and organizations to advance patient safety by improving communication, reporting incidents, and improving prevention strategies through incident analysis and careful changes to existing processes.26
College of Intensive Care Medicine (CICM) of Australia and New Zealand	“Resources”	No	The CICM of Australia and New Zealand provides information on a range of matters from Professional Documents detailing policies, guidelines, and statements to Training Resources Documents.27
European Society of Paediatric and Neonatal Intensive Care (ESPNIC)	“Mission, Objectives, and Strategy”	Yes	ESPNIC aims to promote the delivery of the highest quality health care to critically ill children in Europe by exchanging ideas and knowledge in international and multidisciplinary settings.28
Institute for Healthcare Improvement	“How to Improve”	No	The Institute for Healthcare Improvement uses the Model for Improvement as a framework to guide improvement. This model follows the Plan, Do, Study, Act (PDSA) cycle as a tool to accelerate improvement.29
PICU Collaborative, Division of Critical Care Medicine, Boston Children's Hospital	“Mission Statement”	Yes	The PICU Collaborative aims to compare quality measures and outcomes through the collaboration of PICUs worldwide that review bi‐annual data on 6 ICU quality metrics and display them on a collaborative dashboard. This data is used as a benchmark for individual participating sites to improve the quality of care in their PICUs.30
Program for Patient Safety and Quality, Boston Children's Hospital	“Quality and Patient Safety”	Yes	The Program for Patient Safety and Quality encourages family members and children to voice observations, concerns, and opinions to the care team with a specific focus on key measures of quality, such as surgical site infections and vaccination rates.31
Solutions for Patient Safety	“Our Goals”	No	Children's Hospitals’ Solutions for Patient Safety aims to share safety successes and failures to prevent serious harm to children while they heal by sharing harm reduction goals across hospital networks and aligning organizational goals.32
World Federation of Pediatric Intensive & Critical Care Societies (WFPICCS)	“Vision, Mission, Values”	Yes	WFPICCS compiles resources for families, doctors, and nurses on a wide range of topics related to critical care, from research and the development of new clinical treatments to the promotion of educational programs and multidisciplinary collaboration among pediatric intensive and critical care specialists worldwide.33

DISCUSSION

In this systematic review of the literature we found that only 15% of the recently published pediatric intensive care quality improvement studies reported a sufficiently rigorous methodology (basic elements such as the problem studied, the methods and interventions utilized, the findings or results, and a conclusion supported by the findings) to be potentially reproducible and generalizable to other similar programs. We also found a number of notable academic and professional health care organizations devoting open‐access resources to learning and implementing science in continuous quality improvement.

Quality improvement methodology is fundamentally different from that of clinical investigation. The former seeks to use a rapid cycle assessment of an intervention designed to impact the system of care in an uncontrolled environment (i.e. the real world) while the latter seeks the meticulous control of the experimental environment in order to discern the effectiveness of an intervention on specific patients compared to a control or usual care. While the two methodologies are fundamentally different, it is equally true that standardization of the basic elements of reporting on the methodology used is necessary if knowledge learned in one environment is to be leveraged and scaled to other similar environments. Indeed, while the International Committee of Medical Journal Editors6 for over 20 years have had established standards on the uniform requirements for manuscripts submitted to biomedical journals, our analysis found that 85% of the recent quality improvement literature published in biomedical journals lacked basic elements in scientific reporting of the intervention that was undertaken.

Publication guidelines and standards for quality improvement in health care were first published in 20087 and revised a second time and widely reported in 2015.8 While these guidelines support various methodologies to study system level health care improvement, they also stipulate a standardized format for reporting on these interventions, such as structured presentation that includes: background, local problem, methods, interventions, results, and conclusions.9

Our study does have several limitations concerning for a selection bias in our final sample. First, we excluded studies that examined quality improvement processes exclusively for neonatal or adult patients receiving intensive care, and thus possibly overlooking potentially promising interventions for the pediatric intensive care domain. Second, our search included only studies available in English, another potential source of bias and again potentially overlooking promising quality improvement interventions already underway in non‐English speaking countries.

Going forward, quality improvement programs that seek to improve the care provided to critically ill children will benefit from further research on new ideas that are tested and studied from all corners of the globe. Efforts to accelerate the use of standardized improvement methods to enhance performance of a system, process, and/or outcome, followed by uniform adherence to publication guidelines on such studies, will lead to more effective and efficient sharing of knowledge for the benefit of critically ill children across the world.

CONFLICT OF INTEREST

The authors report no conflict of interests related to the material presented in this article.