Impact of Complex Quality-Interventions on Patient Outcome: A Systematic Overview of Systematic Reviews.

Quality of care and the increasing strategies to its promotion, especially in inpatient settings, led to the question which quality-interventions work best and which do not. The aim was to summarize and critically appraise the evidence on the effects of structure- and/or process-related quality-interventions on patient outcome in predominantly controlled and inpatient settings. A systematic overview of systematic reviews after electronic searches in Medline, Embase, Cinahl, and PsycINFO, supplemented by hand search and expert survey, was conducted. From a total of 1559 identified records, 37 reviews fulfilled the inclusion criteria. 26 reviews assessed process-related quality-interventions, 6 structure-related quality-interventions, and 5 combined structure- and process-related quality-interventions. In all, 19 reviews reported pooled effect estimates (meta-analysis). Based on the evidence of this systematic overview, stroke units and pathways can be recommended. Although patient-relevant improvements for interprofessional approaches and discharge planning have been reported, pooled effect estimated evidence are currently missing for these and other quality-interventions.

What do we already know about this topic?

It is already known that a change of structural and procedural determinants affects the quality and outcome of medical care.

How does your research contribute to the field?

From the perspective of evidence-based health care, it is important to investigate and critically appraise which structure- and/or process-related quality-interventions work best to induce patient benefit and which do not have the intended effects.

What are your research’s implications toward theory, practice, or policy?

Robust, overwhelmingly controlled and pooled estimated evidence was found for the effectiveness of stroke units and clinical pathways on patient outcome.

Introduction

According to the Donabedian framework, structures and procedures of health care provision determine the outcome of care, that is, effectiveness and efficiency of a health care system.[1] The governments of several countries have started to increasingly focus on the quality of health care provision and have implemented various strategies aiming to promote high quality of care in their health care systems. The goals of these strategies to improve quality of care (eg, short waiting times/high access to health care, low mortality rates, and high patient satisfaction), the national commitment to quality (eg, regional, federal, and nationwide), and the reward design (eg, bonus and malus) of pay-for-performance programs vary considerably between nations and programs.[2] Typically, efforts to improve the quality of care focus on the structure, process, or both dimensions of health care provision to eventually improve outcome quality (eg, reduce mortality/morbidity/cost and improve patient quality of life) or value of care.[3,4] Examples for structure-related quality-interventions include the use of reminder systems[5] or telehealthcare.[6] Process-related quality-interventions include, for instance, approaches to achieve a better interprofessional cooperation.[7] Inter-ventions that address both the structural and the procedural levels include multifaceted features (eg, integrated care).[8] From the perspective of evidence-based health care, it is important to investigate which quality-interventions work best to induce patient benefit, which do not have the intended effects, and for which quality-interventions evidence is still scarce or missing. Therefore, we undertook a systematic overview to summarize and critically appraise published systematic reviews on the effects of prospectively planned structure- and process-related quality-interventions on patient outcome.

Methods

This systematic overview was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Checklist (Supplemental Material 1).[9] The review protocol was registered (CRD42017059402) in the International Prospective Register of Systematic Reviews.[10]

Study Inclusion and Exclusion Criteria

Inclusion and exclusion criteria were specified addressing the PICOS categories: population, intervention, comparison, outcome, and study type (Supplemental Material 2). To be included, a systematic review had to predominantly (>50%) include controlled trials with or without randomization and had to be conducted in an inpatient-involved setting. Interventions of interest included all planned interventions on the structure and/or process level with documented effects on outcome quality. Control groups could be standard care or an alternative quality-intervention. All systematic reviews published in English or German between January 1, 2000, and January 24, 2017, were included. The start date of the search was chosen regarding the milestone article “Crossing the quality chasm.”[11]

Searches

Systematic electronic searches were conducted in Medline (Ovid), Embase (Ovid), Cinahl (EBSCOhost), and PsycINFO (EBSCOhost). The search strategy included a combination of free text words and database-specific subject headings (Supplemental Material 3). A referenced search strategy was used to acquire the publication-type systematic review.[12] Additionally, a hand search screening in Google Scholar and reference lists of included reviews was performed. The results of systematic searches and hand search presented to a panel of experts (the SIQ working group “correlation of structural and outcome quality”) highlight missing articles. EndNote X7 was used for creating the literature database and removing duplicate copies.

Study Selection and Data Extraction

Two authors (F.W. and J.S.) independently screened titles/abstracts and full texts for eligibility. Regarding the full-text screening, a serial 2-step process was conducted before a full extraction of characteristics, effects, and quality assessment followed. The first step included a general assessment of contents, publication type and year, and so on. Studies that met the screening criteria at this stage were further screened for included study designs and setting. If a publication was excluded due to a lack of controlled designs or low/missing inpatient/intersectoral setting, a summary of characteristics is followed (Supplemental Material 4). Data extraction and quality assessment of included systematic reviews was conducted by one (F.W.) and double-checked by the second author (D.K.). Disagreements were solved through the consultation of a third reviewer (screening: D.K.; extraction or quality appraisal: J.S.).

Quality Assessment

The R-Amstar-Checklist was used to assess the quality of included reviews instead of AMSTAR 2, which has been published 6 months (September 2017) after data extraction and quality appraisal have been finished (March 2017).[13] One item (9) was only feasible if a meta-analysis was conducted.[14] For this reason, the reviews were separated into those with meta-analysis (max. 44 points) and those without meta-analysis (max. 41 points). Two reviewers independently (F.W. and D.K.) performed quality assessment.

Extraction and Interpretation of Results

Every primary endpoint of the included systematic reviews was extracted. If the primary endpoint was a structure or process measure (and not a true outcome), all secondary outcomes were extracted in addition to the primary outcome parameters. The extracted effects were categorized by content and quality dimensions. Infrastructure (eg, technique) or general organizational aspects (eg, staff-to-patient) were defined as structure quality. Aspects of performance (eg, length of stay) were considered as process quality. Outcome quality was defined according to “5 Ds” of Donabedian: Death, Disease, Disability, Dissatisfaction, and Discomfort,[15] or to typical health economic values (eg, cost-effectiveness). Four effect directions (favors intervention; favors standard care; no difference; unclear) were defined. If an extracted result was quantitatively summarized (meta-analysis), it was possible to directly establish the effect direction. If effects were descriptively summarized, the results of all individual included trials were counted. To conclude a counted direction, the corresponding effect had to be shown in more than 50% of the included trials. Otherwise, an effect was classified as “unclear.”

Results

After removal of duplicate copies (n = 960) and the identification of 12 records through hand search, 1559 records were included in the title/abstract screening. In all, 1421 records were excluded and 138 remaining full texts were assessed for eligibility (Supplemental Materials 4 and 5 for justified exclusions). As presented in Figure 1 after sequential full-text screening, 37 systematic reviews were finally included.

Figure 1.

PRISMA flowchart.

Content of Included Systematic Reviews

Five reviews analyzed the effectiveness of combined structure- and process-related quality-interventions.[16-20] The investigated interventions included consultations of clinical pharmacists,[16] hospital-wide geriatric consultation teams,[17] different kinds of organized stroke unit care,[18] implementation of quality indicators as a reporting/feedback tool[19] and hospital-wide, integrated and facility-overspanning practices (eg, geriatric consultation) to deliver care for older patients.[20] Six reviews investigated the effects of structure-related quality-interventions[21-26] addressing the use of personal digital assistants,[21] clinical decision support systems,[22] computerized advice on drug dosage,[23] geriatric evaluation and management units,[26] and stroke units in mobile[24] and inpatient delivery.[25] Most of the included reviews (26/37; 70%) assessed process-related quality-interventions.[27-51] Eleven systematic reviews evaluated effects of discharge planning[27-37] addressing patient/family engagement,[29] different communication/consultation services,[27,31,36] detailed/individual assessments,[35] early discharge,[32,37] structured multidisciplinary discharge plans with patient/family engagement,[33] and various discharge support designs.[28,30,34] Procedural interprofessional approaches were evaluated in 4 systematic reviews addressing different conditions and designs like comprehensive geriatric assessments,[38] multifaceted care for intensive care unit (ICU) patients,[39] team-based models in end-of-life care,[40] and various interprofessional approaches.[41] Clinical pathways were evaluated in 4 systematic reviews addressing end-of-life care,[42] colorectal surgery,[43] hip fracture,[44] and various settings/designs.[45] Other process-interventions included the use of communication tools to conclusively document end-of-life decisions,[48] shared decision-making,[49] nutritional screening, routine use of patient-reported data,[50,51] and interventions to reduce the number of medication errors for children (eg, checklists)[46] and in-hospital falls (eg, risk assessment).[47] Overall, discharge planning (11 reviews), interprofessional approaches (8 reviews), clinical pathways (4 reviews), and stroke units (3 reviews) were the most investigated interventions. As summarized in Supplemental Material 7, the 37 systematic reviews included a total of 559 trials. A meta-analysis was conducted in 19 of 37 reviews (51%). A controlled design was described in 523 of 559 (93%) of the included trials. Except for 1 systematic review,[34] a primary endpoint was always defined. In 35 of 36 (97%) reviews, the primary evaluation of outcome-relevant effects was reported. And, 26 of 37 (70%) reviews exclusively included controlled studies.

Most of the systematic reviews (35/37) provided complete information regarding characteristics of included studies (item 6). Detailed information about inclusion criteria (item 1), study selection/data extraction (item 2), literature search (item 3), and scientific quality (item 7) were provided in majority. In all, 10 of 37 reviews (27%) partly[16,23,25,35,36,40,46,48] or fully reported[17,45] on the likelihood of publication bias (item 10). Mixed results were shown regarding the inclusion of gray literature (item 4), complete lists of included/excluded studies (item 5), the discussion regarding the scientific quality (item 8) of included primary studies, and if a meta-analysis was conducted methods for pooling of results (item 9). No systematic review reported about conflicts of interest in the primary studies (item 11). Supplemental Material 6 presents detailed results of the quality assessment.

Effects of Interventions

The effects of combined structure- and process-related quality-interventions are given in Table 1. Consulting clinical pharmacists reduced the occurrence of adverse drug events and improved the medication adherence of patients.[16] The outcome-relevant impact of geriatric consultation teams led to (nonsignificant) reductions of mortality, readmissions, and better functional health outcomes for patients.[17] Structure- and process-integrated stroke units led to significant and nonsignificant improvements in mortality and certain composite outcomes (eg, death/institutional care).[18] Interventions to improve care for frail older inpatients have not shown differences in mortality and functional outcome, or remained unclear due to a lack of statistical tested values reported in the primary studies.[20] Using quality indicators[19] did not lead to differences regarding patient outcome compared with standard care.

Table 1.

Effects of Combined Structure- and Process-Related Quality-Interventions.

Intervention	Effect	Significant[a]		Nonsignificant[a]		Descriptive/not rep[a]
		Process	Outcome	Process	Outcome	Process	Outcome
Interprofessional: interventions by clinical pharmacists[16]	Adverse event						+
	Patient-reported						+
	Treatment duration					+
Interprofessional: comprehensive geriatric assessment[17]	Physician-reported				+
	Treatment duration			+
	Mortality				+
	Readmissions				+
Hospital-wide interventions to improve care for frail older inpatients[20]	Adverse events						?
	Readmissions						?
	Mortality						+/-
	Physician-reported						+/-
	Treatment duration					+/-
	Health economic					? ?	?
	Other						?
Organized inpatient (stroke unit) care[18]	Mortality		+ + +		+		+ [b] +
	Death/institutional care		+ +		+ +		+ [b]
	Death/dependency		+		+ +		+ [b] +
Using quality indicators[19]	Physician-reported						+/-

Note. + = favors intervention; ? = unclear/not interpretable; +/- = no difference; - = favors standard care.

If meta-analysis was conducted.

No total meta-analysis: results of subtotals summarized.

The effects of structure-related quality-interventions are shown in Table 2. The use of personal assistants was rated as useful for the clinical setting.[21] Clinical decision support systems in neonatal care led to a nonsignificant reduction of mortality.[22] The computerized advice on drug-dosage led to significant and nonsignificant improvements in therapeutic range and serum concentrations of the applied medication. The occurrence of adverse (medication) events (eg, bleeding) was nonsignificantly reduced.[23] Mobile stroke units were significantly less effective compared with a regular stroke unit regarding mortality and other composite outcomes. Compared with a treatment in a general medicine ward, mobile stroke units led to a nonsignificantly lower 6-month mortality and 12-month death/dependency rate. However, the risks for 12-month mortality and death/institutional care tended to increase.[24] Treatment in acute/comprehensive stroke units led to significant reductions of mortality, institutional care, and improved combined outcomes (death/institutional care) compared with standard care.[25] The implementation of a Geriatric Evaluation and Management Unit led to significant reductions in institutional care and nonsignificant reductions in 12-month mortality, readmissions, and (physician-reported) functional decline of patients.[26]

Table 2.

Effects of Structure-Related Quality-Interventions.

Intervention	Effect	Significant[a]		Nonsignificant[a]		Descriptive/not rep[a]
		Process	Outcome	Process	Outcome	Process	Outcome
Personal digital assistants[21]	Other						+
Mobile stroke unit[24]	Mortality		- -
	Death/dependency		-		+ -
	Death/institutional care		-		-
Stroke unit care[25]	Mortality		+		-
	Other: independency		-
	Other: institutional care		+
	Death/institutional care		+
Clinical decision support system[22]	Mortality				+
Interprofessional: Geriatric Evaluation + Management Units[26]	Mortality				+
	Other: institutional care		+
	Physician-reported				+
	Readmissions				+
Computerized advice on drug dosage[23]	Mortality				+
	Adverse event		+		+ + + + + + - -
	Physician-reported	+ + +	+ + +	+	-
	Treatment duration			+

Note. + = favors intervention; - = favors standard care; +/- = no difference; ? = unclear/not interpretable.

If meta-analysis was conducted.

A total of 26 process-related quality-interventions (Table 3) were grouped in pathways,[42-45] interprofessional approaches,[38-41] discharge planning,[27-37] and other process-interventions.[46-52]

Table 3.

Effects of Process-Related Quality-Interventions.

Intervention	Effect	Significant[a]		Nonsignificant[a]		Descriptive/not rep[a]
		Process	Outcome	Process	Outcome	Process	Outcome
Pathways for
Colorectal surgery patients in recovery[43]	Treatment duration	+
	Other		+
	Readmission				+/-
Dying patients, carers/providers/relatives[42]	Physician-reported						+
Stand-alone and multifaceted pathways in different settings/conditions[45]	Mortality				- +
	Adverse events		+
	Readmissions				+		+
	Other	+
	Health economic		+		+
	Treatment duration			+		+
Hip fracture[44]	Mortality						+
	Adverse events						+
	Readmissions						?
	Other						? +
Interprofessional approach
Comprehensive geriatric assessment[38]	Other: living at home		+
Interprofessional collaboration practice-based interventions[41]	Treatment duration					+ ?
	Health economic						+
Multifaceted care approaches[39]	Mortality						+ + +
	Adverse event						+ + + + +
	Health economic						+
	Physician-reported						+ +
	Treatment duration					? ?
	Other					+ + + ? ?
Team-based models of care[40]	Mortality						+ +
	Patient-reported						+ ? ?
	Physician-reported						? ?
	Treatment duration						?
	Readmissions						+ +
	Other			+			+
Discharge planning
Comprehensive discharge plan, postdischarge support[36]	Readmissions		+
Early postnatal discharge[32]	Readmissions				- -
	Patient-reported				+ +
Computer-enabled discharge communication interventions[29]	Mortality						+/-
	Readmissions						+/-
	Adverse events						?
Effective emergency department- discharge of older patients[35]	Readmissions				+ -
	Mortality				-
Hospital-initiated transitional care[30]	Readmissions						+ ?
	Adverse events						? ?
Family-centered transition processes[27]	Patient-reported						+ + + +
	Health economic						+ + + +/-
Supporting discharge from hospital to home[34] (no stated primary endpoints, effects rated as secondary endpoints)	Mortality				+
	Other: living at home		+
	Other: institutional care						+
	Readmissions						?
Patient handovers from hospital to primary care[28]	Health economic						? ?
	Other					+
	Adverse event						?
	Patient-reported						+
	Physician-reported						? ?
Discharge planning[33]	Treatment duration	+
	Readmissions		+
Structured discharge-process[31]	Readmissions						+ + +
Services for reducing duration of hospital care[37]	Mortality				+
	Death/institutional care		+
	Death/dependency		+
Other process-interventions
Communication tools for end-of-life decision-making[48]	Patient-reported	+ +		+ + -
	Patient-reported[b]		+ +				+
Hospital fall prevention[47]	Adverse event		+		+		?
Reducing medication errors[46]	Adverse event						? ?
Information and communication technologies adoption[52]	Other						+
Shared decision-making[49]	Patient-reported						? +/- + +
Routine use of patient-reported data[50]	Patient-reported						?
Nutritional screening[51]	Patient-reported						+/-
	Adverse events						+

Note. + = favors intervention; - = favors standard care; +/- = no difference; ? = unclear/not interpretable.

If meta-analysis was conducted.

Secondary endpoint.

Pathways in end-of-life care showed improvements in physical symptom severity.[42] Recovery pathways after colorectal surgery led to significant reductions in length of stay and 30-day morbidity with no differences of readmissions.[43] Pathways for hip fractures improved overall rates of mortality, adverse events (medical/multiple complications), and functional recovery/mobility, but the results were not tested for significance.[44] Pathways regardless of inpatient condition led to significantly reduced costs/charges (health economic), complications (adverse events), and a nonsignificantly reduced treatment duration and 6-month readmissions.[45]

Interprofessional interventions: Comprehensive geriatric assessment led to significantly increased ratios of patients living at home after discharge.[38] Through practice-based interventions for delirium-prevention, the number of adverse events (eg, coma days), mortality (eg, 30 days), costs, and worsening physician-reported discharge-diagnoses (eg, patients without ability to return home) were reduced. Mixed effects were shown in treatment duration and other procedural effects (eg, use of benzodiazepine).[39] Team-based models in end-of-life care improved hospital/ICU (re)admission rates, mortality (preferred places of death: nursing home/home), and patient satisfaction. Unclear effects were shown regarding provider-satisfaction, quality of life, and symptom management.[40] General interprofessional collaborations led to reduced total charges and length of treatment with unclear effects regarding length of stay.[41]

Discharge planning: Different forms of family-centered discharge professes were effective regarding health economic (eg, health care utilization/cost) and patient-reported (eg, health status) outcomes.[27] The use of patient handovers from hospital to primary care showed unclear effects regarding adverse events (eg, errors/near misses), health economic (eg, use of hospital care) and physician-reported issues (eg, health care provider status). The general patient status improved.[28] No advantages of computer-enabled discharge communication could be shown regarding mortality, adverse events, and readmission rates.[29] Hospital-initiated transitional care reduced the number of readmissions, but did not lead to homogeneous effects for adverse events.[30] Implementing a structured discharge process for adult patients with community acquired pneumonia reduced 30-, 90-, and 180-day readmission rates after discharge.[31] After implementation of early postnatal discharge programs, the incidence of depressions decreased nonsignificantly, whereas the rates of breastfeeding women (patient-reported) and readmissions increased nonsignificantly.[32] Global discharge planning programs led to significantly reduced lengths of hospital stay (treatment duration) and 3-month readmissions.[33] Supporting discharge from hospital to home in older patients resulted in a significantly higher probability to live at home at the end of follow-up (other) and nonsignificant/descriptive improvements in mortality, readmissions, and institutionalizations.[34] Interventions for an effective discharge of older patients from the emergency department led to nonsignificantly reduced 1-month readmission rates together with nonsignificantly increased 18-month mortality rates and (unplanned emergency department) readmissions.[35] A comprehensive discharge plan with an additional postdischarge support significantly reduced the readmission rate.[36] Services for reducing hospital duration of stroke patients led to significant improvements in combined outcomes (eg, death/institutional care) and nonsignificant decreases in mortality.[37]

Other process-interventions: Different interventions to reduce medication errors of hospitalized children led to unclear effects regarding adverse events.[46] Interventions for fall-prevention significantly reduced the number of falls (adverse event) and nonsignificantly the number of fallers. The effects for the number of recurrent fallers remained unclear.[47] Communication tools for end-of-life care significantly improved processes (eg, documented goals-of-care decisions) and patient-relevant outcomes (eg, end-of-life care literacy).[48] Shared decision-making improved patient adherence and well-being, whereas effects on patient status and satisfaction remained unclear.[49] The routine use of patient-reported data in cancer clinics did not draw a clear conclusion for patient-reported outcomes (eg, satisfaction).[50] Nutritional screening resulted in decreased adverse events (eg, infection rates) and showed no differences in patient-reported outcomes.[51] Interventions to promote the adoption of information/communication technologies into the care process were rated as effective.[52]

In summary, the 37 included reviews reported on a total of 162 outcomes. Additionally, a total of 32 procedural effects were identified and abstracted from the included reviews as they had been declared as primary endpoint. Of 194, 187 (96%) effects were classified as primary endpoints. A meta-analysis was conducted for 101 effects. An I2-test was provided for 67 of 101 of the effects (66%). Based on the Cochrane Handbook,[53] the I2 results were mainly rated minor (I2 ≤ 30%: 35/67; 52%) or moderate (I2 ≤ 50%: 8/67; 12%) heterogeneous. A patient-relevant improvement of structure- and/or process-related quality-interventions could be shown in majority (Supplemental Material 6). The most-frequent contents of the extracted effects were adverse events, mortality, and physician-reported results. Intervention-related improvements were mostly documented regarding combined endpoints (eg, death/dependency), other results (eg, ventilator days), or mortality. Unclear effects were documented regarding health economic issues, treatment duration, and adverse events due to a lack of provided interpretations or values. Both combined structure and process modifications and exclusive structure- or process-related quality-interventions were reported to be effective in most of the systematic reviews.

Discussion

The goals of this systematic overview were to identify the influence on patient outcome of structure- and/or process-related quality-interventions and to generally summarize and critically appraise the existing evidence. The number of results of the search strategy shows that this topic is of great interest, and 37 systematic reviews were included. The quality appraisal of the reviews indicated adequate quality regarding the provision of general information by the authors such as details on inclusion criteria or study selection. However, information on the likelihood of publication bias and conflicts of interest in the included primary studies substantial information was lacking in many articles included. Most of the included reviews reported exclusively on controlled studies. For approximately half of the included reviews and reported effects, a meta-analysis was performed and the I2-values showed no substantial heterogeneity. Especially, inpatient acute/comprehensive stroke units and clinical pathways were effective in terms of patient-relevant outcomes compared with standard care. Due to a lack of significant results in the included meta-analyses, it is difficult to generally recommend or advise against certain structure- or process-related quality-interventions like discharge planning or interprofessional approaches.

Limitations

One limitation was the lack of a standardized definition of the expression “quality.”[54,55] Authors of the reviews have differing understandings of how a quality-intervention is defined. Therefore, they might not have used the expression “quality” although they have investigated a quality-intervention. Complex interventions which cannot be easily assessed in controlled designs may be underrepresented in our overview. The global approach of an overview leads to further methodological limitations. The analysis relied on more or less detailed information in the systematic reviews. An investigation of the single trials and a check for duplet copies were not applied. Regarding other overviews of nontherapy interventions, there is a general problem of heterogeneous content and methods.[6,8,56] One solution to this limitation could be the conduction of systematic overviews for each single intervention, such as interventions to reduce the use of emergency departments.[56]

To our knowledge, this is the first overview assessing the effectiveness of predominantly controlled structure- and process-related quality-interventions in inpatient settings. A clear favor could be shown for stroke units and pathways. Due to a general lack of significant results of discharge planning and interprofessional approaches, a recommendation for these interventions cannot be given. Because of the missing standardized definition of the expression “quality,” it was challenging to include all relevant systematic reviews. Due to a lack of the clear definition of quality, it is necessary to work without this term, and to focus on certain interventions to draw clear conclusions, a systematic overview of meta-analyses is recommendable. However, it is important to set a starting point for following overviews which assess the outcome-related influence of definite quality-interventions (eg, discharge planning) in inpatient settings.

Click here for additional data file.

Supplemental material, Additional_files_Overview_Inquiry for Impact of Complex Quality-Interventions on Patient Outcome: A Systematic Overview of Systematic Reviews by Felix Walther, Denise Kuester and Jochen Schmitt in INQUIRY: The Journal of Health Care Organization, Provision, and Financing