Appropriateness, effectiveness and safety of care delivered in Canadian hospitals: a longitudinal assessment on the utility of publicly reported performance trend data between 2012-2013 and 2016-2017.

OBJECTIVES: To assess the utility of publicly reported performance trend results of Canadian hospitals (by hospital size/type and jurisdiction).
DESIGN: Longitudinal observational study.
SETTING: 489 hospitals in Canada between fiscal years 2012-2013 and 2016-2017. PARTICIPANTS: Analysis focused on indicator results of individual Canadian hospitals. PRIMARY AND SECONDARY OUTCOMES: Eight outcome indicators of hospital performance: in-hospital mortality (2), readmissions (4) and adverse events (2). Performance trend outcomes of improving, weakening or no change over time. Comparators in performance by hospital size/type of above, below or same as average.
RESULTS: At the national level, between 2012-2013 and 2016-2017, Canadian hospitals largely reduced in-hospital mortality: hospital deaths (hospital standardised mortality ratio) -9%; hospital deaths following major surgery -11.1%. Conversely, readmission to hospital increased nationwide: medical 1.5%; obstetric 5%; patients aged 19 years and younger 4.6% and surgical 3%. In-hospital sepsis declined -7.1%. Approximately 10% of the 489 hospitals in this study had a trend of improving performance over time (n=49) in one or more indicators, and a similar number showed a weakening performance over time (n=52). Roughly half of the hospitals in this study (n=224) had no change in performance over time for at least four out of the eight indicators. No single hospital had an improving or weakening trend in more than two indicators. Teaching and larger-sized hospitals showed a higher ratio of improving performance compared with smaller-sized hospitals.
CONCLUSIONS: Analysis of Canadian hospital performance through eight indicators shows improvement of in-hospital mortality and in-hospital sepsis, but rising rates of readmissions. Subdividing the analysis by hospital size/type shows greater instances of improvement in teaching and larger-sized hospitals. There is no clear pattern of a particular province/territory with a significant number of hospitals with improving or weakening trends. The overall assessment of trends of improving and weakening as presented in this study can be used more systematically in monitoring progress. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Assesses performance results for all Canadian hospitals.

Assesses eight hospital performance indicators related to appropriateness, effectiveness and safety.

Assesses performance of hospitals by size, type and provincial/territorial jurisdiction.

Uniform indicator calculation methods and performance trend assignment.

Performance of hospitals is limited to categorical and aggregate outcomes, and did not include absolute quantitative performance.

Introduction

Performance information can aid a range of policy and organisational change levers to facilitate healthcare performance management (such as accountability) and contribute towards quality improvement initiatives.1 The design of coherent and integrated health information systems has ensured the collection, calculation and access to data necessary for performance monitoring.2 The use of these data and evidence in quality improvement science, and for full transparency of performance, is essential as we enter a new generational era in medicine and healthcare.3 But as novel performance data and methods are introduced, their assessment for utility is warranted to ensure they are fit for purpose and actionable.4

Mortality rates,5 readmission rates6 and adverse events7 are frequent publicly reported indicators used to illustrate hospital performance with respect to appropriateness, effectiveness and safety of care. Public-reporting of these performance results aims to, among numerous goals, spur quality improvement initiatives at the hospital and health jurisdiction level.8–10

However, much of the existing scientific literature on hospital performance focuses on a small number of indicators at a time, generally in a narrow care setting (such as teaching hospitals or specific hospital units), and captures a short time span. In the Canadian context, few, if any studies have been published that quantify hospital performance across all Canadian hospitals, across numerous hospital performance domains or cover sizeable time-spans.

This type and level of evidence would inform meso-level and macro-level system initiatives that may hold greater promise of impact. Backed by this evidence, pan-national and provincial/territorial agencies charged with performance improvement mandates would be able to gather (otherwise isolated) best-practices, and target (potentially pooled) resources to address areas of care and services that are most pressing to the performance of Canadian hospitals. It also helps to assess the overall long-term changes in performance of Canadian hospitals.

The Canadian Institute for Health Information (CIHI) holds the mandate to collect hospital admissions data, to perform statistical calculations and to report on the performance of all hospitals in Canada. CIHI has disseminated hospital performance results in a variety of mediums over its 25-year history. In recent years, this has included establishing a consolidated online webtool titled Your Health System (YHS). CIHI’s YHS tool provides detailed results for approximately 45 health system performance (HSP) indicators.11 CIHI’s mandate, similar to other health information agencies, is to report on hospital and HSP, but to refrain from overt ranking of hospitals or health jurisdictions, as there are unintended consequences associated with public release of performance data.12 Nonetheless, the data are often publicly available for others to perform secondary analyses.

In recent years, CIHI has added a dimension to its reporting by delineating hospital results by whether they are improving, weakening or having no significant change in performance over time. Eight hospital-level indicators within CIHI’s YHS tool (covering in-hospital mortality, readmission and safety-related adverse events) show performance results with this dimension of performance trends over time. These eight indicators fall under CIHI’s HSP framework13 quadrant of health system outputs, and cover the themes of (1) appropriateness and effectiveness and (2) safety (see https://www.cihi.ca/en/indicator-library).

In this paper, we explore the utility of CIHI’s publicly reported hospital results data to determine trends and any meaningful findings of performance by different care domains, by hospital type and size and at the national and provincial/territorial levels. Specifically, we explore the following four research questions:

What are the trends in hospital performance at the national level?

How many hospitals are improving or weakening in performance?

Is there a relationship in performance trends by hospital size/type?

What are the trends in hospital performance by province/territory?

Methods

We used the all data export report file from CIHI’s YHS tool14 to perform the analyses. The data file contains results for all indicators published on the YHS website as well as contextual measures and additional variables to assist with analysis and interpretation. The following eight indicators were assessed:

Hospital deaths (HSMR (hospital standardised mortality ratio)).

Hospital deaths following major surgery.

Medical patients readmitted to hospital.

Obstetric patients readmitted to hospital.

Patients aged 19 years and younger readmitted to hospital.

Surgical patients readmitted to hospital.

In-hospital sepsis.

Obstetric trauma (with instrument).

CIHI HSP framework theme: safety

All eight indicators are risk-adjusted by CIHI; all indicator-specific hospitalisations in the country were used to create a reference population in the risk model methodologies for respective indicators (model specifications and coefficients are publicly released by CIHI15). Five singleton fiscal year (1 April–31 March) hospital performance values were available covering 2012–2013 and 2016–2017. CIHI calculates national indicator rates by using the indicator values of all hospitals in the country. The last 3 years (2014–2015, 2015–2016, 2016–2017) of a hospital’s results were used to calculate a performance trend outcome.

National-level performance rates were compared by calculating the percent change difference from 2012–2013 and 2016–2017 data years (national rates aggregate the result of all Canadian hospitals). A linear regression model was used to determine national trend analysis; p values were calculated in a model in which the indicator national rate was the dependent variable and time was the independent variable.

CIHI reports hospital performance trends in three categories: (1) improving; (2) weakening and (3) no change. To determine this trend over time, CIHI’s methodology includes a series of two z-tests to compare the log-odds of a hospital’s results over the most recent 3 years of data (2014–2015, 2015–2016, 2016–2017).16 Additionally, there are three comparator categories for each hospital’s indicator results: (1) same as average; (2) below average and (3) above average; and is calculated by determining whether the hospital’s result was statistically significantly different from its peer group average.

To quantify trends in performance by hospital size/type, we stratified all results across the four hospital peer groups by the three performance categories. To quantify hospital performance trends subdivided by provincial/territorial jurisdictions, we identified each hospital that had either an ‘improving’ or ‘weakening’ indicator result.

Criteria for inclusion/exclusion of hospitals from this study

Not all indicators are applicable to every Canadian hospital; for example, clinical services may simply not be offered at certain facilities. Furthermore, hospitals that underwent a recent reorganisation (ie, a merger) experience a break in time-series, and are thus exempt from trending calculations in the short-term period. Moreover, indicator results may be suppressed due to privacy concerns (ie, small counts, generally when numerator and/or denominator values are between 1 and 4), or due to unstable results (a denominator between 1 and 49, or an expected event less than 1 if the observed numerator count was greater than 0).17 Nonetheless, the reported national indicator values incorporate all patient admissions throughout the country (regardless of small counts or mergers of individual hospitals). The province of Quebec merged many community-large and community-medium hospitals in 2015; as such, CIHI omitted these Quebec hospitals from trending value calculations for the 2016–2017 reporting year. As a result of the above criteria for hospital participation in the CIHI YHS tool and having performance trending values available, 489 hospitals were included in this analysis.

Hospital types

CIHI classifies Canadian hospitals into four distinct types (also referred to as ‘peer groups’): teaching (T) hospitals; community-large (H1) hospitals; community-medium (H2) hospitals and community-small (H3) hospitals. This classification facilitates meaningful comparisons across hospitals of similar structural characteristics, patient volume and clinical complexity.18 The four hospital types are described below:

Teaching hospitals (T)

A hospital is designated as ‘teaching’ by provincial/territorial ministries of health, or were identified as such in the provincial/territorial ministry’s submission to CIHI’s Management Information System database.

Community-large hospitals (H1)

A hospital is classified as ‘community-large’ if it met two of the following three criteria:

More than 8000 inpatient cases.

More than 10 000 weighted cases.

More than 50 000 inpatient days.

Community-medium hospitals (H2)

A hospital is classified as ‘community-medium’ if having 2000 or more weighted cases.

Community-small hospitals (H3)

A hospital is classified as ‘community-small’ if having fewer than 2000 weighted cases.

Analyses were performed on R V.3.5.0 (R Foundation for Statistical Computing, Vienna, Austria).

Patient and public involvement

Patients or public involvement were not included in the design of this study.

Results

Trends in hospital performance indicators at the national level

There are clear trends in hospital performance across the domains of mortality, readmission and safety when comparing the first and last available years (2012–2013 and 2016–2017) (table 1). In-hospital mortality (hospital deaths (HSMR) and hospital deaths following major surgery) have decreased (−9% and −11.1%, respectively). All four hospital readmission indicators show slight increases at the national level. As for hospital safety, in-hospital sepsis has decreased (−7.1%), while obstetric trauma (with instrument) saw initial improvement followed by a recent uptick of the same degree over the study period. Of the eight indicators, only three showed statistically significant trends over time; an improving trend for hospital deaths (HSMR) and hospital deaths following major surgery, and a worsening trend for surgical patients readmitted to hospital.

Table 1

National trends in hospital performance

Indicator	Unit of measurement	Number of hospitals with values	National rate2012–2013	National rate 2016–2017	Change (2012–2013 to 2016–2017) (%)	P value for trend
Hospital deaths (HSMR)	Baseline 100	92	100	91	−9	0.00245*
Hospital deaths following major surgery	%	180	1.8%	1.6%	−11.1	0.0154*
Medical patients readmitted to hospital	%	474	13.5%	13.7%	1.5	0.0577
Obstetric patients readmitted to hospital	%	225	2%	2.1%	5	0.0577
Patients aged 19 years and younger readmitted to hospital	%	185	6.5%	6.8%	4.6	0.0805
Surgical patients readmitted to hospital	%	246	6.7%	6.9%	3	0.0154*
In-hospital sepsis	Per 1000	431	4.2	3.9	−7.1	0.194
Obstetric trauma (with instrument)	%	100	18.9%	18.9%	0	0.880

*Statistically significant.

HSMR, hospital standardised mortality ratio.

Quantifying hospitals that have an improving or weakening performance trend

Table 2 further illustrates the contrast in hospital performance across the domains of hospital mortality, readmissions and safety. The largest ratio of hospitals improving versus weakening occurred for the indicators in-hospital sepsis (17 vs 8), hospital deaths (HSMR) (10 vs 6) and hospital deaths following major surgery (8 vs 1). The four readmission indicators largely saw a weakening of hospital performance; in aggregate, there were 37 instances of hospitals with weakening readmission rates, compared with only 17 instances of improving rates. Readmission of patients aged 19 years and younger was the only indicator to have no hospitals improving.

Table 2

Number of hospitals improving or weakening

Indicator	Number of hospitals (proportion of reported hospitals, %)
	Improving	Weakening
Hospital deaths following major surgery	8 (4%)	1 (1%)
Hospital deaths (HSMR)	10 (11%)	6 (7%)
In-hospital sepsis	17 (4%)	8 (2%)
Medical patients readmitted to hospital	7 (1%)	12 (3%)
Obstetric patients readmitted to hospital	6 (3%)	9 (4%)
Patients aged 19 years and younger readmitted to hospital	0 (0%)	6 (3%)
Surgical patients readmitted to hospital	4 (2%)	10 (4%)
Obstetric trauma (with instrument)	2 (2%)	5 (5%)

HSMR, hospital standardised mortality ratio.

The third performance trend category ‘no change’ can be slightly misleading on its own because it does not elaborate on whether the hospital’s performance was consistently poor or good. It is important to take into consideration whether hospitals with a trend of no change over time were performing above average, below average or same as average as compared with its hospital type (or peer group). Of all instances of ‘no change’ in indicator performance, 112 unique hospitals were performing above average, compared with 96 hospitals that were performing below average. A query of how many hospitals had no change in performance for at least four out of eight indicators produced a list of 224 hospitals (teaching (n=44), community-large (n=54), community-medium (n=85) and community-small (n=41)).

Quantifying performance trends by hospital type

Table 3 shows performance of the four hospital types across the eight indicators. One lens to view these data for any possible trends is to differentiate between instances of a greater number of hospitals improving versus weakening (shown with), instances of a larger number of hospitals weakening versus improving (shown with) and instances where the number of hospitals improving and weakening were the same (shown with). Teaching hospitals had the most instances of improvement (five out of eight indicators), followed by community-large (two indicators) and community-medium (one indicator). Only community-small hospitals did not have an instance of an indicator showing a higher ratio of improvement to weakening performance. The only hospital type with more improving hospitals than weakening in any readmission indicator was teaching hospitals for medical readmission. The surgical and medical readmission indicators had the largest ratio of hospital types (three out of four) with overall weakening hospital performance.

Table 3

Number of hospitals within each performance trend (by hospital size/type)

Indicator	Teaching (T) hospitals		Community-large hospitals (H1)		Community-medium hospitals (H2)		Community-small hospitals (H3)*		National total—all hospitals
Hospital deaths (HSMR)	I: 6 (16%)	✓	I: 4 (8%)	✓	I: 0 (0%)	=	I: N/A	N/A	I: 10 (11%)
	W: 3 (8%)		W: 3 (6%)		W: 0 (0%)		W: N/A		W: 6 (7%)
	NC: 28 (76%)		NC: 41 (85%)		NC: 7 (100%)		NC: N/A		NC: 76 (83%)
Hospital deaths following major surgery	I: 3 (7%)	✓	I: 1 (2%)	=	I: 4 (6%)	✓	I: 0 (0%)	=	I: 8 (4%)
	W: 0 (0%)		W: 1 (2%)		W: 0 (0%)		W: 0 (0%)		W: 1 (1%)
	NC: 40 (93%)		NC: 53 (96%)		NC: 68 (94%)		NC: 10 (100%)		NC: 171 (95%)
Medical patients readmitted to hospital	I: 3 (7%)	✓	I: 0 (0%)	✕	I: 1 (1%)	✕	I: 3 (1%)	✕	I: 7 (1%)
	W: 2 (5%)		W: 2 (4%)		W: 4 (4%)		W: 4 (1%)		W: 12 (3%)
	NC: 38 (88%)		NC: 53 (96%)		NC: 85 (94%)		NC: 279 (98%)		NC: 455 (96%)
Obstetric patients readmitted to hospital	I: 1 (3%)	✕	I: 2 (4%)	=	I: 3 (4%)	=	I: 0 (0%)	✕	I: 6 (3%)
	W: 3 (8%)		W: 2 (4%)		W: 3 (4%)		W: 1 (2%)		W: 9 (4%)
	NC: 32 (89%)		NC: 47 (92%)		NC: 74 (92%)		NC: 57 (98%)		NC: 210 (93%)
Patients aged 19 years and younger readmitted to hospital	I: 0 (0%)	✕	I: 0 (0%)	=	I: 0 (0%)	✕	I: 0 (0%)	=	I: 0 (0%)
	W: 2 (4%)		W: 0 (0%)		W: 4 (7%)		W: 0 (0%)		W: 6 (3%)
	NC: 43 (96%)		NC: 51 (100%)		NC: 57 (93%)		NC: 28 (100%)		NC: 179 (97%)
Surgical patients readmitted to hospital	I: 0 (0%)	✕	I: 1 (2%)	✕	I: 3 (3%)	=	I: 0 (0%)	✕	I: 4 (2%)
	W: 2 (5%)		W: 4 (7%)		W: 3 (3%)		W: 1 (2%)		W: 10 (4%)
	NC: 42 (95%)		NC: 50 (91%)		NC: 84 (93%)		NC: 56 (98%)		NC: 232 (94%)
In-hospital sepsis	I: 7 (15%)	✓	I: 7 (13%)	✓	I: 3 (3%)	=	I: 0 (0%)	=	I: 17 (4%)
	W: 0 (0%)		W: 5 (9%)		W: 3 (3%)		W: 0 (0%)		W: 8 (2%)
	NC: 41 (85%)		NC: 43 (78%)		NC: 84 (93%)		NC: 238 (100%)		NC: 406 (94%)
Obstetric trauma (with instrument)	I: 2 (6%)	✓	I: 0 (0%)	✕	I: 0 (0%)	✕	I: 0 (0%)	=	I: 2 (2%)
	W: 1 (3%)		W: 2 (4%)		W: 2 (11%)		W: 0 (0%)		W: 5 (5%)
	NC: 31 (91%)		NC: 43 (96%)		NC: 17 (89%)		NC: 2 (100%)		NC: 93 (93%)

Number (%) of hospitals improving, weakening or with no change (by hospital size/type and national total).

indicates more hospitals improving (than weakening). indicates more hospitals weakening (than improving). indicates equal number of improving and weakening hospitals.

*Community-small hospital quantifications should be interpreted with caution as these hospitals largely do not meet the minimum number of admitted patients and/or patients requiring high-complexity treatment to qualify for calculation and reporting of performance indicators.

HSMR, hospital standardised mortality ratio; I, improving; N/A, not applicable; NC, no change; W, weakening.

Quantifying hospital performance trends by provincial/territorial jurisdiction

Table 4 shows unique counts of hospitals (by province/territory) with a result of improving or weakening performance in at least one indicator. There is generally an equal distribution of trends across provinces and territories. Five regions had more hospitals weakening than improving; another five regions had more hospitals improving than weakening and the remaining three regions with an equal balance. At the hospital level, there were only five hospitals across the country that had a weakening trend for two indicators: two in each of teaching and community-large hospital types and one in community-medium. Of these five hospitals, three appeared in Ontario. Furthermore, there were five hospitals across the country that had an improving trend for two indicators; of these, four were teaching hospitals.

Table 4

Number of unique hospitals improving or weakening and total number of hospitals within province/territory

Provincial/territorial region	Number of hospitals with improving trends	Improvement trend probability, % (95% CI)	Number of hospitals with weakening trends	Weakening trend probability, % (95% CI)	Total number of hospitals within province/territory (as reported with performance trending results in the YHS tool)
Alberta	5	5 (2 to 12)	10	11 (5 to 19)	92
British Columbia	7	10 (4 to 19)	7	10 (4 to 19)	71
Manitoba	2	4 (0 to 13)	4	7 (2 to 18)	55
New Brunswick	2	11 (1 to 33)	1	5 (0 to 26)	19
Newfoundland and Labrador	3	12 (3 to 32)	4	17 (5 to 37)	24
Northwest Territories	1	25 (1 to 81)	0	0 (0 to 60)	4
Nova Scotia	1	4 (0 to 18)	2	7 (1 to 24)	28
Nunavut	–	–	–	–	0
Ontario	22	18 (12 to 26)	19	16 (10 to 23)	122
Prince Edward Island	0	0 (0 to 46)	1	17 (0 to 64)	6
Quebec*	2	18 (2 to 52)	1	9 (0 to 41)	11*
Saskatchewan	3	5 (1 to 15)	3	5 (1 to 15)	56
Yukon	1	1 (2 to 100)	0	0 (0 to 97)	1
National totals	49		52		489

*Community-large and community-medium hospitals in Quebec underwent mergers in 2015, thus inhibiting calculation of trending data for indicators.

YHS, Your Health System.

Discussion

Reporting of Canadian hospital performance has increased in its complexity and utility over the last decade.11 19 Publicly available tools now provide multiple user interfaces (from decision-makers to the general public) to view and understand how hospitals perform within their jurisdiction and across Canada. While mindful not to ‘name and shame’, health information agencies still endeavour to provide reporting of hospital performance and HSP of increased actionability. In recent years, CIHI has added new dimensions to hospital performance reporting, including performance trends over time, top results, comparisons to the national average and outlier analysis using funnel plots.

To date, few, if any, scientific studies summarise hospital performance across disparate domains for all hospitals in Canada, including any quantification of improving versus weakening trends. This study, therefore, provides an initial overview on the landscape of hospital performance on appropriateness, effectiveness and safety of care.

This secondary analysis of CIHI’s hospital performance reporting shows that in-hospital mortality—indicators of hospital deaths (HSMR) and hospital deaths following major surgery—has significantly declined in Canada, which has been shown elsewhere.20 Conversely, national rates of hospital readmission showed slight increases over time (but mostly non-statistically significant in trend analysis). Moreover, hospital readmission accounted for the majority of instances of weakening hospital performance. With respect to the CIHI HSP framework domain of Safety, in-hospital sepsis is declining; however, obstetric trauma (with instrument) has risen modestly (after a short period of decline). Roughly half of the hospitals in this study, which account for approximately one-third of all Canadian hospitals, did not improve or weaken in performance across at least four out of the eight indicators.

Strengths and limitations of this study

The chief strength of this study is the standardised calculation methods for all eight indicators (applied to all Canadian hospitals) by CIHI. All data used for calculations, from hospital admission abstracts, to statistical methods to perform risk-adjustment, are uniformly applied to all indicators and hospitals. Therefore, results for all eight indicators and the 489 hospitals included in this analysis, are confidently valid having been vetted and assured for data quality by CIHI. Furthermore, the statistical methods used by CIHI to determine performance trends (improving, weakening or no change) employs robust statistical tests to determine directionality of performance.

This study assessed 5 years of data for eight indicators, with a pool of 489 hospitals. Additionally, this study reported results from multiple performance domains, and by the four Canadian hospital peer groups (teaching, community-large, community-medium and community-small hospitals), which is scarce in the scientific literature.

Due to mergers of community-large and community-medium hospitals in the province of Quebec in 2015, these hospitals were excluded from performance trending calculations by CIHI. In subsequent years, when additional years of data are calculated for these hospitals, it will be feasible to include all Quebec hospitals into a similar analysis. Performance of hospitals is largely limited to categorical outcomes, and did not include absolute quantitative performance.

Nearly half of all Canadian hospitals are classified as community-small. Compared with the other three hospital peer groups, community-small facilities treat fewer and less-complex patients. Therefore, it is common for indicator results for these hospitals to be suppressed (or at times not calculated) due to small counts of cases. Similarly, CIHI’s model to calculate trending performance may not render statistically significant indicator results for smaller-sized hospitals due to small counts of patient cases. As such, smaller-sized hospitals overwhelmingly show performance trend outcomes of ‘no change’ over time.

Overall, the generalisability of the results in this study are largely restricted to Canada; this is due to the unique Canadian context of provision of hospital services, geographic distribution of hospitals and populations, risk-adjustment modelling to produce risk-adjusted indicator rates, indicator definitions, hospital size/type definitions and the quality and depth of the underlying clinical administrative data. Nonetheless, the methodology described by CIHI to produce performance trend outcomes can be applied to other national/subnational settings, hospital size/types and indicators.

Reflections on the study’s findings

Mortality, readmissions and adverse events are unfavourable and costly hospital outcomes. A 2015 systematic evaluation and reconfirmation exercise of all HSP indicators reported by CIHI found the eight indicators in this study as having high-utility by health decision-makers across Canada.4 In recent years, hospital mortality and adverse events were at the forefront of the performance management agenda, with programmes of pay-for-performance tying remuneration and hospital funding to results.21 This study shows that in-hospital mortality and adverse events have been in decline. Conversely, there are observed increases in readmission rates, and a greater number of hospitals with declining performance compared with those improving. In addition to the complexities of behavioural, socioeconomic and health factors of patients, readmission to hospital is also indicative of the quality, organisation and delivery of an integrated healthcare system (ie, primary care, home care, mental healthcare).22 Attention at national and provincial levels may now need to focus on addressing the system-level factors that contribute to hospital readmission.

Numerous national and provincial stakeholders occupy key roles in hospital performance improvement. Public reporting of hospital performance trending results provides valuable insight into current and future care domains that require concerted attention to address. National agencies charged with aiding hospital quality improvement (ie, the Canadian Foundation for Healthcare Improvement) would be able to use the findings of system-wide analyses such as this to identify best practices from across the country, and to accelerate their spread and scale. Performance information, such as the type used in this study, can facilitate numerous levers of change in the pursuit of quality improvement in health care.1

Provincial quality councils, hospital associations and ministries of health would be able to address more granular performance issues, such as particularly weakening performance within a select group of hospitals or type/sizes of hospitals. Accordingly, any provincial pay-for-performance schemes would benefit from periodic reviews to determine which hospital care domains have reached a performance plateau (ie, mortality and adverse events), and which should be newly incorporated (such as readmissions).

Community-small hospitals account for roughly half of all Canadian hospitals, yet these facilities generally do not meet the minimum number of patient cases in order to qualify for publicly reported performance results. Understandably, these rates, due to small counts, can fluctuate substantially from year to year, often show wide CIs and generally do not show significant trends over time. Nonetheless, for this sizeable group of Canadian hospitals to have meaningful inclusion in public and privately reported performance indicators, novel performance measurement techniques are required from health information agencies that account for these characteristics of smaller-sized hospitals.

The causality between public reporting of performance results leading to improved quality hospital care has been examined for over two decades; on balance, there is evidence that public reporting has spurred quality improvement activity at the hospital level,9 10 and measurable improvement on process and outcome indicators. However, such inquiries (including systematic reviews) have also concluded an insufficient sample-size, a lacking evidence-base and rigorous evaluation methods to be able to make a conclusive finding on the efficacy and impact of public performance reporting.23 24

This study has shown that there are clear trends in Canadian hospital performance, and that meaningful findings can be gleaned from secondary analysis of publicly reported performance data. Various stakeholders and administrators of hospital performance may find this type of summary analysis of benefit in their planning of quality performance improvement initiatives, and assignment of resources towards prioritised hospital care domains.

Conclusion

This study shows that meaningful findings on hospital performance can be gleaned when assessing publicly reported performance results. Analysis of Canadian hospital performance through multiple indicators shows a reduction of in-hospital mortality and in-hospital sepsis, but slight increases in rates of hospital readmissions. Subdividing the analysis by hospital size/type shows greater instances of improvement in teaching and larger-sized hospitals. The overall assessment of trends of improving and weakening as presented in this study can be used more systematically in monitoring progress. Further research is required on the complementarity of the indicators studied, especially the relationship between in-hospital mortality and hospital readmission trends.25