Prospects for comparing European hospitals in terms of quality and safety: lessons from a comparative study in five countries.

PURPOSE: Being able to compare hospitals in terms of quality and safety between countries is important for a number of reasons. For example, the 2011 European Union directive on patients' rights to cross-border health care places a requirement on all member states to provide patients with comparable information on health-care quality, so that they can make an informed choice. Here, we report on the feasibility of using common process and outcome indicators to compare hospitals for quality and safety in five countries (England, Portugal, The Netherlands, Sweden and Norway). MAIN CHALLENGES IDENTIFIED: The cross-country comparison identified the following seven challenges with respect to comparing the quality of hospitals across Europe: different indicators are collected in each country; different definitions of the same indicators are used; different mandatory versus voluntary data collection requirements are in place; different types of organizations oversee data collection; different levels of aggregation of data exist (country, region and hospital); different levels of public access to data exist; and finally, hospital accreditation and licensing systems differ in each country.
CONCLUSION: Our findings indicate that if patients and policymakers are to compare the quality and safety of hospitals across Europe, then further work is urgently needed to agree the way forward. Until then, patients will not be able to make informed choices about where they receive their health care in different countries, and some governments will remain in the dark about the quality and safety of care available to their citizens as compared to that available in neighbouring countries.

Introduction

There are many reasons why it is important to be able to assess and compare hospitals in terms of quality and safety both within and between countries, for example to help hospitals improve their services and for governments to understand the impact of their policies. As it becomes more common for patients to seek health care in different countries [1], the need for comparative information to enable choice is becoming more important. This has been recognized by the recent European Union (EU) directive on cross-border care [2], placing responsibility on countries to provide relevant information about the health care they provide to help patients make a choice based on the availability, safety and quality.

Here, we report on the challenges identified in developing a framework of available indicators and information that could be used to assess hospitals based on the quality and safety in five European countries, considering the issue involved and making recommendations for the future. The countries are England, Portugal, The Netherlands, Sweden and Norway; countries representing variation in important aspects of health care, for example, funding arrangements and health-care quality [3]. Health services research teams working in leading universities in these countries are participating in the EU-funded Quality and Safety in European Hospitals research study [4, 5], a study in 10 hospitals, 2 in each country [6] exploring the organizational and contextual factors affecting how hospitals implement quality improvement. The purpose of the selection process for the study was to find hospitals at different stages of the quality journey, rather than only those seen to be doing well.

Indicators available in each country to assess and compare hospitals

Ten commonly used quality and safety process and outcome indicators were proposed to identify hospitals with different levels of performance in each country, indicators that are widely regarded in the medical field as good practice (Table 1), including operating on hip fractures within 48 h. The research teams were asked whether the indicators were available; who had access; were they available for every hospital; how robust were the data (including definitions); and what other indicators were available?

Table 1

Quality and safety indicators available in each country highlighting difference between countries

Indicator	England	Portugala	Netherlands	Sweden	Norway	Indicators available in all countries
C-diff or MRSA rates	Yes, available for every hospital	Yes	Data are collected at a national level from all hospitals. Low rates, so information is not used to compare quality in hospitals	Yes, but data are not considered useful to compare hospitals because of the low rates	Data are not available at the hospital level.
Surgical site infection rates	Yes	Yes	Yes	Yes	Yes	Yes
Composite mortality rate	Yes HSMR	Yes	Yes Potentially avoidable mortality rate in proportion of actual mortality rate of the hospital	For some hospitals, not all	Not available in time for this process—now available
Specific mortality rates (AMI, stroke, CABG and AAA)	Yes	Yes	Yes	Yes	Some, but not all available in time for this process— all now available	Yes
Emergency readmission rates	Within a set period	Yes	Only for heart failure readmissions within 12 weeks	Some diagnoses, not all	Not available publicly (could be requested from the Norwegian Patient Register for a fee)
Third and fourth degree perineal trauma rate	Yes	No	Yes, but not available at the hospital level for comparison (voluntary data collection by obstetricians)	Yes	Not available in time for this process—now available
Caesarean section rate	Yes	Yes	Yes	Yes	Yes	Yes
Primary angioplasty rates	Yes	Yes	Yes	Yes	No, but could be requested from the Norwegian Patient Register on payment of a fee (see above)
Hip fractures treated in set time	Yes	Yes	Yes	Yes	Yes	Yes
24 h brain scan rate for stroke	Yes	Yes	No	No	No

AMI, acute myocardial infarction; AAA, abdominal aortic aneurysm; CABG, coronary artery bypass graft; HSMR, hospital standardized mortality indicator.

aIn Portugal, the information was available but was not available publicly by hospital.

Table 1 summarizes the responses received, with only three outcome indicators available in all countries: infection rates; a range of condition-specific mortality rates; and Caesarean section rates and only one process indicator: hip fractures treated in a set period. In general, these indicators were publicly available, but in Portugal, this was more complicated as described later in this paper, and in all cases, the researchers had to know where to look for the information from different organizations and websites.

The data for methicillin-resistant Staphylococcus aureus (MRSA) bacteraemia were not considered to be helpful in the Netherlands and Sweden, where the rates are very low as compared to the UK, where this indicator is considered very relevant and where until recently, the numbers have been much higher. It was apparent that although available in all countries, these indicators were not directly comparable because of differences in definitions (Table 2). For example, the surgical site infection rates in some countries are based on 1 day prevalence surveys, whereas in others, surveillance systems operate, which include every patient, and in each country, different surgical procedures are included, some are mandatory, whereas others are voluntary. The indicator of hip fracture treatment ranged from operating within 18 h in Sweden to 48 h in England and was only available for people aged over 65 in Norway.

Table 2

Examples of differences in the definition of indicators between countries

Indicator	Country	Definition
Surgical site infections	England	Surveillance methodology used. Hospitals are able to choose from 14 categories of surgical procedures. Data are collected on each patient who has a procedure in the category under surveillance. SSIs that meet standard case definitions are identified through active follow-up during the post-operative stay or through readmission
	Netherlands	Percentage of patients with post-operative site infection within 30 days after surgery with at least one of the following symptoms: pain, redness, local swelling and heat. (based on the definitions of European Centre for Disease Prevention and Control)
	Norway	The number of demonstrated hospital infections—of the four most common, urinary tract, lower respiratory, post-operative wound and septicaemia—at a given point of time, in relation to the total number of hospitalized patients at the same time. Post-operative wound infection data collected for: (i) heart surgery, (ii) Caesarean section, (iii) insertion of prosthesis in the hip joint, (iv) removal of gall bladder and (v) surgical intervention on colon
Caesarean section rate	England, Netherlands and Norway	The proportion of all deliveries that were a Caesarean section
	Sweden	Percentage of Caesarean sections at uncomplicated delivery, age 20–34 years
Hip fractures treated within a set period	England	The percentage of all patients with a fracture neck of femur as primary diagnoses that have a related procedure within 2 days.
	Netherlands	Percentage of patients with ASA 1 & 2 classifications operated within 1 day. Also, percentage of patients with ASA 3 & 4 classifications operated within 1 day
	Sweden	Percentage of hip fractures treated within 18 h of arrival at hospital
	Norway	Proportion of patients over 65 years with hip fracture, treated within 48 h after being hospitalized with needs of immediate help
24 h brain scan rate for stroke	England	Proportion of stroke patients who have a brain scan (CT or MRI) performed on the day of admission and within 1 day of admission
	Netherlands	Average time between the admission by TIA or Stroke and the completion of standard diagnostic tests—blood tests, brain imaging, ECG and imaging of the neck vessels.

ASA, American society of anesthesiologists; SSIs, surgical site infections; TIA, trans ischaemic attack.

The following sections provide more detail by country.

England

In England, data are collected centrally from each hospital organization through the Hospital Episode Statistics database. These data are analysed by Dr Foster Intelligence (a partnership between the UK Department of Health and a commercial provider of health-care information), and each year, a hospital guide is made available to the public, setting out the position of each hospital against a range of indicators [7]. Data on every patient with a blood stream MRSA infection are collected with aggregated figures for each hospital made available on a public website [8].

Portugal

Since 2001, the Health Minister has provided the Organisation for Economic Co-operation and Development (OECD) with process and outcome indicators similar to those in Table 1. However, it is not possible to disaggregate the data into regions and hospitals, making individual hospital comparisons impossible. In the last decade, various reforms, such as the merger of hospitals into clusters and changes in the mix of public–private ownership, have led to past performance data being lost in the merged groups, and some are only now producing group wide data. Furthermore, hospital organizations have autonomy to choose the processes and outcomes indicators that they use, and these are their exclusive property. In this context, access to good quality performance data was almost impossible in Portugal, and the indicators included in the Performance Evaluation Reports of Public Portuguese Hospitals [9] were used as a proxy. These reports of raw data have been published annually since 2005 by the National School of Public Health [10], but are not adjusted for age or case mix.

Netherlands

In the Netherlands, there was a wealth of publicly available data about quality in hospitals, predominantly derived from the information of the Dutch Health Care Inspectorate (IGZ) [11] and listed by a newspaper in an annual top 100 hospitals [12]. This is supported by further information of the Dutch Association of Hospitals (NVZ) [13] such as hospital mortality rates. Infection data are separately collected for all hospitals, but presented in a combined way on a health-care comparison website [14]. Process measures were chosen from a wide range of publicly available information, including pressure ulcers and eye examination for diabetes, measures collected by the hospitals, but not externally validated.

Sweden

Sweden has a tradition of collecting data nationally; however, more quality data have been collected over time by county councils (who manage hospitals in Sweden) and private health organizations. There are also many disease-specific quality registries. One of the key issues in Sweden is that most data so far have been published at a county council level, but there were enough data published at a hospital level to satisfy the needs of the project. An increasing number of hospital-specific quality indicators are now being published, and a process is ongoing between the Swedish Association of County Councils and Regions [15] and the National Board of Health and Welfare [16] to analyse and make available national information in an accessible way and for comparative purposes.

Norway

In Norway, hospital indicator data are reported by the hospitals to the Norwegian Patient Register and the Norwegian Institute of Public Health and published on a website called ‘Free Hospital Choice’[17]. The Norwegian Directorate of Health [18] publishes data on a growing number of national quality indicators in acute care. It is mandatory in Norway to report MRSA infections, but this information is not available for each hospital. The process indicators available were different here to other countries and included the number of patients placed on a bed in a corridor, bathroom, laundry room or living room at 7 a.m. and the proportion of patient discharge letters transferred within 7 days to the general practitioner.

Hospital accreditation or licensing schemes in each country

Table 3 describes the hospital licensing, accreditation or similar schemes in each country showing that this type of information is not available to be used to assess or compare the quality of hospital care in Europe.

Table 3

Accreditation or licensing systems for hospitals in each country

Country	Accreditation/licensing schemes for hospitals
England	Care Quality Commission licensing scheme [19] statutory requirement with ‘Annual Health Check’ of all hospitals producing publicly reported quality ratings for each hospital organization [27].
Portugal	No national accreditation scheme, but one is in the validation phase—some hospitals voluntarily accredited with Joint Commission [28] and the Health Quality Service (formerly Kings' Fund) [29]
Netherlands	Voluntary accreditation, so not all hospitals are involved. Accreditation undertaken by the Netherlands Institute for Accreditation in Healthcare [30] based on the Baldridge Award [20]. Some hospitals use other accreditation systems, i.e. ISO and INK (Institute for Dutch Quality).
Sweden	No formal hospital accreditation system. ‘Baldridge’ type award scheme in operation [20]
Norway	No formal hospital accreditation system. A small number of institutions providing clinical services are certified according to ISO 9000 by accredited certification bodies.

ISO, International Organization for Standardization; INK, Instituut Nederlandse Kwaliteit.

Only in England is there is a statutory licensing scheme for all hospitals [19]. In the other countries, accreditation or certification is voluntary, and quality award schemes are often based on the Baldridge Award [20].

Challenges and potential solutions

These findings highlight the differences between countries in the data and information available about the quality and safety of hospital care. In England, the Netherlands and Norway, various hospital data are submitted to national organizations that conduct analysis and make them available to the public by hospitals. In Sweden, the level of aggregation of the data is stronger at the county council level and in the disease-specific quality registries. In Portugal, the information that is collected at the national level is not easy to obtain by region or hospital. In all countries, data on specific procedures or conditions are collected by organizations, including medical societies, but not all make this available to the public.

The differences in the indicators collected by countries highlight the different policy concerns of those countries. In England, for example, data are available on MRSA infections for each hospital, whereas in Norway, no MRSA hospital level data are available, but data are available on the proportion of patients placed in a bed in a corridor or living room at 7 a.m. Each country also has a different approach to the assessment of hospitals in terms of accreditation or licensing with most having voluntary schemes. Only England has statutory licensing and an ‘annual health check’ of hospitals.

Discussion/conclusions

The increasing movement of people across borders for health care, whether by choice or circumstance, is driving the need for more transparency about the quality and safety of health care in European hospitals. The new EU directive on patients' rights [2] places a responsibility on member states to ‘provide relevant information to help individual patients to make an informed choice, including on treatment options, on the availability, quality and safety of the health care they provide’[21].

In recognition of the requirement for all citizens to have access to high-quality, safe health care, the EU has been engaged in a number of projects to support hospitals in this quest and to find ways of monitoring and measuring this. These include the SIMPATIE project [22] that reported in 2007 on the development of indicators and outcome measures and a vocabulary for patient safety across Europe. The OECD [23] and WHO [24] have also funded projects to develop indicators for hospital quality and safety. However, the findings here show that despite all this important work, there are very little data available on the quality and safety of hospital care that is comparable between these five countries in Europe, and we anticipate that this is likely to be the case across the EU and in countries outside Europe, where citizens also travel for health care.

In 2008, the European Observatory on Health Systems and Policies [25] reported that the EU needs to answer the question of how citizens of Europe can be assured that they will receive high-quality care if they need health care beyond their national frontiers. Both OECD [26] and the SIMPATIE [22] project have recommended that the quality of administrative data for health services in the EU needed to improve. Our study indicates that to date these issues have not been addressed and that further work is urgently needed to find a way forward if patients are to have any information to make choices between health-care providers when they travel for treatment in Europe. Clearly, the solution will need to be of low cost with the information easily kept up-to-date. The first step may be for those countries without comparative data about the safety and quality of their hospitals to be required to assemble available information from hospitals within their borders, to analyse this information and place it in the public domain. In the longer term to provide information for comparison between countries, the solution may be to agree to a small set of well-defined indicators to be collected by each hospital organization in Europe with some common standards and methods for assessing these. Alternatively, the solution may be a patient-led hospital equivalent of an internet hotel comparison website such as ‘Trip Advisor’ (www.tripadvisor.com). In the meantime, patients travelling across the borders for their health care in Europe will not be able to make informed choices about where they receive their health care on any other basis than access, and many governments in Europe will remain in the dark about the quality and safety of care currently available to their citizens in their hospitals as compared to that available elsewhere.

Funding

The research leading to this paper received funding from the European Community's Seventh Framework Programme (FP7/2007–2013) under grant agreement no. 241724. The Imperial College and the Kings College Centre for Patient Safety and Service Quality were both supported by the UK National Institute of Health Research. The researchers were independent from the funders. Funding to pay the Open Access publication charges for this article was provided by European Community's Seventh Framework Programme (FP7/2007–2013) under grant agreement no. 241724.

Conflict of interest statement

All authors have completed the Conflict of Interest form, available on request from the corresponding author and declare that there is no conflict of interest with regard to this work.