The Effect of ISO 9001 and the EFQM Model on Improving Hospital Performance: A Systematic Review.

CONTEXT: This study aimed to explore the effect of the International Organization for Standardization (ISO) ISO 9001 standard and the European foundation for quality management (EFQM) model on improving hospital performance. EVIDENCE ACQUISITION: PubMed, Embase and the Cochrane Library databases were searched. In addition, Elsevier and Springer were searched as main publishers in the field of health sciences. We included empirical studies with any design that had used ISO 9001 or the EFQM model to improve the quality of healthcare. Data were collected and tabulated into a data extraction sheet that was specifically designed for this study. The collected data included authors' names, country, year of publication, intervention, improvement aims, setting, length of program, study design, and outcomes.
RESULTS: Seven out of the 121 studies that were retrieved met the inclusion criteria. Three studies assessed the EFQM model and four studies assessed the ISO 9001 standard. Use of the EFQM model increased the degree of patient satisfaction and the number of hospital admissions and reduced the average length of stay, the delay on the surgical waiting list, and the number of emergency re-admissions. ISO 9001 also increased the degree of patient satisfaction and patient safety, increased cost-effectiveness, improved the hospital admissions process, and reduced the percentage of unscheduled returns to the hospital.
CONCLUSIONS: Generally, there is a lack of robust and high quality empirical evidence regarding the effects of ISO 9001 and the EFQM model on the quality care provided by and the performance of hospitals. However, the limited evidence shows that ISO 9001 and the EFQM model might improve hospital performance.

1. Context

There is an increasing concern about the quality and safety of healthcare services. Quality management systems are broadly used in healthcare (1-3). Many hospital departments have applied a quality management system to improve the efficiency and quality of healthcare services (4, 5). One widespread approach that is used is the framework defined by the International Organization for Standardization (ISO), the ISO 9001 standard. ISO was founded in 1947 to provide standardization for technical specifications for products traded in the international marketplace. The ISO 9001 family of quality management system standards can be applied to any type of organization seeking to improve the quality of the production of its goods or services. This standard represents an international agreement on goods management performance, which helps ensure that employees constantly deliver the product or service that meets the quality requirements of the organization’s clients, continuously enhance the organization’s performance, and improve client satisfaction (6-8).

Another international model that is used to conduct quality improvement processes is the European foundation for quality management (EFQM) Excellence Model. EFQM was founded in 1988 with the approval of the European commission. The EFQM Excellence Model is a multidimensional model based on eight concepts of excellence. This model has been well accepted in the public sector in Europe in recent years. Its non-prescriptive framework is based on nine criteria (9). Five of the criteria refer to ‘enablers’ (leadership, people, policy and strategy, partnership and resources, and processes) and four of the criteria refer to 'results’ (people results, customer results, society results, and key performance results) (10). The EFQM Excellence Model is a valuable tool to help organizations recognize quality management gaps and monitor their improvement. It emphasizes the idea of self-assessment and the detection of strengths and weaknesses using the criteria’s guiding principles. This framework is widely used in the industry, as well as in the healthcare sector (11, 12).

There are several systematic reviews regarding the effect of quality improvement methods in healthcare. Some of the published studies in those systematic literature reviews have studied the effectiveness of Six Sigma and Lean Six Sigma (13-15). Other reviews have looked at broader system-wide quality improvement (QI) models or collaborations (16) and have highlighted their context-dependent nature, the degree of overlap between models, and the need for an effective organization-specific implementation method and infrastructures for success (17, 18). Others have examined the literature concerning the effectiveness of QI methodologies, such as plan-do-check-act (PDCA) cycles, statistical process control (SPC) or statistical quality improvement (CQI), total quality management (TQM), Six Sigma, and Lean Six Sigma, in improving the quality of care for surgical patients (19). We did not find any systematic review about the effect of the ISO 9001 standard and the EFQM model, which is the subject of this review. The aim of this systematic review was to examine the literature concerning the effects of ISO 9001 and the EFQM model on improving hospital performance.

2. Evidence Acquisition

The PubMed (1994 to September 2013), Embase through Scopus (1997 to September 2013), and Cochrane Library (Issue 3, 2013) databases were searched. In addition, Elsevier (1996 to September 2013) and Springer (1835 to September 2013) were searched as the main publishers in the field of health sciences.

The search terms were devised to cover hospital performance improvement as well as the names of and synonyms for ISO 9001 or International Organization for Standardization, EFQM or European Foundation for Quality Management. These terms included: ‘‘hospital’’, ‘‘hospital performance’’, ‘‘quality improvement’’, ‘‘quality management, ‘AND’ with each of the following terms: ‘‘ISO 9001’’, ‘‘International Organization for Standardization’’, ‘‘EFQM’’, ‘‘European Foundation for Quality Management’’ in the title or abstract.

We included empirical studies with any design that had used ISO 9001 or the EFQM model to improve hospital performance. Studies that had addressed the use of an intervention, such as ISO 9001 or the EFQM model, and that had assessed the effect of that intervention on hospital performance over a specific period of time, were included. Theoretical studies, editorials, letters, opinions, audits, and reviews were excluded. Studies that assessed healthcare professionals’ perceptions about ISO 9001 or the EFQM model were excluded from this present study. We also excluded self-assessment papers that only addressed hospital scores.

All the papers that were retrieved from the search strategy were imported into an Endnote database. Their titles, abstracts and, if necessary, the full texts were then scanned by a reviewer and checked by a second reviewer against the inclusion criteria. The statistical method used for this analysis was Cohen’s Kappa coefficient, which was calculated as 0.7.

Data were collected and tabulated into a data extraction sheet that was specifically designed for this study. The collected data included: authors’ names, country, year of publication, intervention, improvement aims, setting, length of program, study design, and study outcomes.

The quality assessment for non-experimental studies has not been well developed (20). Therefore, a combination of available checklists was used to design a list of criteria that was applied to appraise the quality of the included studies (20, 21). The following criteria were used to appraise the quality of the included studies: the adequacy of the description of methods, the appropriateness of the research methods to the study question, the quality of the data collection, the quality of the data analysis, and the quality of the data presentation.

3. Results

A descriptive synthesis of the results was performed giving consideration to the risk of bias and the quality of the studies. The search strategy yielded 121 studies. After removing the duplicates, 96 of these studies were excluded after checking the titles and abstracts. The full texts of the 25 studies were checked and seven of them met the inclusion criteria (Figure 1). Three studies described the EFQM model and four were concerned with ISO 9001 (Table 1).

Figure 1.

Summary of the Search Strategy

Table 1.

Summary of the Included Studies

Authors	Year	Country	Setting and Length of Intervention	Study Method	Intervention	Study Aim	Outcomes
Sanchez et al. (22)	2005	Spain	Hospitals 1995 up to 2003	Observational study	EFQM model	Quality, equity, efficiency	Satisfaction increased; mean length of stay and delay on the surgical waiting lists decreased
Leigh et al. (23)	2005	UK	Hospital 2002 up to 2004	Survey	EFQM model	Qualified nurses	Confidence, competence and retention of newly nurses increased
Vallejo et al. (24)	2007	Spain	Hospital Psychiatric ward 2003 up to 2005	Quasi- experimental	EFQM model	Quality	No. of admissions increased; mean length of stay and re-admission rate decreased
Rodriguez-Cerrillo et al. (25)	2012	Spain	Hospital 2007 up to 2008	Quasi- experimental	ISO 9001	Patient satisfaction; mean stay; return to hospitals	Patient satisfaction increased; percentage of unscheduled returns decreased
Vitner et al. (26)	2011	Israel	Hospital NICU unit 2006 up to 2007	Quasi- experimental	ISO 9001	Medical and administrative services	Admission process yields and service satisfaction improved
Beholz et al. (27)	2005	Germany	Hospital 2000 up to 2003	Quasi- experimental	ISO 9001	Control of costs, customer satisfaction	Cost containment and satisfaction improved without loss in quality of medical treatment
van den Heuvel et al. (28)	2005	The Netherlands	Hospital 1998 up to 2001	Quasi- experimental	ISO 9001	Improve the system	Improvement in rate of patient safety and policy and management category

Moreover, three studies were undertaken in Spain, one in the United Kingdom (UK), one in Germany, one in the Netherlands, and one in Israel. Two studies were published after 2010. Five studies had a quasi-experimental design; one study had a survey design and one was an observational study. Four articles reported a follow up period greater than two years, and all of the included studies were performed in hospitals (22-26).

Furthermore, three studies addressed the degree of patient satisfaction. Sanchez et al. (22) reported that the patient satisfaction rate increased to 96.2% and 93.1%, respectively, in outpatient clinics and the emergency department with a four-year follow up; however, the satisfaction rate decreased in inpatients after applying the EFQM model. Rodriguez-Cerrillo et al. (25) also reported that patient satisfaction increased from 92% in 2008 to 98.8% in 2010 after applying the ISO 9001 standard. Vitner and colleagues (26) also reported a service satisfaction improvement ranging between 4.8 and 5 (based on a scale of 1 to 5) in 66% of the items in the patients’ survey after implementing the ISO 9001 standard.

Additionally, three studies addressed the average length of a hospital stay. Sanchez et al. (22) showed that implementing the EFQM guidelines resulted in a reduction in the mean length of hospital stays from 6.1 days in 2000 to 5.9 days in 2003 in acute care hospitals with a four-year follow-up. Vallejo and colleagues (24) also observed a reduction in the mean length of hospital stays from 14.8 in 2003 to 13.8 in 2005, with a three-year follow up after implementing EFQM measures. Rodriguez-Cerrillo et al. (25) reported no change in the mean length of hospital stays in a three-year follow up after implementing ISO 9001.

Two studies addressed the delay in receiving health services. Sanchez et al. (22) reported a reduction in the amount of time a patient has to remain on the surgical waiting list (22). The average delay decreased from 57.1 days in 2000 to 53.8 days in 2003 after using the EFQM model. Vallejo and colleagues (24) found a decrease in the number of medical records that had a delay in reporting from 1 in 2003 to 0 in 2005 after implementing the EFQM model.

Moreover, two studies addressed the hospital re-admission process. Vallejo and colleagues (24) found that emergency re-admission was reduced from 20 in 2003 to 12 in 2005, with a three-year follow up, after implementing the EFQM model. Rodriguez-Cerrillo et al. (25) reported that the percentage of unscheduled returns to the hospital decreased from 7% to 3% after using ISO 9001.

Finally, two studies addressed the hospital admission process. Vallejo and colleagues (24) showed that the number of admissions increased from 282 in 2003 to 297 in 2005, with a three-year follow up, after using the EFQM model. Vitner and colleagues (26) stated that the number of admission improved to 78%, 19 months after using ISO 9001.

Regarding the remaining outcomes, only one study was found for each of those outcomes, and the results of these studies are presented below.

Rodriguez-Cerrillo et al. (25) reported no medical equipment failures and an improvement in the external suppliers' performance. They found that, after implementing ISO 9001, 100% of the patients that required oxygen or aerosol therapy had access to the required equipment in their homes by the time they returned from the hospital, while this was not possible two years before. They also showed that, after implementing ISO 9001, in 97% of the cases protocols were followed, six medication-related incidents were detected, and all patient problems requiring an urgent consultation were satisfactorily resolved.

Leigh et al. (23) observed that confidence, competence, and retention of newly qualified nurses improved and the number of newly qualified nurses leaving in the first 18 months of their employment was reduced after implementing the EFQM model.

Sanchez et al. (22) reported that the duration of primary care medical consultations increased from 7.4 min in 2000 to 8.5 min in 2003, with a four-year follow up. Moreover, after using the EFQM model, the percentage of patients waiting less than one month for specialized care decreased from 63.1% in 2000 to 61.1% in 2003, the prevalence of patients admitted with an infection episode declined from 6.7% in 2000 to 6.6% in 2003, and the percentage of minimally invasive tumors detected in the early detection breast cancer program decreased from 43.3% in 2000 to 40.9% in 2003.

In addition, after using ISO 9001, van den Heuvel et al. (28) reported that patient safety increased from 38% in 1998 to 63% in 2001, and the rate of improvement in the policy and management category was 58% in 2001.

Beholz et al. (27) reported that costs for medical goods decreased by € 187.36 from 2001 to 2003, representing a 6.1% drop; and, from 2001 to 2002 the total absolute savings in laboratory costs was 35.2%. The 32-day mortality rate for all operations showed a slight increase in 2001, reaching 4.2%. In 2002 and 2003, hospital mortality decreased to 3.7%. After using ISO 9001, returns of the questionnaire-based evaluation of satisfaction of cooperating cardiologists continuously increased from 57.1% in 2001 to 65.4% in 2002 and 70.6% in 2003.

4. Conclusions

The EFQM model and the ISO 9001 standard are becoming the most recommended methods and guidelines for creating and continuing good quality clinical services; however, there has been a debate about the advantages and disadvantages of these models (29, 30). These models have been applied successfully to different aspects of hospital performance. This review included studies that assessed the ISO 9001 standard and the EFQM model in order to consider the evidence from the literature to determine how these quality management guidelines might apply to this field. We searched the main databases and publishers in the field of health sciences and tried to include articles covering the major outcomes in order to test the effectiveness of these models from first principles rather than from simple observations.

The result showed that using the EFQM model and ISO 9001 in hospitals was associated with an improvement in the satisfaction rate of outpatient clinics and emergency departments, a reduction in the mean length of stay in acute care hospitals, reduced delays on the waiting list, a decrease in emergency re-admissions, a reduction in the percentage of unscheduled returns, improvement in confidence, competence, and retention of newly qualified nurses, and a reduction in the number of newly qualified nurses leaving in the first 18 months of their employment.

Furthermore, the use of these models was also associated with a decrease in the prevalence of patients admitted with an infection episode, a decrease in the occurrence of medical equipment failures (zero failures), an improvement in the external suppliers’ performance, adherence to protocols in 97% of the cases, the detection of six medication-related incidents, an improvement in admission processes, an increase in patient safety and an improvement in the policy and management rate , a reduction in costs for medical goods and laboratory services, a decrease in 32-day mortality for all operations, and a continuous increase in the return of the questionnaire-based evaluation of satisfaction of cooperating cardiologists.

There are a number of limitations when looking for the results of using ISO 9001 and the EFQM model to assess hospital performance. Only a few of the studies met all of the inclusion criteria for each model. Most of the studies included hospitals that had utilized these models and reported that they can be effectively used without assessing their results, or they presented a self-assessment instead of specifically setting up and designing studies to test the effectiveness of these models from first principles. Many of the quality improvement models require noticeable data collection and staff must be trained in how to implement the model; most models also require a degree of statistical analysis. Obviously, if using these models can result in significant cost savings, there is a good reason to invest in staff training and technology to help support this process. However, without investing in these projects, sufficient evidence is unavailable to suggest sector-wide implementations.

Five studies had a quasi-experimental design and indicated outcomes before and after applying ISO 9001 and the EFQM model, so their results are more reliable than the results of studies with a survey design or an observational design. In the observational studies, we could not correlate the progress of the outcomes to use of the quality improvement models. On the other hand, in the survey design, the results could be subjective.

Studies with a rigorous, low-risk methodology or design are necessary when it comes to reporting results related to the use of these models. Randomized controlled trials (RCTs) are needed or, at least, studies with a non-randomized time series design, in order to truly indicate the important outcomes and potential harms of using these models. We searched the main databases and publishers in the field of health sciences, although this might not guarantee that we have found all the pertinent papers in this field; this is a limitation of any systematic review. These studies showed that the evidence underlying quality improvement is positive, but limited, and the effects cannot be predicted with great certainty, so the results should be interpreted and used with caution.

Generally, there is a lack of robust and high quality empirical evidence regarding the effect of ISO 9001 and the EFQM model on the quality and performance of hospitals. However, the limited evidence shows that, to some extent, ISO 9001 and the EFQM model might improve hospital performance. In addition, no negative evidence was found about the effect of ISO 9001 and the EFQM model on hospital performance.