Emilia-Romagna Regional Blood System accreditation as an example of improvement through application of specific requirements: a retrospective analysis.

BACKGROUND: Institutional accreditation in Italy represents the license given by a region to a public or private facility to provide services in the name and on behalf of the National Health Service. This study aims to evaluate the improvement of the Emilia-Romagna Regional Blood System and to highlight its unresolved issues, analysing non-conformities observed during accreditation and maintenance inspections between 2013 and 2018.
METHODS: All the Emilia-Romagna Regional Blood facilities were invited to participate in this study voluntarily and anonymously. Participants had to access a web application that we developed specifically. For each of the three inspections evaluated in this study, they had to enter data about the state of their organisation branches and non-conformities observed by regional inspectors. All data entered were finally exported from the web application database and analysed with spreadsheets. Statistical analysis was performed using Wilcoxon signed-rank test with continuity correction.
RESULTS: 17 structures took part in the study, with a total of 174 organisation branches. The number of branches changed over the years because of new openings and closures due to reorganisations or non-conformities that were too difficult to correct. Inspectors observed 2381 non-conformities (291 structural, 611 technological and 1479 organisational). As a result of accreditation inspections and consequent improvement actions, non-conformities were reduced by 88%. The most frequent non-conformities concerned the management software and the transportation of blood and blood components.
CONCLUSION: An improvement in the Emilia-Romagna Regional Blood System over time is evident: institutional accreditation certainly pushed it to change and overcome its problems to comply with specific requirements. The remaining non-conformities after the three inspections were mostly organisational and management software was the most critical issue. Despite these non-conformities, all currently active structures are accredited and guarantee high standards of quality and safety of products and services. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Introduction

Institutional accreditation in Italy represents the license given by a region to a public or private facility to provide healthcare services in the name and on behalf of the National Health Service. It aims to guarantee the same quality, safety and efficacy of healthcare throughout the country. In the case of the Blood System, this means ensuring to all donors that blood and blood components are collected in a safe and appropriate way for actual needs, and ensuring to all potential recipients that blood components are always available, effective and free of transmissible pathogens.1–12

Accreditation is issued on verification of compliance with both general and specific requirements.

General requirements are common to all types of healthcare facilities. Their purpose is to check for documented evidence of the quality and safety of organisations, such as a Quality Manual, an annual plan of activities, a documented information control system, a periodic review plan and improvement projects.

Specific requirements are different for each type of health facility and belong to three categories: structural, concerning the premises that host the activities; technological, concerning the devices and materials used and organisational, concerning the quality management system and personnel. For example, according to the specific requirements of the Blood System: (1) premises must be suitable for each activity (blood collection, blood component production and conservation, treatments, etc); (2) automatic tilting scales must be used for blood collection; (3) suitable devices must be used for blood and blood component conservation and transportation.

The accreditation process consists of three main phases: analysis of the documented evidence of the quality management system of the applicant structure; on-site visit by accredited inspectors and preparation and transmission of the inspection report.

Although the National Health System includes all public health organisations and the Ministry of Health issues national decrees and guidelines, such as the Essential Levels of Healthcare, regions are responsible for many aspects of the health service. Hence, the existence of regional accreditation models, which not only have much in common, but also have distinctive characteristics.13–17

In the Blood System, on the other hand, in the latest years, there has been a strong national standardisation effort in compliance with several European Directives and Guidelines.18–53 This is because transfusion medicine is indeed a critical field, since it involves substances of human origin, both blood components and plasma-derived medicinal products.

The Blood System includes Blood Establishments and Blood Collection Units. Blood Establishments are public hospital units that collect, process, store and distribute blood and blood components. Blood Collection Units are organisations run by non-profit blood donor associations that collect blood and blood components and deliver them to a Blood Establishment. They must have a formal agreement with their reference Blood Establishment and operate under its technical control. Both are structured in one or more Organisation Branches, locations where one or more activities (from collection to distribution) are carried out under management control.29 37

For simplicity, Blood Establishments, Blood Collection Units and Organisation Branches are hereinafter referred to as ‘Hospitals’, ‘Associations’ and ‘Branches’.

This study aims to evaluate the improvement of the Emilia-Romagna Regional Blood System as a whole and to highlight its unresolved issues, analysing non-conformities (NCs) observed during accreditation and maintenance inspections of Hospitals and Associations between 2013 and 2018.

Methods

Participants

We initially presented this project to the Regional Blood System Quality Workgroup made up of all the Quality Managers of Hospitals and Associations.

The Regional Blood Centre then invited the Emilia-Romagna Hospitals and Associations to voluntarily and anonymously participate in this study. Participants authorised the use of the data collected for publication in aggregate and anonymous form.

Subject of the study

We reviewed the first accreditation inspection (2013–2014) and the two following maintenance inspections (2015–2016 and 2017–2018). We evaluated the state of Branches (active, newly opened and closed) and NCs observed by the regional inspectors for each of these assessments. The checklist of specific requirements is defined in the Agreement between State and Regions of 16 December 2010,41 which, in our region, was implemented in the Regional Committee Resolution number 819 of 13 June 2011.42 During the inspections, general requirements, common to every health organisation, were also evaluated, but they were not the subject of this study. As for NCs, we specify that in the Emilia-Romagna accreditation model, the level of compliance to each requirement in the checklist can be expressed with a scale of four values: ‘YES’ (compliant), ‘yes’ (partially compliant), ‘no’ (non-compliant—work in progress), ‘NO’ (non-compliant). Requirements can concern an entire organisation or its Branches and can have a value of ‘NA’ if not applicable to that organisation or Branch.54 For this study, we considered only requirements with ‘no’ or ‘NO’ values in the checklist, which, therefore, represent NCs.

Data entry

Participants had to access a web application that we developed specifically. For each of the three inspections evaluated in this study, it was necessary to enter: (1) the number of active branches; (2) the number of branches closed or newly opened since the previous inspection; (3) the new values of NCs observed in the previous inspection (‘YES’, ‘yes’, ‘no’, ‘NO’ or ‘NA’); (4) new NCs (‘no’ or ‘NO’). These values had to be drawn from the inspection reports received by the Regional Health and Social Agency, the technical body that assesses compliance with the accreditation requirements on behalf of the region.

To maintain the anonymity of the participants, they were marked only with their type of facility (hospital or association) and a unique alphanumeric code generated at the first access to the web application. Branches were marked with a progressive number generated by the web application and an optional alphanumeric code, chosen by the participants to distinguish them more efficiently when entering data. Participants could receive technical support from the administrators only through a messaging system in which they were identified by their unique alphanumeric code.

Data were collected between 3 May 2019 and 31 January 2020. Participants could choose to enter data in multiple sessions, each time accessing the web application with their unique alphanumeric code.

Data analysis

All data entered were exported from the web application database and analysed with spreadsheets. The NCs were grouped by the category of their requirements: structural, technological and organisational. The analysis was made both overall and separately for Hospitals and Associations to highlight possible peculiarities of each type of organisation.

Statistical analysis was performed using R software (V.4.1.0); we used Wilcoxon signed rank test with continuity correction to compare the number of NCs in each branch in the first (2015–2016) and second (2017–2018) maintenance inspections versus the number of NCs in each branch in the first accreditation inspection (2013–2014).

Patient and public involvement

It was not appropriate to involve patients or the public in the design, or conduct, or reporting or dissemination plans of our research, since it was based on the analysis of institutional inspections reports.

Results

Seventeen organisations took part in the study, 11 (65%) out of 11 hospitals in the region (100%) and 6 (35%) out of 9 associations in the region (67%).

Participating structures had a total of 174 branches, 42 belonging to hospitals (24%) and 132 belonging to associations (67%).

Organisation branches

The number of branches changed over the years because of new openings and closures due to reorganisations or NCs that were too difficult to correct. Table 1 shows this data in detail.

Table 1

Organisation branches

Inspection	Active branches			Newly opened branches			Closed branches
	H	A	Total	H	A	Total	H	A	Total
2013–2014	41	130	171	–	–	–	–	–	–
2015–2016	41	111	152	0	1	1	0	20	20
2017–2018	41	108	149	1	1	2	1	4	5

A, associations; H, hospitals.

The number of branches for participating hospitals remained essentially unchanged from 2013 to 2018; the number of branches for participating associations went from 130 in 2013 to 108 in 2018 (−17%).

Non-conformities

Table 2 contains the NCs grouped by category (structural, technological or organisational), type of structure (hospital or association) and year of inspection. There were many more NCs for associations, but it must be considered that they also had many more branches than hospitals; therefore, to better compare the two types of structures, in table 3, we report the mean number of NCs per branch.

Table 2

Non-conformities by category, type of structure and year of inspection

Inspection	Structural NCs			Technological NCs			Organisational NCs			All NCs
	H	A	Total	H	A	Total	H	A	Total	H	A	Total
2013–2014	95	160	255	72	409	481	164	905	1069	331	1474	1805
2015–2016	16	11	27	9	78	87	69	178	247	94	267	361
2017–2018	9	0	9	26	17	43	46	117	163	81	134	215
Total	120	171	291	107	504	611	279	1200	1479	506	1875	2381

A, associations; H, hospitals; NCs, non-conformities.

Table 3

Non-conformities by category, type of structure and year of inspection (mean number of NCs per organisation branch)

	Structural NCs			Technological NCs			Organisational NCs			All NCs
	H	A	Total	H	A	Total	H	A	Total	H	A	Total
2013–2014	2.3	1.2	1.5	1.8	3.1	2.8	4.0	7.0	6.3	8.1	11.3	10.6
2015–2016	0.4	0.1	0.2	0.2	0.7	0.6	1.7	1.6	1.6	2.3	2.4	2.4
2017–2018	0.2	0.0	0.1	0.6	0.2	0.3	1.1	1.1	1.1	2.0	1.2	1.4

A, associations; H, hospitals; NCs, non-conformities.

Overall, the inspectors observed 2381 NCs, 21% of which for hospitals and 79% of which for associations. However, the number of NCs per branch was comparable between hospitals and associations since 76% of branches belonged to associations.

Following the accreditation inspections and the consequent improvement actions, the NCs were reduced by 88% overall, more for associations (−91%) than for hospitals (−76%).

Over 50% of NCs were organisational, both for hospitals and for associations. Hospitals were affected by structural NCs more than associations (24% vs 9%), especially in the first 2-year period.

The structural NCs were overall reduced by 97% (89% already by the second inspection). While associations resolved all of their structural NCs, hospitals still had some of these problems as of 2018.

The technological NCs were overall reduced by 91% (82% already by the second inspection). We highlight a significant increase in the number of technological NCs for hospitals in the 2017–2018 period (26 NCs), mostly related to devices for the transportation of blood and blood components.

The organisational NCs were overall reduced by 85% (77% already by the second inspection). Their resolution rate was the lowest of the three categories of observed NCs.

Table 4 shows the most frequent NCs that most affected the Regional Blood System, listed separately by hospitals and associations.

Table 4

Most frequent non-conformities

Structure	Req.	Cat.	Subject	2013–2014	2015–2016	2017–2018	Total
Hospitals	11 063	O	Management software	14	10	13	37
	11 027	T	Devices for transportation	19	4	9	32
	11 022	O	Validation of transportation	14	11	5	30
	11 026	T	Devices for transportation	12	3	14	29
	11 221	O	Validation of storage	13	5	6	24
Associations	11 285	O	Management software	81	65	64	210
	11 267	T	Devices for transportation	127	38	17	182
	11 271	O	Validation of transportation	127	17	17	161
	11 348	O	Validation of transportation	127	17	17	161
	11 257	T	Scales for blood collection	115	17	0	132

Cat, category of requirement; O, organisational; Req, requirement number; T, technological.

The NCs relating to management software were the most critical since they remained substantially unchanged over time. NCs concerning the transportation of blood and blood components and its validation were considerably reduced, although there was still room for improvement. NCs concerning scales for blood collection were eliminated thanks to public tenders carried out by groups of local health authorities (the so-called wide areas).

Table 5 shows the NCs remaining after the last inspection in 2018, grouped by category (total and mean number per branch).

Table 5

Remaining non-conformities as of 2018

	Remaining NCs as of 2018			Remaining NCs as of 2018(mean number per branch)
Category	H	A	Total	H	A	Total
Structural	9	0	9	0.2	0.0	0.1
Technological	26	17	43	0.6	0.2	0.3
Organisational	46	117	163	1.1	1.1	1.1
Total	81	134	215	2.0	1.2	1.4

A, associations; H, hospitals; NCs, non-conformities.

Statistical analysis

We used a paired difference test, so we had to discard 28 of the 174 branches that were not open on all of the three inspections (3 newly opened and 25 closed). We, therefore, made calculations on the remaining 146 branches.

If compared with the first accreditation inspection (2013–2014), the number of NCs in branches was significantly lower both in 2015–2016 (p<2.2e-16) and in 2017–2018 (p<2.2e-16). We repeated the same analysis stratifying according to the type of facility (hospitals and associations), the category of NCs (structural, technological and organisational) and both criteria simultaneously. In all these cases, we got p values ≤0.001. This data show that there was a statistically significant improvement already by the second inspection, not only for the whole regional Blood System but also both for hospitals and associations and in every category of NCs.

We finally tried to perform the same analysis on organisations instead of branches, but the sample became thereby too small to give significant p values for all of the above comparisons. In particular, comparisons concerning the six associations gave p values ranging from 0.05 and 0.07.

Discussion

Current evidence about the impact of accreditation

In 2020, Araujo et al conducted the most recent systematic review about the impact of accreditation on hospital outcomes. They stated that previous literature reviews gave controversial results: not all of them found evidence to support a link between hospital accreditation and measurable changes in healthcare quality indicators. This could be explained in part by the complexity of accreditation programmes and in part by methodological differences between the selected studies. Therefore, they chose a more systematic approach, analysing only studies that quantitatively examined differences in health quality indicators before versus after hospital accreditation or among accredited versus non-accredited hospitals. In particular, they watched for changes in seven healthcare quality dimensions and concluded that accreditation may have a positive impact on efficiency, safety, effectiveness, timeliness and patient-centeredness. In turn, only one study analysed the impact on access, and no study had investigated the impact on equity dimension yet.12

Compliance with requirements and actual improvement

We too followed a quantitative approach in a sense, since we analysed the number of NCs over time. However, we believe that the accreditation process does not only consist in complying with a list of requirements but also has led to an actual improvement of the Blood System quality. Here are some examples: (1) Electronic scales allow for complete traceability of blood units volume, collection time and operators involved. Thanks to this, organisations can quickly identify non-conforming units and verify that operators maintain their competence, (2) continuous monitoring of transportation temperature guarantees the safety and efficacy of blood components, (3) the adjustment of premises for blood collection has made it possible to define paths ensuring the privacy and security of donors, operators and products.

Compliance maintenance over time

Compliance is maintained on a daily basis, even outside the inspections, through continuous monitoring of the activity with: (1) process and outcome indicators; (2) internal audit following relevant NCs, changes, developments, and in any case, at least every year. There are, in fact, some organisational requirements that explicitly dictate: (1) periodic management reviews to highlight quality problems and the need for corrective and preventive actions and (2) internal audits to verify the compliance with current legislation, standards and procedures.

Differences between hospitals and associations

Hospitals had already undergone accreditation since the early 2000s but limited to general requirements. Thanks to this, they had already developed a quality management system and were more ready than associations to meet specific organisational requirements. In fact, in the first 2-year period, hospitals had four organisational NCs per branch compared with seven for associations. At the end of the analysed period, however, both types of structures had 1.1 organisational NCs for branch: this testifies for the commitment of the associations and a solid organisational synergy between the two kinds of structures, which is one of the objectives of the Blood System accreditation.

In the first 2-year period, on the other hand, associations were less affected by structural NCs than hospitals (1.2 per branch vs 2.3 per branch) and resolved them more effectively than hospitals, partly through reorganisations that led to the closure of some branches. In fact, public premises are more complex to modify because they also host healthcare activities other than blood collection. In addition, each structural change requires a bureaucratic procedure and a public tender for transparency reasons. On the contrary, blood donor associations are private subjects with a more agile organisation than public health service: they can more readily approve structural changes through a governing council, subject to financial availability.

For other aspects, the possibility of carrying over public tenders is one of the strengths of the hospitals and also the associations can take advantage of them. For example, regional or wide area tenders contributed effectively and uniformly to resolving some technological issues, especially the ones concerning the scales for blood collection.

Future prospects

We believe that more public tenders for goods and services could prove helpful for closing the remaining organisational and technological NCs, such as those concerning the management software and the transportation of blood and blood components. However, this approach alone will not be enough: also the discussion and collaboration between head physicians and quality managers of hospitals and associations will be essential to close the remaining organisational NCs, contributing to the regional standardisation of processes and, therefore, to the maturity of the whole system.

Strengths and limitations

To our knowledge, this is the first study in Italy to evaluate the Blood System’s change over time and to analyse so many blood organisations altogether.

Three associations out of nine did not participate in the study; therefore, the collected data are significant but not complete. In addition, during the last maintenance inspection (2017–2018), the region did not visit all branches on-site to check the status of NCs observed in the previous inspection (2015–2016). In these cases, the Regional Health and Social Agency settled for the self-assessment checklists submitted by hospitals and associations in the preinspection phase. Compliance values for these requirements are, therefore, possibly biased.

Conclusions

An improvement of the Emilia-Romagna Regional Blood System over time is evident. The institutional accreditation certainly pushed hospitals and associations to change and overcome their problems to comply with specific requirements; in the process, they got used to working with a view to continual improvement, which will help them to face future challenges.

Our work also highlighted some remaining NCs as of 2018, which were mostly organisational. The management software was the most critical issue, and, in this regard, the region carried out a public tender to soon provide all hospitals and associations with the same software that meets all requirements. Also devices for transportation of blood and blood components and the validation of the transportation process still had room for improvement. Despite these NCs, it should be noted that all currently active structures are accredited and guarantee high standards of quality and safety of products and services.