Improvement in hospital Quality of Care (QoC) after the introduction of rotavirus vaccination: An evaluation study in Belgium.

During each winter period hospital emergency rooms and pediatric wards are often overwhelmed by high patient influx with infectious diseases leading to chaotic conditions with poor quality of care (QoC) delivery as a consequence. The conditions could be improved if we were able to better control the influx by introducing for instance better prevention strategies against some of the most frequent infectious diseases. New prevention strategies using vaccination against rotavirus infection were introduced in Belgium in November 2006. We developed a measure of hospital QoC suitable for assessing the impact of pediatric rotavirus vaccination. The study is retrospective collecting routine data on bed and staff management in one pediatric hospital in Belgium. The data were divided in pre- and post-vaccination periods during rotavirus-epidemic and non-epidemic periods. The scores were constructed using Explanatory Factor Analysis (EFA). All patients enrolled were admitted to the pediatric ward over the period from 1 January 2004 to 31 December 2009. The results of the epidemic period indicated that bed-day occupancy, bed-day turnover and unplanned readmissions for acute gastroenteritis were lower in the post-vaccination compared with the pre-vaccination periods. The QoC scores were therefore significantly lower (indicating improved QoC) after the introduction of rotavirus vaccination, compared with pre-vaccination. The data suggests that the reduction in the winter peak of rotavirus-related hospitalizations after the introduction of the vaccine reduces pressure on hospital resources and improves the quality of hospital care. The findings should be further tested in similar settings.

quality of care

explanatory factor analysis

respiratory syncytial virus

acute gastro-enteritis

United States

bed-day occupancy

bed-day turnover

unplanned readmissions

full-time equivalent staff per day

overtime hours worked per day

rotavirus gastro-enteritis

RVGE nosocomial infection rate

RVGE specific death rate

Hospitalization rate

Staff Leave Time

Staff replacement per day

Introduction

Rotavirus disease places a high demand on European healthcare systems, accounting for 56.2% of hospitalizations and 32.8% of emergency department visits for community-acquired gastroenteritis during the winter epidemic seasons in children aged <5 years. The seasonal rotavirus peak coincides with other pediatric infections, such as respiratory syncytial virus (RSV), influenza, pneumococcal disease, and other causes of acute gastro-enteritis (AGE). Each winter, those infections cause a high influx of children with communicable diseases into pediatric hospital wards and emergency rooms.

These winter increases in hospitalizations for infectious diseases in young children place a heavy burden on the delivery of medical care. Overcrowding and excess workload in hospital care is recognized as a serious problem for patients and staff. High patient-to-nurse ratios are related to unfavourable patient outcomes and increased self-reports of staff burnout and job dissatisfaction. In addition to staff stress, the influx of infectious disease cases may result in wards crowded over capacity, facilitating pathogen transmission and increasing nosocomial infections. This has been documented by epidemiological data, but the consequences have not been fully investigated. Overcrowding leads to high turnovers, with consequent potential for premature discharge and high readmission rates. Over-stressed conditions for staff may be associated with high sickness rates requiring recruitment of temporary personnel, which in turn may be associated with a risk of incorrect diagnosis and/or treatment. Thus, the seasonal influx of patients with communicable diseases during a short time risks a cascade of sequential problems each winter.

Rotavirus vaccination was introduced in 2006 and offers the potential for better control of the influx of infectious disease cases. Rotavirus vaccination in countries with established vaccination programmes has consistently and significantly reduced the incidence of rotavirus gastroenteritis (RVGE) and associated hospitalizations, emergency department visits and outpatient/physician office visits in the United States (US), Europe and Australia.

Belgium introduced rotavirus vaccination in November 2006, with an uptake of 85% in the first year and 89% in the second year. This high vaccine uptake is maintained throughout subsequent years where parents are asked for a co-payment of the vaccine of 11.6€ per dose. We have previously investigated the change in winter pediatric hospitalizations from before to after the introduction of rotavirus vaccination. By reducing the seasonal influx of urgent RVGE cases into pediatric hospitals, the vaccine could help to reduce the winter pressure on healthcare resources and staff, offering potential wider benefits beyond reduced healthcare costs and quality-of-life gains in improving hospital quality of care (QoC). Such benefits would accrue to patients and their families, staff and hospital managers. Patients, who at present may be exposed to overcrowded hospitals with staff operating under stressful conditions, could benefit from better care provided by a less pressured service. Staff could benefit from reduced work stress and thus better conditions for optimal delivery of care, and could be less prone to infections. Health authorities and management may benefit from more efficient overall operation of the healthcare system, and will be better able to maintain a high quality of hospital care throughout the year.

Can these additional potential benefits of rotavirus vaccination on QoC be measured, and could such a measure be used as a benchmark for assessing the value of a new vaccine to hospital care? The objective was to obtain a daily QoC score based on easily accessible variables that reflect the management of hospital beds and staff. The score should indicate when care management is at risk for a quality drop potentially affecting patients and caregivers, and should be able to measure the impact of a vaccine introduction. The following hypothesis was therefore tested. The average daily QoC score in hospital management during winter epidemic seasons is significantly/relevantly worse in pre-vaccination compared with post-vaccination periods with a large score difference among the AGE population, a moderate score difference in the infection-only population and a marginal difference score overall.

Results

Descriptive results

Rotavirus infections

Testing for rotavirus started on 1 June 2005. The percentage of rotavirus-positive tests in the winter decreased from 56.9% (165/290) pre-vaccination to 23.0% (48/209) post-vaccination ().

Table 1.

Number of rotavirus tests and rotavirus-positive tests by study period during the winter

RV tests	Pre-vaccination (2005–2006)	Post-vaccination (2007–2009)
Positive (%)	165 (56.9%)	48 (23.0%)
Total	290	209

RV, rotaviru

Bed management variables

shows the results for bed-day occupancy (BDOR), bed-day turnover (BTOR) and unplanned readmissions (UnPln). During the observation period (1st of January 2004 to 31st of December 2009) the total number of bed-days occupied in the pediatric ward with 34 beds was 56,451 days (76%), of which 25,973 days (46%) were due to infectious diseases. Of these, 7,697 days (29.6%) were due to AGE. During the winter the mean number of occupied beds per day for AGE (BDOR) was much higher pre-vaccination than post-vaccination (7.52 bed-days vs. 4.47 bed-days, respectively). The AGE BTOR rate and AGE UnPln rate were also higher pre-vaccination than post-vaccination (0.048 versus 0.028 for BTOR and 0.56 vs. 0.16 for UnPln, respectively).

Table 2.

Bed management variables (2004–2009) for overall, infection-only, and AGE patient groups by study period during the winter

Study period	Value	Overall	Infection-only	AGE
Occupied beds per day (BDOR)
Pre-vaccination	Mean	30.59	16.96	7.52
	N	271	271	271
	SD	7.05	4.29	3.51
	Sum	8,289	4,595	2,039
Post-vaccination	Mean	28.89	14.80	4.47
	N	361	361	361
	SD	7.25	3.95	2.19
	Sum	7,828	4,010	1,212
Bed turnover rate per day (BTOR)
Pre-vaccination	Mean	0.253	0.079	0.048
	N	271	271	271
	SD	0.132	0.052	0.041
	Sum	69	21	13
Post-vaccination	Mean	0.284	0.065	0.028
	N	361	361	361
	SD	0.143	0.048	0.031
	Sum	77	18	8
Unplanned readmission rate per day (UnPln)
Pre-vaccination	Mean	0.93	0.76	0.56
	N	271	271	271
	Cases	48	36	29
	SD	0.908	0.880	0.691
	Sum	253	207	152
Post-vaccination	Mean	0.38	0.29	0.16
	N	361	361	271
	Cases	25	19	9
	SD	0.685	0.529	0.404
	Sum	136	106	43

AGE, acute gastroenteritis; SD, standard deviation; N, Number of days

Staff management variables

Surprisingly, the mean number of full-time equivalent staff per day (FTEs) and overtime hours (OTW) were higher in post- than pre-vaccination (, 14.468 FTEs versus 14.945 FTEs and 6.95 h vs. 7.55 h OTW). However, the minimum numbers were much lower in post-vaccination periods (9.1 FTEs versus 6.9 FTEs and 3 h vs. 0 h for OTW). It was not possible to split FTEs and OTW by patient group. Only a few days had overtime hours. Pre-vaccination, the number of days with overtime hours was 38/271 (14%), compared with 57/361 [16%] post-vaccination. shows data on staff sick leave (SLT). The pre-vaccination period had higher average and maximum values for sick leave than the post-vaccination period (5.25 h versus 4.37 h).

Table 3.

Staff management variables (2004–2009) for overall study period during the winter

Study period	N	Sum	Mean	SD	Maximum	Minimum
Staff numbers per day (FTEs)
Pre-vaccination	271	3920.9	14.468	3.064	21.3	9.1
Post-vaccination	361	5395.3	14.945	3.634	22.9	6.9
Overtime hours worked per day (OTW)
Pre-vaccination	38	264	6.95	2.770	14	3
Post-vaccination	57	431	7.55	2.608	16	0

FTE, full-time equivalent; SD, standard deviation; N, Number of days.

Table 4.

Staff sick leave by season and study period

Study period	Value	Sick leave, hours	Sick leave, persons	Sick leave, FTE
Pre-vaccination	N	271	271	271
	Sum	1423.28	308	187.274
	Mean	5.252	1.14	0.691
	SD	5.990	0.95	0.788
	Maximum	25.47	4	3.351
	Minimum	0.00	0	0.000
Post-vaccination	N	361	361	361
	Sum	1579.33	311	207.807
	Mean	4.375	0.86	.575
	SD	5.148	0.858	.677
	Maximum	20.90	4	2.750
	Minimum	0.00	0	0.000

FTE, full-time equivalent; SD, standard deviation; N, Number of days.

Analytical results

shows average Factor 1 (bed management score), Factor 2 (staff management score) and overall QoC scores (Factor 1 + Factor2) pre- and post-vaccination. QoC scores were significantly lower (improved QoC) post-vaccination than pre-vaccination in all 3 groups (AGE, Infection only, Overall). The largest difference was in the AGE group, as expected. It may be surprising that the QoC improvement post-vaccination in the infection-only group was not larger than in the overall group. This may be because the AGE group comprised a large proportion of the infection-only group, and there were relatively few non-AGE infections to benefit from improvements in time and bed-space. This is not the case when the overall pediatric ward is considered.

Table 5.

Average QoC scores pre-and post-vaccination for each patient group in winter

Patient group	Factor	Pre-vaccination	Post-vaccination	Mean difference	t-test	p-value (2-tailed)
Overall	Factor 1	−0.061	0.065	−0.127	−1.034	0.30
	Factor 2	0.514	−0.554	1.069	10.332	0.000*
	QoC score	0.453	−0.488	0.941	5.767	0.000*
Infectious-only	Factor 1	0.506	−0.546	1.052	10.188	0.000*
	Factor 2	−0.100	0.108	0.209	−1.718	0.087
	QoC score	0.406	0.053	0.352	5.107	0.000*
AGE	Factor 1	0.501	−0.544	1.046	10.125	0.000*
	Factor 2	0.333	−0.361	0.695	6.152	0.000*
	QoC score	0.834	−0.906	1.741	11.153	0.000*

AGE, acute gastroenteritis; QoC, quality of care.

significant differences.

compares the daily Factor 2 (staff management) scores during the winter of 2006 and 2008 as an example. In 2006 (pre-vaccination) there was a period of stress, indicated by high Factor 2 scores, with no equivalent in 2008 (post-vaccination). shows as another example of the impact of the vaccine, the absolute number of bed-days occupied per month in 2005 (1 year pre-vaccination) and 2009 (2 years post-vaccination). Post-vaccination, BDOR stayed below the ideal threshold throughout the winter rotavirus season (January–March), whereas pre-vaccination the BDOR was above this threshold for several months. The increase in November–December in both years is partly explained by the hospitalization of RSV cases.

Figure 1.

Daily Factor 2 (staff management) scores in the winter of 2006 (pre-vaccination) and the winter of 2008 (post-vaccination).

Figure 2.

Bed occupancy number per month pre-vaccination and post-vaccination Footnote: Nbr: number.

Discussion

Introducing a new vaccine with an immediate high uptake (>85% coverage) and an explicit focus on reducing hospitalizations during epidemic seasons into an established healthcare system would be expected to affect hospital management. Overcrowding as indicated by excess BDOR has been associated with increased hospital infections and patient mortality, and exposure to BDOR of >10% in excess of the recommended limit of ≤85% for >6 months has been associated with antidepressant treatment in hospital staff. Overcrowding typically occurs in the winter, when influenza, RSV, pneumococcal disease and rotavirus all circulate together, and can be exacerbated by rapid pathogen spread within the hospital causing high rates of nosocomial infection. However, although the problem of winter pressure is familiar to hospital staff, until now there has been no method to quantify that phenomenon into one single measure. The present study is a first attempt to develop a single QoC score that can quantify stress in healthcare services and can assess the impact of interventions, such as rotavirus vaccine introduction, on service stress.

As this was a retrospective analysis, we used data that were already available. This should allow replication of this analysis in other hospital settings collecting similar data which is the case for bed day management variables but could be more difficult for staff management variables that most often are not fully electronically available over a long enough period during pre-vaccination or before 2006 in Belgium. Overcrowding and its adverse effects on staff stress should be visible in measures of bed management and staff management, so we concentrated on variables in these areas. Some of these variables should be correlated, e.g. when BDOR is high, BTOR is also likely to be high, which in turn increases the risk of unplanned readmission. The technique of Explanatory Factor Analysis (EFA) allows identification of links between the variables and the integration of several variables into a new measure. In the present study, we pre-defined the number of Factors we wanted to work with, one on bed management and one on staff management, to facilitate the construction of an overall QoC score with the right weighting values for each variable in the EFA equation. As expected, the difference in winter QoC scores pre- and post-vaccination was highest among AGE patients. Notably, there was also a substantial difference for the overall group. This indicates that improved QoC after introduction of rotavirus vaccination benefited the pediatric ward as a unit, and not only patients with the specific infection targeted by the vaccine. We also explored an analysis of calculating QoC-scores in the non-winter periods expecting no much of a difference between the pre- and post-vaccination periods for the 3 patient groups considered and obtaining much lower scores than during the winter periods. Our hypothesis was confirmed and indicated a way to validate the construction of the score-composition. We report these particular data in the Supplementary Appendix 1.

The analysis can also be used to identify periods of stress by plotting the daily scores. Our results showed that the daily Factor 2 (staff management) scores indicated an extended period of high stress in the winter of 2006, and bed occupancy indicated an extended period of overcrowding in the winter of 2005. Such information could be used to predict the development of problems and to implement remedial actions.

It should be emphasized that this analysis does not support reductions in personnel numbers in pediatric wards during the winter period now that rotavirus vaccination has been introduced. Instead, it demonstrates that there was considerable stress in pediatric wards prior to vaccine introduction. This stress could have detrimental effects on patients, staff and the efficient operation of the healthcare system. Introducing the rotavirus vaccine has reduced the seasonal peak in rotavirus admissions, thus reducing the winter stress on healthcare services and improving QoC, which should benefit patients, staff and hospital management overall. But QoC scores are likely to be dynamic, since they are composed of several variables, and it is likely that they will continue to evolve over time.

The technique of EFA to build scores is not new and has especially been applied in the world of sociology. In the medical world it has been used to help classifying patient-groups in function of disease severity levels based on scores constructed through the collection of data from questionnaires.

Currently, cost-effectiveness analyses of new vaccines have not been able to include any QoC benefits of vaccination. The present analysis offers a way to quantify the QoC benefit, which may allow its potential impact on hospital costs and quality-adjusted life-years (QALYs) to be incorporated in economic analyses. We are exploring this in future research. However we need to be sure to make links between the level of QoC-scores and the cost of disease management in hospital care or the impact on QALYs among the personnel. The latter should preferentially be investigated with prospective data collection. That process doesn't facilitate the implementation of such a study.

The present study has some limitations. First, it was conducted at a single center, and the results should be confirmed in other settings. We are presently investigating the possibility of repeating the project in another hospital in Belgium with access to similar data. However, if we want to expand our research in other countries it will be difficult. As of today not many countries in Europe can confirm the analysis results we have as only a few of them have currently accepted the rotavirus vaccination as a universal mass vaccination program in children including the UK, Austria, and Finland. Moreover by limiting the study to one center in Belgium it was also not possible to include all the variables we selected for the analysis. For instance we were unable to introduce nosocomial infection rates which are normally an appropriate indicator of stress situations in hospital management. That specific event underwent a dramatic observed reduction after the introduction of the vaccine. But because of too low numbers registered during the observation period in this center, we could not include the variable in the analysis. Another option to circumvent the problem of too low observation units is to change the time unit of observation from day to week or months. This is an option we would like to further explore. Moreover, the EFA approach remains a subject of debate among experts. One challenging item is about the fitting procedures to obtain the regression coefficients among the variables and the Factors in between, called the Factor loadings. Different approaches exist but there is no method that indicates the best or the worst approach in calculating these values. In this study here we have taken a conservative approach by selecting appropriate data and trying to make the analysis as uniform as possible. Full details are provided in the Supplementary Appendix 1 to allow other researchers to replicate the analysis.

Introducing a new vaccine that affects a major public health problem could have many healthcare benefits beside the improvements of clinical benefits such as mortality and morbidity reduction. The new vaccine may also unexpectedly have an impact on other domains of the health care program if that program suffers as well about overcrowded hospital services during certain periods, stress in care delivery by health care professionals, bad management of the beds to be used. These are conditions we often have difficulties to measure correctly about the deviations of normal practice. The analysis method here described as EFA has just the facility to be able to collect that new information quantitatively in an easy to apply way and it does not require the collection of additional data. It integrates several aspects of healthcare service stress by including both bed-day management and care of staff, which are linked and should therefore be evaluated together. The result of such an approach is that we are now in a position to measure the additional hidden benefit a vaccine can offer, showing an improved summary score in QoC with better patient care, more staff time, reduced spread of infectious disease in hospitals and more resources to apply on other disease areas. The QoC score can be used to predict periods of stress and identify problems to tackle during day to day management. Using the quantitative QoC score may help to find more objective ways of analyzing the issues of healthcare service stress and the benefit impact of new interventions.

Methods

Hospital setting

The study was conducted at the Jessa Hospital in Hasselt, Belgium, which is also part of another rotavirus vaccine study. Rotavirus vaccination was introduced in the region following the recommendation of the Flemish High Committee for Public Health Services (Hoge Gezondheidsraad). The hospital has 34 pediatric beds, and its database was electronically accessible. Its catchment area was uniform during the study period and there were no major management changes, allowing comparisons between pre- and post-vaccination periods. Ethical approval for the study was obtained in December 2012.

Variables

The hospital's existing database provided data on the following variables relating to management of patient beds and staff, measured per day that could be used for the construction of the QoC score:

Bed-day occupancy number and rate (BDOR): Number of beds occupied/number of beds available per day;

Bed-day turnover rate (BTOR): Number of patients discharged/number of beds available per day;

Unplanned readmission rate (UnPln): Number of readmissions ≤7 days after discharge/total number of discharges per day;

Rotavirus test rate and positive results (RVT): Number of tests performed per day and rate of positive test results;

RVGE nosocomial infection rate (RVNR): number of RVGE nosocomial infections/total number of RVGE hospitalizations per day

RVGE specific death rate (RVDR): number of RVGE specific deaths/total number of RVGE hospitalizations per day

Hospitalization rate (HR): number of hospitalizations/total number of children in the catchment area per day

Full-time equivalent staff per day (FTE): Total number of hours worked by the personnel per day/7.6;

Over-time work by the staff per day (OTW): Total number of extra hours taken by the personnel/total number of regular hours per day;

Sickness leave by the staff per day (SLT): Total number of paid and unpaid sick days/total number of FTEs per day.

Staff replacement per day (SR): number of replacement of staff/total number of staff present per day.

These variables were analyzed descriptively and used to construct the QoC scores. First, the numbers per day were assembled to observe whether their distributions were normal. If not, the data were transformed using log-transformation, square-root or other methods until a normal or near-normal distribution was reached. Quintiles were then calculated in 8 groups (5th, 10th, 25th, 50th, 75th, 90th, 95th, >95th) and each variable value per day was given a score of 1–8 according to its quintile category. We selected 8 categories in order to get enough granularity in the spread of the variable values. By not selecting too few or too many categories it may impact the analysis with too many empty cells or not enough sensitive correlation between variables. It should be noted that the end results (average daily QoC-scores) were analyzed by grouping period of pre-vaccination and post-vaccination, pooling therefore the numbers of several years. We have also split the evaluation period within a year as rotavirus epidemic period or not. It is expected that the difference in QoC scores are enhanced if we consider those periods separately instead of a full year analysis that may dilute the impact with a period where no difference in results is expected when comparing pre-post vaccination in the non-epidemic seasons.

Meanwhile the following variables (RVNR; RVDR; HR; SR) were deleted from the first analysis. The event numbers were too small (no deaths and no replacement during the observation period) or showed no meaningful change over time, so these variables were not considered further. This was especially the case for RVNR that with 87 nosocomial infection observations over a 2557 day period was too low to obtain a meaningful result. Pooling results from different hospitals should enhance the analysis or considering a shorter time period for the assessment such as week observations instead of day observation could have been another approach to circumvent the obstacle of too few observations. Meanwhile changing the unit of observation may impact the calculation process when the data have been collected on a day to day basis. But it is certainly an option to consider when the analyzing the data of only one center.

Study period

Data were collected for the period between 1 January 2004 and 31 December 2009, divided into a pre-vaccination period (before November 2006) and a post-vaccination period (after November 2006). Winter epidemic months were defined as 1 January to 31 March.

Between 2004 and 2009, there were 2,557 days of which 632 (24.7%) days were during the winter. Of the winter days, 271 (42.9%) days were during the pre-vaccination period and 361 (57.1%) days were during the post-vaccination period. In the current analysis presented here we only demonstrate the results of the winter or the rotavirus epidemic period. We did an analysis of the non-epidemic period as well but that is reported in the Supplementary Appendix 1.

Patients

The study included data on all patients admitted to the pediatric ward over the study period, stratified into 3 groups:

Overall population (all admissions);

Infection-only population (patients admitted for infectious disease of any kind);

AGE population (patients in the infection-only group admitted specifically for AGE).

Data analysis

Descriptive

Summary descriptive statistics are reported for each variable selected for calculating the QoC scores with no statistical significance level reported per specific period evaluated for each variable. We did this on purpose as the focus of the study is on the measurement of the QoC scores and not on the individual variable results used to calculate the QoC scores.

Analytical

A summary QoC score for hospital care should integrate data from several variables to provide a useful overall measure. Explanatory Factor Analysis (EFA) is a statistical method that assesses whether a number of observed variables are linearly related to a smaller number of unobservable Factors. We used EFA to derive 2 Factors from the observed variables. A scree plot indicated that only 2 Factors could be constructed from the data. These two Factors can be summed to produce a summary QoC score if the Factors are independent from each other. Before conducting the EFA, the observed data were standardised into 8 quintiles as described above in order to be able to assemble them correctly together.

The EFA was conducted in several steps. First, the dataset was analyzed with specific tests to assess whether EFA can be applied, constructing a correlation matrix for the pairwise correlation coefficients between the variables. The EFA should meet 3 objectives: it should be parsimonious (minimum number of new explanatory Factors created to explain the observed data); the new Factors should be independent of each other as far as possible; and the Factor scores should make sense (e.g., a Factor score about staff management should be driven by variables such as overtime and sick leave). Second, the Principal Component Method (PCM) confirmed that 2 explanatory Factors could be derived. Third, each Factor was constructed with specific factor loadings using varimax rotation because of the independence of each Factor. Finally, the weighting coefficient of each variable in the equation that produces the Factor was adjusted after rotation.

Using the 7 variables listed above, 2 new measures were constructed, Factor 1 (bed management) and Factor 2 (staff management). The daily score for each Factor was calculated with a regression equation of all selected variables. The sum of the 2 Factors is the summary QoC score. Higher scores indicate worse QoC. The Factor scores and QoC scores were compared between pre- and post-vaccination periods using the T-test, for each patient group in each season (winter and non-winter). The P-value cut-off was set at 0.05. All analyses were conducted using IBM SPSS Statistics v22.0.

Summary results for each Factor and QoC scores are presented in this paper. A full EFA analysis showing the calculation of the regression coefficients and Factors for one period (winter) and one patient group (overall) is provided in the Supplementary Appendix 1.