Impact of the COVID-19 Pandemic on the Hospital: Inpatient's Perceived Quality in Spain.

Although the Coronavirus disease 2019 (COVID-19) pandemic has generated a large amount of studies, the patient-perceived quality of care (PQ) in this context is still not well known, so more studies intending to focus on this issue are strongly needed. This study assesses changes on PQ in patients hospitalized in Spain during the first month of the COVID-19 pandemic and investigates differences between those admitted for this cause and the rest a descriptive study using the "Net Promoters Score" and the hospital regular monitoring plan. Due to this point of view, ethical approval is not applicable. Four PQ dimensions (nurse, physician, and nurse assistant actions [NA], and discharge information [DI]) were measured in all COVID patients (57) and in a sample of non-COVID patients (60) discharged at home during the first month of the pandemic, and also compared with another sample (384) from an immediately previous period. The COVID patients scored worse (8.2) than non-COVID ones (9.0; P < .0001), especially in NA and DI, and were more likely to be detractors (odds ratio [OR]: 3.05, P < .0001) and less to be promoters (OR: 0.64, P < .05). Global and DI net promoters score values before the pandemic were higher than afterward. In conclusion, the COVID-19 pandemic negatively and significantly influenced the health care quality as perceived by inpatients, both in COVID and in non-COVID ones, but more intensely in the former. As a health care organization, this knowledge meant an opportunity from improvement and to be better qualified to face the pandemic.

Introduction

It is well known that patient experience is positively associated with clinical effectiveness, thus supporting its inclusion as one of the central pillars of health care quality (1). In consequence, it is common for health care organizations to monitor this issue using patient surveys as a way to collect a large amount of useful information to improve the quality of care (2,3).

Health emergencies cause challenges for health organizations in terms of care demands and resources availability that cover not only clinical but also ethical aspects (4). The Coronavirus disease 2019 (COVID-19) pandemic forced a drastic redesign of health care processes in order to ensure adequate care for patients who, due to this or another health problem, had to use care services. Nevertheless, available data about this subject usually only refer to operational, clinical, or epidemiological points of view (as the occupancy rate, the available beds in intensive care units, the average length of stay, the death rate, etc) (5). Unfortunately, elements related to the patient-perceived quality of care are not usually among them, so studies intending to focus on this issue are strongly needed. For this reason, the aim of this study is to assess changes in perceived quality in patients hospitalized during the first month of the COVID-19 pandemic and to investigate differences between those admitted for this cause and the rest.

Methodology

Descriptive study carried out in a third-level hospital in Murcia (Spain), using a survey and an adaptation of its perceived quality monitoring plan that applies the “Net Promoters Score” (NPS) methodology. Due to this point of view of regular monitoring, ethical approval is not applicable.

Net promoters score is a method for measuring customer loyalty introduced by Reinheld (6) and recently extended to clinical assessment (7,8). The NPS is an easy-to-use questionnaire based on the question “How likely is it, that you would recommend the service to a friend or colleague?”. Participants were asked to score on a 0 to 10 numeric rating scale, with 10 being extremely likely to recommend the service. The percentage of participants whose response was between 0 and 6 was subtracted from the percentage of those whose scores were 9 to 10 to calculate the NPS. Participants with the values 7 and 8 were assumed to be passive. Therefore, the NPS can be as low as −100 if everybody is a detractor, or as high as +100 if everybody is a promoter.

Perceived Quality Dimensions Measured and NPS Methodology Adaptation

The programme for evaluation and improvement of healthcare quality (EMCA) is a Murcia institutional initiative aimed to foster activities related to quality management in its Health System (9). Regarding perceived quality, this programme identified the aspects and quality dimensions more strongly related to the care organizational processes in Murcia setting. Based on this, the hospital monitoring plan for the hospitalization process prioritized and measures periodically 3 quality dimensions: (10)

Nurse action, as the main agent of the hospitalization process.

Physician action, as providers of the convenient diagnostics and treatments.

Patient discharge information about his health situation, treatment and care, as basic elements for the patient to reintegrate effectively in the community.

In this study, a fourth dimension referred to nurse assistant action was added because in Spain this professional is in close touch with the patients (personal and room cleaning, sheets replacement, food service, etc) and so it could be of interest in a pandemic context.

According to NPS methodology, a question asking about the likelihood for recommending the hospital to a friend or colleague was built for every quality dimension. Thus, every patient included in the study was enquired about the 4 of them.

Sample Determination and Patient Selection

A list of all patients (57) who were admitted in the period from March 16 to April 15, 2020, for COVID-19 and discharged at home was obtained.

Additionally, a random sample from the 394 patients admitted for any other cause in adult units and discharged at home during the same period was chosen. The sample was made up of 60 cases and was stratified into 2 cohorts of 30 patients for each hospitalization block: maternal, which had a total of 185 discharges (50.8% of the 394 discharges) and general.

The survey was administered to these 3 groups of patients by trained staff by phone. Verbal informed consent was obtained from the patients for their anonymized information to be published in this article. The cases with no contact or those that refused to answer were noted.

Furthermore, data about nurse and physician actions and discharge information from hospitalization process monitoring obtained before the onset of the pandemic (February 2020) were used in the study. In that period, a sample of 384 patients randomly selected (16 patients for each of the 24 existing hospitalization units) was studied.

Data Analysis

The mean values, NPS, and their confidence intervals were calculated per quality dimension in the 2 hospitalization blocks and also for COVID-19 and non-COVID patients. Possible differences between them and between the quality dimensions were tested using the Chi square, Student’s t, analysis of variance or Kruskal-Wallis tests, depending on the type of variable. In order to find out about other factors, the relationships between the scores and age, gender, and length of stay were analyzed using the Student’s t test. Finally, logistic regression analysis was used to determine whether they were related to the probability that the patient was a promoter or a detractor, (“forward conditional” method, input for P equal to .05, and outputs for .10). In both cases, the continuous variables (age, stays) were dichotomized taking their median value as a reference.

Eventually, for the 3 common quality dimensions, these scores were compared with those from the regular perceived quality hospitalization process monitoring (data collected in February 2020).

Results

Of the 57 COVID patients, 49 could be contacted (86%) and all of them agreed to answer the survey (response rate, 100%). Their age ranged from 19 to 84 years (mean 53.8, deviation 18.5) and 57.1% of them were male. The length of stay ranged from 0 to 24 days (mean 5.8, deviation 5.0).

On non-COVID patients, contact effectiveness was also 86% and also all of them (60) agreed to answer the survey. Their age ranged from 18 to 89 years (mean 49.3, deviation 21.3) and 40% of them were male. The length of stay ranged from 0 to 60 days (mean 5.2, deviation 8.2). No significant differences were found in terms on age, gender, or length of stay between COVID and non-COVID patients.

Pandemic Period

The global perceived quality adjusted mean value was 8.9, but there were significant differences between dimensions, nurse and physician actions scoring higher than “discharge information,” and “nurse assistant action” dimensions (P < .0001). The global perceived quality score was also lower in COVID than in non-COVID patients (P < .0001), with no significant differences between blocks. By quality dimension, on COVID patients, the scores were lower in “nurse assistant action” (P < .0001) and “discharge information” dimensions (P < .05). The distribution is shown in Table 1.

Table 1.

Strata Distribution of Perceived Quality Scores (Global and by Dimension).a

Quality dimension	Stratum	N	Mean	Standard deviation	Standard error	95% confidence interval		Minimum	Maximum
						Inferior limit	Superior limit
Nurse assistant action	COVID	49	6.78	3.23	0.46	6.43	7.12	0	10
	GB	30	8.63	1.38	0.25	8.18	9.08	5	10
	MB	30	9.27	1.23	0.22	8.86	9.67	5	10
Nurse action	COVID	49	8.98	1.48	0.21	8.82	9.14	4	10
	GB	30	9.30	0.95	0.17	8.99	9.61	6	10
	MB	30	9.47	0.82	0.15	9.20	9.74	7	10
Discharge information	COVID	49	8.02	2.14	0.31	7.79	8.25	0	10
	GB	30	9.00	1.41	0.26	8.54	9.46	5	10
	MB	30	8.60	2.08	0.38	7.92	9.28	3	10
Physician action	COVID	49	9.18	1.29	0.18	9.05	9.32	5	10
	GB	30	8.73	2.52	0.46	7.91	9.56	0	10
	MB	30	9.33	1.06	0.19	8.98	9.68	6	10
Total	COVID	196	8.24	2.36	0.17	8.12	8.36	0	10
	GB	120	8.92	1.67	0.15	8.65	9.19	0	10
	MB	120	9.17	1.40	0.13	8.94	9.40	3	10

Abbreviations: BG, general hospital block; COVID-19, Coronavirus disease 2019; MB, maternal hospital block.

a The confidence interval has been adjusted as the sample sizes were greater than 10% of the stratum size. SCORES were lower in COVID than in non-COVID patients, both globally and in the “nurse assistant action” and “discharge information” dimensions. No significant differences were found between non-COVID patients from the general (BG) and maternal (MB) hospital blocks.

Table 2 includes all the NPS values and their confidence intervals. The global NPS adjusted value was 65, lower in COVID than in non-COVID patients (P < .001). By dimension, “nurse assistant action” and “discharge information” also scored lower in COVID patients (P < .001 and P < .05, respectively).

Table 2.

Strata Distribution of NPS Values (Global and by Quality Dimension).a

	Nurse Action			Physician Action			Nurse assistant Action			Discharge information			Global perceived quality
Stratum	NPS	95% CI		NPS	95% CI		NPS	95% CI		NPS	95% CI		NPS	95% CI
		Inferior limit	Superior limit		Inferior limit	Superior limit		Inferior limit	Superior limit		Inferior limit	Superior limit		Inferior limit	Superior limit
MB	87	54	100	77	45	100	77	45	100	53	24	82	73	58	89
GB	80	48	100	70	39	100	43	16	70	70	39	100	66	50	81
Non-COVID	83	61	100	73	51	96	60	39	81	62	40	83	70	59	81
COVID	67	57	77	76	65	86	6	2	10	39	30	47	47	42	51
Total (adjusted)	80	64	96	74	58	89	49	35	63	57	42	71	65	57	73

Abbreviations: BG, general hospital block; COVID-19, Coronavirus disease 2019; NPS, Net Promoters Score; MB, maternal hospital block.

a The total values and confidence intervals have been adjusted. Scores were lower in COVID than in non-COVID patients, both globally and in the “nurse assistant” dimension.

Bold values signifes P <.05.

Related Factors Analysis

During the pandemic period, significant differences were found between the global perceived quality score and the age of the patients: Patients older than the median (51 years) scored worse (P < .001). There were also difference between the “nurse assistant action” score and the gender (female patients scored worse, P < .01).

The likelihood for a COVID patient to be a detractor was about 3 times higher than for a non-COVID one (P < .0001). Reciprocally, it was also one and a half times lower to be a promoter (P < .05). This relationship was particularly strong in “nurse assistant action” dimension, where the likelihood for a COVID patient to be a detractor was almost 7 times higher (P < .0001) and almost a half for to be a promoter (P < .05). When considering globally both the COVID and the non-COVID patients, to be younger than 51 and, with regard to “nurse assistant action” dimension, to be a female made less probable to be a promoter (P < .05). Logistic regression analysis is described in Table 3.

Table 3.

Logistic Regression Analysis.a

			Related factors
	Quality dimension		COVID	Length of stay	Age	Gender
Promoter	Nurse action	OR (95% CI) signification (P)	−	−	−	−
			−	−	−	−
	Physician action	OR (95% CI) signification (P)	−	−	−	−
			−	−	−	−
	Nurse assistant action	OR (95% CI) signification (P)	0.42 (0.19-0.95) < .05	−	−	0.37 (0.17-0.84) < .05
				−	−
	Discharge information	OR (95% CI) signification (P)	−	−	−	−
			−	−	−	−
	Global	OR (95% CI) signification (P)	0.64 (0.42-0.99) < .05	−	0.63 (0.41-0.97) < .05	−
				−		−
Detractor	Nurse action	OR (95% CI) signification (P)	−	−	−	−
			−	−	−	−
	Physician action	OR (95% CI) signification (P)	−	−	−	−
			−	−	−	−
	Nurse assistant action	OR (95% CI) signification (P)	6.97 (2.36-2.55) < .0001	−	−	−
				−	−	−
	Discharge information	OR (95% CI) signification (P)	−	−	−	−
			−	−	−	−
	Global	OR (95% CI) signification (P)	3.05 (1.66-5.61) < .0001	−	−	−
				−	−	−

Abbreviation: COVID-19, Coronavirus disease 2019.

aFactors related (P < .05) to the likelihood of being a promoter (scoring 9 or 10 out 10) or a detractor (scoring 6 or less out 10).

Comparison With Regular Monitoring Perceived Quality Data Before the Onset of the Pandemic

Global adjusted NPS value in immediately before to onset of the pandemic period was better than in its first month afterward (P < .05). By quality dimensions, better numbers were also observed in “discharge information” before the onset of the pandemic (P < .05), while there was no significant difference neither in the rest. There were no “nurse assistant action” data before the onset of the pandemic, so comparisons could not be established. All these values are summarized in Figure 1.

Figure 1.

Adjusted net promoters score values (NPS) before the onset of the pandemic (January and February 2020, Jan-Feb) and during its first month. Both global and “discharge information” NPS were lower after the onset of the pandemic. There are no measurements in “nurse assistant action” dimension before the onset of the pandemic. the arrows indicate the width of the 95% confidence intervals.

Discussion

Our study shows that the perceived quality on discharged patients during the first month of the pandemic has worsened when compared to patients previously discharged, especially in the “discharge information” dimension. This is important because this dimension is related to a better self-understanding of the health status, of the kind of care that is needing and, therefore, of the subsequent clinical outcomes (11). Nevertheless, it was not possible to compare “discharge information” to the “nurse assistant action” dimension, which showed the worst scores in COVID versus non-COVID patients after the onset of the pandemic. In addition, it is known that there is a strong relationship between the nurse’s quality of care and the health care quality as a whole (12 –14). Therefore, it is also worth noting that, in the study, the perceived quality of both physician and nursing care dimensions did not show significant variation during the first month of the pandemic.

The study also evidenced that, during the pandemic, there was worse perceived quality scores in COVID patients when compared to non-COVID ones. This gap is mainly caused by the “nurse assistant action” dimension (referred to the help offered for personal hygiene, meals, room cleaning, etc). In this way, in COVID patients, the NPS value in this dimension was 6, the likelihood for to be a detractor 7 times higher and 0.43 times lower for to be a promoter, numbers that certainly leaves room for the improvement (6). Furthermore, and although the worse score on “discharge information” dimension for the patients as a whole after the onset of the pandemic has been already referenced, it is also important to note that this difference also has been detected between COVID and non-COVID patients.

These results may be influenced by the fact that health care workers are aware of their belonging to a high-risk group for COVID infection and experience a significant stress level because of the changes in the hospital policies after the onset of the pandemic (15). Thus, in a study on the influenza H1N1 pandemic, a significant proportion of professionals experienced moderately high levels of concern and psychological distress (16). These concerns could affect their overall effectiveness in a pandemic and, therefore, should be addressed by incorporating strategies to manage them in pandemic planning, making sure they had availability of personal protective equipment and an adequate training on their use (17,18). It is possible that, in the first pandemic month, our hospital strategy was incomplete or insufficient for the nurse assistants (along with they could have less theoretical knowledge about this new disease), thus influencing the perceived quality fall reported by COVID patients in “nurse assistant action” dimension.

It is also noteworthy the decrease in the “nurse action” dimension while the “physician action” one is maintained, although the two experienced a fall in the pandemic period both in COVID and non-COVID patients when compared to the period prior to the pandemic. The same effect was detected in “discharge information”: this dimension showed similar scores in COVID and non-COVID patients after the onset of the pandemic, but worse that those taken in the pre-COVID period measurement.

Using NPS method to assess perceived quality and health patient experience is more and more commonplace for health care organizations (19 –21) and for specific clinical situations (22,23). On one hand, because assessing satisfaction usually needs to apply extensive and complex questionnaires more frequently based on expectations rather than on the real experience (24). On the other, because using a standard question structure makes it possible to establish comparisons between different processes or clinical situations (25). Also, the feedback gained from this method is actionable at a lower cost than traditional ones and can be used to make strategic improvements that will impact health care outcomes (24). It is especially useful when, instead of a single question, specific approaches (the quality dimensions in the study) have been employed (26,27). Furthermore, the construct validity of the perceived quality dimensions selected in the study is supported by the accuracy of the original questionnaires, their empirical verification and practical implementation (10).

Regarding confounders, some of the more commonly related to perceived quality have been controlled in the study, such as age, gender, or length of stay (9). This forethought is important because a recent systematic revision found that the strength and directions of these associations were inconsistent (28). Nevertheless, neither socioeconomic nor cultural status influence could be taken into account because data were not available.

Despite all these considerations, there are also a host of other factors, unevenly distributed across hospitals, able to influence the results (22). Therefore, their generalization has to be tackled with caution. On this matter, the redesign of health care procedures, the occupancy rate, the incidence of the pandemic, the size and type of the workforce, or the kind and amount of resources available, including the personal protective equipment, could be considered.

Conclusion

The study evidences that the COVID-19 pandemic is able to negatively and significantly influence the health care quality as perceived by inpatients, both in COVID and non-COVID ones, but more intensely in the former. Hence, it meant an opportunity from improvement that our organization was able to take advantage of. Thus, health care procedures were revisited and redesigned incorporating what is learnt and paying especial attention to the training of nurse assistants and discharge information. As an organization, the experience was supposed to be better qualified to face conditions like this in the future.