What Has Been the Impact of Covid-19 on Safety Culture? A Case Study from a Large Metropolitan Healthcare Trust.

Covid-19 has placed an unprecedented demand on healthcare systems worldwide. A positive safety culture is associated with improved patient safety and, in turn, with patient outcomes. To date, no study has evaluated the impact of Covid-19 on safety culture. The Safety Attitudes Questionnaire (SAQ) was used to investigate safety culture at a large UK healthcare trust during Covid-19. Findings were compared with baseline data from 2017. Incident reporting from the year preceding the pandemic was also examined. SAQ scores of doctors and "other clinical staff", were relatively higher than the nursing group. During Covid-19, on univariate regression analysis, female gender, age 40-49 years, non-White ethnicity, and nursing job role were all associated with lower SAQ scores. Training and support for redeployment were associated with higher SAQ scores. On multivariate analysis, non-disclosed gender (-0.13), non-disclosed ethnicity (-0.11), nursing role (-0.15), and support (0.29) persisted to a level of significance. A significant decrease (p < 0.003) was seen in error reporting after the onset of the Covid-19 pandemic. This is the first study to investigate SAQ during Covid-19. Differences in SAQ scores were observed during Covid-19 between professional groups when compared to baseline. Reductions in incident reporting were also seen. These changes may reflect perception of risk, changes in volume or nature of work. High-quality support for redeployed staff may be associated with improved safety perception during future pandemics.

1. Introduction

The provision of safe, high-quality care should always be an essential aim of a healthcare system, even during crises such as natural disasters, conflicts, or pandemics. To date, there has been no systematic evaluation of the impact of the Covid-19 pandemic on patient safety.

In addition to its effect on patients, Covid-19 has also placed a significant burden on healthcare systems, with drastic ramifications for the way healthcare is delivered. Rapid changes in models of care delivery were seen during the Covid-19 pandemic including increased workload, redeployment of staff to unfamiliar clinical environments, cancellation of routine services, and the requirement to treat patients suffering from a novel disease about which little was known. Working in these challenging conditions may impact on the ability of staff to deliver safe and effective care. Previous work has identified organisational breakdown, inadequate staffing, increased production pressures and provider fatigue as contributors to poor patient safety [1]. Conceivably, all these factors may have been present during the pandemic, and as such patient safety during the Covid-19 pandemic merits further investigation. A way of obtaining insight into the state of patient safety is via assessing safety culture and incident reporting.

Safety culture refers to “managerial and worker attitudes and values related to the management of risk and safety [2]” and is positively associated with both patient safety [3,4,5] and clinical outcomes [6,7]. A recent systematic review revealed positive correlations between safety culture and patient outcomes, although the researchers also concluded that further interventional studies are needed, with important consideration given to measurement tools, level of analysis and outcome measures [8]. Incident reporting is an important aspect of safety culture [9]. Reporting systems do more than just measure harm at the organisational level; they identify system vulnerabilities, promote learning, and indicate willingness of staff to raise concerns.

Aim

This study aims to assess institutional safety culture during the Covid-19 pandemic using a large London healthcare trust as a case study.

2. Methods

2.1. Study Design

This is a cross-sectional, survey-based study. Study participants were invited to complete a self-administered questionnaire during the Covid-19 pandemic, with available historical data collected the same way as baseline.

Incident reporting data during the Covid-19 pandemic were collected as well, and compared to pre-pandemic figures.

As this was a study of clinical staff, no patient or public involvement was performed in the design of this study. A multidisciplinary group of clinicians was involved in the survey design. Results have been disseminated to the public using a series of free webinars available to view on the PanSurg platform.

All data are available in a de-identified format upon reasonable request to the Imperial College Big Data and Analytical Unit (BDAU) or the corresponding author.

2.2. Setting

Local institutional ethical approval was obtained from Imperial College Research Ethics Committee (ICREC Ref:20IC5890). Imperial College Healthcare NHS Trust (ICHNT) is a group of 5 hospitals located in central London. The trust has over 1.2 million patient contacts per year. Over 12,000 staff work at the trust, of which 2700 are doctors, 4800 are nurses or midwives and 770 are allied health professionals. The proportion of staff from Black, Asian or Mixed Ethnic (BAME) backgrounds is 52%.

In response to the escalating Covid-19 pandemic, changes included introduction of telemedicine outpatient clinics, mass usage of online off-site access for clinicians, cessation of elective surgical services, and expansion of adult critical care capacity by repurposing operating theatre recovery areas and paediatric intensive care beds. Additionally, significant changes were made to the workforce including redeployment to in-demand areas such as critical care or general medical wards. The nature and clinical characteristics of the Covid-19 caseload at ICHNT have been described elsewhere [10].

2.3. Data Collection

Baseline data collection was conducted using Survey Monkey (SurveyMonkey LLC, San Mateo, CA, USA) between 12 October 2016 and 10 January 2017. Data for the Covid-19 group were completed using Google forms (Google LLC, Mountain View, CA, USA) between 22 April 2020 and 22 May 2020. In both instances, invitations were sent by email to all staff, regardless of job role. Two reminder emails were sent and 3 days of in-person convenience sampling was performed. Questionnaires were anonymous to encourage open responses.

2.4. Questionnaire Items

2.4.1. Demographic Questions

The baseline questionnaire included questions related to job role, type of contract, and department (Supplementary Materials). For the purpose of analysis, staff were categorised into doctors, nurses, and other clinical staff (including allied health professionals, healthcare support workers, pharmacists). During the Covid-19 data collection period, questions relating to personal characteristics were expanded to include: age, ethnicity, gender, and redeployment (Supplementary Materials).

2.4.2. Safety Attitudes Questionnaire (SAQ)

The Safety Attitudes Questionnaire (SAQ) was used to assess perceptions of safety across six domains: safety climate, teamwork, stress recognition, perception of management, working conditions and job satisfaction. The SAQ is a reliable psychometric instrument [11] and is one of the most commonly used tools for measuring safety climate. It has been validated in a variety of settings including operating theatres, emergency care, intensive care, as well as medical and surgical wards [7]. The SAQ is widely utilised in different countries and has been adapted for multiple different languages [7,12,13,14,15,16,17]. The tool is effective at comparing across locations, staff groups, and time. On all scales, higher scores represent a more positive perception of safety [11], which have been associated with positive clinical outcomes [18].

SAQ scores were calculated based on the 33 questions common to both surveys (Supplementary Materials). Each question is associated with a given subscale or the overall score and includes a statement followed by a 5-point Likert scale from “strongly disagree” to “strongly agree”. Scores are expressed as the proportion of respondents that “agree” or “strongly agree” with a given statement. Vice versa for negatively worded questions. Questions left blank or answered “not applicable” are excluded from the analysis.

2.5. Statistical Analysis

To achieve at least a precision of 5% on the SAQ a minimum sample size of 400 was required.

Group characteristics were demonstrated with simple descriptive statistics using Pearson chi-square tests to compare the distribution of characteristics among groups. Data were analysed for normal distribution using the Shapiro–Wilk test. SAQ data were found to be non-parametric, so Kruskal–Wallis tests were used to compare SAQ across groups. Violin plots comparing SAQ scores among professions were created using RStudio v1.2 (Version 1.2.5042. 2020 RStudio Inc., Boston, MA, USA).

For the Covid-19 cohort, univariate quantile regression was performed to identify which characteristics influence SAQ score and the direction of any effect. Factors statistically associated with SAQ score in the univariate analysis (ethnicity, role, training, support) were assessed further in the multivariate model using a backwards stepwise approach. The final multivariate model controlled for factors that were significant or deemed important to control for (age, gender, ethnicity, role, redeployment). Redeployed staff were analysed as a subgroup to identify the impact of training and support in relation to redeployment. Confidence intervals were obtained by using 20 bootstrap samples. Statistical significance was set at p ≤ 0.05. Data were analysed using Stata v14 (StataCorp. 2015. Stata Statistical Software: Release 14. StataCorp LP, College Station, TX, USA).

2.6. Incident Reporting

Incident reporting is another important indicator of safety climate and willingness to learn safety lessons. There is extensive research demonstrating the benefits of reporting.

Incidents reported through the Datix reporting system (Datix Limited, London, UK) between 1 April 2019 and 10 May 2020 were examined. These were classified as resulting in “harm”, “no harm”, or “near misses [19]”. These were plotted using a statistical process control (SPC) XmR chart. SPC charts are a tool used to graphically depict and statistically analyse incident reporting, allowing users to identify whether changes are due to natural (common cause) variation, or special cause variation. A detailed explanation of the methodology for plotting SPC charts can be found elsewhere [20]. The upper and lower confidence intervals were set at 3 standard deviations based on the first 20 observations in order to define inherent variation in the process [20]. The process mean and control limits were then recalculated when 8 consecutive points fell above or below the control line [20].

3. Results

At baseline, there were 1580 responses from 8353 staff (19%). This included 166 doctors, 616 nurses, and 374 other clinical staff. Nonclinical staff (424) and those that filled in none of the 33 SAQ questions (117) were excluded, resulting in 1039 eligible responses, with an eligible response rate of 12.4%. The median overall SAQ score at baseline was 0.70.

During Covid-19, 455 responses were received, representing approximately 4% of staff working at the trust. This included 183 nurses, 105 doctors and 127 other clinical staff. A total of 40 non-clinical staff responded, who were excluded from the analysis, resulting in 415 eligible responses and an eligible response rate of 3.5%. Respondent characteristics for both groups are shown in Table 1. The median overall SAQ score was 0.73.

Table 1

Respondent Characteristics.

Covariates	Baseline Survey	Covid-19 Survey	Chi-Square p-Value
	n (%)	n (%)
Role
Other clinical staff	325 (31)	127 (30)	<0.001
Doctor	157 (15)	105 (25)
Nurse	557 (54)	183 (44)
Division
Medicine	536 (34)	188 (45)	<0.001
Surgery	445 (28)	112 (27)
Other	599 (38)	115 (28)
Age, years; mean (SD)		36.8 [10.5]
18–29	Not available	142 (34)
30–39	Not available	131 (32)
40–49	Not available	96 (23)
50+	Not available	46 (11)
Gender
Male	Not available	100 (26)
Female	Not available	288 (73)
Not stated	Not available	5 (1)
Ethnicity
White	Not available	212 (51)
Black, Asian and Minority Ethnic (BAME)	Not available	182 (44)
Not stated	Not available	21 (5)
Redeployed
No	N/A	250 (60)
Yes	N/A	165 (40)
Redeployed to
Not redeployed	N/A	249 (60)
Covid-19 ward medicine	N/A	50 (12)
Intensive treatment unit (ITU)	N/A	70 (17)
Other	N/A	46 (11)
Symptoms of Covid-19
Asymptomatic	N/A	367 (88)
Symptomatic	N/A	48 (12)
Covid-19 Testing
Not tested	N/A	258 (62)
Tested	N/A	157 (38)
Training for redeployment *
None/Very Poor/Poor	N/A	54 (32)
Neutral/Good/Very Good	N/A	113 (68)
Support during redeployment **
None/Very Poor/Poor	N/A	40 (25)
Neutral/Good/Very Good	N/A	120 (75)
Total	1039 (100)	415 (100)

* n = 167, ** n = 160 SD: Standard deviation; BAME: Black, Asian and Minority Ethnic; ITU: Intensive treatment unit.

3.1. Comparison

Distribution of overall SAQ scores by professional group is demonstrated in Figure 1. Scores for SAQ subscales at baseline and during Covid-19 are shown in Figure 2. In terms of the distribution across professional groups, overall SAQ scores for nurses were highest in 2017 (0.73), with doctors and other clinical staff scoring 0.69 and 0.67, respectively. At baseline, there was no statistically significant difference in median overall SAQ score between staff groups (χ2(2) = 2.896, p = 0.2351) During the Covid-19 pandemic, nurses had the lowest overall SAQ scores at 0.61, with doctors and other clinical staff scoring 0.82 and 0.73, respectively. There was a statistically significant difference in median overall SAQ score between staff roles in the Covid-19 cohort (χ2(2) = 35.499, p = 0.0001).

Figure 1

Violin plot demonstrating Safety Attitudes Questionnaire (SAQ) scores at baseline and during Covid-19, by profession. Width represents the proportion of respondents with a given score. Central line represents the median. Box represents interquartile range. Lines represent the range of scores. Dots represent outliers (more than 1.5 times the interquartile range). Scores represented as percentages.

Figure 2

Radar plots demonstrating SAQ subscale scores by professional group in the baseline 2017 cohort and the Covid-19 cohort. At baseline, nurses and other clinical staff had similar subscale scores.

3.2. Regression Analysis

On univariate analysis of the responses from the Covid-19 cohort, age 40–49 years (p < 0.01), non-disclosed gender (p < 0.001), BAME ethnicity (p < 0.001), nursing job role (p < 0.001), other clinical staff job role (p < 0.01) were predictors of lower SAQ scores. On subgroup analysis of those redeployed, age 40–49 (p < 0.05), BAME ethnicity (p < 0.01), nursing job role (p < 0.01) and having symptoms (p < 0.01) predicted lower SAQ scores. Training for redeployment (p < 0.001) and support during redeployment (p < 0.001) predicted higher SAQ scores.

On multivariate analysis, controlling for all other covariates in the model, non-disclosed gender (−0.13, −0.26–0.00), non-disclosed ethnicity (−0.11, −0.22–0.00) and nursing role (−0.15, −0.24–0.06), remained significant predictors of low SAQ scores. In the redeployed subgroup (n = 139), support (0.29, 0.14–0.44) and nursing role (−0.12, −0.23–(−0.01)) persisted to significance. Detailed results of the quantile regression analysis are depicted in Table 2.

Table 2

Quantile regression of SAQ scores during the Covid-19 pandemic.

Covariate (Baseline)	Median	Univariate Analysis (All Respondents)			Multivariate Analysis (All Respondents)			Univariate Analysis (Redeployed Subgroup) §			Multivariate Analysis (Redeployed Subgroup) §
	SAQ Score	Coefficient	95% CI		Coefficient	95% CI		Coefficient	95% CI		Coefficient	95% CI
Age (18–29)	0.76	-	-	-	-	-	-	-	-	-	-	-	-
30–39	0.70	−0.06	−0.14	0.02	−0.03	−0.14	0.08	−0.03	−0.28	0.22	−0.01	−0.16	0.14
40–49	0.64	−0.13 **	−0.23	−0.03	−0.06	−0.13	0.01	−0.21 *	−0.41	−0.01	−0.06	−0.23	0.11
50+	0.73	−0.03	−0.11	0.05	−0.04	−0.14	0.06	0.00	−0.14	0.14	−0.01	−0.16	0.14
Gender (Male)	0.76	-	-	-	-	-	-	-	-	-	-	-	-
Female	0.73	−0.03	−0.11	0.05	0.00	−0.08	0.08	0.11	−0.04	0.26	0.01	−0.13	0.11
Not stated	0.42	−0.34 ***	−0.42	−0.26	−0.13*	−0.26	0.00	-	-	-	-	-	-
Ethnicity (White)	0.77	-	-	-	-	-	-	-	-	-	-	-	-
BAME	0.61	−0.16***	−0.23	−0.09	−0.08	−0.18	0.02	−0.18 **	−0.29	−0.07	−0.08	−0.21	0.05
Not stated	0.74	−0.03	−0.13	0.07	−0.11 *	−0.22	0.00	-	-	-	-	-	-
Role (Doctor)	0.82	-	-	-	-	-	-	-	-	-	-	-	-
Nurse	0.61	−0.21 ***	−0.32	−0.10	−0.15 ***	−0.24	−0.06	−0.15 **	−0.27	−0.03	−0.12 *	−0.23	−0.01
Other clinical staff	0.73	−0.09 **	−0.16	−0.02	−0.05	−0.13	0.03	0.00	−0.07	0.07	−0.04	−0.16	0.08
Division (Medicine)	0.68	-	-	-	-	-	-	-	-	-	-	-	-
Surgery	0.62	−0.08	−0.17	0.01	-	-	-	−0.15	−0.31	0.01	-	-	-
Other	0.69	−0.02	−0.09	0.05	-	-	-	0.03	−0.11	0.17	-	-	-
Redeployed (No)	0.75	-	-	-	-	-	-	-	-	-	-	-	-
Yes	0.70	−0.04	−0.11	0.03	−0.07	−0.14	0.00	-	-	-	-	-	-
Redeployed to (Not redeployed)	0.74	-	-	-	-	-	-	-	-	-	-	-	-
Covid-19 ward medicine	0.67	−0.07	−0.23	0.09	-	-	-	-	-	-	-	-	-
ITU	0.73	−0.01	−0.07	0.05	-	-	-	0.09	−0.03	0.21	-	-	-
Other	0.65	−0.11	−0.25	0.03	-	-	-	−0.01	−0.20	0.18	-	-	-
Had symptoms (Asymptomatic)	0.73	-	-	-	-	-	-	-	-	-	-	-	-
Symptomatic	0.66	−0.09	−0.27	0.09	-	-	-	−0.26 **	−0.46	−0.06	−0.10	−0.22	0.02
Tested (No)	0.73	-	-	-	-	-	-	-	-	-	-	-	-
Yes	0.67	−0.06	−0.15	0.03	-	-	-	−0.09	−0.27	0.09	-	-	-
Training (None/Very Poor/Poor) n = 167	0.52	-	-	-	-	-	-	-	-	-	-	-	-
Neutral/Good/Very Good	0.76	-	-	-	-	-	-	0.21 ***	0.07	0.35	0.07	−0.04	0.18
Support (None/Very Poor/Poor) n = 160	0.40	-	-	-	-	-	-	-	-	-	-	-	-
Neutral/Good/Very Good	0.74	-	-	-	-	-	-	0.34 ***	0.22	0.46	0.29 ***	0.14	0.44

* p < 0.05; ** p < 0.01; *** p < 0.001 § n = 139, BAME: Black, Asian and Minority Ethnic; ITU: Intensive treatment unit.

3.3. Error Reporting

Weekly patient safety incidents are demonstrated as an XmR Control chart in Figure 3. Following the arrival of the first Covid-19 case in the trust, error reporting significantly reduced. In the 8 weeks following the first Covid-19 patient arriving at the trust, the number of weekly error reports consistently fell below the 52-week mean. For 6 of the 8 weeks, the rate was more than 3 standard deviations below the weekly mean. When broken down by type, “No Harm” and “Near Miss” reports were significantly reduced (p < 0.003). However, the number of “Harm” incidents reported did not significantly change.

Figure 3

Incident Reporting Year to Date.

4. Discussion

To the authors’ knowledge, this is the first study investigating safety culture during the Covid-19 pandemic. Due to changes in the clinical context and staffing, a direct comparison between the baseline cohort and the Covid-19 cohort may be subject to limitations. However, having SAQ data from 2017 allowed for the establishment of baseline scores within the trust, and comparisons between groups at each time point will still provide valuable insights. During Covid-19, nurses demonstrated lower SAQ scores compared with doctors and other clinical staff. This contrasts with the baseline cohort where nurses had the highest safety attitude scores, albeit with no statistically significant difference in overall SAQ score between staff roles. In the Covid-19 cohort, whilst controlling for other covariates, non-disclosure of gender, non-disclosure of ethnicity and the nursing job role were predictive of a poor SAQ score. In contrast, provision of support was predictive of a high SAQ score.

The results may reflect the unequal accessibility and effectiveness of current efforts to support staff. There have been both national and local initiatives to support staff during the Covid-19 pandemic such as access to free food, staff counselling services, and transport allowances. Nationally, the “clap for carers” campaign, commercial discounts, and social media initiatives may have also contributed. Other possible explanations include a sense of solidarity in which multiple staff came together to overcome the challenge of a novel disease. The lower scores for working conditions and job satisfaction in the nursing staff group compared to other roles may suggest that not all staff had equal access to support initiatives or that such initiatives were not equally effective for all staff.

The finding that staff who did not disclose gender or ethnicity had lower SAQ scores, may reflect a wish to avoid being identified for negative views or responses on a survey, especially on questions relating to perceptions of management. That being said, this group of respondents were very few in number and as such the true significance of this finding is unclear.

Environmental factors, working conditions, cognitive load, stress, and fatigue are all factors that impact on individual performance and may have differentially affected staff during the Covid-19 pandemic. Safety climate and perception of management decreased in nurses, which may reflect experiences of topical issues such as provision of personal protective equipment provision or testing. Staff groups spend different amounts of time with Covid-19 patients and therefore may have differing requirements, concerns and experiences of such resources or initiatives. Interestingly, there was no significant association between staff testing and SAQ score.

Support during redeployment was associated with significantly higher SAQ scores. This suggests that in addition to the intended effects of improving staff wellbeing, support programmes may be associated with improved perceptions of safety. However, a prospective design would be needed to demonstrate causality. The specific aspects of support perceived positively by staff also warrant further investigation.

4.1. Incident Reporting

There was a significant reduction in the overall rate of incident reporting following the onset of Covid-19. However, no statistically significant change was seen in the number of incidents reporting patient “harm”. Incident reporting is well recognised as an indicator of safety culture [21], however it is unclear if that association would persist under conditions of the pandemic.

This reduction in reporting may be explained by: staff having less time to file incident reports due to increased workload, a change in the perceived relative importance of error reporting, or fewer patient safety incidents occurring due to changes in the nature of work, such as the cessation of routine services. Irrespective of aetiology, the reduction in incident reporting represents potential lost opportunities to learn and improve safety, which is of importance at such an unprecedented time for service delivery.

4.2. Recommendations for Future Pandemics

Further information is needed to understand why personal characteristics were associated with perceptions of safety. However, through well designed and targeted support, it may be possible to mitigate some of these differences.

Organisational factors such as shift patterns, redeployment opportunities, and occupational risk management may represent targets for interventions. Besides that, incident reporting is an area of particular importance during a change in service provision, as this provides opportunities to understand what is impacting on safety in a particular healthcare context. Initiatives should therefore be considered that facilitate staff to incident report even during pandemics.

Quality improvement and patient safety teams can support the pandemic response. Gurses et al. [22] have described the importance and potential benefits of using human factors experts as part of a pandemic response. The group suggested that integration of human factors principles could help “design, adapt and reconfigure work systems, maximise individual and team performance under high-risk, high-stakes environments, while minimizing the introduction of new significant safety risks or unintended consequences into the work system”. Staines et al. have proposed a five-step strategy to support organisations’ responses during the Covid-19 pandemic [23]. Steps proposed place emphasis on clear communication, just-in-time guidelines and training, adequate support for patient and staff wellbeing, improving operational flow, and risk management. The strategy provides a framework for rapid implementation should further peaks of the Covid-19 pandemic occur and can be utilised in other pandemics or protracted major incidents. Such initiatives may contribute to improved safety culture.

4.3. Limitations

Both surveys had a limited response amongst staff introducing a possible response bias. This was more pronounced during the Covid-19 sample (5% compared with 15%). Although the response rate was low, the absolute number of respondents represented a large number and diverse range of staff. Prior research has highlighted the poor response rates common in surveys of clinicians [24]. This may have been accentuated by changes in the nature of work, increased workload, time pressure, or survey fatigue during the Covid-19 pandemic. In instances where response rates are likely to be low, previous work [25] has highlighted the importance of collecting data using a combination of methods. To address potential response bias, we made use of both email invitations and in-person sampling to capture responses from individuals that might not have otherwise participated.

The clinical context and workforce were different at the two points in time. Whilst differences between professional groups at either point in time are likely to be valid, drawing inferences regarding the changes in absolute scores over time is subject to limitations. Nevertheless, the sample was from a large teaching hospital with a diverse workforce, and our sample was representative of the underlying workforce demographics. In the Covid-19 cohort, for example, 73% of respondents were female. Broadly in line with the NHS workforce, which is 77% female [26].

The categories and rate of incident reporting have been described; however, the underlying incidents have not been analysed. It is therefore not possible to identify whether the decrease in incident reporting suggests changing safety culture, attitudes about reporting or a change in the underlying nature of work.

Finally, this study investigates safety attitudes which, while providing valuable insight, may not provide the complete picture on patient safety during the Covid-19 pandemic. Further research on clinical outcomes during this period is warranted.

5. Conclusions

This is the first study to investigate safety culture during the Covid-19 pandemic. Using a cross-sectional survey-based design alongside incident reporting data, we have identified significant differences in safety attitudes between professional healthcare worker groups, factors affecting safety attitudes, as well as changes in incident reporting before and during the Covid-19 pandemic. Within the Covid-19 cohort, nursing role was associated with lower SAQ scores as compared to other roles. Support was associated with better SAQ scores. The rate of incident reporting reduced following the onset of the pandemic. These findings underline the importance of targeted high-quality support during future pandemics.