Quality improvement in a crisis: a qualitative study of experiences and lessons learned from the Irish National Ambulance Service response to the COVID-19 pandemic.

OBJECTIVES: The COVID-19 pandemic has produced radical changes in international health services. In Ireland, the National Ambulance Service established a novel home and community testing service that was central to the national COVID-19 screening programme. This service was overseen by a multidisciplinary response room. This research examined the response room service, particularly areas that performed well and areas requiring improvement, using a quality improvement (QI) framework.
DESIGN: This was a qualitative study comprising semi-structured, individual interviews. Maximum variation sampling was used. The data were analysed using an established thematic analysis procedure. The analysis was guided by the framework, which comprised six QI drivers.
SETTING: Response room employees, including clinicians, dispatchers and administrators, were interviewed via telephone.
RESULTS: Leadership for quality: participants valued person-oriented leadership, including regular, open communication and consultation with staff. Person/family engagement: participants endeavoured to provide patient-centred care. Formal patient feedback mechanisms and shared decision-making could be beneficial in the future. Staff engagement: working in a response room could affect well-being, though it also provided networking and learning opportunities. Staff require support and teambuilding. Use of improvement methods: improvements were made in a relatively informal, ad hoc manner. The use of robust methods based on improvement science was not reported. Measurement for quality: data were collected to improve efficiency and accuracy. More rigorous measurement would be beneficial, especially formally collecting stakeholder feedback. Governance for quality: close alignment with collaborators and clear communication with staff are essential. Information and communications technology for quality: this seventh driver was added because the importance of information technology specially designed for pandemics was frequently highlighted.
CONCLUSIONS: The study provides insights on what worked well and what required improvement in a pandemic response room. It can inform health services, particularly emergency services, in their preparation for additional COVID-19 waves, as well as future crises. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

The qualitative design of this study facilitated the collection of rich data about the experiences of staff in a National Ambulance Service (NAS) in the midst of a pandemic.

This study was conducted in a recently established COVID-19 Response Room, which oversaw the delivery of a national COVID-19 home and community testing service, and thus provided unique insights on an entirely new service within a NAS.

The study employed several techniques for enhancing the rigour of qualitative research, such as peer debriefing and disconfirming evidence analysis.

This was a novel application of the Health Service Executive Framework for Improving Quality, which is a healthcare quality improvement (QI) framework, in the context of pandemic-imposed change.

A potential limitation of this study was that it used deductive thematic analysis, which could mean that important concepts or patterns that did not fit with the QI framework were overlooked.

Introduction

The COVID-19 crisis, which the WHO declared a pandemic in March 2020, has required health services around the world to undergo radical and rapid changes.1 2 COVID-19 is a highly transmissible disease that can have an array of short-term and long-term adverse health consequences, including respiratory, cardiovascular, neurological and psychiatric consequences.3 4 It has necessitated the introduction of numerous healthcare policies and practices, including infection control procedures, screening and vaccination programmes, and telehealth interventions.5 It has also prompted the development of quality improvement (QI) methods and tools suited to evaluating pandemic-imposed change, such as rapid learning cycles and after action reviews.1 6 QI typically entails using a systematic approach to design, test, measure and implement changes to improve the safety, effectiveness and experience of care for the benefit of patients and healthcare workers.7 In times of crisis, QI approaches must be adapted and streamlined so that improvements are implemented quickly without unnecessarily burdening strained services and staff.1 6 However, it is important that the validity and scientific rigour of these QI approaches is preserved, despite the urgency of the crisis, which has seen the rapid dissemination of large volumes of healthcare research of variable quality.8–11

In Ireland, the first known case of COVID-19 was reported on 29 February 2020.12 In subsequent weeks, the Irish Government introduced various public health measures to curb the spread of the virus, including travel restrictions, closure of schools and non-essential businesses, cessation of large indoor gatherings, contact tracing for potential cases, and stay-at-home and social distancing measures, in addition to increasing funding for the Health Service Executive (HSE) and re-registering former healthcare workers.13 Ireland has since undergone several waves of COVID-19, including a substantial third wave during the 2020 Christmas period.14 Public health measures, including strict lockdown procedures and mask/face covering regulations, were implemented to mitigate these waves.12 At the time of writing, the nation had recently been heavily impacted by the Delta variant, which brought the total number of deaths from COVID-19 to over 5600 individuals, representing a death rate of approximately 1.1%.15 However, Ireland has implemented one of the strongest vaccination campaigns in the European Union, which has resulted in over 89% of the eligible population (ie, those aged over 12 years) being fully vaccinated since its launch in December 2020.16

The National Ambulance Service (NAS) has been at the forefront of the HSE response to the COVID-19 pandemic in Ireland since the first wave. This required NAS to undergo swift and substantial changes, including the urgent implementation of a COVID-19 testing service, which was central to the national COVID-19 screening programme.17 18 To oversee the delivery of this service, NAS established the COVID-19 Response Room in its headquarters in Dublin. This was the only centre in Ireland dedicated to managing all referrals for COVID-19 tests and organising home testing for patients who cannot attend mass testing centres, as well as on-site testing for individuals living and working in residential facilities, such as nursing homes. It is staffed by a single team who were drawn from a variety of disciplines and professions, including general practitioners (GPs), nurses, administrators, dispatchers and call-takers. Some were recruited from within NAS. Others were recruited from different health service sectors or the Irish Defence Forces and thus had not previously worked in an ambulance dispatch environment. The response room structure and processes have evolved over the course of the pandemic and will continue to evolve while the pandemic endures. Moreover, the response room could be reinstituted in the event of future pandemics or similar emergencies. Therefore, this research aimed to examine the quality of the response room service by identifying practices and innovations that worked well, in addition to those that did not work well, in order to prepare for further COVID-19 waves or new crises, as well as to improve routine service. This examination was guided by a national QI framework: the HSE Framework for Improving Quality.19 The research will inform the preparation of health services, particularly Emergency Medical Services (EMS), for future crises.

Methods

Design

This was a qualitative study that was designed to explore staff experiences and perceptions of the NAS COVID-19 Response Room service. The research paradigm was pragmatism, as this study was a component of a mixed methods evaluation of the NAS response to the pandemic.20 The full protocol of this evaluation was published prior to the present qualitative study being conducted.17 The qualitative method used in this study was the semi-structured, individual interview. This technique entails a discussion between a researcher and a participant that is guided by a flexible interview schedule comprising a shortlist of opening questions and a range of potential follow-up questions and probes.21 The schedule is typically designed to ensure that data are captured relating to core topics while maintaining sufficient flexibility to allow participants to bring their own perspectives and personalities to the fore.22 Amendments can be made to the schedule as the study progresses, including removing ineffective questions or adding probes to explore new issues raised by participants in previous interviews.21 The semi-structured approach enables the researcher to develop rapport, discuss sensitive or complex topics, explore unexpected responses and identify issues valued by the participants.21 23 In this study, the interview schedule was reviewed and, where necessary, refined following each interview, though its core content remained the same across the interviews (online supplemental file 1). The interviews took place between 7 December 2020 and 5 January 2021, when Ireland had entered a third COVID-19 wave.14 By this time, NAS had screened many tens of thousands of patients for COVID-19. Physical distancing regulations and travel restrictions meant that participants were interviewed via telephone. The study has been reported in line with the Standards for Reporting Qualitative Research (online supplemental file 2).24

QI framework

This research was informed by the HSE Framework for Improving Quality, which aims to guide the thinking, planning and delivery of healthcare in order to foster a culture of quality that continuously seeks to provide safe, effective and person-centred care.19 It comprises six QI drivers.

Leadership for quality

This entails cultivating a culture of continual improvement that places patients at the centre of care planning and delivery and that enables staff to provide safe, effective and compassionate care. It includes setting clear aims, regularly communicating with and listening to patients and staff, seeking evidence about service quality and demonstrating a commitment to a culture of quality.

Person and family engagement

This means listening to and learning from patients and their families and involving them in designing, planning and delivering care so that it meets their needs, considers their preferences and focuses on outcomes they value.

Staff engagement

This involves supporting teamwork, promoting a culture of respect, creating a healthy workplace, facilitating learning and development, and partnering with staff to design, deliver and improve services.

Use of improvement methods

QI should entail using established improvement methods, prioritising proven solutions to reduce harm, focusing on standardisation and availing of the unique knowledge of frontline staff.

Measurement for quality

Measuring and analysing data are central to QI by supporting the identification of areas that progressed or underperformed. This includes measuring patient experiences and outcomes, measuring only what matters and building measurement into routine practice.

Governance for quality

Boards and executive management teams should implement the structures, processes, standards and oversight needed for safe, person-centred and effective services. This includes clear leadership and accountability, a culture of learning and strong relationships with patients and staff.

The framework has been applied to hospital services, a national safety programme for pressure ulcers and a social care programme for intellectual disabilities, though it has not previously been used in a pandemic setting.19 25 The current study explored the perceptions and experiences of COVID-19 Response Room staff regarding the six QI drivers.

Patient and public involvement

Patients and the public were not involved in this study.

Participants

In qualitative research, sampling typically concludes when sufficient information has been gathered to produce rich and convincing account of the phenomenon of interest, which differs from quantitative research, where sampling normally ceases when a predetermined sample size that can produce generalisable results has been achieved.26 27 As such, qualitative research can be conducted with relatively small samples, particularly when the topic is clearly defined and the participants are chosen carefully based on their substantial knowledge and experience of the topic.26 In this study, the target sample was the staff of the NAS COVID-19 Response Room. They were recruited via a study invitation email composed by the university research team and sent on their behalf by a NAS manager to former and current staff. They were informed that, if they agreed to participate, they would take part in a confidential interview with an external researcher from a university. They were also informed that the interview would not be used to evaluate their individual work performance, they would receive a copy of their transcript and they would not be identifiable in any subsequent reports. They could contact the university researchers directly for more information and to arrange participation.

It was ensured that recruitment closed as soon as it had been determined that the two criteria of saturation and maximum variation had been fulfilled. This minimised the burden placed on the COVID-19 Response Room staff, who were overseeing a national COVID-19 testing service during the third wave of the pandemic in Ireland at the time of the study. Saturation occurs when no new themes or salient patterns that are relevant to the research aims are identified from the data.28 Saturation was assessed through preliminary data analysis and research team discussions. Maximum variation referred to diversity in the key characteristics of gender and role/occupation.28 29 This process led to the recruitment of nine participants, which represents approximately half of the staff in the response room (table 1). One other person, who was a medical doctor, declined participation due to clinical work commitments during the third wave of COVID-19.

Table 1

Participant demographic information

Demographic category		N
Gender	Female	5
	Male	4
Role in COVID-19 Response Room	Dispatcher	2
	Call-taker	1
	Administrator	2
	Nurse advisor	2
	Clinical staff	2
Previous role(s)	NAS emergency dispatcher	2
	NAS emergency call-taker	1
	Health service manager	3
	Nurse	2
	Clinical student	1
	Member of Irish Defence Forces	1
Status in COVID-19 Response Room at time of interview	Current staff member	4
	Former staff member	5

NAS, National Ambulance Service.

Procedure

The research team gave the participants an information sheet containing detailed information about the study, as well as their contact details for any questions. Informed, written consent was then obtained. Participants were interviewed at their preferred time and location (eg, home, office). Each interview lasted approximately 1 hour. They were conducted by the first or second author, who were respectively a postdoctoral researcher and a research assistant in a medical school in an Irish university. They had no prior connection to the participants. The first author had formal training and experience in conducting qualitative research, including interviewing healthcare workers. The second author received training from the first author and had previous experience of interviewing patients. They maintained field notes and reflexive notes throughout. They met virtually after each interview to determine whether any amendments were required to the schedule. For example, after the initial interviews, they shortened the schedule slightly to ensure that all core topics were covered within an hour. They also made minor adjustments to enhance the clarity of some questions and identified topics that had arisen during the initial interviews that could be further explored in subsequent interviews. The interviews were audio-recorded and transcribed verbatim. All data collected were treated confidentially and stored securely.

Data analysis

Reflexive thematic analysis was conducted in accordance with an established procedure developed by Braun and Clarke (2006; 2019; 2020).30–32 This procedure comprises six main analytical phases: familiarising oneself with the data, generating initial codes, generating initial themes, reviewing themes, defining themes and producing the written report. A theme represents a pattern of response or meaning within the dataset that is salient to the research question.30 In this procedure, the analysis is conducted at a point on the spectrum between primarily inductive analyses, which prioritise data-based meaning, and primarily deductive analyses, which prioritise analyst-based or theory-based meaning.32 33 Inductive analyses are firmly grounded in the data, though they are not purely and exclusively data-driven, as researchers cannot completely free themselves from their assumptions, aims and knowledge of the literature.30 32 Deductive analyses use existing research and theory as a lens through which the data are interpreted, which includes exploring evidence for themes identified by previous research or using an established framework to inform the coding of the data.32 They often include inductive elements, such as inductive coding, generating inductive subthemes within deductive themes, mapping inductive themes onto deductive themes or generating both inductive and deductive themes.33–36 The deductive approach is commonly used to evaluate, explicate, amend or challenge an existing framework, rather than to simply endorse it.36 37 The importance of the framework and its components can increase or decrease as the analysis progresses depending on the researcher’s assessment of its relevance and utility.36 37

In the current study, the analysis was conducted by the first author and was supported by QSR International NVivo V.12 software. The analysis was primarily deductive, as it was underpinned by the Framework for Improving Quality.19 30 This framework had not been developed for or previously applied to pandemic-imposed change. The analysis explored experiences and perceptions of the QI drivers from the framework within a pandemicresponse room. The framework could be amended as part of the analysis because the aim was to ensure that all key patterns from the dataset were adequately represented, rather than to uncritically support the framework. For example, new drivers could be added to the framework or existing drivers could be re-contextualised or removed from the framework. The analysis began with immersion in the interview recordings and transcripts. Subsequently, the entire dataset, including extracts that appeared to be unrelated to the framework, was coded inductively to ensure that nothing of importance was overlooked and that the meaning communicated by the participants was captured.30 34 The codes were then refined and assigned to relevant themes. Peer debriefing enhanced the trustworthiness of the analysis.38–40 Debriefing entailed the first and second authors meeting to compare their interpretations of a subset of four transcripts. No substantial discrepancies were identified, indicating that the analysis was not restricted to the perspective of a single researcher. Additionally, the research team met regularly to discuss the data analysis. Disconfirming evidence analysis was also used to bolster trustworthiness.38–40 This involved returning to the dataset to ensure that the themes had sufficient evidence and that any pertinent contradictory evidence was reported.38–40 Ultimately, six themes were derived from the QI drivers of the framework. They represented the participants’ experiences and interpretations of these drivers in the context of a pandemic response room. Furthermore, a seventh theme, which was also a QI driver, was added so that all salient patterns from the dataset were captured.

Results

There were seven themes that each represented a key QI driver for the COVID-19 Response Room (table 2).

Table 2

Thematic analysis summary

Index	Theme/QI driver	Key points
1	Leadership for quality	Leaders should be person-oriented, rather than task-oriented, and should regularly consult staff.
		Any staff member can provide leadership, such as by finding solutions to problems and by supporting and training peers.
2	Person and family engagement	Staff frequently listened to and learnt from patients and families, though this was usually done informally.
		Formal engagement of patients and families can be difficult but valuable in a pandemic.
3	Staff engagement	Well-being can be affected by working in a pandemic response room, especially during case surges or following difficult cases.
		Peer and organisational support, teambuilding, training and opportunities for variety and networking are valued by staff.
4	Use of improvement methods	Improvement initiatives were implemented, though they did not normally entail the use of established and rigorous methods.
		Improvement initiatives, especially those involving robust and defined methods, were considered important for tackling current and future crises.
5	Measurement for quality	Data collection helped to improve efficiency and accuracy, though it was often unsystematic, especially during the first wave.
		Measurement should be rigorous, especially the collection of feedback from key stakeholders (eg, patients, collaborators).
6	Governance for quality	Strong collaboration and coordination with other organisations (eg, nursing homes, laboratories) is essential.
		Clear and continual communication from senior management to response room staff is crucial.
7	Information and communications technology for quality	Information and communications technology that is specifically designed for pandemics is needed.
		In particular, the technology should be able to process large case numbers and track patients across different databases.

Leadership for quality

Several participants stated that good leadership is essential in a pandemic response room where staff must frequently adapt to new and amended directives and procedures. In particular, the participants appreciated a leadership style that is person-oriented, rather than task-oriented, which includes ensuring staff feel comfortable sharing ideas and reporting mistakes and looking after staff well-being. Interviewee 1 said: ‘That would be the top thing: that [leaders] are not task-orientated but that they … value people and they … allow people to have a good idea and raise it … and that they are just empathetic’. Interviewee 8 stated:

The leadership approach … on the clinical team was brilliant … Very understanding, very approachable … very appreciative of everyone’s contribution. [They] would encourage us to voice our opinions and … to continuously assess … As humans, we all are more inclined to hide [errors] if … we are going to be judged … An approach where, rather than scorning someone for a mistake, ask them: “Okay how can we improve this?”… is invaluable … That’s going to build up their skills [and] create a more effective and efficient environment.

Many reported that the response room manager provided strong leadership, which included securing resources, providing support and boosting morale, thus enabling staff to perform well in a demanding environment. Interviewee 7 explained:

She was absolutely amazing … She got to know people [and] their personalities … It was majorly busy … but we were well looked after … She would bend over backwards to help people out so they knew they could go to her … If we needed extra resources … the resources were provided … She looked after people, and everybody got their head down and got their work done … and it was always done to the highest standard.

Additionally, the participants themselves often provided leadership within the response room, such as by training and supporting their peers, as well as becoming self-sufficient and identifying solutions to problems: ‘I’ve had to train [staff about] how the COVID Response Room works … I really enjoy teaching people … They’ll know the system inside-out’ (Interviewee 6). Interviewee 9 said:

‘We were able to … manage ourselves at times … Leadership isn’t just management. Leadership is getting people to lead themselves as well … One or two leaders that were in there … would teach you … so you can go off and manage it and then maybe teach … the other people’.

Communication and consultation with staff was one aspect of leadership that several participants highlighted as an area that was often done well:

The manager … was very caring and very direct and she allowed you [flexibility] around what you were doing. I never felt I was being told to follow a line I didn’t agree with … I could … make suggestions and she listened … If I found a way of doing it that was better: I did it. (Interviewee 1)

However, some reported that there remained scope for the response room leaders to improve their consultation with staff. In particular, a more formal means of obtaining and implementing their feedback would be of benefit:

‘[It sometimes] felt like a very top-down approach … Whoever is on the ground has most experience with patients … Surely their input should be extremely valuable? … A platform where they can feedback [is needed]. Also maybe an audit … to make sure that input is actually … being taken into consideration’ (Interviewee 8).

Person and family engagement

Many reported listening to and learning from patients and families and providing person-centred care: ‘If a family member rang the room [needing] special arrangements, that was certainly done … There was a lot of communication … with patients and their families’ (Interviewee 7). Interviewee 1 recalled:

This lovely … woman rang … in tears. Her father … was in a nursing home … She spent [his] last hours looking through the window at him … She didn’t know what he died of … The GP wasn’t available and …the laboratory [was] busy … I went on a search … I would ring her: “I haven’t forgotten”. … I eventually got the result … She was enormously grateful that I cared.

It was difficult to formally gather patient and family feedback and involve them in decision-making during a pandemic: ‘Especially in the first wave … it was so fast-paced … Everyone was running off adrenaline … I don’t think there was time to ask for feedback … I don’t think there was going to be a benefit … when we knew it wasn’t perfect … at the time … It was such a scramble’ (Interviewee 8). However, feedback was often obtained informally: ‘[We] would have taken a lot of calls from patients, a lot of emails … We would have talked to patients to explain … if there was a delay or … ask them how they were doing … That was … the feedback mechanism … but it was … informal’ (Interviewee 8). Most agreed that formal engagement could be beneficial in future pandemics, especially in terms of clarifying the role, capacity and procedure of the testing service:

In regards to testing … patient feedback could have helped … People are aware of what is involved now but certainly at the beginning … there was a lack of awareness. [Feedback on] how information could have been delivered better or what they thought of the entire process … could have a benefit … A lot of the descriptions that were going out were from medical experts … Not everyone does understand that terminology. (Interviewee 9)

Staff engagement

A key element of staff engagement is the creation of a healthy workplace. Several participants reported that well-being can be affected by working in the response room, especially during surges of COVID-19 cases or following difficult calls: ‘In the height of [the pandemic], staff were getting … long, long hours … It tires out the crew … Everyone is very down … You are taking call after call … But it’s what … we love to do … Just to help people’ (Interviewee 6). Several noted that well-being support was available from managers and peers. Interviewee 9 said: ‘I would have had one or two bad phone calls … You were able to take that step back … There wasn’t a shortage of people to go to or resources to look after you’. Interviewee 3 said: ‘You had lots of support … Lots of people checking in on you … There was good collegial interaction’. Adequate staffing is also important for well-being: ‘If we knew we were going to be busy, if we could get [extra] cover: that would be a great help’ (Interviewee 4).

Another important element of staff engagement is supporting teamwork and facilitating development. Several commented that the response room provided opportunities for learning and variety: ‘I’m really enjoying what I’m doing. It’s … totally different … I’ve learnt an awful lot’ (Interviewee 4). Interviewee 7 said: ‘Being able to help out and feel like you [are] making a difference was good’. There were also networking opportunities: ‘It’s good for the CV: the connections that you are making … You have more time to get to know the staff, the managers, the officers out on the road’ (Interviewee 2). Some noted that additional training, teambuilding and engagement could help staff adjust to the environment and cooperate more effectively. This is especially important for those who were unfamiliar with an ambulance dispatch environment:

There was segregation: a little bit of … “You are this team, I’m that team”. … At the same time, generally, there was no friction … What struck me was … how regimental it was … There was … a plan for us to have a … training day … but it just didn’t happen … It would have probably helped people to understand … how the structure works, how it’s organised … We would have been able to slot in … faster, (Interviewee 5)

Interviewee 8 said: ‘Everything … stems back to relationships and communication and just to make sure that everyone is in the same boat and ensure that patient care is the end goal’. Unfortunately, teambuilding can be difficult during a pandemic: ‘I would hope when everything quietens down that we could … sit down and have dinner and … talk about war stories … We are not there just yet’ (Interviewee 4).

Use of improvement methods

The participants reported that they had encountered little use or adaption of established and rigorous improvement methods, such as rapid learning cycles, in the response room. Several stated that implementing improvement initiatives can be difficult given the considerable upheaval, unpredictability and increased demands brought about by a pandemic: ‘During the first wave, there was very little time to … look back and see how things could be improved. It was more so a case of … just fighting against the wave … certainly from our level’ (Interviewee 9). Interviewee 6 said: ‘I’m sure there is plenty of ways of improving it and obviously we don’t know [what’s coming]. After Christmas we could be absolutely [inundated] or it could be … calm … We are on the top of the roller-coaster about to tip down … We don’t know what way it’s going to go’.

Nevertheless, improvements were possible in the response room, though they tended to be made on a relatively informal, ad hoc basis: ‘The clinical team … towards the end of the first peak … created a few standard operating procedures and we created lists … with all the important contacts [and] what we thought worked, what we thought didn’t work … That was a really invaluable document … if they were training anyone’ (Interviewee 8). Many provided examples of improvements they had introduced or in which they had been involved, often out of necessity, during the course of their work. Interviewee 1 said: ‘We had to design a [referral] form for … the nursing homes because we were getting the two hundred medications the elderly clients were on … We are not interested in that. We wanted the bare bones … So we designed that and that made life easier’. Interviewee 7 recalled:

We would have … hundreds of emails … on a daily basis for all the referrals … Initially, it was two people looking at the same email account … We didn’t know … if the other person had … actioned the email … I set up a process that, if I … actioned it, … I put a little tick beside it … I did … a process flow chart … that was followed then by anybody that took over the email account … They were too important not to know whether they were definitely actioned or not.

Despite the challenges posed the by pandemic, most were in favour of the implementation of improvement processes, especially more defined and stringent processes, given their importance for managing both current and future crises: ‘It’s good that people are looking to see what worked and what didn’t … That’s always a positive’ (Interviewee 7). Interviewee 8 commented: ‘Continuous assessment … was a good way … to try and find the learnings and findings within the process itself or while doing it’.

Measurement for quality

Data were collected and analysed regarding response room activities in order to support QI, especially enhancing efficiency and streamlining, as well as minimising errors and delays. For instance, Interviewee 8 said: ‘The clinical team … were constantly assessing … “Could we do this more effective and efficiently?” … If anything fell through the cracks, [like] a patient had been waiting longer than … expected, [They] wouldn’t … hide that there was some error or issue … They would always assess … if there’s something that was avoidable’. However, the collection and analysis of data for QI was often unsystematic, especially in the early days of the response room, when procedures and resources were still being put in place. For example, Interviewee 5 recalled efforts to record information about patient care:

NAS were using their [emergency] call system … There was a notes section on that … Otherwise we were using a communications log book [for] sharing information … in relation to the care … being provided … Informally [the manager] did actually call us … to say: “How are we getting on?” … Informally at break times you were talking about how things could improve or better processes.

Interviewee 6 reported that the measurement of call data, especially response times, became more systematic with time:

At the start … they used to print off sheets with all the calls and it would get very messy … Now … everything is recorded … We have to make sure that we don’t miss anybody … It’s all about … ways of improving … We have a system now where … we time each call and [measure] the distance of each call to improve … the speed of the swabbing.

Several participants identified areas where measurement for quality could be improved, including facilitating more rigorous and routine measurement. For example, Interviewee 9 suggested regularly collecting and reviewing appointment data to maximise COVID testing: ‘Make sure that every available opportunity there was to provide a COVID test … was being used … Have a look at was every [test] slot filled … Make sure … people were showing up or [find out] what was going on’. Several recommended formally gathering and assessing feedback from a range of stakeholder groups, including key collaborators: ‘They should [get] feedback … from GPs, from the public health, from the … lab … to try and make sure that we are … meeting the needs’ (Interviewee 5). Patients were another important stakeholder group:

[In] the patient-centred service that we are trying to provide … their input is hugely important … There was one scenario of a lady … Since she had been referred for a COVID test, the home help stopped coming … I called … to ask: “Was she eating? Could she shower?” … When you are so focused on getting a COVID swab, it’s easy to [overlook] the holistic approach to the patient care … Another reason why patient feedback would be extremely important. (Interviewee 8)

Staff were also an important stakeholder group: ‘With [this interview], it’s good to see someone on the outside talking to someone on the inside … You’d be wording it differently if you were speaking to someone … in the job … It’s good to see other people … are interested in … what we do’ (Interviewee 6). This measurement would benefit not only the response room but also improve the management of future crises: ‘Be prepared for future public health shocks … That’s absolutely key … I don’t think you can just close the room and say it no longer exists … There’s a lot of … ongoing work to be done … around evaluation of those processes … Other QIs will emerge out of that’ (Interviewee 3).

Governance for quality

When the response room was established during the first wave, its standards and processes were still being developed. Interviewee 2 explained:

When we started off, it was kind of madness … We just had to learn new procedures, put new policies in place … We had to learn new protocols [about] COVID symptoms … We had a lot of people ringing in thinking they had it … It was all up in the air about who gets tested, who doesn’t … We were looking at maybe a thousand calls … a day.

Interviewee 8 said:

‘A few different … areas … were all trying to come together, which was extremely difficult under the circumstances … There was NAS and the clinical team and [staff] back out of retirement … It was difficult … at the start trying to come up with a strategy and … a system, trying to understand all the different software … and talk to external stakeholders’.

Over time, the governance evolved:

The COVID room was … a combination of people from the [NAS] college, from the Air Corps, from the [NAS] corporate office … and from [ambulance] control … It was well governed … It was good … once that all settled into place … When it started, we were all learning what we needed to do … We’d never dealt with this before but … it was dealt with very positively. (Interviewee 7)

One particular area that required improvement was collaboration and coordination with other organisations, including general practices, residential facilities, laboratories and contact tracing: ‘There was no communication between the ‘Test and Trace’ process and ourselves in the early days. [We] asked them for their communication strategy … I went down through it … There’s nothing about the home test … by the National Ambulance Service, so we made that change’ (Interviewee 3). Some felt that, occasionally, communication from management required improvement: ‘Everyone in a nursing home was going to be tested as part of mass screening … We had found that out through watching the news … Communication in that regard was lacking’ (Interviewee 9). At other times, there was good communication: ‘In the morning … you got a handover and … the urgent stuff flagged to you … The directive would come down. The communication was quite clear’ (Interviewee 3).

Information and communications technology for quality

Most asserted that suitable information and communications technology (ICT) is essential for optimal performance in the response room. Initially, the response room relied heavily on paperwork: ‘We generated so much paperwork … because the ICT infrastructure wasn’t … sufficient … We must have … damaged a rainforest’ (Interviewee 3). Over time, the ICT infrastructure improved, enhancing efficiency and accuracy: ‘We used to have to handwrite all the lists … We’d write down each call individually … Then we started typing up the lists … That cut the workload a bit and then they brought in a report so basically you can … pull all the calls that are active … That cut down the workload’ (Interviewee 2). Several recommended developing ICT infrastructure specifically for managing pandemics:

The system was … built for the dispatchers to deal with accidents … around the country but the COVID-19 cases were all going into that as well so the system hadn’t been designed to actually tackle [that] amount of people … A system developed specifically for … pandemics … would help greatly, or at least designed to have a greater capacity. (Interviewee 9)

Ideally, different health service computer systems and databases should be integrated within the ICT infrastructure, including giving each patient an identification code that can track them across systems:

Everybody [needs] a unique identifier … There’s no doubt … We had a reliance on two numbers. I don’t think you should rely on two … The systems should have spoken to each other … When somebody went up on [the referrals system] it should have populated … straight through and onto [the NAS] system … We created a lot of work in relation to that. (Interviewee 3)

Discussion

This study examined the quality of service in a pandemic response room dedicated to delivering the Irish national COVID-19 home and residential testing programme. It provided unique insights on QI at a time of crisis. Specifically, it explored the experiences and perceptions of pandemic response room staff regarding key QI drivers from the Framework for Improving Quality. The first such driver was leadership for quality. The results underscore the importance of person-centred, rather than task-centred, leadership that entails attending to staff well-being and regularly communicating and consulting with staff, which includes enabling them to openly share ideas and report errors. This finding aligns with best practice recommendations for healthcare leadership in a pandemic, which include distributed leadership and deference to expertise, frequent and open communication, and meeting physical and psychological safety needs.1 41 42 Moreover, the results indicate that any employee can display leadership in this environment by being self-sufficient, using their initiative to solve problems, and by training and supporting colleagues. This somewhat corresponds with the Safety Differently approach to improving safety, which entails creating a culture where staff are viewed as a source of solutions, rather than problems, and where all employees, including frontline workers, are encouraged to take ownership of safety processes.43

The second QI driver was person and family engagement. The findings show that staff endeavoured to provide person-centred care, including listening to patients and their families and tailoring the service to their individual needs. However, pandemic pressures impeded formal engagement, such as requesting feedback and making shared decisions about care design and delivery. In line with published guidance, formal engagement was recommended for future pandemics.44 45 This engagement should be meaningful, rather than tokenistic, such that it entails partnering with patients to design, plan and produce healthcare and sharing leadership, power and decision-making.44 45 This has the potential to improve the clarity of pandemic-related communications, increase adherence to pandemic isolation measures, and enhance the support provided to caregivers during the crisis.45 The third QI driver was staff engagement. The results suggest that working in a response room can affect well-being, especially during case surges. Previous investigations confirm that the well-being of EMS professionals can be impaired by the pandemic, though they largely focused on paramedics and first responders, rather than dispatch or response room staff.46–49 In the current study, staff reported benefiting from organisational and peer support, as well as networking and learning opportunities. Greater teambuilding and training would have bolstered integration and cooperation, particularly for staff recruited from external organisations. Such activities should be virtual, outdoors or distanced to protect staff during the pandemic.

The fourth and fifth QI drivers were use of improvement methods and measurement for quality. In the response room, there were efforts to make improvements and to collect data to enhance efficiency and accuracy. However, there was little utilisation or adaptation of robust improvement methods and measurement that had a foundation in improvement science or established QI models. The substantial workload and uncertainty created by the pandemic, especially the first wave, was frequently cited as the reason for this lack of rigour. This corresponds with reports that the rapid conduct and dissemination of healthcare research during the pandemic frequently diminished its quality and integrity.8–10 For example, the COVID-19 crisis increased the use of novel, untested methods and established but less stringent methods.8 9 However, it is possible to employ and adapt valid QI methods and models during a pandemic, especially where there is appropriate governance and resources, as demonstrated by several recent studies that evaluated new telemedicine services and rapid response capabilities.9 50–52 It is also possible to learn from the repository of literature about previous pandemics, including outbreaks of other respiratory viral infections.53 Quality standards must be upheld, even during crises, in the interests of patient safety, resource allocation, and the trustworthiness of clinical research and practice.9 It should be noted that the current study is part of a project examining the NAS response to the pandemic, which represented a move towards a more formal approach to measurement and evaluation.17 This project also entailed assessing the dissemination of newsletters containing pandemic-related evidence and guidance to NAS personnel and evaluating the sharing of pandemic-related information within an international EMS network.

The sixth QI driver was governance for quality. The findings highlight the importance of close cooperation with collaborating organisations (eg, laboratories, contact tracing and nursing homes) coupled with clear and continual communication with staff. It has previously been recommended that pandemic-related communication between senior leadership and frontline staff should involve establishing a centralised team to review the latest developments and coordinate among departments, disseminating staff-wide information daily and hosting virtual town halls or staff forums approximately two times per week.41 In Ireland, NAS regularly distributed newsletters to staff summarising the latest COVID-19 literature and established a COVID-19 information-sharing network with international EMS representatives.17 Finally, a seventh QI driver was identified in this study: ICT for quality. Specifically ICT that has been designed to manage a pandemic, including processing large case numbers during waves and tracking cases across different databases and systems, is required to optimise efficiency and minimise errors. This finding suggests that the Framework for Improving Quality should be expanded to include this driver, especially when applied to similar contexts.

A limitation of this study was that, though participants from a range of roles, including clinical staff, were interviewed, a medical doctor was not recruited. A representative of this profession could have provided unique insights. However, increased workloads due to the third wave of COVID-19 in Ireland made it difficult to recruit a physician. Another limitation was that the participants were recruited via an email sent by a NAS manager on behalf of the university research team. The university researchers did not have access to the contact details of current and former response room staff due to data protection concerns. Additionally, NAS advised that response room staff would be unlikely to respond to an external email from the university, especially when dealing with the third wave. However, a disadvantage of this approach is that the participants could have been concerned about their anonymity, which could have affected their willingness to openly discuss their experiences and opinions. In particular, they could have felt uncomfortable providing negative feedback. To help counter this, they were assured that the interviewers were external to NAS, the interviews would be confidential and would not be used for performance evaluation, and the participants would not be identifiable in any reports produced. They also received a copy of their transcript, which provided an opportunity to raise concerns or make clarifications, though none availed of this. Additionally, both current and former response room staff were recruited, as the latter could be especially willing to speak freely. Moreover, the results show that the participants were willing to highlight practices requiring improvement. Nevertheless, at least some participants could have remained reluctant to be completely open during the interviews.

Another limitation of this study was that it used a predominantly deductive analysis approach whereby the themes were largely derived from the Framework for Improving Quality. Deductive approaches are especially suited to investigations of a highly specific research question or of a phenomenon of interest that can be illuminated by a relevant, existing framework.30 33 54 In contrast, inductive approaches are especially suited to more explorative studies in which the specific research question evolves during the analysis or in which the aim is to develop a new theory or model.30 Consequently, deductive approaches tend to provide a more detailed analysis of a particular aspect of the dataset, while inductive approaches tend to produce a more comprehensive and rich analysis of the entire dataset.30 Additionally, deductive approaches risk being constrained by the selected framework. For example, important themes that do not match the framework could be overlooked. Conversely, inductive approaches typically produce results that are more strongly linked to the participants’ responses. However, in contrast with this perspective, it has been argued that existing frameworks and orienting concepts can sensitise researchers to relevant issues and interpretations that they might not have identified themselves using an inductive approach, resulting in a rich description of the phenomenon.36 Furthermore, no qualitative analysis is entirely inductive because they cannot be conducted in a vacuum isolated from the researcher’s preconceptions, including their knowledge of relevant literature and theory.30 55 56 In both inductive and deductive reflexive thematic analysis, themes are not viewed as residing within the dataset waiting to be discovered by the researcher or to emerge during the analysis.27 31 Instead, the researcher is regarded as having a central and active role in the construction of themes at the intersection of the data, the analytic process, the research aims and the researcher’s subjectivity, theoretical understanding, training, and experience.27 31

In the current study, efforts were made to minimise the risk of important themes being overlooked due to the deductive nature of the analysis. Specifically, the entire dataset was coded openly, including responses that seemed unrelated the Framework for Improving Quality.30 34 This led to amendment of the framework through the addition of a seventh QI driver, ‘Information and communications technology for quality’. However, it remains possible that some key concepts or patterns were missed. An alternative approach (eg, inductive thematic analysis, grounded theory) would likely have produced different results that were more data-driven, rather than theory-driven. Despite this limitation, deductive analysis has become an established approach in healthcare research, including QI and emergency care research.37 57–60 For example, a recent study applied an existing health psychology framework to the analysis of interviews with first response volunteers in order to identify barriers to automated external defibrillator use.60 Ultimately, both deductive and inductive analyses are acceptable provided that the researcher is transparent about which approach is used and what its implications and limitations are so that the findings can be interpreted and contextualised appropriately.30 35 54

Conclusion

This study examined the quality of service in pandemic response room that coordinated the Irish national COVID-19 home and residential testing programme. It provided unique insights on staff experiences in a NAS in the midst of a crisis and represented a novel application of the Framework for Improving Quality in the context of pandemic-imposed change. The findings indicate that the framework, which comprises six QI drivers (eg, staff engagement, governance), should be expanded to include a seventh driver (ie, ICT), especially when used to address pandemics and similar emergencies. The findings also suggest that, while there were many areas that performed well within the response room (eg, person-centred leadership, patient-centred care), there were also areas requiring improvement. In particular, there was a need for a more formal and rigorous approach to patient and staff engagement and the implementation of QI methods and measurement. This follows calls for research that improves the rigour and quality of improvement practices and evaluations.11 Additionally, there have been calls for clinical research and practice during the pandemic to balance the urgent implementation of change and rapid conduct of studies with the preservation of quality and integrity.9 This study can inform EMS and other health services in their QI activities and in preparation for and response to future public health crises.