The value of qualitative data in Quality Improvement Projects in the care of older adults: the case of frailty scores in the emergency department.

This commentary discusses the role and value of qualitative data when undertaking quality improvement (QI) focussing on the care of older adults. To illustrate this, we reflect on our own experiences of planning a QI project to improve the documentation of Clinical Frailty Scale (CFS) scores in the emergency department (ED) during the coronavirus disease of 2019 (COVID-19) pandemic. National clinical guidance for COVID-19 states that all adults over the age of 65 should be given a CFS at the first point of contact during hospital admission. Therefore, there is a need to improve CFS documentation, specifically in acute care settings. We describe how qualitative methods facilitated an understanding of the barriers to CFS documentation in ED. Staff see the CFS as a useful tool for inter-professional communication, though there are tensions between clinical guidance and their beliefs. Staff had moral concerns about how an ED-allocated CFS might limit available treatment options for older adults. Our findings demonstrate how qualitative methods can illuminate the important social and moral dimensions of why improvement does or does not occur. © Crown copyright 2022.

Value of qualitative data when undertaking Quality Improvement Projects.

Clinical Frailty Scale (CFS) in the Acute Care Setting.

Understanding staff belief’s when implementing change.

Introduction

In this commentary, we argue that qualitative data is important when undertaking Quality Improvement Projects (QIP) in the care of older adults in complex health care systems. To illustrate this, we reflect on our experiences of planning a QIP to improve the documentation of frailty scores in the emergency department (ED) during the coronavirus disease of 2019 (COVID-19) pandemic. We start by describing the use of the Clinical Frailty Scale (CFS) scores during the pandemic, before outlining the qualitative methods we used to explore ED staff’s understanding, beliefs and concerns about the CFS [1]. The qualitative data collected reveal multiple barriers to the documentation of CFS in the ED. Interestingly, we draw attention to tensions between clinical guidance and professionals’ beliefs. In doing so, we demonstrate the importance of methodology and theory in QIP by showing how qualitative data can illuminate the social and moral dimensions of a seemingly simple clinical task, such as documenting a CFS.

Driver diagram for improving accuracy of CFS recording in ED.

Frailty scoring during COVID-19

On the 20th March 2020, the National Institute for Health and Care Excellence published rapid guidance advising that all adults over the age of 65 should be allocated a CFS score on admission to hospital and that clinicians use this score to guide decisions around admission to critical care [2]. This guidance was produced in response to the anticipated unmanageable increase in demand for the National Health Service in the United Kingdom, with particular concern about the availability of critical care beds and ventilators [3].

There is broad agreement that in the context of health system overwhelm, a strategy for resource allocation is necessary. [4] However, academics and practitioners alike have raised concerns about using the CFS to guide decisions about withholding treatment. [5] Attention has been drawn to the limited training of clinical staff and lack of expert oversight of the CFS’s widespread implementation; the potential of the CFS to augment existing socio-economic health inequalities and the ethical implications of using the CFS to ration access to treatment for older people in the absence of predicted resource scarcity [3, 4]. Rockwood called for more research on the CFS’s predictive validity in critical care and the impact of rationing on the CFS’s acceptability to people who live with frailty [4].

Despite these concerns, there is a drive in UK healthcare for the documentation of a CFS score to those over the age of 65: The Specialised Clinical Frailty Network advises a CFS score should be given on ‘first point of contact’ to those over the age of 65, for example [1]. Adherence to these guidelines has been the focus of Quality Improvement and Audit in NHS acute care settings. Practitioners use quantitative targets and analysis to focus on the proportion of older people given a CFS score and the accuracy of such scores [6, 7].

QI processes tend to use univariate quantitative data to measure outcomes. This is partly due to QI’s roots in the financial sector, where ‘success’ is commodifiable [8]. It also reflects the practicalities of doing QI in clinical settings where time, resources and experience can be scarce. Such quantitative measurements can overlook the complex moral, social and political context of healthcare. QI leads should be mindful of what dimensions of care their methodological choices draw attention to, and what is overlooked. Using qualitative theory and methods can excavate the nuances and complexities within healthcare systems providing care to older people. Ultimately, this leads to a better understanding of ‘why improvement does or does not occur’ [9].

Methods

This considered, before designing a QI ‘intervention’ to improve compliance with CFS documentation in the acute setting, we wanted to explore ED staff’s opinions, beliefs and understanding about documenting CFS. This was especially important given the contestations surrounding the CFS’ use during the pandemic. We focussed on the ‘plan’ stage of the Plan, Do Study Act cycle. We spent time in the ED of a tertiary NHS hospital, shadowing individuals involved in allocating CFS scores, including paramedics, nurses and doctors and undertook a face-to-face mixed-methods survey. Questions focused on: approach to CFS scoring; concerns about the CFS; and beliefs about the advantages of the CFS. Responses were either numerical-rating questions between 1 and 10 or free form responses. The survey led to further conversation about the processes around CFS in the ED.

Findings

Shadowing key processes, such as triage and medical reviews, identified how CFS scores were produced, communicated and documented. By observing these processes, we were able to identify some of the barriers to CFS documentation in specific settings. Triage coordinators and nurses pointed out the difficulty of accurately obtaining information about baseline functional state within the confines of triage: ‘there is not enough background patient information’ and ‘we have no means of calculating this in ED nursing documentation’. Recognising the affront to working ‘rituals’ that QI can instigate, we explored how individuals might react to new procedures and steps [10]. Recording CFS at triage would require evolution or redesign of the triage role, which may challenge individuals who had defined, understood and refined their role over years of experience.

The qualitative survey data elucidated staff’s beliefs and opinions about using CFS in the ED. Though most ED staff believed the CFS is a useful tool that aids inter-professional communication, there was concern about how the CFS might be used in decision-making. Some questioned the usefulness of CFS in the ED:

‘From an ED nurse’s perspective, CFS does not really change how care is given in the department’.

Many used the CFS as part of decisions about what treatment should be offered to older people:

‘I’ve only used it since Covid and the need for rapid escalation decisions’.

‘A [person with] CFS 7 with acute surgical presentation, this has impacted surgical vs conservative management, clinically their incarcerated hernia wasn’t treated or imaged’.

Others had moral concerns about how an ED-allocated CFS score might limit available treatment options for older adults:

‘I’m worried about getting wrong score, which may affect outcome’.

We used thematic analysis to analyse our qualitative results. From this we were able to derive primary drivers of successful and accurate documentation and map specific areas of concern amongst staff (here termed secondary drivers) to them. The QI group was then able to devise a series of ‘Change ideas’, which directly related to the concerns, ideas and wishes of staff groups. We produced a driver diagram, which was presented to an expert committee tasked with re-evaluating frailty processes throughout the hospital.

Discussion

Healthcare systems that care for older adults contain a level of complexity often left unsolicited by conventional quantitative QI projects, which primarily focus on a narrow range of numerical targets and analysis. [10] Using qualitative methods can elucidate hard-to-reach complexities and help understand the processes that resist or facilitate improved care quality. By being attentive to the practical and moral considerations of the ED staff, we drew attention to the tensions between their beliefs and national clinical guidance.

Individuals were concerned about CFS documentation’s moral and political dimensions—namely, the extent to which CFS should influence decisions about ceiling of care. This echoes existing literature [5, 11]. Clinicians’ and researchers’ questions about how CFS should feature in decisions about ceiling of care in the wake of COVID are ongoing. It may be that until the role of frailty scores in decision-making has been clarified, there will be resistance to allocating CFS scores to all people over the age of 65. This tension highlights the imperative for national guidance and policymakers to involve staff of all grades when implementing changes to practice.

The longer form responses allowed us to focus on a more nuanced perspective of the context in which we might facilitate change and introduce interventions. The quantitative data we collected failed to do this. By moving away from the usual top-down approach to process change in healthcare, it is anticipated that qualitative QI projects might achieve more sustainable change, though this is an area in need of research.

The perspective of older people is missing from our work and vital to the better fitting of CFS in systems of care. Kuylen et al.’s work illustrates how this can be formulated into our ethical understanding of triage [12]. We did not engage with a specific observation framework, such as Spradley 1979, which would have further developed our study [13]. Ultimately, we hope that by offering a commentary of our experiences, we encourage others undertaking QI to consider how methods suppress and spotlight particular aspects of a complex issue. Engaging with qualitative methodologies and identifying a specific theory of change to structure our data allowed us to better design QI steps in complex systems caring for older adults.