Exploring two distinct gentamicin prescribing protocols in UK hospitals: a mixed-methods realist evaluation.

OBJECTIVES: Gentamicin is the aminoglycoside antibiotic of choice in the UK. It has a narrow therapeutic index: underdosing results in inefficacy while overdosing is characterised by nephrotoxicity and ototoxicity. To improve patient safety, hospitals have protocols for the prescription of gentamicin, which vary in complexity and approach. This study aimed to explore two distinct protocols for prescribing gentamicin in hospital settings, in order to understand the mechanisms they trigger and the outcomes they achieve.
SETTING: A mixed-methods realist evaluation explored gentamicin prescribing protocols in two hospital surgical admissions units in South West England between January and August 2018. Site 1 had a traditional, complex protocol, while site 2 took a simplified protocol. PARTICIPANTS: Testing the initial programme theory (IPT) involved semi-structured audio-recorded interviews of a volunteer sample of healthcare professionals (HCPs) involved in the prescribing and administering process, alongside a clinical audit reviewing accuracy of gentamicin prescribing. OUTCOME MEASURES: Three sequential phases were used to identify factors in a successful protocol: IPT generation; testing; refinement of the IPT. The IPT was generated by literature search and analysis of existing protocols of sites 1 and 2. Refinement of the IPT synthesised the results of the quantitative and qualitative research to identify the key characteristics of a successful protocol.
RESULTS: One hundred gentamicin prescriptions were reviewed, with a mean accuracy of gentamicin prescribing at site 1 of 65.67% and at site 2 of 78.79% (p<0.01). Thirty HCPs were interviewed. Key contexts were identified including prescriptiveness, experience and availability of patient information. These triggered hidden mechanisms including uncertainty, fear, confidence and frustration leading to both intended outcomes but also unintended outcomes such as deviation from protocol and unnecessary gentamicin levels.
CONCLUSIONS: A simplified prescribing protocol for gentamicin is better accepted by prescribers, leading to better adherence to protocol and more accurate prescribing. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Interviews with staff enabled us to uncover implicit assumptions about how the prescribing protocols work.

Semistructured interviews were undertaken in order to offer rich insights and enabling discussion of a topic which participants may have otherwise felt uncomfortable to talk about.

The smaller numbers of patients being prescribed gentamicin at site 2 created challenges in terms of data collection.

The incidence of gentamicin associated nephrotoxicity cannot be determined from this study.

This project did not have the scope to repeat interviews at a later stage due to resource limitations and staff turnover; however, additional questions were asked in further interviews with different respondents, which contributed to theory refinement.

Introduction

Approximately one-third of hospital inpatients are prescribed an antibiotic at some point during their hospital stay.1 The error rate of those prescriptions has been reported to affect 7% of prescriptions and up to 50% of hospital admissions.2 Gentamicin is one of the most frequently prescribed aminoglycoside antibiotics3 and also one of the most frequently associated with prescription error.1 Despite its common use, gentamicin is challenging to prescribe due to its narrow therapeutic index. This means that small differences in dose or blood concentration can lead to very different outcomes—slightly too low and it will not be effective; slightly too high and toxicity can result. Perhaps unsurprisingly, clinicians have tended to err on the side of caution and the major problem is underdosing of gentamicin,4 5 which can result in increased patient morbidity and mortality through undertreatment of sepsis. However, overdosing is a real concern too, with reported rates of gentamicin-associated nephrotoxicity varying from 1.2%6 to 55% of prescriptions,7 thus it is common to monitor serum gentamicin levels during treatment.

In response to these challenges, hospitals have developed different approaches for gentamicin prescribing and monitoring. The transition of evidence-based medicine over recent decades has resulted in the development of many treatment protocols and an abundance of guidelines. In secondary care, local guidelines for antimicrobial prescribing are the norm. Traditional practice is to dose gentamicin at 5 mg per kilogram of ideal body weight, with the drug being given once daily, but the evidence basis is quite limited and dated. Some alternative approaches use actual body weight with a dose of 5–7 mg per kilogram or use patient height to calculate the dose instead.8

Also, although hospitals are highly regulated environments, practice ‘on the ground’ may not be the same as that described by the local guidelines. As reported by Public Health England, ‘the emerging evidence on “prescribing etiquette” demonstrates a complex social environment where roles and hierarchy intersect with respect for autonomy and clinical judgement’.9 Key influences on the decision-making process include social and intrinsic factors. The structure of the hospital multidisciplinary team and interactions play a significant role in the process of prescribing.10 Lack of knowledge and training in prescribing (local protocols, drug monitoring), lack of familiarity with the drug or patient, time pressure and heavy workload have all been identified as reasons behind prescribing errors.4 The feasibility of a protocol must also take into account the realities of the nature of clinical practice. In the face of such complexity, solutions addressing a single cause are likely to have only limited benefit and therefore research is needed to establish what works, when and for whom.

This study explored two distinct approaches to prescribing gentamicin in two different hospitals, assessing their impact in terms of clinical outcomes and medication errors, and their feasibility in terms of healthcare professional’s (HCP) experiences of the process. The findings provide a greater understanding of the process of gentamicin prescribing; what works, when and for whom. We hope that these findings will be used to standardise prescribing protocols, making them more likely to be adhered to and shared across settings, improving the quality of care that patients receive.

Methods

Aim and objectives

The aim was to explore two distinct protocols for prescribing gentamicin in hospital settings in order to understand the mechanisms they trigger and the outcomes they achieve through a realist evaluation. There were three objectives:

To elicit an initial programme theory (IPT) by analysis of the protocols that articulate the intended process of gentamicin prescribing in two hospital settings.

To test the IPT using empirical data from a mixed-methods study in order to identify the important contexts, mechanisms and outcomes associated with the actual process of gentamicin prescribing.

To refine the programme theory in order to understand what works (outcome), how (mechanisms) and under what conditions (context) in terms of gentamicin prescribing.

Study design

A mixed-methods realist evaluation design was used.11 The topic of prescribing was well suited to a realist approach as whenever a programme (eg, the prescribing protocol) is implemented it is testing a theory about what might cause change even though that theory may not be explicit.11 One of the tasks of realist evaluation is to make the theories within a programme explicitly developing clear hypotheses about how, for whom and why programmes might work.11 The site Trust protocols were analysed first in order to elicit an IPT (which describes how and why interventions are expected to work). Then a convergent mixed-methods study approach was used to collect empirical data to assess the context, mechanisms and outcomes involved in the ‘actual’ process. Quantitative data collection was in the form of a clinical audit reviewing the accuracy of gentamicin prescribing at each site. Qualitative research involved semistructured audiorecorded interviews of HCPs’ experiences of prescribing gentamicin in those settings. The qualitative and quantitative data were analysed and integrated together. Finally, the data were synthesised to create a final programme theory.

Patient and public involvement

Not applicable in this study. This project reviewed clinical staff’s experience of prescribing gentamicin.

Study setting

The selected sites for this research project were the adult surgical admissions units at two acute National Health Service Trust Hospitals in South West England with different prescribing methods for gentamicin, one based on patient weight (site 1) and one based on patient height (site 2). Site 1 was a larger, urban teaching hospital with 797 inpatient beds; site 2 was a smaller, rural District General Hospital with 423 inpatient beds. Study participants for the qualitative interviews included all grades of doctors, nurses and pharmacists working on the adult surgical admissions units at the two sites.

Analysis of protocols

The IPT was established through ‘desk-based research’ including a literature review and identifying the relevant protocols in current practice at each study site. This documentary analysis enabled us to uncover the ‘intended’ process of prescribing gentamicin.

The process of developing the IPT started by creating a process map as a visual representation of the ‘intended’ prescribing process at site 1 and site 2. The IPT was elicited by a combination of analysis of these local protocols that articulate the intended process of gentamicin prescribing in the two hospital settings, and a literature search.

The literature search was conducted in August–September 2017. The following five bibliographic databases were searched: Embase (Ovid), MEDLINE (Ovid), MEDLINE-in-process (Ovid), the Cochrane Database of Systematic Reviews and the HTA database (all via the Cochrane Library). A combination of free-text and indexing terms were used, including ‘gentamicin’, ‘aminoglycoside’ ‘antimicrobial’, ‘prescribing’, ‘junior doctors’, ‘protocol’, ‘stewardship’ and synonymous terms.

This research enabled us to identify what appeared to be important contexts, mechanisms and outcomes in this process of prescribing. Thus, aiding the process of theory formulation and development of the IPT through analysis of the ‘intended’ process of gentamicin prescribing in the two hospital settings, as described by protocols.

Qualitative data collection

The qualitative data comprised semi-structured audio-recorded interviews of gentamicin prescribing experiences of staff members working on the surgical admissions units at the two sites. Initial contact to potential participants was made by email via rota co-ordinators, attaching a participant information sheet and consent form, inviting participants to contact the lead researcher if they would like to be involved in the study.

The interview schedule was trialled among volunteer staff to develop the research questions in response to topics referenced by the volunteers. This was also used as an opportunity to discuss ideas on how to recruit study participants and how best to disseminate the results. The lead researcher then attended junior doctors teaching sessions to publicise the study. Thirty participants were recruited, 15 at each site.

The interviews were undertaken by the lead researcher and took place in a private office on site, at the participant’s convenience. The interviews were transcribed using the company UK Transcriptions and then uploaded into NVivo V.10 (QSR International Pty, Doncaster, Vic, Australia) (qualitative data analysis software) for analysis.

Quantitative data collection

The quantitative data comprised a clinical audit, which commenced at the same time as the IPT was developed and continued alongside the qualitative research (see figure 1: Study Flow Chart). The retrospective clinical audit reviewed both the accuracy of initial dose gentamicin prescribing according to local hospital protocol, and the renal function of patients receiving gentamicin, over the period January–August 2018. This study was interested in identifying whether ‘actual’ gentamicin prescribing deviated from the ‘intended’ prescribing. Relevant data were recorded as appropriate for the protocol used in order to determine if a dose was accurate. This included the initial prescribed dose of gentamicin; the patient’s sex, height and weight; the patient’s renal function on admission and at 24, 48 and 72 hours postadministration of gentamicin and gentamicin level. Where all the required information was not available, that data set was not included in analysis. On average, data were collected once per week over the study period. Data were reviewed for any subsequent potential negative clinical outcomes, such as acute kidney injury (AKI), that might have been related to the gentamicin. To ensure consistency of approach, gentamicin prescriptions were regarded as ‘accurate’ if they were within 10% of the expected dose according to hospital protocol. This is because a patient’s weight or the subsequently calculated dose of gentamicin, is often rounded up or down12 to allow for ease of administration of gentamicin (which tends to be in 80 mg vials). The data were analysed using a two-tailed t-test in Microsoft Excel.

Figure 1

Study flow chart. IPT, initial programme theory.

Data analysis

Realist evaluation enables the relationship between mechanisms, outcomes and context to be determined and explored.13 The context-mechanism-outcome configuration (CMOC) is used as the main structure for realist analysis,14 and this is how the results (analysis) are described. The IPT was refined based on the findings of the convergent mixed-methods study. The study team moved iteratively from analysis of interviews and initial audit results, developing theories throughout the period of study and subsequently refined the programme theory. To answer the question ‘what works, when and for whom’, we identified and examined underlying generative mechanisms (M) associated with the programme, the conditions or contexts (C) under which the mechanisms operate and the pattern of outcomes (O) produced. Iterative cycles of close reading identifying points of interest in the transcripts and sharing the developing ideas with subsequent interview participants helped with theory refinement. Theories were developed and working propositions through analysis and interpretation of interview extracts. Throughout the analysis, emerging CMOCs were continually compared and contrasted with the developing programme theory, so as to understand and then test the relationships between each CMOC and the programme theory. The theories and working propositions (ie, CMOCs) were then refined through further interview data analysis and interpretation. As this evaluation was a ‘snap shot’ of gentamicin prescribing at the two sites, the analysis stage was used to test and refine propositions between site visits and then, in the final stages, across data sets and sites. A set of thematic codes based on the initial framework was applied to the transcripts using the QSR Nvivo V.10 application to organise the data. The final programme theory describes gentamicin prescribing in the hospital setting: what works (outcome), how (mechanisms) and under what conditions (context).

Ethics

This project adhered to the six core principles for undertaking good research practice as stated by the Economic and Social Research Council.15 The consolidated criteria for reporting qualitative research5 was followed to ensure standards of research were maintained.

Results

Initial programme theory

The IPT was elicited by analysis of the protocols that articulate the intended process of gentamicin prescribing in the two hospital settings. The process maps of the ‘intended’ prescribing process at site 1 and site t2 are depicted in online supplemental material 1 and 2. The important contexts, mechanisms and outcomes identified through the literature research and through analysis of the local policies for gentamicin prescribing are shown in table 1, the IPT.

Table 1

Initial programme theory

Contexts	Length and prescriptiveness of protocol Availability of patient information Complexity of calculations involvement of multiple individuals to acquire all relevant data Involvement of different teams Length of policy
Mechanisms	Fear of negative outcomes (such as incorrect dose, damage to kidneys, delay in prescribing in unwell patients) Frustration Confidence
Outcomes	Adherence to protocol Appropriate dosing (as inappropriate dose or an error such as allergy could lead to serious negative clinical outcomes) Underuse of policies

Qualitative data

Through the semi-structured interviews, the study team gained an understanding of both the intended prescribing protocol and the actual prescribing process. The demographics of the 30 participants are shown in online supplemental table 1. The average interview was 11 minutes in length (range 3–17 minutes) and in total 4 hours and 49 minutes of audio data were collected. Initial inductive analysis enabled us to develop preliminary theories and identify themes which were then tested and refined through realist analysis. The three themes identified were: clinician experience; properties of the protocol and the prescribing environment.

Theme 1: clinician experience

From the literature, it is clear that clinician experience impacts on prescribing habits and errors. The EQUIP Study2 found that foundation doctors were the single group accounting for the largest number of prescribing errors. However, when Trust specific protocols were discussed during interviews, it was found that more senior doctors tended to articulate assumed knowledge about prescribing, which appeared more general and lacked local specific knowledge. Concerns regarding gentamicin and risk of nephrotoxicity were expressed as a recurrent unease. There was also acknowledgement that the association with nephrotoxicity may be historic. Five CMOCs relating to theme 1 were identified:

Clinicians with less experience (C) may feel a greater need to review protocols (M1) or seek advice when uncertainty remains (M2) and therefore may be more likely to follow protocol (O).

As training progresses, more senior clinicians (C) may feel they have gained experience (M1) and have more confidence (M2) prescribing, which may result in reluctance to review protocols (O).

Previous clinical experience of adverse side effects (C) (such as AKI) may cause concern (M) and therefore lead to more cautious prescribing (O).

When colleagues of other health professional teams are anxious about gentamicin prescribing (C), then prescribers may also become concerned (M) leading to more cautious dosing (O1) and unnecessary monitoring of gentamicin levels (O2).

In hierarchical relationships (C), the expectations that juniors know how to prescribe gentamicin (M) leads to juniors either avoiding asking for assistance (O1) or seeking advice from other HCPs (O2).

Theme 2: properties of the prescribing protocol

At site 1, calculating the initial gentamicin dose and subsequent gentamicin levels were cited as being the most difficult and ‘painful’ part of the prescribing process. Delays in renal function and availability of other patient information (height, weight) were also stressed as posing a risk to delaying prescribing. Participants at site 2 reported issues around knowledge or awareness of when gentamicin levels were required. It was clear from the interviews that levels were taken at inappropriate or unnecessary times for both sites. Three CMOCs relating to theme 2 were identified:

When prescribing policies are long, overly specific, prescriptive and/or inaccessible (C), frustration can result (M) leading to the policy being underused (O).

Prescribing policies that are simplified (C1) and easily accessible (C2) are more likely to be engaged with (M) and therefore lead to adherence of local policy (O).

Operational inefficiencies such as IT not working (C) causes a reluctance (M) among clinicians to use it again and therefore it is not used (O).

Theme 3: prescribing environment

Antibiotic prescribing is often time sensitive or in a pressured environment. The evidence from the interviews suggests that despite this, junior clinicians did not feel they had insufficient time to prescribe in a timely manner. However, delay in the availability of patient information appears to lead to prescribing without adequate information despite knowledge of the protocol at site 1. Participants reported that the fear of antibiotic omission was greater than the fear of overprescribing or adverse effects of an inappropriate or incorrect dose. Participants at site 2 acknowledged concerns regarding renal function, however still appeared to be comfortable with the prescribing protocol. Four CMOCs relating to theme 3 were identified:

When prescribing under time pressure (C), clinicians may fear consequences of not administering antibiotics promptly (M) and therefore do not adhere to the prescribing protocol (O).

Prescribing when you do not know the patient (C), causes concern (M) and can lead to unnecessary monitoring of gentamicin levels (O).

Missing clinical information (C), leads to uncertainty about how to proceed (M) but typically leads to treatment even if not per protocol (O).

When patients are reviewed again (C), clinicians often feel fear of criticism by colleagues (M1) or fear of patient deterioration (M2) leading to greater likelihood to prescribe antibiotics (O).

Exemplar quotes to support each CMOC can be found in the online supplemental material.

Quantitative data

Seventy-three prescribing episodes (data sets) were collected at site 1 and 36 prescribing episodes at site 2. The final data set consisted of 67 prescribing episodes from site 1 and 33 from site 2. At site 1, data were collected across 38 days over the period January–July 2018. Between one and four patients on the adult, surgical admissions unit were typically prescribed gentamicin on the data collection days. Each point on the run chart (online supplemental material 1) is the mean accuracy of gentamicin prescribing on each data collection day. In one-third of collection days, there were no ‘accurate’ gentamicin prescriptions completed. When the total data set was reviewed, 44 of the 67 (65.7%) gentamicin prescription doses at site 1 were accurate, with 10.45% overdosed and 23.88% underdosed. At site 2, data were collected on 28 days over the period January–August 2018. On 12 data collection days, no patients were prescribed gentamicin. On three quarters of data collection days, every dose of gentamicin prescribed was ‘accurate’ (online supplemental material 2). When the total data set was reviewed, 26 of the 33 (78.8%) gentamicin prescription doses at site 2 were accurate, with 9.09% overdosed and 12.12% underdosed. Site 2 was more likely to adhere to prescribing gentamicin according to hospital protocol than site 1 (78.79% accurate prescriptions at site 2 in comparison to 65.67% at site 1) and this was statistically significant with a p value of 0.0038.

Seven patients at site 1 received an overdose dose of gentamicin. Of these seven patients, six were subsequently found to have safe levels of gentamicin at 6–14 hours. None of these patients developed an AKI. At site 2, one patient was found to develop an AKI; however, this patient had received the correct dose of gentamicin according to hospital protocol. At site 1, seven patients were found to have an AKI on renal function monitoring at 48 and 72 hours. However, two of these patients had an AKI on admission (prior to administration of gentamicin). Two of the patients developed an AKI at 48 hours of monitoring, however these two patients had also received a contrast CT on admission (contrast is known to be nephrotoxic).16 Four of the patients had received the correct dose of gentamicin according to hospital protocol, and two of the patients had been underdosed. Although the numbers in this audit are too small to determine if there is a statistically significant difference in the number of patients that developed an AKI at each site, it is pertinent to note that of the small number of patients who did develop an AKI the majority had received the correct dose of gentamicin. There was no correlation between dose of gentamicin and effect on renal function.

Final programme theory

Prescribing ‘in reality’ varied from the ‘intended’ process, due to the triggering of (often hidden) mechanisms in certain contexts. The overarching final programme theory focuses on gentamicin prescribing behaviour from the perspective of more junior doctors as these are the clinicians who most often prescribe it. Online supplemental material 3 provides an overview of the final programme theory, consolidating the relationships of the 12 CMO configurations that emerged from the data. This programme theory demonstrates why or why not, and in what circumstances, clinicians do or do not adhere to the gentamicin prescribing protocol. It references important contexts including both characteristics of the prescribing protocol itself (length and accessibility), and also the involvement of other HCPs, patient factors and the environment in which prescribing occurs. The programme theory identifies what drives reluctance or willingness to follow protocol and the outcomes (intended and unintended) that result from the complex inter-relationships of these contexts and the embedded mechanisms (fear, confidence, frustration, etc.) they trigger.

Discussion

This realist evaluation explored two distinct protocols for prescribing gentamicin in two hospitals, in order to understand the mechanisms they trigger and the outcomes they achieve. The ‘actual’ process of prescribing gentamicin often varied substantially from the ‘intended’ process. As found in other studies, the existence of guidelines is often insufficient to ensure appropriate prescribing and monitoring.17 Although all participants acknowledged the existence of a gentamicin prescribing protocol, there was varying knowledge of the details of their hospital specific protocols at both sites.

Quantitative data found gentamicin prescribing was not always accurate, and the qualitative data have provided some insight into why this might be. The three key themes identified across both sites were: clinician experience, properties of the prescribing protocol and the prescribing environment. The audit data appears to support the suggested mechanisms. For example, as cited in the literature and noted from this study’s interviews, fear of side effects leads to underdosing of gentamicin.4 5 Clinicians were cautious not to overdose and instead were more likely to underdose: site 1: 24% underdosed, 10% overdosed; site 2: 12% underdosed, 9% overdosed.

Concerns regarding gentamicin and nephrotoxicity are well established. From the data collected in this study regarding gentamicin and AKI, it is not possible to draw conclusions that these were directly correlated; 57% of those that developed an AKI had in fact received what was deemed a ‘safe’ (accurate) dose of gentamicin according to hospital protocol. An AKI was also seen to develop in two patients that were underdosed with gentamicin. It is clear that an AKI can develop in surgical patients for a multitude of reasons, for example sepsis, blood or fluid loss, contrast from scans, other nephrotoxic medications, etc.

The prescribing protocol at site 1 was longer, more prescriptive and was also referenced many times as having operational issues, causing frustration and reluctance to use the policy. The prescribing protocol at site 2 was shorter and simplified, it was also more positively received by clinicians. The relationship of these contexts, mechanisms and outcomes is reflected in the audit findings: 66% of prescriptions reviewed at site 1 adhered to the hospital protocol, whereas 79% of gentamicin prescriptions adhered to the local hospital protocol at site 2.

This study has established that there is much variability in gentamicin prescribing, both between the two study sites and within each site despite established protocols. Both properties of the prescribing protocol and also the prescribing environment, clinician experience and interaction with the wider healthcare team all trigger mechanisms including uncertainty, fear, confidence and frustration. These can lead to both the intended outcome of adherence to the prescribing protocol and also unintended outcomes: intentional deviation from the protocol, unintentional non-adherence and unnecessary gentamicin levels being taken.

A simplified prescribing protocol is better received by those involved in the process of prescribing and therefore leads to better adherence to protocol and consequently more accurate prescribing.

Interviews with staff enabled us to uncover implicit assumptions about how the protocol works. However, the participants were volunteers and therefore could be a biased sample of staff with an interest in the study area.

Semistructured interviews were undertaken in order to offer rich insights and enabling discussion of a topic which participants may have otherwise felt uncomfortable to talk about. We do however appreciate that they may lack the greater insight into social interaction, and, potentially hierarchies, that focus groups might have provided.18

The smaller numbers of patients being prescribed gentamicin at Site 2 created challenges in terms of data collection.

The incidence of gentamicin associated nephrotoxicity is not known from this study as the study team appreciates that when assessing clinical outcomes it is important to consider confounders (such as other nephrotoxic drugs, fluid status of the patient, etc.) before drawing conclusions.

This project did not have the scope to repeat interviews at a later stage due to resource limitations and staff turnover throughout the period of study. However, different and additional questions were asked in further interviews with different respondents, which contributed to theory refinement.19

Implications for policy, practice and further research

By undertaking this realist evaluation, the study team has been able to suggest the following recommendations for practice to improve gentamicin prescribing:

Simplifying a protocol leads to better adherence to protocol and more accurate prescribing.

When technology is required for a prescription, ensuring this is working and if not, then an alternative is available.

Hospital Trust inductions should alert new staff members to protocols and where to find them.

Regional consensus on prescribing of gentamicin would ensure more confidence in prescribing and likely improve accuracy of prescriptions.

When adequate information is not readily available (height, weight, etc.), a system should be in place to aid prompt prescribing.

Policy-makers might benefit from applying theory-driven evaluation to clarify the design of a programme prior to implementation and/or establish a performance monitoring framework.

Future research could repeat this study across other hospitals to establish the extent to which the final programme theory is transferable to other clinical environments. Evidence from the interviews demonstrates frustration that such a commonly prescribed drug should have a different prescribing protocol at each hospital and a standardised simplified approach could lead to improved patient outcomes. It would also be helpful for future research to identify the relationship between patients that receive gentamicin and subsequently develop an AKI which was beyond the scope of the current work.