Increasing the use of perioperative risk scoring in emergency laparotomy: nationwide quality improvement programme.

BACKGROUND: Emergency laparotomy is associated with high morbidity and mortality. The early identification of high-risk patients allows for timely perioperative care and appropriate resource allocation. The aim of this study was to develop a nationwide surgical trainee-led quality improvement (QI) programme to increase the use of perioperative risk scoring in emergency laparotomy.
METHODS: The programme was structured using the active implementation framework in 15 state-funded Irish hospitals to guide the staged implementation of perioperative risk scoring. The primary outcome was a recorded preoperative risk score for patients undergoing an emergency laparotomy at each site.
RESULTS: The rate of patients undergoing emergency laparotomy receiving a perioperative risk score increased from 0-11 per cent during the exploratory phase to 35-100 per cent during the full implementation phase. Crucial factors for implementing changes included an experienced central team providing implementation support, collaborator engagement, and effective communication and social relationships.
CONCLUSIONS: A trainee-led QI programme increased the use of perioperative risk assessment in patients undergoing emergency laparotomy, with the potential to improve patient outcomes and care delivery.

Introduction

Emergency abdominal surgery encompasses a variety of high-risk surgical procedures that require the timely delivery of resource intensive care. Optimal care requires individually tailored treatments, for which early identification of high-risk patients is a priority. In recent years surgical and anaesthetic improvement programmes have focused on emergency laparotomy as it is associated with significant morbidity and a 30-day mortality rate ranging from 5 to 21 per cent[1-9]. Such a wide range reflected differences in both surgical indications[9,10] patient co-morbidities[11], and factors related to care delivery[12], including ageing[13], lack of insurance, and belonging to ethnic minorities or lower socioeconomic groups[14,15]. Numerous reports from the UK’s National Emergency Laparotomy Audit (NELA) emphasized the usefulness of a systematic preoperative assessment likely to identify adverse postoperative outcomes[16-18]. Patients without a risk assessment score had a 30-day mortality of 5.4 per cent, setting them in a high risk category according to NELA standards[16]. Components of care delivery, such as direct consultant involvement critical care support, may vary according to whether the risk has been assessed and retrieved before surgery. As their implementation is associated with improved survival after an emergency laparotomy[19-21], preoperative risk documentation may be considered a surrogate indicator of good quality of care.

Several approaches to improving outcomes for patients undergoing emergency laparotomy have been described, including audit[16], peer comparison[9], best practice tariffs[16], ‘name and shame’ strategies[22], and local improvement activities[19]. Facilitating healthcare professionals to address key issues in their work area is a common low-cost strategy for achieving improvements[23,24], but has recognized deficiencies[25]. Surgical trainees (residents) are the primary frontline staff within the Irish healthcare system but are frequently overlooked and remain largely isolated from hospital-based improvement programmes[26,27]. Trainee involvement in improvement activities may be an effective strategy for developing a ‘local’ culture of process improvement.

The aim of this study was to develop the structures to support a nationwide surgical trainee-led quality improvement (QI) programme and evaluate interventions in the implementation to increase the use of perioperative risk scoring in laparotomy.

Methods

In the Republic of Ireland, a state-funded healthcare system is organized and delivered by the Health Service Executive (HSE). State-funded hospitals are classified into four levels (termed ‘models’) with increasing care complexity. Accordingly, model-three hospitals provide care to undifferentiated surgical patients, whereas model-four hospitals provide acute undifferentiated care and accept tertiary referrals[28]. Twenty-four hospitals provide acute surgical services on a 24-h basis. On average, 2068 emergency laparotomies are performed annually[9]. Ireland does not have a formal national NELA framework. General surgery training is an 8-year structured postgraduate training pathway separated into a 2-year ‘core’ programme followed by competitive entry into a 6-year ‘specialty’ training programme. General surgery trainees (years 3–8) work across model 3 and 4 hospitals providing 24-h emergency on-call under the supervision of a consultant general surgeon.

Conceptual framework

This study was a prospective QI programme, called ‘Know the Score,’ involving 25 surgical trainees and one surgical physician associate working in state-funded Irish hospitals. The programme was structured using the active implementation framework (AIF)[29], which provides mechanisms and strategies to adopt and embed evidence-based practices in a specific context[30], and plan–do–study–act (PDSA) cycles. A distributed leadership model was used, combining local programme leads at individual sites supported by a central implementation team to provide competency, organizational, and leadership support. The programme implementation team consisted of a senior surgical trainee to provide near-peer clinical leadership, a post-specialty training fellow in surgical QI to deliver technical expertise, and a senior surgeon within the Irish National Clinical Programme in Surgery for the sponsorship. Surgical trainees were voluntarily recruited as local leads in each hospital and provided structured coaching, mentorship, and resources. The study was endorsed within the HSE National Quality Assurance and Improvement System (NQAIS) Clinical Governance Framework and registered with the Audit/QI Committee at the institution of the senior author. Data on individual hospitals were de-identified and anonymized to ensure confidentiality. A minimum of 6 months of participation was required for inclusion in data analysis. This report complies with the SQUIRE 2.0 (Standards for QUality Improvement Reporting Excellence) guideline[31].

Interventions

The full QI programme was run over 15 months from September 2018 to December 2019. After exploration and installation phases, the final 12 months of the study interval was split to analyse the impact of rotation of local leads. Specific interventions were designed and tested based on the AIF implementation stage: exploration, installation, initial, and full implementation[29] ().

The exploration phase interventions aimed at highlighting variation in outcomes after emergency laparotomy with the implementation of preoperative assessment and included multiple presentations at national surgical meetings and a media campaign.

Interventions during the installation stage included written correspondence to all hospital clinical directors and departmental chairs outlining the improvements programme, installation of posters with a quick response (QR) code to access risk-scoring apps in participating hospitals, a participant-wide secure messaging group to support communication, dissemination of a standardized presentation slide set to support local education, and a formal invitation to all current Irish surgical trainees to participate in the change initiative.

During the initial implementation phase, interventions consisted of twice-monthly teleconferences for local leads, local educational presentations given by programme participants, and the biweekly distribution of the programme run-chart as a means of ‘real-time’ feedback. Trainees were supported to develop improvement cycles depending on the needs identified in their hospital. Performance data were shared internally in the hospital but routinely de-identified for national input.

As per , the interventions during the full implementation phase include participants’ certificates for training portfolio/logbook, support for local leads rotating posts, recruitment of new local leads, QI education sessions, and surveys of participants’ experience.

Evaluation of interventions

Programme participants completed a questionnaire during the initial implementation phase to evaluate QI knowledge before and after the programme and evaluate the effectiveness of the interventions used. A 12-item questionnaire was designed using web-based survey software (Qualtrics™, Seattle, Washington, USA). Three baseline questions assessed the current scope of practice and previous experience and understanding of QI. Five multiple-choice and ranked-order questions assessed which programme interventions participants found most valuable and applicable in driving change locally. Four questions using a mixture of Likert scale and free text captured the experiences and feedback of participants at local sites. Questionnaires were analysed using non-parametric descriptive statistics, the weighting of ranked responses to calculate the highest average ranking, and qualitative analysis of key themes from free-text responses.

Programme communication

E-mail communication between local leads and the steering group was used to disseminate materials, and key programme initiatives. Phone calls were used at the discretion of the steering group to contact local leads, whereas routine communication was conducted using a ‘WhatsApp’ messaging group. The volume, content and temporal pattern of messages were recorded during the study and analysed as a proxy measurement of the engagement of local leads with the programme. Moreover, de-identified data regarding laparotomy volume and risk scoring performance were submitted to the steering group using the ‘WhatsApp’ platform at each biweekly data collection time point.

Data collection and statistical analysis

The primary outcome was the rate of patients undergoing emergency laparotomy at each hospital site who had documented preoperative risk score (NELA score or P-POSSUM). Two arbitrary targets were set over the data collection interval—50 per cent of reported cases achieved by month 5 (January 2019) and 70 per cent of reported cases achieved by month 10 (June 2019).

The volume of emergency laparotomies performed for biweekly data collection time points was recorded for each study site. The median number of laparotomies per data collection interval was used to dichotomize study sites into ‘low’- and ‘high’-volume hospitals. Biweekly arithmetic means of patients with a risk score performed at each site were submitted centrally, plotted on a run-chart, and shared with local study leads. A baseline was constructed for the first six data points during the exploration phase of the programme (from September 2018 to December 2018). The primary outcome was assessed for evidence of improvement using a Shewhart statistical process control (SPC) chart developed using Microsoft Excel 2019. This technique was deemed most appropriate in a real-world, large-scale change in which the control of independent variables is not always possible[32]. The chart was inspected for common cause variation (random fluctuation) and special cause variation (changes due to external factors). Special cause variation was considered a clinically and statistically significant change once a substantial change not due to natural variation was identified, when the mean biweekly performance breached the upper or lower control limits or when eight consecutive biweekly performances lay on one side of the mean line. Parametric continuous data are described by mean(s.d.) and non-parametric data are described with median and ranges. Pearson and Spearman tests were used to assess correlation in scoring performance and programme communication respectively. Statistical significance was set at P = 0.05

Results

Performance of risk scoring

A total of 15 hospitals were recruited into the programme, of which 8 (53 per cent) were model-four and 7 (47 per cent) were model-three hospitals. The participants’ practice grade is reported in . A run-chart with preoperative risk scores is shown in . Within the exploration phase, patients scored ranged from 0 to 11 per cent. Progressive interventions led to an increase in the rate of performed scores ranging from 8 to 35 per cent in the installation phase (months 1–3), to 21 to 67 per cent in the initial implementation phase, and achieved 35–100 per cent from 6 months onwards with the full implementation phase. The interim target of 50 per cent of patients scored was not consistent at 5 months; however, the full implementation phase resulted in achievement of the second interim target of 70 per cent of preoperative scores.

The rate of patients receiving a preoperative score was also assessed for variability using an SPC chart (). Breach of the lower control limits indicating a statistically significant change was found at three points within the exploration and installation phases of the programme. Variation was reduced as the programme proceeded through the subsequent phases of implementation.

Programme statistical process chart

Programme implementation at local study sites

Study sites participated for varying lengths of time (mean(s.d.) 12.4(4.66) months) with a mean(s.d.) number of data collection time points for individual sites of 16.1(5.5). The number of laparotomies varied from 4 to 70 intervention (mean(s.d.) 33.4(21.6) laparotomies), with an operating volume at individual sites ranging from 0.44 to 4.12 laparotomies per data collection time point. Risk scores were performed in 7–100 per cent of patients requiring emergency laparotomy for study sites without correlation between overall participation time and risk-scoring performance (r = –0.16, P = 0.592).

The median value of 1.96 laparotomies per data collection was used to dichotomize this variable into low- and high-volume study sites. An increasing but not significant trend in preoperative scoring was highlighted in low-volume compared with high-volume centres (mean 0.63 versus 0.42, P = 0.19) (). Similarly, study sites with one local lead seemed slightly more successful in performing risk scoring with a mean risk assessment of 0.76 compared with 0.38 and 0.50 of centres with two and three local leads respectively (P = 0.07) ().

The rate of patients with a recorded preoperative risk score before emergency laparotomy according to study site factors

Considerable variation was observed in the number of messages sent by individual local leads (median 15.2, range 1–64). No significant difference was observed between trainee grades and messaging volume. No correlation was identified between the total number of messages from local leads and the total proportion of patients scored (r = 0.02, P = 0.95). shows the temporal pattern of messages according to the variation in participant performance and engagement during the programme. A surge in messaging activity was observed at crucial times, such as programme commencement and approaching the July 2019 rotational date. Peaks in risk-scoring performance were also preceded by increased message activity. The differential purpose of instant messaging between steering group members and local leads is summarized in .

Temporal relationship of risk score performance and volume of mobile instant messaging per month

Impact of rotation of local leads

The contribution of individual hospitals to data collection was compared between 10 hospital sites for two study intervals (January to July 2019 and July to December 2019) (). Eight sites contributed fewer data collection time points after the July 2019 rotation; one site contributed more, and one was unaffected by the changeover (). The rate of performed scores increased in 6 out of 10 study sites after July 2019 and decreased in three sites, whereas performance was unaffected in one site.

Local lead engagement was reflected in the volume of mobile messages before and after rotation () with most sending higher numbers of messages before July 2019.

Experience of programme participants

The programme questionnaire had an 88.5 per cent response rate. Overall, 60.8 per cent of participants had no previous experience or training in QI. In comparison, 39.4 per cent considered themselves to have a ‘good’ understanding of QI before starting the programme, increasing to 94.1 per cent after 4 months of participation. Participants reported that trainee teleconferences, reminder messages regarding data collection, and visible display of programme goals in the hospital helped drive change during the programme (). Participants also highly valued the contribution of the programme to their professional logbook. The theme of communication was identified as the most effective local intervention. Participants highly valued inclusion in a network of collaborators with frequent reminders and encouragement from colleagues on site and programme coordinators. Common themes of feedback to the programme highlighted a concern amongst surgical teams that implementing risk scoring would not alter clinical management. A frequent suggestion for programme improvement was incorporating risk scoring into standard preoperative documentation.

Importance of programme interventions ranked by participants

Discussion

This study describes the structure and implementation of a nationwide surgical trainee-led QI programme to increase the use of perioperative risk scoring in emergency laparotomy using an AIF. Throughout the programme, the proportion of patients who received perioperative risk scoring increased from 0–11 per cent during the exploratory phase to 35–100 per cent during the full implementation phase. Secondary analyses suggested that implementation support, trainee engagement, communication, and hospital context were important determinants of programme success.

Two large-scale prospective emergency laparotomy QI programmes from the UK, have been published by the Emergency Laparotomy Collaborative (ELC)[20] and the Enhanced Peri-Operative Care for High-Risk Patients (EPOCH) trial groups[33]. The ELC study involved the implementation of a six-point care bundle in 28 hospitals and found a reduction in the unadjusted mortality rate and the duration of hospital stay and changes in some of the care bundle metrics. This programme used the Institute for Healthcare Improvement Breakthrough Series collaborative as an implementation framework. Although perioperative risk assessment was not part of the care bundle, it demonstrated that QI methodology could improve outcomes in emergency laparotomy[20]. A Bath perioperative ‘Boarding Card’ project used an individualized card to prompt and promote high-quality perioperative care, including risk assessment scoring[34].

The EPOCH trial intervention consisted of an extensive 37-point evidence-based pathway in 93 hospitals. Preoperative documentation of risk was one of the process measures included as part of the bundle. Overall, the study investigators found modest improvements in 10 patient-level process measures; however, this did not translate to overall outcome measures[33]. There were wide variations in intervention fidelity between hospitals, with differences in the targeted processes, change rate, and eventual success. In a hospital-level evaluation of the EPOCH trial, no hospital reliably implemented all 10 processes[35]. These findings indicated that change at scale and the context in which change was implemented were more complex than initially anticipated. As in the present study, context was central to the success (or failure) of QI programmes.

Successful implementation may also be influenced by the intervention’s content and the implementation process. Some publications have reported the effects of small-scale QI projects to improve outcomes in emergency laparotomy. The ELPQuiC group assessed the implementation of a bundle of five interventions in four hospitals and reported a reduction in risk-adjusted mortality (risk ratio 0.61, 95 per cent c.i. 0.45 to 0.84)[19]. Compared with the EPOCH trial, the difference in findings may relate to the simpler intervention and stronger pre-existing relationships between staff leading implementations in these early adopter hospitals. The EPOCH trial set ambitious targets in hospitals where there may have been a less favourable organizational readiness for change than in ELPQuicC hospitals. Reports of QI programmes in other clinical areas have delivered mixed results[36-38], suggesting that more focused, discrete clinical interventions might be more successfully implemented than complex processes.

The findings of the present study also suggested that study sites with relatively lower operative volume and an individual QI lead may be more capable of effecting local change. Greater personal engagement and a sense of ownership of a QI project may be instrumental in effecting change, whereas larger study sites may see effectiveness reduced by dilution of responsibility.

The Royal College of Anaesthesia (UK) Quality Improvement Compendium identifies risk assessment of emergency laparotomy patients as a crucial target intervention[39]. The present study demonstrated that surgical trainees can deliver cross-disciplinary improvement at a national level within a structured programme. Postgraduate surgical trainees are adaptable and resourceful and, with the proper support, are an ideal group of healthcare professionals to drive change at a local level. As frontline staff, trainees can also influence the intervention, engage staff, and counteract resistance to externally led change[40]. A simple, explicitly designed measure with a system for data collection, monitoring, and feedback could be a valuable improvement tool.

Social and technical aspects seemed crucial to the success of this programme. The central expert implementation team oversaw the development of local programme teams, building local QI capacity. The leadership group provided the structure for the local team’s progress through the stages of implementation successfully. Local leads capitalized on social QI strategies to drive change by utilizing a participant messaging platform and monthly participant teleconferences. Building and maintaining effective social relationships is time-consuming and challenging, particularly with senior colleagues and hospital administration[41]. This may be even more challenging for trainees who usually have temporary training appointments.

This programme was unfunded, with limited resources and time available to coach local QI leads, allowing adaptability and replicability if successful. As in other reported QI interventions[42], a higher intensity coaching programme might have led to greater fidelity[43,44].

There are some inherent limitations to this study. The primary outcome was a recorded preoperative risk score for patients undergoing an emergency laparotomy at each included hospital; without evaluating intervention-related changes in patient outcomes. There is, however, evidence from the literature that standardized clinical pathways are associated with improved survival[21,45]. Variations in the total number of laparotomies performed for the biweekly measures sometimes interfered with signals in the data. For example, in a 2-week interval with a small denominator, fewer patients scored may create a data point that breaks a signal that would otherwise indicate a move towards improvement. In combination with the time-bound nature of the analyses, this may have led to some real-world improvements not being identified using the run charts.

A nationwide surgical trainee-led QI programme has been developed leading to increased perioperative risk scoring in emergency laparotomy, using the AIF, which demonstrated a sustained improvement in risk assessment during the full implementation phase. Such an approach is feasible and could lead to better patient outcomes and delivery of care.

Collaborators

SURGical Improvement Network

S. Abd Elwahab (Galway University Hospital, Galway, Ireland); G. Bass (Tallaght University Hospital, Dublin, Ireland); E. Burke (St. Lukes General Hospital, Kilkenny, Ireland); D. Cagney (Cork University Hospital, Cork, Ireland); A. Canas (Connolly Hospital, Blanchardstown, Ireland); C. Cronin (Cork University Hospital, Cork, Ireland); C. Cullinane (Cork University Hospital, Cork, Ireland); L. Devane (The Mater Misericordiae University Hospital, Dublin, Ireland); N. Fearon (St. Vincents University Hospital, Dublin Ireland); A. Fowler (Portiuncula University Hospital, Ballinasloe Ireland); A. Fullard (Midland Regional Hospital Tullamore, Tullamore, Ireland); D. Hechtl (Galway University Hospital, Galway, Ireland); M. Kelly (St. James Hospital, Dublin, Ireland); J. Lenihan (Beaumont Hospital, Dublin, Ireland); E. Murphy (The Mater Misericordiae University Hospital, Dublin, Ireland); C. Neary (Galway University Hospital, Galway, Ireland); R. O'Connell (Mayo University Hospital, Castlebar, Ireland); M. O'Neill (University Hospital Kerry, Tralee, Kerry); C. Ramkaran (St. Vincents University Hospital, Dublin Ireland); A. Troy (Beaumont Hospital, Dublin, Ireland); R. Tully (Our Lady of Lourdes Hospital, Drogheda, Ireland); C. White (Our Lady of Lourdes Hospital, Drogheda, Ireland); H. Yadav (Our Lady of Lourdes Hospital, Drogheda, Ireland).

Click here for additional data file.

Table 1

Interventions by implementation phase

Exploration(pre-intervention)	Installation(months 1–3)	Initial implementation(months 4–6)	Full implementation(>6 months)
Presentations on emergency laparotomy outcomes at national surgical meeting	Communication to hospital clinical directors, departmental chairs describing programme	Participant-wide monthly teleconferences	Participant certificate for training portfolio/logbook
Media coverage of emergency laparotomy	Programme theatre posters with QR code to access scoring apps	Local presentations (grand rounds, anaesthetics, perioperative nursing)	Support for local leads rotating posts
Creation of mobile messaging group	E-mail invitation to all surgical trainees to participate	Biweekly progress feedback	Recruitment of new local leads
Creation of data analysis files	Recruitment of study sites encouraged through steering group and local lead contacts	Reminder messages for on-call doctors	Quality improvement education session
Baseline data collection at limited study sites	Programme slide presentation for education sessions by local leads	Frequent communication to encourage local lead discussion and feedback	Survey of participant experience