Mixed-methods evaluation of the Perioperative Medicine Service for High-Risk Patients Implementation Pilot (POMSHIP): a study protocol.

INTRODUCTION: Perioperative complications have a lasting effect on health-related quality of life and long-term survival. The Royal College of Anaesthetists has proposed the development of perioperative medicine (POM) services as an intervention aimed at improving postoperative outcome, by providing better coordinated care for high-risk patients. The Perioperative Medicine Service for High-risk Patients Implementation Pilot was developed to determine if a specialist POM service is able to reduce postoperative morbidity, failure to rescue, mortality and cost associated with hospital admission. The service involves individualised objective risk assessment, admission to a postoperative critical care unit and follow-up on the surgical ward by the POM team. This paper introduces the service and how it will be evaluated. METHODS AND ANALYSIS OF THE EVALUATION: A mixed-methods evaluation is exploring the impact of the service. Clinical effectiveness of the service is being analysed using a 'before and after' comparison of the primary outcome (the PostOperative Morbidity Score). Secondary outcomes will include length of stay, validated surveys to explore quality of life (EQ-5D) and quality of recovery (Quality of Recovery-15 Score). The impact on costs is being analysed using 'before and after' data from the Patient-Level Information and Costing System and the National Schedule of Reference Costs. The perceptions and experiences of staff and patients with the service, and how it is being implemented, are being explored by a qualitative process evaluation. ETHICS AND DISSEMINATION: The study was classified as a service evaluation. Participant information sheets and consent forms have been developed for the interviews and approvals required for the use of the validated surveys were obtained. The findings of the evaluation are being used formatively, to make changes in the service throughout implementation. The findings will also be used to inform the potential roll-out of the service to other sites. © Author(s) (or their employer(s)) 2018. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Perioperative Medicine Service for High-risk Patients Implementation Pilot is the first pilot of a perioperative medicine service using a team comprising anaesthesia and intensive care trainees. The concurrent robust mixed-methods evaluation will therefore explore important questions regarding potential future models of working.

The evaluation is based on a controlled before and after research design and a qualitative process evaluation, thus capturing different aspects of the outcomes and implementation of the service.

The implementation of the service and evaluation are taking place in one site, thus limiting the generalisability of the findings.

Introduction

In the UK, an estimated 10 million procedures occur per year, and worldwide, this figure stands at 313 million per annum.1 Mortality varies between patient populations, depending on patient characteristics, the type and urgency of surgery undertaken, and the quality of care delivered. Estimates of short-term mortality after major surgery (defined as surgery which requires an inpatient hospital stay) currently stand at around 0.5%–1%; however, serious postoperative complications (or morbidity) occur in around 15% of such patients.2 Such complications can have a lasting effect on health-related quality of life and long-term survival.3–6 National guidance recommends various approaches to the management of surgical patients considered to be at high risk of death or major complications.7 However, audit and research data show that compliance with many of the recommended processes is low.8 9

A significant amount of resource is invested in the intraoperative care of patients undergoing major surgery, and relatively little is allocated to their postoperative care by comparison. Bundled payments for surgery in the UK reward increased activity (more surgery, more outpatient appointments) and neglect the role of postoperative care of inpatients. The Royal College of Anaesthetists has proposed the development of perioperative medicine (POM) services as an intervention aimed at improving postoperative outcome, by providing better coordinated care for high-risk patients. An overall strategy has been developed and published10; however, detailed plans for the implementation of POM services have not yet been developed or tested.

One underpinning theory of change for POM services proposes that if high-risk patients are identified before surgery (through individualised risk stratification), and then monitored with appropriate vigilance and provided extra support throughout their perioperative pathway, that deterioration can be predicted and possibly prevented. An important supporting mechanism for this comes from the literature surrounding the concept of ‘failure to rescue’, which uses data from the USA to show that while risk-adjusted complication rates may vary two or threefold between different institutions, risk-adjusted mortality rates after developing complications can vary more than 10-fold.11 12 This implies that system-level differences in structure and process are influencing surgical outcome, more than the individual skills of the operating surgeon, surgery staff or anaesthesia teams. Further research from the USA has explored the role of communication, team working and safety culture in the avoidance of ‘failure to rescue’.13 14

We hypothesise that the implementation of a POM service for the care of high-risk surgical patients will reduce postoperative morbidity, failure to rescue, mortality and cost associated with hospital admission. The implementation pilot comprises two elements: the new patient-facing service and its concurrent evaluation. Data collection commenced in February 2016 and will continue until August 2018, with an expected recruitment of 2000 patients.

The new service

The Perioperative Medicine Service for High-risk Patients Implementation Pilot (POMSHIP) will run at University College Hospital at Westmoreland Street from May 2016 to August 2018. This is a satellite hospital performing only scheduled thoracic and urological surgery. The service comprises three main components (summarised in figure 1).

Figure 1

Diagram of the POMSHIP pathway. AKI, Acute Kidney Injury; ARISCAT, Assess Respiratory Risk in Surgical Patients in Catalonia; ASA, American Society of Anesthesiologist’s Physical Statust Score; DASI, Duke Activity Status Index; EFS, Edmonton Frail Scale; NSQIP, National Surgical Quality Improvement Program; PACU, Post-Anaesthetic Care Unit; POD, postoperative delirium; POM, perioperative medicine; POMSHOP, Perioperative Medicine Service for High-Risk Patients Implementation Pilot; RCRI, Revised Cardiac Risk Index; VATS, Video-Assisted Thoracoscopic Surgery.

Individualised preoperative risk assessment of all eligible patients by the POM team. We are using a combination of risk-prediction tools which have been individually validated (see the Recruitment section) for organ-specific morbidity, global postoperative complications or mortality. Patients scoring as high risk in any of these assessment methods will be recruited into the pathway and receive the second two components of the service.

Postoperative admission to a critical care unit. This is a recommended standard of care for patients considered high risk of perioperative complications, but there is disagreement over the thresholds which should be used to trigger admission.15

Ward-based postoperative follow-up by a POM team member until fit for discharge from hospital. The POM team will be staffed by senior trainees in anaesthesia, supported by intensive care consultants. A POM service consisting of geriatricians was shown to be beneficial in the urology population,16 but we are not aware of any previous trials of POM services delivered by anaesthesia and intensive care trainees.

Inclusion criteria for study population

The study population is determined on the basis of risk factors which have been previously associated with high perioperative risk, namely: surgical magnitude, age and comorbidities. Inclusion criteria for the study population are presented in table 1.

Table 1

Inclusion criteria for study population

Criteria	Description
Surgical magnitude	Any urological surgical patient undergoing cystectomy, Mitrofanoff procedure; formation of neobladder, nephrectomy or any urological surgical procedure involving a laparotomy; any thoracic surgical patient undergoing lobectomy (VATS or open), pneumonectomy, thoracotomy, mediastinal tumour resection or any thoracic procedure involving a sternotomy.
Or
Age	Any patient ≥80 years of age undergoing surgery with a planned length of stay ≥2 nights.
Or
Comorbidities/functional status	Any patient with ASA grade ≥3 (or with comorbidities causing concern to the preassessment team) undergoing surgery with a planned length of stay ≥2 nights.

All patients, who fulfil eligibility criteria on the basis of screening of type of surgery, comorbidities and age, will be assessed for inclusion in the POMSHIP pathway through risk assessment (as described below) in the preoperative assessment clinic.

Exclusion criteria

Surgical patients with an expected length of stay (LOS) less than two nights. Any patient who becomes ‘of concern’ to ward staff while in hospital can be referred to the POM team for an opinion, but these patients will not be included in the analysis if they have not received the complete POMSHIP service.

Recruitment to POMSHIP

Most of the patients on this site are undergoing expedited cancer surgery. They are referred to the preoperative anaesthetic assessment nurse-led clinic at the time of decision to operate. They would normally be preassessed and operated on within 30 days. All patients who are deemed eligible according to the inclusion/exclusion criteria described above are referred to the POM team by the preassessment nurses and will undergo an individualised objective risk assessment as described below. Those patients, who meet the criteria of ANY individual risk-prediction score or whom the POM team suspect have risks not captured by the risk-prediction tools, will be included in POMSHIP. Multiple risk assessment tools will be used to ensure we detect as many high-risk patients as possible, to allow for variation in their accuracy and for the purposes of risk adjustment in our subsequent analysis. Where the POM team feel the calculated risks are underestimated, they may use their discretion to deem the patient high risk.

Individualised risk assessment

Patients referred to the POM team undergo individualised risk assessment using eight previously published risk assessment tools, after the patient has undergone routine preoperative assessment with nursing staff. These risk assessment tools were selected by reviewing the literature to find tools validated for common or important postoperative complications.

Four validated tools which predict organ-specific complications were found and are being used with thresholds for referral into the POMSHIP postoperative pathway as described below:

Postoperative acute kidney injury has been shown to be predicted by the American College of Surgeons’ National Surgical Quality Improvement Program acute kidney injury risk score.17 A class ≥3 or other risk factors (as identified by the UK’s National Institute for Health and Care Excellence (NICE)—namely exposure to contrast or nephrotoxic drugs) is the threshold to refer patients into the POMSHIP pathway.

Lee’s Revised Cardiac Risk Index is a validated and well-established tool for estimating perioperative risk of major adverse cardiac events.18 19 The presence of two or more risk factors triggers inclusion in the POMSHIP pathway.

The Assess Respiratory Risk in Surgical Patients in Catalonia model has been externally validated for predicting postoperative pulmonary complications.20 A score ≥45 denotes ‘high risk’ and therefore entry to the POMSHIP pathway.

Postoperative delirium (POD) is a common complication in the older surgical population. The American Geriatrics Society include risk assessment criteria within their guidelines for best practice in the management of POD.21 NICE has also published guidelines and a list of recognised risk factors for developing delirium in hospital (not specifically postoperatively).22 If a patient scores two or more risk factors sourced from the predictors in one or both sets of guidelines, they are deemed to be at high risk for POD and therefore referred onto the POMSHIP pathway.

Additionally, four risk prediction tools have been identified as validated ways of predicting overall patient outcome:

The Surgical Outcome Risk Tool (SORT) is a UK-developed and internally validated, parsimonious risk assessment tool for 30-day mortality.23 A predicted mortality ≥1% has been chosen as the entry criteria to the pathway.

The Edmonton Frail Scale is an objective measure of frailty which has been validated as predictive of postoperative complications.24 25 A score ≥8 has been chosen as the referral trigger to POMSHIP as this was shown to reflect a higher risk of complications and a lower risk of being discharged home.

Functional capacity has been repeatedly shown to correlate well with perioperative mortality and with risk of major adverse cardiovascular events. The Duke Activity Status Index can be used to record patient-reported functional capacity in metabolic equivalents.26 A score <4 is indicative of high risk and triggers inclusion in POMSHIP.

Cardiopulmonary exercise testing (CPET) has been found to be predictive of outcome in patients having intra-abdominal surgery.27 An anaerobic threshold <11 mL/kg/min were found to be predictive of postoperative complications and will therefore trigger inclusion in POMSHIP. Referral for CPET is decided by the regular preassessment team and results considered by the POM team only if available.

Postoperative critical care admission

All patients determined to be high risk on the basis of the above screening criteria are booked for admission to the postoperative anaesthetic care unit (PACU). This unit cares for levels 2–3 patients, is staffed by intensive care doctors and nurses (under the supervision of an on-site intensive care consultant) and aims to provide 24 hours of high intensity care in the immediate postoperative period. The PACU was established in 2001, providing postoperative levels 2–3 support for cardiothoracic surgical patients. In 2015, there was a reorganisation of services and urology and thoracic surgery moved to the Westmoreland Street site. The main goals of PACU admission are to optimise pain management, mobilise early and return to oral intake, and careful attention to fluid management. There are no specific protocols regarding goal-directed haemodynamic optimisation although devices such as the oesophageal Doppler which facilitate this type of therapy are available should the clinical team decide it to be necessary. Standard discharge practices based on Trust protocols will be used for all patients, regardless of whether they are in the POMSHIP pathway: review by the PACU Consultant prior to discharge to the ward; return to preoperative physiological parameters with no ongoing requirement for organ support greater than 4L/min oxygen therapy; all critical care only medication discontinued; acute pain score less than two for two consecutive hours.

Ward-based postoperative follow-up

Post-PACU discharge, patients are followed up by the POM team which comprises senior trainees in anaesthesia and/or intensive care with clinical support from intensive care consultants. This is a novel intervention aiming to avoid the phenomenon of ‘failure to rescue’, as described above.11 12 It has a number of discrete objectives:

Thorough clinical assessment to identify signs of postoperative deterioration early in the patients’ postoperative course.

To recommend clinical actions to the parent surgical team with the intention of instituting interventions to prevent clinical deterioration and ‘failure to rescue’.

Surveillance of existing complications and the coordination of communications between the multidisciplinary team.

To facilitate the readmission of patients to critical care when deemed necessary.

All POMSHIP patients receive daily clinical follow-up on the postoperative wards until discharge. This process has been structured and developed into a protocol to minimise variations in practice and to enable the delivery of consistent, high-quality care. Clinical assessment takes the form of clinical examination and case note review. Once clinical reviews of all patients have been completed, the clinical fellow liaises with the POM/PACU consultant to discuss all recommendations to be communicated to the parent team. Following this discussion, a consultant ward round is conducted of relevant patients. All POMSHIP patients will have a daily POM review until discharge. When patients are fit for medical discharge from hospital but remain admitted for non-medical reasons, a ‘fit for discharge’ box on the daily review proforma will be ticked.

Methods and analysis

Aim

To evaluate the implementation and impact of the new service using mixed methods.

To evaluate clinical effectiveness (morbidity, health-related quality of life and ‘failure to rescue’) of this pathway.

To analyse and compare admission-associated costs before and after implementation and estimate the cost-effectiveness of the new service.

To explore staff and patient beliefs, attitudes and behaviours before and after implementation of the service.

Study design

This study evaluates the impact of the new POM service using a mixed-methods approach comprising three elements:

Clinical effectiveness

We will evaluate the impact of the POM service on patient outcomes using a ‘before and after’ analysis. The primary outcome is the presence of morbidity on postoperative day 7, as measured using the PostOperative Morbidity Survey (POMS). A previously validated and widely used instrument, the POMS captures morbidity of sufficient magnitude to require inpatient care.28 Prolonged postoperative morbidity, defined using the POMS, has been associated with reduced long-term survival.4 Secondary outcomes include: length of postoperative hospital stay (LOS); POMS morbidity on day 3 and 14; unplanned admission to critical care; unplanned return to the operating theatre; inpatient mortality and ‘failure to rescue’ defined as the proportion of patients who die in hospital after developing complications (inpatient mortality occurring after POMS morbidity on day 7). Patient-reported outcomes are being collected, including: the Quality of Recovery-15 (QoR-15) Score,29 measured at baseline (preoperatively) and on day 3 after surgery and the EQ-5D (a quality of life metric) measured at baseline and at 6 months postoperatively.30

Costs and cost-effectiveness

We will undertake a ‘before and after’ cost analysis using data from 3 months preimplementation and 3 months postimplementation. If the service is shown to be clinically effective then a ‘before and after’ cost-effectiveness analysis will be conducted using EQ-5D to reflect quality of life.

Qualitative process evaluation

In-depth qualitative studies have been increasingly recognised as valuable evaluations of complex healthcare interventions.31 We have been interviewing and observing staff to help understand their perceptions and experiences with the POM service. Data are being collected in two waves: first at baseline and during a follow-up period after launch of the new service. Additionally, a sample of patients recruited to the pathway is being interviewed during the follow-up period to gather their experiences of the POM service. The qualitative data will be subjected to framework analysis to elucidate staff and patient views of: the strengths and weaknesses of the existing system; the benefits and risks of the new service; the barriers and enablers to implementation of the new service and the potential for sustainability and spread of the new service. Findings from this element of the evaluation will be fed back formatively to the POM team to enable iterative adaptation of implementation strategy.

Quantitative and health economic data collection

Evaluation of clinical effectiveness

Data collection began in February 2016, 3 months before implementation of the service in May 2016. Objective risk factors for adverse outcomes are being collected from patients’ health records by the POM team and research nurses. These include the variables within the Portsmouth-Physiological and Operative Severity Score for the Enumeration of Mortality and Morbidity (P-POSSUM),32 the most widely validated risk adjustment model for heterogeneous surgical cohorts and the SORT.23 The patient-reported outcome measures are being collected by a member of the POM team at baseline (preoperatively) and in hospital on day 3 after surgery (QoR-15). Patients are being contacted by email or telephone to collect the 6-month EQ-5D data.

Evaluation of costs

Resource use and costs will be determined using: the upfront cost of staffing the service, Patient-Level Information and Costing System data provided by the Trust finance department and the National Health Service National Schedule of Reference costs. Using a diary-monitoring exercise, we will calculate the extra staff time required per patient to provide a POM service such as the model we describe.

Qualitative data collection

Evaluation of staff and patients’ perceptions and experiences of the new service

Data are being gathered during two waves of semistructured interviews and non-participant observation of staff meetings: the first wave will be conducted at baseline and the second wave will begin approximately 4 months after implementation. Interview topic guides are being used to explore topics such as: perceived strengths/weaknesses of the baseline system; knowledge, expectations and experiences of the new service; perceived barriers/enablers of implementation; perceived transferability of the new service (the full topic guides are presented in online supplementary appendices 1–3).

Interviews are being conducted by an anaesthetic registrar (DuW) as part of his academic clinical fellowship training, under the guidance of experienced qualitative researchers (CV-P). He has previously worked at the Trust (at a different site) and so may know some of the staff being interviewed.

Interviews are being conducted in private and last approximately 30–60 min. They are being audio recorded and professionally transcribed. After each interview, the researcher is writing a memo reflecting on what was learnt.

A sample (using the sampling strategy defined below) of patients is being approached for recruitment during the second wave of data collection. The interviews are semistructured. The interview topic guide is presented in the online supplementary appendix 3. Interviews are conducted in private and last approximately 30–60 min. The interviews are audio-recorded and professionally.

Relevant meetings for observation are being identified by discussion with local stakeholders. A Structured Observation guide (see online supplementary appendix 4) is being used to record field notes at the meetings. After each meeting, the researcher is writing a memo reflecting on what was learnt. Data collection from meetings is being regarded as complete when a substantial and full account of the issues has been generated such that a rich depth of material has been generated for analysis.

Sampling of staff and patients for interviews

Purposive sampling across professional groups is being used to ensure representation of all major groups of stakeholders in staff interviews. These groups include: preassessment nurses and doctors; urology and thoracic surgeons (consultants, registrars and junior doctors); urology and thoracic ward nurses and clinical nurse specialists; anaesthetists and intensivists; allied health professionals (eg, physiotherapists, pharmacists, dieticians); the acute pain team; ICU outreach nurses and general managers. There are two waves of interviews with staff members, including a maximum of 15 staff members per wave. The research is aiming to interview the same participants during both waves of data collection.

Once the second wave of data collection began, patients on the new pathway have been continuously sampled for interview until theoretical saturation is achieved (this is likely to occur with 10–15 participants). Patients are being identified from existing POMSHIP inclusion data. Urology and thoracic surgery patients are being recruited in similar numbers.

Recruitment for interviews

Staff members of relevant teams are being approached by the researcher to see if they would like to receive further information about the study. This information is sent to them through the hospital internal email system 48 hours before any interview and includes a staff participant information sheet and a staff informed consent form. The researcher follows up the email invitation with a face-to-face approach, allowing questions to be answered and interest to be confirmed. The PIS includes contact information for the chief investigator (CI) and principal investigator (PI) should any further information be required. Interviewees are selected according to the sampling strategy described above. If the interviewee agrees to take part in the interview, they are asked to sign a consent form.

Patients are approached in person by the researcher on the postoperative surgical ward, have the study explained to them and given the patient information sheet and consent form. They are allowed at least 24 hours to consider their decision to take part in the study. The patient information sheet includes contact information for the CI and PI, should any further information be required. Patients are selected according to the sampling strategy described above.

Relevant meetings for observation are identified through discussion with local staff members. Before attending meetings, the researcher contacts and seeks permission to attend from the meeting Chair. A PIS is circulated to all participants 48 hours before the meeting along with routine papers. The PIS is also available at the beginning of the meeting when participants are given the opportunity to opt-out of having their contributions recorded should they so wish.

Data analysis

We will evaluate the impact of the POM service on patient outcomes using a ‘before and after’ analysis. Logistic (or in the case of LOS, linear) regression will be used to account for potential confounders, with variable selection based on the P-POSSUM and SORT models, and individual variables selected on the basis of forward stepwise regression, dropping variables with p>0.1. In order to prevent overfitting of the risk model, variables with incidence <1% in the population will be excluded. The final model will provide a predicted likelihood (risk score) of the outcome of interest (day 7 POMS morbidity) for each patient. These risk scores will then be used as covariates (defining patient risk) in a multilevel regression model. Comparison will be made for risk-adjusted population outcomes (incidence of day 7 POMS morbidity) before and after implementation of the pathway; thus, the ‘after’ group will include patients within the study population who were screened but not subsequently selected for inclusion in the POMSHIP pathway based on the results of their risk assessment. Outcomes will be compared between patients before and after implementation of POMSHIP, and also between those receiving and not receiving the POMSHIP intervention. Data collected in the 6 months after cessation of the service will also be analysed.

Similar to clinical outcomes, we will undertake a ‘before and after’ analysis for costs, presenting the total difference before versus after, as well as the relative cost per patient and per spell. Univariate sensitivity analyses will be conducted on incidence of morbidity and mortality, costs of treatment and investigations, length of total hospital stay, and number of (re)admissions to critical care. These factors will be investigated to examine their effect on the total cost and cost-effectiveness of the service model.

Evaluation of staff and patient perceptions and experiences of the new service

The interview transcripts and observation notes will be subjected to framework analysis.33 Framework analysis allows systematic and comprehensive review of material collected, and between and within-case analysis.34 35 The data will be analysed according to the themes identified in the research objectives and will also include additional themes that emerge from the collected data. A codebook will be developed to maintain consistency in the coding and to carry out triangulation of interview and observation data. Qualitative data will be analysed using NVivo (QSR International V.10, 2014). Two researchers will compare a sample of transcripts to check the coding strategy. Findings will be shared with interview participants to check accuracy.

Integrating findings

Qualitative findings from the interviews with, and observations of, staff and patients will be formatively fed back to the POM team to enable iterative adaptation of their implementation strategy.

Patient and public involvement

This research addresses the following priorities identified by the James Lind Alliance Priority Setting Partnerships: anaesthesia/perioperative care, intensive care, dementia and pressure ulcers.

We are grateful for feedback on the interview topic guide and structured observation guide from the Embedded Research Team Patient and Public Involvement Panel at the Department of Applied Health Research, UCL.

A draft manuscript detailing the results of this study will be circulated to all participants involved in the qualitative research, including patients, giving them an opportunity to feedback any comments for inclusion in the final manuscript which we aim to publish in a peer-reviewed journal.

Ethics and dissemination

Permission was granted by EuroQOL to use the EQ-5D in our evaluation. Patients will be asked to sign a consent form stating that they were happy to be contacted by email or telephone to be asked EQ-5D questions after leaving hospital. Patients and staff will be asked to sign consent forms to record their interviews.

Dissemination

Findings are being shared with the POM team so processes and pathways can be reviewed and refined throughout the implementation. Manuscripts detailing the quantitative and qualitative evaluations will be submitted to peer-reviewed journals for publication. Presentations will be prepared for professional and lay audiences.