SNAP-2 EPICCS: the second Sprint National Anaesthesia Project-EPIdemiology of Critical Care after Surgery: protocol for an international observational cohort study.

INTRODUCTION: The admission of high-risk patients to critical care after surgery is a recommended standard of care. Nevertheless, poor compliance against this recommendation has been repeatedly demonstrated in large epidemiological studies. It is unclear whether this is due to reasons of capacity, equipoise, poor quality clinical care or because hospitals are working creatively to create capacity for augmented care on normal surgical wards. The EPIdemiology of Critical Care after Surgery study aims to address these uncertainties. METHODS AND ANALYSIS: One-week observational cohort study in the UK and Australasia. All patients undergoing inpatient (overnight stay) surgery will be included. All will have prospective data collection on risk factors, surgical procedure and postoperative outcomes including the primary outcome of morbidity (measured using the Postoperative Morbidity Survey on day 7 after surgery) and secondary outcomes including length of stay and mortality. Data will also be collected on critical care referral and admission, surgical cancellations and critical care occupancy. The epidemiology of patient characteristics, processes and outcomes will be described. Inferential techniques (multilevel multivariable regression, propensity score matching and instrumental variable analysis) will be used to evaluate the relationship between critical care admission and postoperative outcome. ETHICS AND DISSEMINATION: The study has received ethical approval from the National Research Ethics Service in the UK and equivalent in Australasia. The collection of patient identifiable data without prior consent has been approved by the Confidentiality Advisory Group (England and Wales) and the Public Privacy and Patient Benefit Panel (Scotland). In these countries, patient identifiable data will be used to link prospectively collected data with national registers of death and inpatient administrative data. The study findings will be disseminated using a multimedia approach with the support of our lay collaborators, to patients, public, policy-makers, clinical and academic audiences. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2017. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

A comprehensive study which has recruited patients from 95% of eligible UK hospitals and we expect similar high levels of coverage in Australasia.

Cost-effectiveness through usage of research networks. Through engagement with a variety of networks including the Quality Audit and Research Coordinator network led by the NIAA Health Services Research Centre, the NIHR Clinical Research Network, and trainee research networks (including the UK’s national overarching anaesthesia network, Research and Audit Federation for Trainees, RAFT) this study has been delivered for less than £65 000.

The ethical approach to data capture, which has enabled us to collect patient data without explicit consent, has reduced the risk of sampling bias in the high-risk population of surgical patients.

The highly pragmatic nature of this study means that there may be a risk of inter-rater reliability in the data capture. We have mitigated this risk by using objective and validated measures.

The snapshot methodology means that issues we identify pertaining to systems pressures (eg, lack of critical care capacity) may not be generalisable throughout the year.

Introduction

Surgical morbidity and mortality is a well-described public health issue. An estimated 313 million operations take place worldwide each year.1 In the UK, mortality in the first month after major elective surgery is around 0.5% to 1%; however, particularly high-risk procedures such as emergency laparotomy have a population mortality of over 11%.2 3 Among short-term survivors, major morbidity is common and associated with increased long-term mortality, even after accounting for known preoperative patient-related risk factors.4

Surgical outcome depends on more than an individual surgeon’s skills in the operating theatre. Patient selection, optimisation and perioperative management delivered by the multidisciplinary team all influence patients’ perioperative pathways and outcomes. Data demonstrating variation in ‘failure to rescue’ rates between hospitals support this assertion. While complication rates may vary twofold or threefold, the mortality rates after developing major complications can vary by 10-fold or 15-fold.5 6 These findings indicate that structures and processes aimed at detecting and treating postoperative complications are delivered more effectively in some hospitals than others. Such interventions might include pre-emptive critical care admission, enhanced postoperative monitoring in postanaesthetic care units or active follow-up by critical care outreach teams. To that end, planned postoperative critical care admission is recommended for the ‘high-risk surgical patient’.7 Cohort studies reveal that despite this, the majority of patients undergoing non-cardiac, non-neurological surgery who die in hospital are discharged from the postoperative recovery room directly to the general ward.8 This is also likely to be true even for the high-risk patient.2 3

There are four likely reasons for this apparent failure to adhere to national recommendations. The first is that clinicians may not always correctly identify patients who are at risk of postoperative complications. While it is true that elderly patients undergoing emergency surgery constitute a high-risk surgical population,9 it is also true that as a result of the much larger volume of elective surgery, the absolute numbers of patients who die postoperatively may be greatest in patients undergoing less acute surgery.8 This observation, known as the ‘Prevention Paradox’, is well-described throughout medicine.10 11 Thus, individualised assessment to identify high-risk patients on the basis of their specific comorbidities, fitness and planned surgical procedure is of importance when considering any patient’s perioperative pathway. Ongoing research comparing the accuracy of different methods of risk stratification (eg, exercise testing versus biomarkers versus risk models) should be informative;12 however, even when accurate methods are developed, their uptake in clinical practice can be poor.13 Thus, developing an understanding of clinicians’ views and practice with regard to perioperative risk stratification is likely to help us contextualise data about critical care admission after surgery.

The second reason is capacity. The UK has fewer critical care beds per head of population than other high-income nations14 15 and thus the hospital system in the UK, and other countries with similar constraints, may lack the capacity to provide critical care even to those patients identified as high risk.

Third, there may be issues around definitions of critical care. Some hospitals may be delivering some aspects of critical care (such as 1:2 nurse: patient ratios) to patients outside the walls of the traditional critical care unit. Patients managed in these ‘enhanced’ settings may not be classified as receiving critical care in large pragmatic studies and thus the potential benefits afforded to these patients not captured in subsequent analyses.

Lastly, some clinicians may not believe routine critical care admission offers benefit to surgical patients. While some patients may be obliged to receive critical care treatment (because they are receiving interventions such as invasive blood pressure monitoring, ventilation or continuous haemodynamic therapy which cannot be delivered on normal wards), many others may only require enhanced monitoring and closer nursing support. The postoperative critical care ‘intervention’, particularly for those patients who do not require specific therapies, has not been tested in a large multicentre randomised controlled trial. This lack of evidence must be balanced against national recommendations that critical care admission should be a standard of care for high-risk patients (>5% predicted mortality).7 It is therefore possible that some clinicians feel justified in not sending their patients to CCU postoperatively (because of equipoise) while others may believe that it would be unethical not to do so (because of guidelines).

We have therefore designed the EPIdemiology of Critical Care after Surgery (EPICCS) study to explore these four issues and to provide a contemporary estimate of the provision of critical care for high-risk surgery in the UK and Australia.

Methods

Study design

EPICCS is a 1-week, prospective observational cohort study of patients and anaesthetists in UK National Health Service (NHS) and Australian hospitals. The study has two components:

Patient study: an observational cohort study of patients undergoing inpatient surgery in participating hospitals to describe the case mix, quantify the risk of morbidity and mortality and explore the effect of pre-emptive critical care admission

Clinician substudy: an observational cross-sectional study of clinician’s perceptions of critical care and risk stratification.

A pilot study has been completed in two UK centres: University College Hospital London and Derriford Hospital (Plymouth). The pilot study has enabled us to refine and modify the study protocol and the final version to be delivered across the UK and Australasia is presented here.

Research questions (RQ)

RQ1. How do clinicians determine the risk of postoperative mortality in clinical practice?

RQ2. Do previously validated risk stratification tools accurately predict postoperative mortality?

RQ3. On what basis do clinicians refer patients for planned postoperative critical care?

RQ4. What factors influence whether patients actually receive planned postoperative critical care?

RQ5. Does immediate critical care admission reduce postoperative morbidity and mortality?

To collect data on all patients undergoing inpatient surgery for 1 week in UK NHS and Australian hospitals (RQ1, 2, 4, 5).

To measure and analyse patient-level estimates of perioperative risk using previously validated risk prediction tools to determine their accuracy (discrimination and calibration) in a comprehensive national sample (RQ2, 3).

To use three different analytic techniques (regression, instrumental variable (IV) and propensity score matched analyses) to measure the relationship between patient risk factors, postoperative critical care admission and patient outcomes (morbidity and mortality) (RQ5).

To survey anaesthetists and surgeons on their attitudes and behaviours regarding risk prediction and postoperative critical care admission (RQ3).

Eligibility criteria

Hospital level

All UK NHS and Australian hospitals which undertake inpatient surgery will be eligible to take part. Hospitals in the UK will be recruited using the NIAA-HSRC’s Quality Audit and Research Coordinator (QuARC) network, aiming for 100% coverage across the UK. There is precedent for this aspiration, from the first Sprint National Anaesthesia Project (SNAP-1) study16 (97% of eligible UK hospitals participation) and the National Audit Projects (100% of eligible UK hospital participation).17 18 In addition to the week of prospective patient-level data collection, we asked all hospitals to complete an organisational survey, aimed at helping us to understand institutional provision of critical care for surgical patients, and to capture information about ‘enhanced care’ provision which may help us to understand the patterns of critical care referral and admission between different centres. Completion of this survey was a stipulation of hospital participation in the main EPICCS study.

Patient level

Inclusion criteria

Adult (≥18 years) patients undergoing surgery, or other interventions, that require the presence of an anaesthetist, and who are expected to require overnight stay in hospital. These would include all procedures taking place in an operating theatre, radiology suite, endoscopy suite or catheter laboratory for which inpatient (overnight) stay is planned, including both planned and emergency/urgent surgery of all types, caesarean section, surgery for complications of childbirth, endoscopy and interventional radiology procedures.

Exclusion criteria

Patients who indicate they do not want to participate in the study, ambulatory surgery, non-surgical obstetrics, ASA-PS (American Society of Anesthesiologists Physical Status score) grade VI, non-interventional diagnostic imaging (eg, CT or MRI scanning without interventions), emergency department or critical care interventions requiring anaesthesia or sedation but no interventional procedure

Clinician substudy

All anaesthetists, intensivists and surgeons, of all grades, who undertake perioperative care for inpatient surgery during the study period will be invited to participate.

Consent

All patients undergoing inpatient surgery (elective or emergency) during the study week will be enrolled onto the main EPICCS study, with an ‘opt-out’ approach to consent. In England and Wales, we have been granted Section 251 exemption by the Confidentiality Advisory Group to include patients without the need to provide written informed consent, for the purpose of data linkage with national registries. In Scotland, we have been granted support from the Public Benefit and Privacy Panel for Health and Social Care for the same permissions. In Northern Ireland, individual Health and Social Care Trusts have agreed to provide non-identifiable data for analysis. Similar provisions exist in Australasia. The reason for an opt-out approach is an attempt to avoid sampling bias. It is likely that patients who are of higher perioperative risk will be unable or unwilling to provide consent—we have data to support this assertion from the SNAP-1 study19 which followed a similar methodology of trying to achieve participation from 100% of eligible UK hospitals for a short-term study. There are also data from other settings which support the notion that the requirement for informed consent can introduce bias into studies where the target population may be critically unwell, and therefore jeopardise the results.20 This therefore poses the risk that we may not have data on exactly the group of patients who may most benefit from critical care admission; hence, introducing bias into our analyses. We will provide patient information sheets and posters inpatient areas informing them of the study and the methods of analysis and data linkage to be used, and information of how to contact the study team to withdraw participation. The provision of patient information leaflets, as well as being good practice and in the interests of patients, was a requirement of the NIHR to enable the EPICCS study to access the support of the local clinical research networks. The Participant Information Sheet is available from the study website (http://www.niaa-hsrc.org.uk/SNAP-2-EpiCCS-Study-Documents#pt).

Clinician level

We are using an implied consent approach, with completion of the questionnaire as evidence of consent. Every perioperative anaesthetist, intensive care clinician and surgeon available in each hospital during the study week will be asked to complete a questionnaire which explores their approach to risk stratification and postoperative care (eg, which risk prediction measures used, if any; what they estimate to be the risk of postoperative death or complications, where they proposed the patient should be cared for postoperatively). Given that there are national guidelines regarding these questions, we are keen to provide reassurance to perioperative anaesthetists and surgeons that their responses will be used in confidence, without risk of litigation or reprisal. We will provide an explanation of this on the front page of the questionnaire, including the fact that their information will be non-identifiable at the analysis stage. The Clinician Perceptions Questionnaire can be found on the study website.

Data collection and follow-up procedures

Perioperative anaesthetists will complete a Case Report Form (CRF; see online supplementary appendix 1) for every patient undergoing surgery during the study week, unless the patient has indicated that they would like to opt-out of the study. As there is the possibility particularly in high-risk or emergency cases, that perioperative anaesthetists may be unable to complete the CRF at the time of surgery, local principle investigators will compare the CRFs completed against local records of patients undergoing surgery on a daily basis to ensure that all eligible patients are included. If a patient is found not to have a form completed, local investigators will be responsible for completing the patient data retrospectively, through accessing the patient notes/hospital results system. The sections on risk stratification will be left blank. All patients who opt-out will be recorded in a site log so that local investigators ensure that their data is not subsequently collected.

For patients who alive but not yet discharged from hospital on day 7 after surgery, inpatient follow-up will take the form of completion of the Postoperative Morbidity Survey (POMS) by local investigators. Postoperative morbidity on day 7 will be assumed absent for those discharged alive before then. The POMS is a commonly used measure of postoperative morbidity and has been validated for both prospective and retrospective data collection.21 22 CRFs will be closed on death or discharge from hospital with the completion of length-of-stay data with inpatient data censoring at 60 days postsurgery. Longer-term follow-up will be achieved through data linkage with central national registries, including NHS Digital (hospital episode statistics data on readmission to hospital and Office of National Statistics data on mortality) in England and Wales, and the NHS Central Register in Scotland. Mortality tracking will continue for 10 years after recruitment to the study ends.

In addition to the patient level data required for the main EPICCS study, the Principle Investigator at each site will be responsible for documenting critical care occupancy two times per day on a structured data entry form (see online supplementary appendix 2) to facilitate the planned IV analysis (see Analysis plan section for details).

No follow-up beyond completion of the clinicians’ perceptions questionnaire will be required.

Analysis plan

Descriptive statistics

The descriptive epidemiology of decision-making, referral and admission to critical care after surgery will be reported. The discrimination and calibration of four previously validated risk stratification tools (the ASA-PS, the Portsmouth Physiological and Operative Severity Score for the enUmeration of Mortality (P-POSSUM) score, the Surgical Risk Scale and the Surgical Outcome Risk Tool)23 24 will be compared using receiver operating characteristic curves and using the Hosmer-Lemeshow X2 statistic. The accuracy of these tools will be compared with clinician estimates of postoperative mortality.

Outcome measures

Our primary outcome will be on day 7 inpatient morbidity as measured by the POMS.

Secondary outcome measures will be mortality during inpatient stay, and at 30 days, 90 days and 1 year; longer-term survival (up to 10 years); cost analysis of planned critical care versus planned ward care; length of hospital stay.

Inferential statistics

We will conduct multivariable regression, propensity score matched and IV analyses to answer the question: Does immediate postoperative critical care admission improve outcome after surgery?

Multivariable regression

Logistic regression will determine independent predictors for critical care referral and admission, including both patient risk factors and structure/process level indicators with attention being paid to the hierarchical structure of the data (patients nested within hospitals and countries). Multilevel multivariable regression will also be used to investigate the relationship between structures (eg, critical care bed capacity as a proportion of hospital beds, staffing ratios, hospital type and size), hospital processes (eg, critical care admission) and patient outcomes adjusted for patient-level risk factors (comorbidity and surgery related). We recognise the risk of ‘confounding by indication’ using traditional regression analysis to examine these relationships, and therefore will also conduct propensity score matching and IV analysis to provide more accurate estimates.

Propensity score matched analysis

Propensity score matching is a well-established method for taking into account selection bias in observational settings. Our dataset draws on the existing literature and previous propensity score matched studies of surgical outcome to include a comprehensive list of variables which should be considered in analyses.23 24

Instrumental variable analysis

As described above, evaluating the effect of direct admission to critical care following surgery in an observational study is difficult because of indication bias. The most unwell patients are most likely to be admitted, and we rely on risk adjustment to compare the outcome of these patients to those not admitted directly. Such comparisons assume that the measurements used to adjust for risk, completely capture all the factors that go into the clinical assessment that was used to allocate treatment. This is equivalent to saying that there is no added value in the ‘end of the bed’ clinical assessment beyond that captured in the preoperative risk score. Unsurprisingly, most clinicians would dispute this.

One solution is to substitute the link between allocation by indication with allocation by randomisation in a controlled trial. Where randomisation is not possible, IV analysis is an alternative technique widely used in the econometric and social sciences literature. IV analysis is a method of estimating the effect of a natural randomisation procedure. There is evidence that because critical care units run at near capacity, that occupancy affects access. This is unlikely to be an issue for the most unwell who would be admitted directly regardless. However, for others, then IV analysis would argue that the number of beds occupied on the CCU at the time of surgery should not affect the outcome of that surgery except through altering the chances of being directly admitted to critical care. This is of particular relevance in the UK where the ratio of critical care to hospital beds is low.

Therefore, information will be collected on occupancy of the critical care unit at the time of surgery. Both physical occupancy, and the available staffing will be considered. The first stage model will examine how occupancy affects the decision making by clinicians, and the delivery of direct admission to critical care. Attention will be paid to the structural factors that affect this, in particular, the size of critical care units. Two IV models will be investigated.

A linear IV model, which ignores the non-linearity of the dependent variable (mortality), will be used to estimate the local average treatment effect (LATE). This is the effect of direct admission on morbidity for the subpopulation of patients that would have been treated if an intensive care unit (ICU) bed been available.

A bivariate probit model, in which mortality is represented as a latent linear variable, will be used to estimate the effect measure on the complete population of eligible patients.

Additional analyses and data sharing

We have developed a process for enabling us to consider requests from investigators outside the core study team and steering group to conduct secondary analyses on EPICCS data. This includes formal consideration by the EPICCS project team and steering committees using a standard data sharing request form; if the request comes in after these structures have been disbanded, the responsibility for reviewing such requests will rest with the Chief Investigator and the Executive Management Board of the NIAA Health Services Research Centre which is overseeing the study management.

Sample size calculation

The sample size calculation is based on the EPICCS main patient study.

There are no realistic estimates available to guide us on the potential effect which critical care admission may have on reducing postoperative morbidity. This issue is further complicated by the likelihood that critical care admission may have a different effect on postoperative outcome depending on the patient and surgical risk factors, and that there is unlikely to be a linear relationship between risk and benefit.

We have used previous studies to guide the allocation proportions and estimates of baseline morbidity. Based on a morbidity rate of 30% on day 7 inpatients admitted postoperatively to the general ward and a 15% relative risk reduction for patients electively admitted to critical care, an allocation ratio of 1:10 (ICU: ward care), R-squared (multiple correlation between the exposure and other covariates)=10% and a dropout rate of 1%, the minimum sample size (n) required is 8177. The 1% drop-out rate in our calculation has been included in order to account for the risk that in some cases, data linkage with the NHS Digital mortality registry may not be possible (because of incorrect patient identifiable data being entered). In 1 week, based on data from the SNAP-1 study,19 we estimate that we should be able to recruit at least 12 000 patients therefore comfortably achieving our sample size.

Study management and funding

The project team is chaired by the Chief Investigator and meets monthly to deliver the day-to-day organisation of the study. A study steering committee with an independent chair meets quarter annually. This steering committee provides multidisciplinary and lay representation on study design and conduct. The study team is responsible to the NIAA Health Services Research Centre Executive Management Board. The study sponsor is University College London. The research costs for the study have been supported by the National Institute for Academic Anaesthesia (Association of Anaesthetists of Great Britain and Ireland Project grant), the Royal College of Anaesthetists and the UCLH NIHR Biomedical Research Centre. In the UK, the study is adopted onto the NIHR Clinical Research Portfolio and equivalents in the devolved nations, and supported by NIHR Local Clinical Research Networks.

Involvement of patients and public

We have had patient and public involvement from inception of this study. Our patient representative has commented on the study design, was a coapplicant on the grant, is a member of the study steering committee and will provide input to all aspects of the study including dissemination.

Dissemination and transparency policy

We intend to present the results online via the study website, in peer-reviewed scientific journals and in the form of conference presentations. In addition to academic publications, we will provide specific summary reports for the following groups:

Healthcare policy-makers—this will include medical and nursing Royal Colleges, specialist societies, Department of Health, NHS England, NHS Wales, NHS Scotland and Health and Social Care Ireland and equivalent in Australasia.

Patients and the Public—our lay representative and the Lay representative group at the Royal College of Anaesthetists will provide support in our dissemination to the non-medical audience.

Participating NHS Trusts and Health Boards—all NHS Chief executives of participating organisations will be sent a summary of the key findings.

The study investigators and steering committee have agreed a policy for authorship and contributor status for all manuscripts which arise from this study, which is published on our study website.

Conclusions

EPICCS will use a pragmatic study design to obtain data on a comprehensive sample of patients undergoing major surgery in the UK and Australasia, to investigate the provision of critical care after surgery and its impact on patient outcomes. In addition, we will define clinicians’ perceptions on the value, utility and effectiveness of risk prediction systems and critical care in surgical patients. Through our comprehensive recruitment strategy, we also hope to achieve excellent engagement with clinicians which should strengthen the dissemination of our findings and the likely impact on colleagues and on patients.